A method, device, storage medium and program product for detecting semi-submerged sea litter interfering with light

Abstract

This application provides a method, apparatus, storage medium, and program product for detecting semi-submerged marine debris that is resistant to light interference, relating to the field of computer vision technology. The method includes: segmenting and temporally processing a sequence of images continuously acquired from a target sea area; using temporal fluctuation feature values ​​characterizing brightness fluctuations and temporal shift feature values ​​characterizing color shifts to divide the sea surface spatial location into a low-frequency fluctuation stable region, a high-frequency fluctuation stable region, and a dynamic interference region; based on whether the grayscale curve across the waterline meets a preset continuous transition condition, associating the low-frequency fluctuation stable region and the high-frequency fluctuation stable region originating from the same marine debris object as suspected marine debris areas; and performing morphological feature detection on the suspected marine debris areas to obtain the semi-submerged marine debris detection result. This solves the technical problem of low detection accuracy of marine debris in related technologies, achieving the technical effect of improving the detection accuracy of marine debris.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, device, storage medium, and program product for detecting semi-submerged marine debris that is resistant to light interference. Background Technology

[0002] With the increasing awareness of marine ecological protection, the monitoring and cleanup of marine debris is a key part of coastal environmental governance. The efficient and accurate identification of marine debris is of great significance for protecting the marine ecological environment and promoting social and economic development.

[0003] In related technologies, visual detection methods based on conventional convolutional neural networks are commonly used. Specifically, the process begins by acquiring images of the sea surface using an image acquisition device, and then labeling these images with garbage categories and location tags to construct a training dataset. This training dataset is then input into a pre-defined convolutional neural network model for iterative training to extract visual features such as color, texture, and contour from the sea surface images, resulting in a well-trained convolutional neural network model. Next, during the actual detection phase, the sea surface images to be detected, acquired in real-time by the image acquisition device, are input into the trained convolutional neural network model. Finally, the bounding box regression and classification algorithms within the trained convolutional neural network model are used to extract features from the sea surface images to output the predicted category and location coordinates of the marine debris.

[0004] However, when using the aforementioned visual detection method based on conventional convolutional neural networks, the image acquisition device may capture images of the sea surface to be detected, including reflections from the sea surface, due to the influence of strong light. When the marine debris in the image to be detected is in a semi-submerged state and is superimposed with sea surface reflections, the trained convolutional neural network model may have difficulty accurately identifying the real and effective features of the marine debris, and may easily misjudge the sea surface reflections as marine debris or miss the real marine debris, thus resulting in a low detection accuracy of marine debris in related technologies. Summary of the Invention

[0005] This application provides a method, apparatus, storage medium, and program product for detecting semi-submerged marine debris that is resistant to light interference, in order to improve the detection accuracy of marine debris.

[0006] In a first aspect, this application provides a method for detecting semi-submerged marine debris that resists light interference, applied to a marine debris detection device. The method includes: acquiring a sequence of images to be detected continuously acquired by an image acquisition device over the same target sea area under a stable field of view; dividing each frame of the sea surface image in the image sequence into multiple image blocks to obtain multiple sets of image block sequences; performing brightness temporal processing on each set of image block sequences to obtain temporal fluctuation feature values; and performing color temporal processing on each set of image block sequences to obtain temporal offset feature values; and dividing the sea surface spatial location corresponding to the multiple sets of image block sequences into low-frequency fluctuation stable areas and high-frequency fluctuation stable areas based on the temporal fluctuation feature values ​​and the temporal offset feature values. The process involves defining a fixed area and a dynamic interference area. The adjacent common boundary, excluding the dynamic interference area, within a preset neighborhood between the low-frequency fluctuation stable area and the high-frequency fluctuation stable area is identified as a candidate waterline. The grayscale values ​​of the first pixel within a first preset length range are extracted along the normal direction of the candidate waterline towards the low-frequency fluctuation stable area, and the grayscale values ​​of the second pixel within a second preset length range are extracted towards the high-frequency fluctuation stable area. A cross-waterline grayscale curve is generated based on the first and second pixel grayscale values. If the cross-waterline grayscale curve satisfies a preset continuous transition condition, the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are associated as a suspected marine debris area. Morphological feature detection is performed on the suspected marine debris area to obtain the detection results of semi-submerged marine debris.

[0007] By employing the aforementioned technical solution, and utilizing the temporal characteristics of continuously acquired image sequences of the same target sea area under a stable field of view, the temporal fluctuation feature value and the temporal shift feature value, which can respectively characterize the degree of brightness fluctuation, are obtained by performing brightness temporal processing and color temporal processing on each group of image block sequences. Since the stable imaging portion of marine debris exposed above the water surface remains stable in both brightness and color, the semi-submerged imaging portion of marine debris periodically covered by the water surface exhibits periodic fluctuations in brightness, but the color remains primarily that of the marine debris itself. However, the imaging portion of the sea surface undergoes drastic changes in both brightness and color due to instantaneous specular reflection caused by strong light. Therefore, based on the temporal fluctuation feature value and the temporal shift feature value, the sea surface space can be... The location is accurately divided into low-frequency fluctuation stable region, high-frequency fluctuation stable region, and dynamic interference region, achieving the removal of strong light interference components. Furthermore, the adjacent common boundary (excluding the dynamic interference region) within a preset neighborhood range between the low-frequency and high-frequency fluctuation stable regions is determined as a candidate waterline. Based on whether the grayscale curve across the waterline along the normal direction of the candidate waterline meets a preset continuous transition condition, it is determined whether the low-frequency and high-frequency fluctuation stable regions originate from the same marine debris object. This associates low-frequency and high-frequency fluctuation stable regions originating from the same marine debris object as suspected marine debris areas, avoiding misjudging sea surface reflections as marine debris or missing actual marine debris. Finally, morphological feature detection is performed on the suspected marine debris areas to output the detection results of semi-submerged marine debris. This solves the technical problem of low detection accuracy of marine debris in related technologies, achieving the technical effect of improving the detection accuracy of marine debris.

[0008] Optionally, based on temporal fluctuation feature values ​​and temporal offset feature values, the sea surface spatial locations corresponding to multiple sets of image patch sequences are divided into low-frequency fluctuation stable regions, high-frequency fluctuation stable regions, and dynamic interference regions. This includes: obtaining preset fluctuation discrimination thresholds and preset offset discrimination thresholds; performing a first comparison between the temporal fluctuation feature values ​​of each set of image patch sequences and the preset fluctuation discrimination thresholds to obtain a first comparison result; and performing a second comparison between the temporal offset feature values ​​of each set of image patch sequences and the preset offset discrimination thresholds to obtain a second comparison result; if the first comparison result indicates that the temporal fluctuation feature value is less than the preset fluctuation discrimination threshold, and the second comparison result indicates that the temporal offset feature value is less than the preset offset discrimination threshold, then the sea surface spatial location corresponding to each set of image patch sequences is determined as a low-frequency fluctuation stable region. The low-frequency fluctuation stable region includes the stable imaging portion of marine debris exposed above the water surface. When the first comparison result indicates that the time-domain fluctuation feature value is greater than or equal to the preset fluctuation discrimination threshold, and the second comparison result indicates that the time-domain offset feature value is less than the preset offset discrimination threshold, the sea surface spatial position corresponding to each group of image block sequences is determined as the high-frequency fluctuation stable region. The high-frequency fluctuation stable region represents the semi-submerged imaging portion of marine debris periodically covered by the water surface. When the first comparison result indicates that the time-domain fluctuation feature value is greater than or equal to the fluctuation discrimination threshold, and the second comparison result indicates that the time-domain offset feature value is greater than or equal to the offset discrimination threshold, the sea surface spatial position corresponding to each group of image block sequences is determined as the dynamic interference region. The dynamic interference region represents the instantaneous specular reflection imaging portion of the sea surface caused by strong light.

[0009] By adopting the above technical solution, a first comparison is performed on the temporal fluctuation feature values ​​of each group of image block sequences using a preset fluctuation discrimination threshold, and a second comparison is performed on the temporal offset feature values ​​using a preset offset discrimination threshold, to obtain the first comparison result and the second comparison result respectively. Utilizing three types of feature differences—small brightness fluctuations and small color shifts in the stable imaging portion of marine debris exposed above the water surface, large brightness fluctuations but small color shifts in the semi-submerged imaging portion of marine debris periodically covered by the water surface, and large brightness fluctuations and large color shifts in the instantaneous specular reflection imaging portion of the sea surface caused by strong light—based on the combined logic of the first comparison result and the second comparison result, the spatial location of the sea surface corresponding to each group of image block sequences is accurately divided into a low-frequency fluctuation stable region, a high-frequency fluctuation stable region, or a dynamic interference region, providing a reliable regional division basis for subsequent candidate waterline determination and association of suspected marine debris areas.

[0010] Optionally, if the grayscale curve across the waterline satisfies a preset continuous transition condition, the low-frequency fluctuation stable region and the high-frequency fluctuation stable region are associated as suspected marine debris areas. This includes: locating the waterline position point corresponding to the candidate waterline on the grayscale curve across the waterline, and dividing the grayscale curve across the waterline into a first grayscale sub-curve located on one side of the low-frequency fluctuation stable region and a second grayscale sub-curve located on one side of the high-frequency fluctuation stable region based on the waterline position point; determining the grayscale value of the first endpoint of the first grayscale sub-curve at the waterline position point and the grayscale value of the second endpoint of the second grayscale sub-curve at the waterline position point, and determining the grayscale break value at the waterline position point based on the first endpoint grayscale value and the second endpoint grayscale value, wherein the grayscale break value characterizes the low-frequency wave... The degree of grayscale jump between the dynamic stable region and the high-frequency fluctuation stable region at the candidate waterline is determined; the first grayscale mean of all grayscale values ​​on the first grayscale sub-curve and the second grayscale mean of all grayscale values ​​on the second grayscale sub-curve are determined, and the grayscale transition difference between the two sides is determined based on the first grayscale mean and the second grayscale mean. The grayscale transition difference between the two sides characterizes the degree of deviation between the overall grayscale level of the low-frequency fluctuation stable region and the overall grayscale level of the high-frequency fluctuation stable region; the grayscale break value and the grayscale transition difference between the two sides are used to determine whether the grayscale curve across the waterline meets the preset continuous transition condition, and if the grayscale curve across the waterline meets the preset continuous transition condition, the low-frequency fluctuation stable region and the high-frequency fluctuation stable region are associated as suspected marine debris areas.

[0011] By adopting the above technical solution, the grayscale curve of the waterline is divided into a first grayscale sub-curve and a second grayscale sub-curve, with the waterline position point corresponding to the candidate waterline as the boundary. The grayscale curve of the waterline is quantitatively characterized from two dimensions: the grayscale break value at the waterline position point and the grayscale transition difference between the two sides formed by the grayscale mean values ​​on both sides. The grayscale break value represents the degree of grayscale jump at the candidate waterline, and the grayscale transition difference between the two sides represents the degree of deviation of the overall grayscale level between the low-frequency fluctuation stable area and the high-frequency fluctuation stable area. The two together reflect the spatial continuity of the grayscale curve of the waterline. Based on the grayscale break value and the grayscale transition difference between the two sides, it is determined whether the grayscale curve of the waterline meets the preset continuous transition condition. This can accurately identify whether the low-frequency fluctuation stable area and the high-frequency fluctuation stable area originate from the same marine debris object, and avoid misassociating spatially adjacent areas but belonging to different objects as suspected marine debris areas.

[0012] Optionally, based on the grayscale break value and the grayscale transition difference between the two sides, it is determined whether the grayscale curve across the waterline meets the preset continuous transition condition. If the grayscale curve across the waterline meets the preset continuous transition condition, the low-frequency fluctuation stable region and the high-frequency fluctuation stable region are associated as suspected marine debris areas. This includes: obtaining a preset break threshold and a preset transition difference range; performing a third comparison between the grayscale break value and the preset break threshold to obtain a third comparison result; and performing a fourth comparison between the grayscale transition difference between the two sides and the preset transition difference range to obtain a fourth comparison result; if the third comparison result indicates that the grayscale break value is greater than... If the grayscale value is equal to the preset fracture threshold, or if the grayscale transition difference between the two sides indicated by the fourth comparison result is not within the preset transition difference range, it is determined that the grayscale curve across the waterline does not meet the preset continuous transition condition, and the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are marked as non-associated sea surface areas; if the grayscale fracture value indicated by the third comparison result is less than the preset fracture threshold, and the grayscale transition difference between the two sides indicated by the fourth comparison result is within the preset transition difference range, it is determined that the grayscale curve across the waterline meets the preset continuous transition condition, and the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are associated as suspected marine debris areas.

[0013] By adopting the above technical solution, an upper bound constraint is imposed on the grayscale fracture value using a preset fracture threshold, and a bidirectional interval constraint is imposed on the grayscale transition difference on both sides using a preset transition difference range. The third and fourth comparison results are combined to determine whether the grayscale curve across the waterline meets the preset continuous transition condition. When the grayscale fracture value is too large or the grayscale transition difference on both sides exceeds the preset transition difference range, the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are marked as non-associated sea surface areas. Otherwise, they are associated as suspected marine debris areas. In this way, robust discrimination of the association relationship of suspected marine debris areas is achieved in the context of complex sea surface, reducing the probability of false association.

[0014] Optionally, luminance temporal processing is performed on each of the multiple image block sequences to obtain temporal fluctuation feature values, and color temporal processing is performed on each of the image block sequences to obtain temporal offset feature values, including: color space conversion is performed on each frame image block in each image block sequence to obtain the luminance channel component and chrominance channel component of each frame image block in a preset color space; the luminance channel component of each frame image block is calculated using a first intra-block spatial mean to obtain the image block luminance mean, and the chrominance channel component of each frame image block is calculated using a second intra-block spatial mean to obtain the image block chrominance mean; The average brightness values ​​of each image patch sequence are arranged in a first temporal sequence to obtain a brightness average time sequence. The discrete statistics of the brightness average time sequence in the time dimension are determined and the discrete statistics are identified as temporal fluctuation feature values. The average chromaticity values ​​of each image patch sequence are arranged in a second temporal sequence to obtain a chromaticity average time sequence. A preset sea surface reference chromaticity parameter is obtained, and the chromaticity distance sequence between each chromaticity average value in the chromaticity average time sequence and the preset sea surface reference chromaticity parameter in the preset color space is determined. The chromaticity distance that satisfies the preset maximum value condition in the chromaticity distance sequence is identified as a temporal offset feature value.

[0015] By employing the aforementioned technical solution, color space conversion is used to decouple the luminance and chrominance information of each frame image block into luminance channel components and chrominance channel components. Intra-block spatial mean calculation and temporal arrangement are then performed on both channels to obtain luminance mean temporal sequences and chrominance mean temporal sequences. In the luminance channel, the discrete statistics of the luminance mean temporal sequence in the time dimension are used as temporal fluctuation feature values ​​to characterize the degree of periodic luminance fluctuations at the corresponding sea surface spatial location of the image block sequence. In the chrominance channel, the chrominance distance sequence between each chrominance mean in the chrominance mean temporal sequence and a preset sea surface reference chrominance parameter is calculated. The chrominance distance satisfying a preset maximum value condition is taken as the temporal offset feature value to characterize the maximum deviation of the corresponding sea surface spatial location from the typical sea surface color. This makes the temporal fluctuation feature values ​​sensitive to luminance changes caused by the periodic coverage of marine debris by the water surface, and the temporal offset feature values ​​sensitive to color abrupt changes caused by strong light, providing a characteristic basis for the accurate division of low-frequency fluctuation stable regions, high-frequency fluctuation stable regions, and dynamic interference regions.

[0016] Optionally, morphological feature detection is performed on the suspected marine debris area to obtain the semi-submerged marine debris detection results, including: extracting connected components from the suspected marine debris area to obtain candidate connected components; determining the principal axis direction of the intersection of the candidate connected components and the candidate waterline, and using the principal axis direction as the waterline extension direction of the candidate waterline; dividing the candidate connected components into an exposed portion located on the side of the low-frequency fluctuation stable region and a semi-submerged portion located on the side of the high-frequency fluctuation stable region, using the candidate waterline as the boundary line; determining a first size parameter of the exposed portion along the waterline extension direction, and determining a second size parameter of the semi-submerged portion along the waterline extension direction, wherein the first size parameter includes the waterline extension direction. The exposed portion projection length and the exposed portion projection width perpendicular to the waterline extension direction are defined. The second dimension parameter includes the semi-submerged portion projection length along the waterline extension direction and the semi-submerged portion projection width perpendicular to the waterline extension direction. The waterline dimension continuity index and overall morphological proportion parameter of the candidate connected region are determined according to the first dimension parameter and the second dimension parameter. If the waterline dimension continuity index is greater than or equal to a preset continuity threshold and the overall morphological proportion parameter is within the preset morphological proportion range, the candidate connected region is determined as semi-submerged marine debris. The candidate connected region position, the first dimension parameter, and the second dimension parameter are used as the semi-submerged marine debris detection result.

[0017] By employing the above technical solution, candidate connected regions are extracted from suspected marine debris areas. These regions are then divided into exposed and semi-submerged portions using the candidate waterline as a boundary. The projected length, width, and projected width of the exposed and semi-submerged portions are calculated along the waterline extension direction. Furthermore, the continuity of the dimensions of the candidate connected regions when crossing the waterline is evaluated using a waterline-crossing dimension continuity index, and the overall morphology of the candidate connected regions is evaluated using an overall morphological proportion parameter to determine if the overall morphology conforms to typical marine debris characteristics. If the waterline-crossing dimension continuity index is greater than or equal to a preset continuity threshold, and the overall morphological proportion parameter is within a preset morphological proportion range, the candidate connected region is identified as semi-submerged marine debris. The location of the candidate connected region, the first dimension parameter, and the second dimension parameter are output as the semi-submerged marine debris detection result. This eliminates non-target objects such as scattered reflective material and isolated floating objects from a geometric morphology perspective, ensuring the accuracy of the semi-submerged marine debris detection results.

[0018] Optionally, after using the candidate connected component location, the first size parameter, and the second size parameter as the detection result of semi-submerged marine debris, the method further includes: extracting a target image region sequence corresponding to the candidate connected component from the image sequence to be detected based on the candidate connected component location; and dividing each frame of the target image region sequence into an exposed portion sub-image located on one side of the low-frequency fluctuation stable region and a semi-submerged portion sub-image located on one side of the high-frequency fluctuation stable region, using the candidate waterline as the dividing line; and constructing a semi-submerged ruler based on the first length ratio of the exposed portion projection length to the semi-submerged portion projection length and the second width ratio of the exposed portion projection width to the semi-submerged portion projection width. The prior vector of the exposed part sub-image, the semi-submerged part sub-image, and the semi-submerged size prior vector are input into a pre-trained marine debris recognition model to obtain the category recognition result and confidence score output by the marine debris recognition model. The category recognition result represents the debris category to which the semi-submerged marine debris belongs, and the confidence score represents the reliability of the category recognition result. If the confidence score is greater than or equal to the preset confidence threshold, the category recognition result and confidence score are added to the semi-submerged marine debris detection result. If the confidence score is less than the preset confidence threshold, the candidate connected components are marked as regions to be reviewed, and multi-frame temporal review operations are performed on the regions to be reviewed.

[0019] By adopting the above technical solution, each frame of the target image region sequence is divided into exposed part sub-images and semi-submerged part sub-images, with the candidate waterline as the boundary. A semi-submerged size prior vector is constructed based on the first length ratio of the exposed part projection length to the semi-submerged part projection length and the second width ratio of the exposed part projection width to the semi-submerged part projection width. The exposed part sub-images, semi-submerged part sub-images, and semi-submerged size prior vector are input into a pre-trained marine debris recognition model, so that the marine debris recognition model outputs category recognition results and confidence scores based on the fusion of appearance features and waterline size prior information. The confidence scores are graded by setting a pre-set confidence threshold. High confidence results are directly accepted and added to the semi-submerged marine debris detection results, while low confidence results are re-discriminated through multi-frame temporal verification, further reducing the probability of misjudgment caused by instantaneous interference in a single frame and improving the reliability of marine debris detection.

[0020] Secondly, embodiments of this application provide a marine debris detection device, which includes: one or more processors and a memory; the memory is coupled to one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the marine debris detection device to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, embodiments of this application provide a computer-readable storage medium including program instructions that, when executed on a marine debris detection device, cause the marine debris detection device to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a marine debris detection device, cause the marine debris detection device to perform the method described in the first aspect and any possible implementation thereof.

[0023] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0024] 1. By spatially segmenting and extracting temporal features from the image sequence to be detected, the spatial location of the sea surface is divided into a low-frequency fluctuation stable region, a high-frequency fluctuation stable region, and a dynamic interference region from two dimensions: brightness fluctuation and color shift. This achieves feature-level separation between the instantaneous specular reflection imaging part of the sea surface caused by strong light and the imaging part of the semi-submerged marine debris, effectively distinguishing between sea surface reflection and semi-submerged marine debris.

[0025] 2. By determining candidate waterlines and verifying the continuous transition of the grayscale curve across the waterline, and combining the geometric constraints of the cross-waterline size continuity index and the overall shape ratio parameter of the candidate connected domain, we can achieve joint identification of semi-submerged marine debris from the dual dimensions of imaging physical continuity and buoyancy geometric constraints. This effectively avoids the situation of misjudging sea surface reflections as marine debris or missing real marine debris due to relying solely on a single appearance feature, thereby improving the detection accuracy of semi-submerged marine debris.

[0026] 3. By inputting the exposed part sub-image, the semi-submerged part sub-image, and the semi-submerged size prior vector into the pre-trained marine debris recognition model, and classifying the confidence score based on a pre-set confidence threshold, and performing multi-frame temporal verification operation on candidate connected components with low confidence, the category recognition takes into account both appearance features and waterline size prior information. The temporal verification further suppresses occasional interference, improves the robustness of marine debris category recognition and the verifiability of detection results, and reduces the probability of misidentification caused by instantaneous changes in illumination or short-term wave disturbances in a single frame. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a method for detecting semi-submerged marine debris that resists light interference, as described in this application.

[0028] Figure 2 This is a schematic diagram of the physical structure of a marine debris detection device in the embodiments of this application. Detailed Implementation

[0029] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0030] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0031] This application provides a method for detecting semi-submerged marine debris that is resistant to light interference. (See reference...) Figure 1 , Figure 1 This is a flowchart illustrating a method for detecting semi-submerged marine debris resistant to light interference, as described in this application, including the following steps:

[0032] Step S101: Obtain the sequence of images to be detected continuously acquired by the image acquisition device in the same target sea area under the stable field of view, and divide each frame of sea surface image in the sequence of images to be detected into multiple image blocks to obtain multiple sets of image block sequences.

[0033] Step S102: Perform brightness temporal processing on each of the multiple image block sequences to obtain temporal fluctuation feature values; and perform color temporal processing on each of the multiple image block sequences to obtain temporal offset feature values.

[0034] Step S103: Based on the temporal fluctuation feature value and the temporal offset feature value, the sea surface spatial location corresponding to multiple sets of image block sequences is divided into a low-frequency fluctuation stable area, a high-frequency fluctuation stable area, and a dynamic interference area.

[0035] Step S104: The adjacent common boundary within the preset neighborhood range between the low-frequency fluctuation stable region and the high-frequency fluctuation stable region, excluding the dynamic interference region, is determined as the candidate waterline. The first pixel gray value within a first preset length range is extracted along the normal direction of the candidate waterline to the low-frequency fluctuation stable region and the second pixel gray value within a second preset length range is extracted to the high-frequency fluctuation stable region. A cross-waterline gray curve is generated based on the first pixel gray value and the second pixel gray value.

[0036] Step S105: If the grayscale curve across the waterline meets the preset continuous transition conditions, the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are associated as suspected marine debris areas.

[0037] Step S106: Morphological feature detection is performed on the suspected marine debris area to obtain the detection results of semi-submerged marine debris.

[0038] In this context, "field-of-view stability" refers to the stability of the image acquisition device's optical axis pointing, focal length, field of view, and spatial pose relative to the target sea area throughout the acquisition duration of the image sequence to be detected. This ensures that the pixel coordinates corresponding to the same target sea area remain essentially unchanged in each consecutively acquired frame of sea surface images. Field-of-view stability can be determined by the inertial measurement unit detecting that the attitude angle deviation of the image acquisition device is less than a preset attitude threshold, or by matching feature points of static reference targets such as shorelines, fixed buoys, and coastal structures in adjacent frames of sea surface images. If the average pixel displacement of the matched feature point pairs is less than a preset displacement threshold, the image acquisition device is considered to be in a field-of-view stability state. The purpose of setting a field-of-view stability state is to ensure that the same spatial location on the sea surface corresponds to the same set of pixel coordinates in the time dimension, making the statistical analysis of subsequent temporal fluctuation feature values ​​and temporal offset feature values ​​physically comparable. The target sea area refers to the water surface area covered by the field of view of the image acquisition device. The spatial range of the target sea area includes nearshore sea surface, harbor basin water surface, aquaculture area water surface, or other open or semi-open water areas requiring marine debris monitoring. The image sequence to be detected refers to a time series consisting of multiple frames of sea surface images continuously acquired by the image acquisition device under a stable field of view and arranged in chronological order. The preset frame rate is not less than 15 frames / second, and the duration of the image sequence to be detected is not less than 3 times the typical wave cycle of the sea surface (i.e., usually not less than 6 seconds), so that the time series statistics can cover a sufficient number of wave cycles and suppress random fluctuation components.

[0039] In this context, an image patch refers to a rectangular pixel region obtained by non-overlapping segmentation of a single-frame sea surface image along the row and column directions according to a preset size. The segmentation size of the image patch matches the projection size of the typical wave wavelength in the target sea area onto the image plane, allowing a single image patch to cover the complete local structure from wave crest to trough. This results in the brightness variations generated by the waves within the image patch exhibiting a periodic modulation of the spatial mean within the single image patch, rather than a phase difference between different image patches. Each image patch can range from 16×16 pixels to 32×32 pixels, with the specific size determined by conversion based on the focal length, pixel size, and observation distance of the image acquisition device according to imaging geometry. An image patch sequence refers to a temporal sequence composed of image patches located at the same pixel coordinate positions in the image sequence to be detected, arranged chronologically, corresponding one-to-one with a fixed spatial location on the sea surface in the target sea area. Brightness temporal processing refers to the process of performing intra-block spatial statistics on the brightness information of each frame image patch in a set of image patch sequences, followed by serialization and quantization in the temporal dimension; color temporal processing refers to the similar processing of chromaticity information. Temporal fluctuation feature value refers to a quantitative indicator that characterizes the degree of periodic fluctuation in brightness of the corresponding sea surface spatial location in the temporal dimension; temporal offset feature value refers to a quantitative indicator that characterizes the maximum degree to which the color of the sea surface spatial location deviates from the typical color of the sea surface in the temporal dimension.

[0040] The low-frequency fluctuation stable region refers to an area in the sea surface with small brightness fluctuations and small color shifts. Physically, this region includes, but is not limited to, the following imaging objects: stable imaging portions of marine debris fully exposed above the water surface, stable imaging portions of calm sea backgrounds, and stable imaging portions of distant coastlines or fixed structures. In this embodiment, the low-frequency fluctuation stable region itself is not directly equivalent to marine debris, but rather serves as a reference area on one side of the candidate waterline determination process. Only when the low-frequency fluctuation stable region and the high-frequency fluctuation stable region are spatially adjacent and pass the verification of continuous transition conditions in the grayscale curves across the waterline are they associated as suspected marine debris areas. Therefore, although non-debris imaging objects such as calm sea backgrounds are also included in the low-frequency fluctuation stable region, they are naturally excluded in subsequent steps because they do not meet the continuous grayscale transition conditions with the high-frequency fluctuation stable region (a pure sea background and a partially submerged debris portion do not have the continuous transition characteristics of the same object in grayscale). The high-frequency fluctuation stable region refers to an area with large brightness fluctuations but small color shifts. Physically, it includes the imaging portion of marine debris that is periodically covered or partially submerged by waves. Its brightness fluctuates with the wave cycle, but the main color remains the inherent color of the marine debris itself. The dynamic interference region refers to an area with large brightness fluctuations and large color shifts. Physically, it corresponds to the local imaging abrupt changes caused by instantaneous specular reflection from the sea surface under strong light. The candidate waterline is the boundary line between the exposed and partially submerged parts of marine debris on the image plane. In this embodiment, it is initially located by the adjacent common boundary between the low-frequency fluctuation stable region and the high-frequency fluctuation stable region, excluding the dynamic interference region. The waterline-crossing grayscale curve is a one-dimensional grayscale sequence formed by the spatial arrangement of pixel grayscale values ​​extracted along the candidate waterline normal direction across both the low-frequency fluctuation stable region and the high-frequency fluctuation stable region. The preset continuous transition condition is a discrimination condition used to determine whether the waterline-crossing grayscale curve represents continuous imaging of the same physical object under both water and air media. Its core is that the grayscale curve presents a smooth transition rather than an abrupt jump on both sides of the candidate waterline. A suspected marine debris area refers to a merged region of low-frequency and high-frequency stable fluctuation areas that are identified through correlation analysis as potentially originating from the same marine debris object.

[0041] In the above embodiments, long-term observation of the sea surface scene revealed that various visual objects on the sea surface exhibit significantly different temporal behaviors in terms of brightness fluctuation and color shift: the stable imaging portion of marine debris exposed above the water surface remains stable in both brightness and color (low brightness fluctuation and low color shift); the semi-submerged imaging portion of marine debris periodically covered by the water surface shows periodic modulation of surface brightness with the thickness of the water surface cover layer as the wave cycle changes (high brightness fluctuation), but because the inherent color of the marine debris itself is not completely obscured by the seawater (the color of the marine debris under the shallow cover layer still dominates the imaging), the degree of color deviation of the semi-submerged imaging portion from the sea surface reference chromaticity is still relatively low; while the instantaneous specular reflection imaging portion of the sea surface caused by strong light has a brightness close to the saturation value of the imaging sensor (severe brightness fluctuation), and the imaging light comes directly from the specular reflection of sunlight. This spectral distribution is significantly different from the diffuse reflection spectrum of the seawater itself, and it shows a significant abrupt change in the chromaticity plane that deviates significantly from the sea surface reference chromaticity (high color shift).

[0042] In the above embodiments, based on the separability of the three types of imaging objects on the two-dimensional temporal feature plane, this embodiment constructs a discrimination feature space with brightness temporal fluctuations and chromaticity temporal shifts as orthogonal dimensions. (This discrimination feature space has a significant advantage in physical separability compared to commonly used methods such as frame difference, optical flow, and brightness difference in modeling methods. A single brightness difference can only distinguish between stable and unstable objects, but cannot distinguish between semi-submerged objects with periodic brightness fluctuations but stable colors and instantaneous reflections with abrupt changes in both brightness and color. This is the fundamental reason for the low accuracy of semi-submerged marine debris detection under strong light.) In specific implementation, the image acquisition device can be an industrial visible light camera fixedly installed on a shore observation tower, offshore buoy platform, or harbor railing support. It acquires the image sequence to be detected in the target sea area at a preset frame rate (e.g., 25 frames / second), and the sequence duration is T seconds (e.g., T is 8 seconds to cover several complete cycles of typical wave cycles of 2 to 6 seconds under Beaufort sea states 3 to 5). The inertial measurement unit integrated into the device body monitors the changes in pitch, yaw and roll angles in real time. When the changes in all three axes are less than 0.2 degrees, the image acquisition device is determined to be in a stable field of view. If the device deviates from the stable field of view at any moment during the sequence acquisition, the current sequence is discarded and acquisition is restarted.

[0043] In the above embodiments, each frame of the sea surface image is divided into several image blocks along the row and column directions, with non-overlapping B×B pixels. The value of B (i.e., the side length of the image block) is determined by imaging geometric parameters: Let the focal length of the image acquisition device be f, the pixel size be p, the slant distance to the target sea area be d, and the typical wave wavelength be λ. Then, the actual size of the sea surface corresponding to a single image block should satisfy... ≥ To ensure that each image patch includes a brightness modulation component of at least half a wave wavelength, in this embodiment, f=25 mm, p=3.45 μm, d=80 m, λ=2 m, substituting into the above formula yields B≥9.1. Considering subsequent temporal statistical stability and alignment to integer powers of 2 for parallel computation, B=16 is chosen. Image patches located at the same pixel coordinates in the image sequence to be detected are arranged in chronological order to form an image patch sequence. Each group of image patch sequences corresponds to a fixed sea surface spatial position in the target sea area. Brightness temporal processing is performed on each group of image patch sequences to obtain temporal fluctuation feature values, and color temporal processing is performed on each group of image patch sequences to obtain temporal offset feature values. Based on the temporal fluctuation feature values ​​and temporal offset feature values, the sea surface spatial position corresponding to each group of image patch sequences is classified into a low-frequency fluctuation stable region, a high-frequency fluctuation stable region, or a dynamic interference region.

[0044] In the above embodiments, the process of determining the candidate waterline is as follows: traverse each image block in the low-frequency fluctuation stable region, and search for high-frequency fluctuation stable region image blocks within a 3×3 image block neighborhood around each image block. If a high-frequency fluctuation stable region image block is found, record the common boundary between the low-frequency fluctuation stable region image block and the high-frequency fluctuation stable region image block as the initial candidate boundary. Subsequently, with the initial candidate boundary as the center, expand a preset neighborhood range to both sides along the normal direction of the initial candidate boundary (i.e., the direction perpendicular to the common boundary). (For example, the preset neighborhood range can be 2 image block widths, i.e., expanding 2 image block widths to the low-frequency fluctuation stable region side and 2 image block widths to the high-frequency fluctuation stable region side), forming a strip-shaped neighborhood verification area with the initial candidate boundary as the central axis. Within the strip-shaped neighborhood verification area, the region category labels of all image blocks are checked one by one. If there are no image blocks marked as dynamic interference areas within the strip-shaped neighborhood verification area, the initial candidate boundary is determined as a candidate waterline. If there are at least one image block marked as dynamic interference areas within the strip-shaped neighborhood verification area, the initial candidate boundary is discarded and is not used as a candidate waterline.

[0045] In the above embodiments, for a real semi-submerged marine debris object, the boundary between its stable surface portion and its semi-submerged underwater portion should be smoothly transitioned and should not be subject to dynamic interference caused by instantaneous specular reflection. If a dynamic interference area image block appears in the strip-shaped neighborhood verification area near the candidate waterline, it indicates that the two sides are likely just seemingly adjacent independent objects separated by strong reflective stripes and should not be simply associated. It should be noted that the value of the preset neighborhood range is related to the image block size and the width of a typical sea surface reflective stripe. In this embodiment, two image block widths (i.e., 2B = 32 pixels) are used, which can cover the projection width of the sea surface specular reflective stripe on the image plane under typical strong lighting conditions. Along the normal direction of the candidate waterline (obtained by rotating the tangent direction of the candidate waterline by 90 degrees on the image plane), a first preset length L1 is extended towards the low-frequency fluctuation stable area to extract the first pixel gray value sequence, and a second preset length L2 is extended towards the high-frequency fluctuation stable area to extract the second pixel gray value sequence. In this embodiment, L1=L2=2B=32 pixels to ensure that the grayscale curve across the waterline covers at least one image block width of grayscale change information on both sides of the candidate waterline. The grayscale value sequence of the first pixel is arranged in reverse order away from the candidate waterline, and the grayscale value sequence of the second pixel is arranged in forward order away from the candidate waterline. The grayscale curve across the waterline is obtained by splicing the sequences together with the candidate waterline as the origin.

[0046] In the above embodiments, the stability of low-frequency fluctuations and high-frequency fluctuations is determined based on whether the grayscale curve across the waterline meets the preset continuous transition conditions. The physical basis here is that when the same marine debris object crosses the water-air interface, although the image grayscale of its above-water and underwater parts is shifted overall due to the difference in the refractive index of the medium and the absorption of the water, the grayscale change process should be a continuous optical gradient. Conversely, if the two sides belong to different objects (e.g., a piece of driftwood on the water and a reflective surface underwater), the grayscale abrupt change at the boundary will exceed the jump range allowed by the transition of the real physical medium. Morphological feature detection is performed on the suspected marine debris area to further exclude non-target objects with unreasonable shapes (such as curved strip-shaped wave clusters, scattered foam groups, etc.), and finally the detection results of semi-submerged marine debris are obtained.

[0047] It should be noted that in actual sea surface scenarios, there may be situations where a calm sea surface background (classified as a low-frequency stable wave zone) and a distant wave area (classified as a high-frequency stable wave zone) are spatially adjacent. In such cases, the common boundary between the two may be initially identified as a candidate waterline. However, such erroneous candidate waterlines will be effectively eliminated in the subsequent verification of the continuous transition condition of the cross-waterline grayscale curve. The elimination mechanism is based on the following physical differences: For the adjacent boundary between a pure sea surface background and a pure wave area, the cross-waterline grayscale curve is characterized by a grayscale transition difference between the two sides that is close to zero or extremely small (because the overall grayscale levels of the calm sea surface and the wave area are similar, both being diffuse reflection grayscale of seawater). This grayscale transition difference between the two sides will fall below the lower bound of the preset transition difference range, failing to meet the lower bound constraint of the grayscale transition difference between the two sides in the preset continuous transition condition (the lower bound of the preset transition difference range requires that there is a systematic grayscale difference caused by water absorption on both sides). Therefore, it will be judged as not meeting the preset continuous transition condition and marked as a non-associated sea surface area. Furthermore, even if the grayscale continuity transition condition is accidentally met in extreme cases, the cross-waterline size continuity index and overall morphological proportion parameter constraints in the subsequent morphological feature detection steps will exclude such large-area, irregularly shaped regions from semi-submerged marine debris (because the connected domains of pure sea surface areas are usually extremely large and their morphological proportions far exceed the upper limit of the preset morphological proportion range). Therefore, the multi-level cascaded verification in this embodiment can effectively avoid erroneous associations caused by the adjacency of calm sea surfaces and wave regions.

[0048] Through the above steps, utilizing the temporal characteristics of continuously acquired image sequences of the same target sea area under a stable field of view using image acquisition equipment, and by performing brightness temporal processing and color temporal processing on each group of image block sequences, temporal fluctuation feature values ​​and temporal shift feature values ​​that can respectively characterize the degree of brightness fluctuation and color shift are obtained. Since the stable imaging portion of marine debris exposed above the water surface remains stable in both brightness and color, the semi-submerged imaging portion of marine debris periodically covered by the water surface exhibits periodic fluctuations in brightness, but the color remains primarily the color of the marine debris itself. However, the imaging portion of the sea surface undergoes drastic changes in both brightness and color due to instantaneous specular reflection caused by strong light. Therefore, based on the temporal fluctuation feature values ​​and temporal shift feature values, the spatial position of the sea surface can be determined. The system accurately divides the area into low-frequency fluctuation stable region, high-frequency fluctuation stable region, and dynamic interference region to remove interference components caused by strong light illumination. Furthermore, it defines the adjacent common boundary (excluding the dynamic interference region) within a preset neighborhood between the low-frequency and high-frequency fluctuation stable regions as candidate waterlines. Based on whether the grayscale curve across the waterline along the normal direction of the candidate waterline meets a preset continuous transition condition, it determines whether the low-frequency and high-frequency fluctuation stable regions originate from the same marine debris object. This associates low-frequency and high-frequency fluctuation stable regions originating from the same marine debris object as suspected marine debris areas, avoiding misjudging sea surface reflections as marine debris or missing actual marine debris. Finally, morphological feature detection is performed on the suspected marine debris areas to output the detection results of semi-submerged marine debris. This solves the technical problem of low detection accuracy of marine debris in related technologies, achieving the technical effect of improving the detection accuracy of marine debris.

[0049] The entity performing the above steps may be a device or system, or a controller or processor in the device or system, or a separate controller or processor, or other processing devices or processing units with similar processing functions, but is not limited thereto.

[0050] In an optional embodiment, based on temporal fluctuation feature values ​​and temporal offset feature values, the sea surface spatial locations corresponding to multiple sets of image patch sequences are divided into low-frequency fluctuation stable regions, high-frequency fluctuation stable regions, and dynamic interference regions. This includes: obtaining preset fluctuation discrimination thresholds and preset offset discrimination thresholds; performing a first comparison between the temporal fluctuation feature values ​​of each set of image patch sequences and the preset fluctuation discrimination thresholds to obtain a first comparison result; and performing a second comparison between the temporal offset feature values ​​of each set of image patch sequences and the preset offset discrimination thresholds to obtain a second comparison result; if the first comparison result indicates that the temporal fluctuation feature value is less than the preset fluctuation discrimination threshold, and the second comparison result indicates that the temporal offset feature value is less than the preset offset discrimination threshold, then the sea surface spatial location corresponding to each set of image patch sequences is determined as a low-frequency fluctuation region. The dynamic stable region includes the stable imaging portion of marine debris exposed above the water surface. When the first comparison result indicates that the temporal fluctuation characteristic value is greater than or equal to a preset fluctuation discrimination threshold, and the second comparison result indicates that the temporal offset characteristic value is less than a preset offset discrimination threshold, the sea surface spatial location corresponding to each group of image blocks is determined as the high-frequency fluctuation stable region. The high-frequency fluctuation stable region represents the partially submerged imaging portion of marine debris periodically covered by the water surface. When the first comparison result indicates that the temporal fluctuation characteristic value is greater than or equal to the fluctuation discrimination threshold, and the second comparison result indicates that the temporal offset characteristic value is greater than or equal to the offset discrimination threshold, the sea surface spatial location corresponding to each group of image blocks is determined as the dynamic interference region. The dynamic interference region represents the instantaneous specular reflection imaging portion of the sea surface caused by strong light.

[0051] The preset fluctuation discrimination threshold is a critical value used to classify time-domain fluctuation characteristic values ​​into low fluctuation and high fluctuation; the preset offset discrimination threshold is a critical value used to classify time-domain offset characteristic values ​​into low offset and high offset. Neither threshold is a fixed empirical constant, but rather a dynamic parameter adaptively calibrated based on the current sea state level and ambient light conditions of the target sea area. The first comparison refers to the numerical comparison process between the time-domain fluctuation characteristic values ​​and the preset fluctuation discrimination threshold. The second comparison refers to the numerical comparison process between the time-domain offset characteristic values ​​and the preset offset discrimination threshold.

[0052] In the above embodiments, a two-dimensional joint discriminant table is used instead of independently concatenating and comparing each feature with its respective threshold. The construction of the two-dimensional joint discriminant table is based on the following: a discriminant plane is constructed with time-domain fluctuation feature values ​​on the horizontal axis and time-domain offset feature values ​​on the vertical axis, and a preset fluctuation discrimination threshold T is used. F Compared with the preset offset discrimination threshold T DThe plane is divided into four quadrants. The low-fluctuation-low-offset quadrant corresponds to the low-frequency fluctuation stable region, which physically represents the stable exposed parts of marine debris and areas with stable brightness and color, such as the calm sea surface background. The high-fluctuation-low-offset quadrant corresponds to the high-frequency fluctuation stable region, which physically represents the semi-submerged periodically covered parts of marine debris (the water surface cover changes brightness periodically, but the inherent color of the marine debris still dominates the chromaticity imaging). The high-fluctuation-high-offset quadrant corresponds to the dynamic interference region, which physically represents the instantaneous specular reflection of the sea surface under strong light (specular reflection simultaneously causes brightness saturation and chromaticity abrupt changes towards the solar spectrum). For the fourth quadrant (low-fluctuation-high-offset), which is the case where the temporal fluctuation characteristic value is less than the preset fluctuation discrimination threshold but the temporal offset characteristic value is greater than or equal to the preset offset discrimination threshold, this case physically corresponds to an area with stable brightness but continuous color deviation from the sea surface reference, which may be caused by the stable imaging of fixed colored underwater objects (such as colored reefs, underwater pipeline markers, etc.).

[0053] In the above embodiments, because these regions have low brightness fluctuations and do not possess the brightness modulation characteristics caused by the periodic coverage of semi-submerged marine debris by the water surface, they are not classified into the high-frequency fluctuation stable region. Simultaneously, due to their high color shift, they do not meet the criteria for classifying as a low-frequency fluctuation stable region. In this embodiment, for image block sequences falling into the fourth quadrant, their corresponding sea surface spatial locations are classified into the low-frequency fluctuation stable region for unified management. The technical basis for this is that the brightness stability characteristics of these regions prevent them from forming effective adjacent common boundaries with the high-frequency fluctuation stable region in the subsequent candidate waterline determination step (because the candidate waterline requires one side to be a low-frequency fluctuation stable region and the other side to be a high-frequency fluctuation stable region; when the fourth quadrant region is adjacent to the high-frequency fluctuation stable region, the verification of the continuous transition condition of the grayscale curve across the waterline will naturally exclude erroneous associations due to the mismatch in color features). Therefore, incorporating them into the low-frequency fluctuation stable region will not interfere with the subsequent detection process. In another embodiment, the fourth quadrant region can be separately marked as a static dissimilar region. In the subsequent candidate waterline determination step, it can be treated as a dynamic interference region and excluded accordingly. That is, both dynamic interference regions and static dissimilar regions are excluded within a preset neighborhood of the candidate waterline to further reduce the probability of false associations. Neither of these processing methods affects the effectiveness of the core discrimination process.

[0054] In the above embodiments, a preset fluctuation discrimination threshold T is used. F The calibration method is as follows: During the silent learning phase after the image acquisition equipment is deployed (e.g., in a clean state where there is no known marine debris on the sea surface), a sea surface image sequence of no less than 10 minutes is acquired. A distribution histogram is constructed for the temporal fluctuation characteristic values ​​of each image block sequence, and the 95th quantile of this histogram is taken as T. FThe initial value. The physical meaning of this calibration mechanism is that, under clean sea conditions, more than 95% of the image patches correspond to a pure wave background, and the distribution of its temporal fluctuation characteristic value reflects the upper envelope of the brightness fluctuation of the pure wave background under the current sea conditions; using this upper envelope of brightness fluctuation as a threshold can make the real wave background fall into the low fluctuation range, while the semi-submerged part of marine debris will be classified into the high fluctuation range because the brightness of the water surface cover layer is modulated and superimposed on the wave fluctuation.

[0055] In the above embodiments, a preset offset discrimination threshold T is used. D The calibration method is as follows: Calculate the temporal offset feature value distribution of each image patch sequence during the same silent learning phase, and take the 99th percentile of this distribution as T. D The initial value. The technical consideration for setting the quantile to 99 instead of 95 is that instantaneous reflections under strong sunlight are a low-probability but high-intensity anomaly. Using a lower quantile would cause normal wave color fluctuations to be mistakenly classified into high-offset ranges, thus incorrectly classifying the stable high-frequency fluctuation region into the dynamic interference region. Furthermore, T... F With T D The threshold needs to be dynamically updated according to the sea state level and lighting conditions at different times: During the detection operation, the quantiles of the feature value distribution of all image block sequences in the recent period are recalculated and the threshold is updated every preset period (e.g., every 10 minutes); when meteorological data or inertial measurement unit detects a sudden change in sea state level (e.g., a change in Beaufort scale exceeding level 1), or when the ambient light sensor detects an irradiance change exceeding 50% (e.g., from cloudy to sunny or during sunrise and sunset), the threshold recalibration process is immediately triggered to avoid large-area misclassification caused by threshold mismatch due to environmental changes.

[0056] In an optional embodiment, if the grayscale curve across the waterline satisfies a preset continuous transition condition, the low-frequency fluctuation stable region and the high-frequency fluctuation stable region are associated as suspected marine debris areas. This includes: locating the waterline position point corresponding to the candidate waterline on the grayscale curve across the waterline, and dividing the grayscale curve across the waterline into a first grayscale sub-curve located on one side of the low-frequency fluctuation stable region and a second grayscale sub-curve located on one side of the high-frequency fluctuation stable region based on the waterline position point; determining the grayscale value of the first endpoint of the first grayscale sub-curve at the waterline position point and the grayscale value of the second endpoint of the second grayscale sub-curve at the waterline position point, and determining the grayscale break value at the waterline position point based on the first endpoint grayscale value and the second endpoint grayscale value, and a grayscale break value table. The study identifies the degree of grayscale jump between the low-frequency fluctuation stable region and the high-frequency fluctuation stable region at the candidate waterline; it determines the first grayscale mean of all grayscale values ​​on the first grayscale sub-curve and the second grayscale mean of all grayscale values ​​on the second grayscale sub-curve, and determines the grayscale transition difference between the two sides based on the first and second grayscale mean values. The grayscale transition difference between the two sides characterizes the degree of deviation between the overall grayscale level of the low-frequency fluctuation stable region and the overall grayscale level of the high-frequency fluctuation stable region; it judges whether the grayscale curve crossing the waterline meets the preset continuous transition condition based on the grayscale break value and the grayscale transition difference between the two sides, and associates the low-frequency fluctuation stable region and the high-frequency fluctuation stable region as suspected marine debris areas if the grayscale curve crossing the waterline meets the preset continuous transition condition.

[0057] In this context, the waterline location point refers to the curve sampling point on the cross-waterline grayscale curve that corresponds to the spatial position of the candidate waterline on the image plane. In this embodiment, it is located at the center of the cross-waterline grayscale curve (because this curve is composed of pixels of lengths L1 and L2 on both sides of the candidate waterline). The first grayscale sub-curve refers to the portion of the cross-waterline grayscale curve located on the side of the low-frequency fluctuation stable region, with the waterline location point as the boundary. The second grayscale sub-curve refers to the portion of the cross-waterline grayscale curve located on the side of the high-frequency fluctuation stable region, with the waterline location point as the boundary. The first endpoint grayscale value and the second endpoint grayscale value are the endpoint pixel grayscale values ​​of the first and second grayscale sub-curves at the waterline location point, respectively, representing the pixel grayscale values ​​of the pixels adjacent to the candidate waterline on both sides. The grayscale break value is the absolute value of the difference between the first endpoint grayscale value and the second endpoint grayscale value, used to quantify the degree of grayscale jump at the candidate waterline. The first grayscale mean is the arithmetic mean of all grayscale values ​​on the first grayscale sub-curve, representing the overall grayscale level on the side of the low-frequency fluctuation stable region. The second grayscale mean is the arithmetic mean of all grayscale values ​​on the second grayscale sub-curve, representing the overall grayscale level on one side of the high-frequency fluctuation stability zone. The grayscale transition difference between the two sides refers to the difference between the first grayscale mean and the second grayscale mean (in this embodiment, the signed value of the first grayscale mean minus the second grayscale mean is taken, which reflects the relative level of the overall grayscale on both sides).

[0058] In the above embodiments, the continuous transition of the grayscale curve across the waterline is decoupled and quantified into two physically complementary dimensions: the degree of local grayscale jump at the candidate waterline (characterized by the grayscale break value) and the degree of overall grayscale deviation on both sides (characterized by the grayscale transition difference on both sides). The physical basis for this decoupling is that for the above-water and underwater parts of the same marine debris object, their grayscale relationship should meet two necessary conditions: First, the grayscale at the waterline should not jump drastically, because the optical transition of the same object near the water surface is continuous, and the small refraction caused by the capillary waves and surface tension film on the water surface only causes a few grayscale levels of local disturbance; Second, there should be a systematic difference in the overall grayscale on both sides, but it should not be too large. The underwater part is affected by water absorption and Rayleigh scattering, and its overall grayscale will systematically shift to a lower and bluish-green direction compared with the above-water part. However, the magnitude of this shift is constrained by the water clarity and does not exceed 30%–40% of the grayscale dynamic range in typical nearshore seawater conditions. If only the mean difference dimension is used for discrimination, it will be impossible to identify false associations where there is a sharp jump at the waterline but the means on both sides are close (for example, two independent objects with similar gray levels but separated by white waves at the waterline); conversely, if only the endpoint jump dimension is used for discrimination, it will be impossible to identify false associations where the endpoint gray level gradually changes but the overall gray level difference on both sides is too large (for example, the above-water part of a white floating object and the underwater part of a dark reef shadow are incorrectly spliced).

[0059] In the above embodiments, the waterline position point (i.e., the center point of the curve) is located on the grayscale curve spanning the waterline, and the curve is divided into a first grayscale sub-curve and a second grayscale sub-curve according to this point. For the calculation of the grayscale break value, to reduce the impact of endpoint grayscale jitter caused by waterline position positioning errors, in one embodiment, instead of strictly taking the single-pixel grayscale value of the endpoint, the average grayscale value of the three pixels near the endpoint is taken as the representative grayscale of the endpoint; grayscale break value. For the calculation of the grayscale transition difference between the two sides, ,in, It is the arithmetic mean of all gray values ​​on the first gray sub-curve; This is the arithmetic mean of all grayscale values ​​on the second grayscale sub-curve. It should be noted that in some specific applications, the cross-waterline grayscale curve can be further constructed based on the luminance channel (such as the Y component of the YCbCr space) rather than the original RGB composite grayscale, in order to reduce the RGB composite grayscale disturbance caused by color shift in the underwater part. This is particularly suitable for scenarios where the water turbidity is low and the color shift in the underwater part is obvious.

[0060] In an optional embodiment, the grayscale curve crossing the waterline is judged to meet a preset continuous transition condition based on the grayscale break value and the grayscale transition difference between the two sides. If the grayscale curve crossing the waterline meets the preset continuous transition condition, the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are associated as suspected marine debris areas. This includes: obtaining a preset break threshold and a preset transition difference range; performing a third comparison between the grayscale break value and the preset break threshold to obtain a third comparison result; and performing a fourth comparison between the grayscale transition difference between the two sides and the preset transition difference range to obtain a fourth comparison result; if the third comparison result indicates a grayscale break... If the value is greater than or equal to the preset fracture threshold, or if the grayscale transition difference between the two sides indicated by the fourth comparison result is not within the preset transition difference range, it is determined that the grayscale curve across the waterline does not meet the preset continuous transition condition, and the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are marked as non-associated sea surface areas; if the grayscale fracture value indicated by the third comparison result is less than the preset fracture threshold, and the grayscale transition difference between the two sides indicated by the fourth comparison result is within the preset transition difference range, it is determined that the grayscale curve across the waterline meets the preset continuous transition condition, and the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are associated as suspected marine debris areas.

[0061] The preset fracture threshold refers to the critical value used to constrain the upper limit of grayscale fracture values. Its physical meaning is the maximum allowable local grayscale jump at the water-air interface for the same marine debris object due to capillary waves, surface tension film, and thin water film refraction. The preset transition difference range refers to the allowable range of grayscale transition differences between the two sides. Its upper limit corresponds to the maximum physical amplitude of grayscale attenuation of the underwater part relative to the surface part, and its lower limit corresponds to the minimum physical amplitude where the grayscale difference between the two sides is too small to distinguish the water-air boundary. If the grayscale transition difference between the two sides is close to zero, it indicates that there is no real water-air boundary, and the so-called candidate waterline may just be a meaningless partition within the same object on the surface. The third comparison refers to the comparison process between the grayscale fracture value and the preset fracture threshold. The fourth comparison refers to the comparison process between the grayscale transition difference between the two sides and the preset transition difference range. The non-associated sea surface area refers to the combination of low-frequency and high-frequency stable fluctuation areas that, after identification, are determined not to originate from the same marine debris object. These two areas will no longer be considered as a whole in the subsequent processing of suspected marine debris areas.

[0062] In the above embodiments, an asymmetric criterion combining upper bound constraints and bidirectional interval constraints is used to quantify the continuous transition conditions. The physical basis for this asymmetry is that the grayscale break value only requires an upper bound constraint, because smaller local jumps indicate a smoother transition, and there are no abnormal situations with excessively small jumps; while the difference in grayscale transition between the two sides requires bidirectional interval constraints, because a large difference implies that the two sides are not the same object (the grayscale attenuation in the underwater part should not exceed the physical upper limit), while a small difference implies that there is no true water-air boundary (the two sides cannot be distinguished). A preset break threshold is used. The calibration basis is as follows: Under nearshore seawater conditions, the local grayscale jumps caused by capillary waves and thin water film refraction generally do not exceed 10% of the 256 levels of the 8-bit grayscale dynamic range, i.e., approximately 25 grayscale levels. In this embodiment, A value of 25 can be used; when strong winds on the water surface cause capillary wave enhancement, The limit can be appropriately relaxed to 35; when the water surface is calm (Beaufort scale below level 2), it can be tightened to 15. The physical upper limit is determined by the variation of the Fresnel refractive index at the water-air interface, and it cannot be exceeded.

[0063] In the above embodiments, a preset transition difference range is defined as follows: , The calibration basis for ] is: lower bound The upper bound should be greater than the random grayscale fluctuation level of the wave background itself (e.g., 10) to avoid misjudging random partitions within the waves as water vapor boundaries; It should be less than the maximum allowable difference in grayscale value due to the attenuation of near-infrared and red light components by water absorption (e.g., 80 for moderately turbid nearshore seawater; 100 for clear seawater; and tightened to 60 for highly turbid seawater (such as estuaries)). The physical basis for this upper bound is the grayscale attenuation caused by the absorption coefficient of visible light by water at a typical semi-submerged depth of 10–30 cm under the Beer-Lambert law. When the grayscale breakpoint... ≥ Or the difference in grayscale transition between the two sides [ , When any condition is not met, the grayscale curve across the waterline is determined to not meet the preset continuous transition condition, and the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are marked as non-associated sea surface areas; if and only if < and [ , When the preset continuous transition conditions are met, the two are associated as suspected marine debris areas.

[0064] In an optional embodiment, luminance temporal processing is performed on each of the multiple image block sequences to obtain temporal fluctuation feature values, and color temporal processing is performed on each of the image block sequences to obtain temporal offset feature values. This includes: performing color space conversion on each frame image block in each image block sequence to obtain the luminance channel component and chrominance channel component of each frame image block in a preset color space; calculating the first intra-block spatial mean of the luminance channel component of each frame image block to obtain the image block luminance mean; and calculating the second intra-block spatial mean of the chrominance channel component of each frame image block to obtain the image block chrominance mean. The process involves: arranging the average brightness values ​​of each image block sequence in a first temporal sequence to obtain a brightness average time series; determining the discrete statistics of the brightness average time series in the time dimension and defining these discrete statistics as temporal fluctuation characteristic values; arranging the average chromaticity values ​​of each image block sequence in a second temporal sequence to obtain a chromaticity average time series; acquiring a preset sea surface reference chromaticity parameter and determining the chromaticity distance sequence between each chromaticity average value in the chromaticity average time series and the preset sea surface reference chromaticity parameter in a preset color space; and defining the chromaticity distances in the chromaticity distance sequence that satisfy a preset maximum value condition as temporal offset characteristic values.

[0065] The preset color space refers to the target color space used to map the image from the original RGB space to a decoupled representation of luminance and chrominance. It can be the YCbCr color space, where the Y component is the luminance channel and the Cb and Cr components are the chrominance channels; or it can be the Lab color space, where the L component is the luminance channel and the a and b components are the chrominance channels. The luminance channel component is the luminance value of a single-frame image block in the preset color space. The chrominance channel component is the chrominance value of a single-frame image block in the preset color space. The spatial mean within a block refers to a single scalar value obtained by arithmetically averaging the values ​​of all pixels in the corresponding channels within a single-frame image block. Replacing the original pixel-level values ​​with the spatial mean aims to suppress random texture noise within the block and highlight the temporal variation characteristics of the block as a whole. The luminance mean temporal sequence is the sequence of the image block luminance mean over time, and the chrominance mean temporal sequence is the sequence of the image block chrominance mean over time. Discrete statistics refer to statistical indicators characterizing the dispersion of sequence values. In this embodiment, standard deviation, variance, or range can be used as quantification forms. The preset sea surface reference chromaticity parameter refers to the chromaticity coordinates representing the typical seawater color of the target sea area in a preset color space, determined by the median of the chromaticity mean values ​​of clean sea surface image patches collected during the silent learning phase. Chromaticity distance refers to the Euclidean or Mahalanobis distance between the chromaticity mean value of each frame in the chromaticity mean time series and the preset sea surface reference chromaticity parameter in the preset color space.

[0066] The preset maximum condition refers to the screening condition used to select the representative maximum chromaticity deviation in the chromaticity distance sequence. Specifically, the process of satisfying the preset maximum condition is as follows: First, all chromaticity distances in the chromaticity distance sequence are sorted in descending order of value; then, the top k chromaticity distance values ​​(k is a preset positive integer, generally 5% to 10% of the length of the chromaticity distance sequence) are selected from the sorted sequence; finally, the arithmetic mean of these top k chromaticity distance values ​​is calculated, and this arithmetic mean is used as the chromaticity distance that satisfies the preset maximum condition, i.e., the time-domain offset feature value. In other words, determining the chromaticity distance in the chromaticity distance sequence that satisfies the preset maximum condition as the time-domain offset feature value means: determining the arithmetic mean of the top k chromaticity distances with the largest values ​​in the chromaticity distance sequence as the time-domain offset feature value. The technical significance of using the mean of the first k chromaticity distances instead of a single maximum chromaticity distance value as the temporal offset feature value is to suppress the contamination of the temporal offset feature value by single-frame outlier noise (such as camera CCD instantaneous bad pixels, flying insects and other isolated abnormal frames), so that the temporal offset feature value has statistical robustness while retaining the physical meaning of the maximum chromaticity deviation.

[0067] In the above embodiments, the selection of the preset color space has a crucial impact on distinguishing between semi-submerged periodic coverage and instantaneous reflection under strong light. The physical basis for selecting the YCbCr color space as the preferred implementation is that the YCbCr color space concentrates luminance information in the Y component and chromaticity information in the Cb and Cr components. The two are statistically approximately orthogonal, which allows the temporal statistics of luminance fluctuations and chromaticity shifts to be performed independently without mutual interference. More importantly, the specular reflection of the sea surface under strong light directly reflects the solar spectrum, and its landing point on the CbCr chromaticity plane will be significantly shifted towards the chromaticity coordinates corresponding to the solar color temperature (approximately the position of CCT 5500K on the CbCr plane), while the chromaticity coordinates of diffusely reflected seawater are located in the blue-green quadrant. The Euclidean distance between the two on the CbCr plane is usually more than 20 digital units (8-bit encoding), which is significantly separable. In contrast, if the H component of the HSV color space is used as a chromaticity measure, the definition of H in the gray region (low saturation S) is singular, leading to unstable H values ​​in reflective imaging (high brightness, low saturation), which is detrimental to stable chromaticity shift measurement. In practice, the following processing is performed on each frame of image blocks in each image block sequence: First, the image blocks are converted from RGB to YCbCr space using the standard ITU-R BT.601 transformation matrix, obtaining the Y channel components, Cb channel components, and Cr channel components of each frame of image blocks in YCbCr space; second, the arithmetic mean of all pixels in the Y channel component of each frame of image blocks is calculated to obtain the average brightness of the image block. The mean chromaticity of the image block is obtained by performing an arithmetic mean on all pixels of the Cb and Cr channel components respectively. , ), where t is the frame time index.

[0068] In the above embodiments, the same image block sequence Arranged chronologically, the mean luminance time series is obtained. , ,..., } where N is the number of frames in a single image patch sequence. The standard deviation of this mean brightness time series is calculated in the time dimension as the temporal fluctuation characteristic value. ,in, Let be the average brightness value of the image block in frame t; , This represents the arithmetic mean of the brightness mean time series over time. The discrete statistic is the standard deviation of the brightness mean time series, or the difference between the maximum and minimum values ​​(i.e., the range). The choice depends on the lighting stability requirements in practical applications. (The remaining text appears to be incomplete and requires further context.) , Arrange the chromaticity mean values ​​in chronological order to obtain a time series. Preset sea surface reference chromaticity parameters ( , The chromaticity mean was obtained by taking the median of the chromaticity mean of all image patch sequences under clean sea surface conditions in the target sea area during the silent learning phase (the median was used instead of the arithmetic mean to ensure robustness to a small number of potentially contaminated samples). The Euclidean distance between the chromaticity mean of each frame in the chromaticity mean time series and the reference chromaticity was calculated. The chromaticity distance sequence { is obtained , ,..., }

[0069] In the above embodiments, the preset maximum condition is defined as the arithmetic mean of the k largest values ​​in the chroma distance sequence, where k is generally taken as 5%-10% of the sequence length (e.g., k=15 when N=200). The technical significance of using the mean of the k largest chroma distances, rather than a strict single-point maximum value, as the temporal offset feature value is that single-point maximum values ​​are easily contaminated by isolated outliers (such as instantaneous dead pixels on the camera CCD, insects flying by, etc.), while the mean of the k largest chroma distances retains the physical meaning of the maximum chroma deviation while exhibiting statistical robustness to isolated noise. The chroma distances in the chroma distance sequence that satisfy the preset maximum condition are determined as the temporal offset feature value D. The temporal offset feature value D physically represents the degree of the most significant chromaticity deviation of the sea surface spatial location from the sea surface reference throughout the entire observation period. It has a sensitive response to instantaneous reflections under strong light (with extremely strong chromaticity shifts occurring in a very few frames), but a weaker response to half-submerged marine debris (with consistently small chromaticity deviations but large brightness fluctuations). Thus, it forms a positive interactive complement with the temporal fluctuation feature value, together constituting the input feature pair of the two-dimensional joint discrimination table.

[0070] In an optional embodiment, morphological feature detection is performed on a suspected marine debris area to obtain a semi-submerged marine debris detection result, including: extracting connected components from the suspected marine debris area to obtain candidate connected components; determining the principal axis direction of the intersection of the candidate connected components and the candidate waterline, and using the principal axis direction as the waterline extension direction of the candidate waterline; dividing the candidate connected components into an exposed portion located on the side of the low-frequency fluctuation stable region and a semi-submerged portion located on the side of the high-frequency fluctuation stable region, using the candidate waterline as the boundary line; determining a first size parameter of the exposed portion along the waterline extension direction, and determining a second size parameter of the semi-submerged portion along the waterline extension direction, wherein the first size parameter includes the size parameter along the waterline extension direction. The projected length of the exposed portion along the extension direction and the projected width of the exposed portion perpendicular to the extension direction of the waterline are defined. The second dimension parameter includes the projected length of the semi-submerged portion along the extension direction of the waterline and the projected width of the semi-submerged portion perpendicular to the extension direction of the waterline. The waterline dimension continuity index and the overall morphological proportion parameter of the candidate connected region are determined based on the first dimension parameter and the second dimension parameter. If the waterline dimension continuity index is greater than or equal to a preset continuity threshold and the overall morphological proportion parameter is within a preset morphological proportion range, the candidate connected region is determined as semi-submerged marine debris. The location of the candidate connected region, the first dimension parameter, and the second dimension parameter are used as the detection result of semi-submerged marine debris.

[0071] Connected component extraction refers to the process of merging spatially adjacent image patches in suspected marine debris areas into a single connected region through four-neighbor or eight-neighbor connectivity analysis. Candidate connected components are several connected regions obtained through connected component extraction, each corresponding to a possible semi-submerged marine debris object. The principal axis direction refers to the geometric principal extension direction of the candidate connected component on the image plane, obtained by decomposing the eigenvector direction corresponding to the largest eigenvalue of the covariance matrix of all pixel coordinates in the connected component. The rules for determining the direction of extension across the waterline are as follows: First, calculate the angle θ between the principal axis direction of the candidate connected domain and the candidate waterline direction; if the angle θ is greater than a preset angle threshold (e.g., 45 degrees), that is, the angle between the principal axis direction and the candidate waterline direction is large enough, it indicates that the principal extension direction of the candidate connected domain does indeed cross the candidate waterline, and the principal axis direction is taken as the direction of extension across the waterline; if the angle θ is less than or equal to the preset angle threshold, that is, the principal axis direction is approximately parallel to the candidate waterline direction, it indicates that the principal extension direction of the candidate connected domain extends along the candidate waterline direction rather than crossing the candidate waterline, and the normal direction of the candidate waterline (i.e., the direction perpendicular to the candidate waterline) is taken as the direction of extension across the waterline. The physical basis of this processing rule is as follows: For typical semi-submerged marine debris (such as plastic bottles, planks, etc.), their main body usually extends across the waterline, with the principal axis direction forming a large angle with the waterline direction; however, for some flat floating objects (such as film sheets, flat foam boards, etc.), their principal axis direction may be approximately parallel to the waterline direction. In this case, using the candidate waterline normal direction as the direction of extension across the waterline more accurately reflects the geometric characteristics of the object crossing the water-air interface. The direction of extension across the waterline serves as the coordinate reference direction for subsequent dimensional parameter measurements. The exposed portion refers to the sub-region in the candidate connected domain located on one side of the low-frequency fluctuation stable region, with the candidate waterline as the boundary. The semi-submerged portion refers to the sub-region in the candidate connected domain located on one side of the high-frequency fluctuation stable region, with the candidate waterline as the boundary. The waterline dimensional continuity index is a quantitative indicator characterizing whether the projected widths of the exposed and semi-submerged portions of the candidate connected domain are smoothly connected in the direction perpendicular to the waterline extension. The overall morphological proportion parameter is a quantitative indicator characterizing whether the overall geometric shape of the candidate connected domain conforms to the characteristics of typical marine debris.

[0072] In the above embodiments, the physical basis for introducing morphological feature detection lies in the physical constraint of Archimedes' law of buoyancy on the geometric shape of semi-submerged marine debris: a real floating object exhibits stable geometric continuity between its above-water and semi-submerged parts under the force balance of its center of gravity, displacement, and its own weight. This is specifically manifested in two aspects: first, the dimensions at the waterline crossing should be smoothly connected (the cross-section of the same object above and below water should not change abruptly); second, the overall shape should exhibit a finite range of aspect ratios (scattered reflective clusters usually exhibit extreme elongated or extreme circular shapes, while the aspect ratios of common marine debris such as plastic bottles, foam boxes, and wooden boards generally fall within a finite range of 1:1 to 1:8). This dual morphological constraint is a hard constraint based on physics, not an empirical statistical law, and therefore has cross-scene robustness. In specific implementation, firstly, the suspected marine debris area is extracted by eight-neighbor connectivity, and small connected components with an area smaller than the preset lower bound (e.g., the area of ​​3 image blocks) are filtered out to obtain candidate connected components. For each candidate connected component, the coordinates (x, y, x) of all its pixels are calculated. i ,y i Construct the covariance matrix from the second-order central moments. ,in, The total number of pixels within the candidate connected component; Let be the column vector of coordinates of the i-th pixel within the candidate connected component; Let be the arithmetic mean vector of all pixel coordinates within the candidate connected region. Perform eigenvalue decomposition on this covariance matrix to obtain the result that satisfies... ≥ Two eigenvalues, and The corresponding feature vector and its relation to The corresponding feature vector. Wherein Larger eigenvalues The corresponding feature vectors represent the principal axis directions of the candidate connected components; For smaller eigenvalues The corresponding feature vector, and Orthogonal, representing the direction of the secondary axis of the candidate connected region; The length of the semi-major axis of the candidate connected component along the principal axis direction; Let be the semi-major axis length of the candidate connected region along the secondary axis direction. After determining the principal axis direction, calculate the angle θ between the principal axis direction and the candidate waterline direction: if θ is greater than 45 degrees, the principal axis direction is taken as the waterline extension direction; if θ is less than or equal to 45 degrees, the normal direction of the candidate waterline is taken as the waterline extension direction. In this embodiment, since semi-submerged marine debris objects usually extend across the waterline (i.e., the main body of the object extends from above water to below water), in most cases the angle between the principal axis direction of the candidate connected region and the candidate waterline direction is greater than 45 degrees, therefore the principal axis direction is the waterline extension direction.

[0073] In the above embodiments, the candidate connected region is divided into an exposed portion (located on the side of the low-frequency fluctuation stability region) and a semi-submerged portion (located on the side of the high-frequency fluctuation stability region) using the candidate waterline as the dividing line. Along the direction of extension across the waterline... and its vertical direction Pixel projection is performed on the two parts separately: the exposed part along... Projection length (The maximum extension distance of the exposed portion in that direction) and along Projection width The partially submerged part along Projection length Along Projection width Waterline dimensional continuity index Defined as: The value range of this indicator is [0,1]. The closer the value is to 1, the smoother the connection between the width above and below the waterline. The reason for using width (perpendicular to the direction of extension across the waterline) rather than length (along the extension direction) as the continuity measure is that length reflects the extent of an object's extension in both the water and air media (which exhibits meaningful differences due to buoyancy / center of gravity), while width reflects the cross-section of the object near the waterline and should be approximately continuous; therefore, the width difference is the true physical quantity characterizing the continuity of the water-air boundary for the same object. A preset continuity threshold T is used. C We set it to 0.7, which means that the maximum difference in width between the surface and underwater areas is allowed to be 30% (including the difference in observation width caused by water refraction distortion).

[0074] In the above embodiments, the overall shape proportion parameter Defined as the ratio of the overall length of the candidate connected component along the principal axis to its width in the vertical direction, i.e. = Preset shape ratio range [ , The range is set to [1.0, 8.0]: a lower bound of 1.0 excludes nearly circular scattered reflective clusters; an upper bound of 8.0 excludes strip-shaped white wave lines or long strips of foam gathered by wind. The continuity index of the waterline size of the candidate connected region is used to determine this. ≥T C And the overall shape proportion parameters ∈[ , When the candidate connected component is identified as partially submerged marine debris, it is determined that the candidate connected component is the image plane pixel coordinate center (or the coordinates of the upper left and lower right corners of the smallest bounding rectangle). Finally, the position of the candidate connected component is determined by the first size parameter ( , ) and the second dimension parameter ( , The combined output is the detection results of semi-submerged marine debris, which can be used by downstream cleanup operation scheduling or law enforcement evidence collection systems.

[0075] In an optional embodiment, after using the candidate connected component location, the first size parameter, and the second size parameter as the detection result of semi-submerged marine debris, the method further includes: extracting a target image region sequence corresponding to the candidate connected component from the image sequence to be detected based on the candidate connected component location; and dividing each frame of the target image region sequence into an exposed portion sub-image located on one side of the low-frequency fluctuation stable region and a semi-submerged portion sub-image located on one side of the high-frequency fluctuation stable region, using the candidate waterline as the dividing line; and constructing a semi-submerged portion sub-image based on the first length ratio of the exposed portion projection length to the semi-submerged portion projection length and the second width ratio of the exposed portion projection width to the semi-submerged portion projection width. The submerged size prior vector is used to input the exposed sub-image, the semi-submerged sub-image, and the semi-submerged size prior vector into a pre-trained marine debris recognition model to obtain the category recognition result and confidence score output by the marine debris recognition model. The category recognition result represents the category of the semi-submerged marine debris, and the confidence score represents the reliability of the category recognition result. If the confidence score is greater than or equal to a preset confidence threshold, the category recognition result and confidence score are added to the semi-submerged marine debris detection result. If the confidence score is less than the preset confidence threshold, the candidate connected components are marked as regions to be reviewed, and multi-frame temporal review operations are performed on the regions to be reviewed.

[0076] It should be noted that using the candidate connected component location, first size parameter, and second size parameter as the detection result for semi-submerged marine debris is a preliminary detection result based on geometric morphological constraints. This preliminary detection result indicates that the candidate connected component meets the feature requirements of semi-submerged marine debris in the geometric morphological dimension. Subsequent category identification and multi-frame temporal verification steps are further refinements of this preliminary detection result, aiming to supplement the preliminary detection result with category information and confidence assessment. Specifically, the detection result of semi-submerged marine debris is a dynamically updated structured data record: In the geometric shape detection stage, the record includes the location of candidate connected components, the first size parameter, and the second size parameter. At this time, the detection result is in the state of geometric confirmation and pending classification. In the category identification stage, if the confidence score is greater than or equal to the preset confidence threshold, the category identification result and confidence score are added to the record, and the detection result is updated to classification confirmation. If the confidence score is less than the preset confidence threshold, the multi-frame temporal review stage is entered. If the review passes, the multi-frame fused category and multi-frame fused confidence score are added, and the detection result is updated to review confirmation. If the review fails (i.e., the multi-frame fused confidence score is less than the preset review confidence threshold), the detection result of the candidate connected component is updated to questionable and awaits manual review. Its location information is retained for manual inspection reference, but it is not used as the trigger for automatic cleanup scheduling. The above processing flow embodies a progressive decision strategy of geometric coarse screening → classification fine judgment → temporal verification. Geometric morphology detection provides preliminary candidates with high recall, while category recognition and temporal verification gradually improve accuracy. The two work together to achieve a balance between detection accuracy and false negative rate.

[0077] The target image region sequence refers to the time series composed of image regions that are extracted frame by frame from the image sequence to be detected based on the position of the candidate connected components, and which correspond one-to-one with each frame of the image sequence to be detected in the time dimension. The pixel coordinate range of the target image region in each frame is based on the minimum bounding rectangle of the candidate connected components and expanded by a preset margin of pixels (e.g., expanded by 4 pixels) to ensure that the target does not exceed the boundary of the target image region in the event of slight inter-frame drift of the candidate connected components. The exposed part sub-image refers to the sub-image region of the target image region located on the side of the low-frequency fluctuation stable region with the candidate waterline as the dividing line, which mainly carries the surface appearance characteristics of marine debris; the semi-submerged part sub-image refers to the sub-image region of the target image region located on the side of the high-frequency fluctuation stable region with the candidate waterline as the dividing line, which mainly carries the underwater appearance characteristics of marine debris. The semi-submerged size prior vector is a two-dimensional vector composed of a first length ratio (the ratio of the projected length of the exposed portion to the projected length of the semi-submerged portion) and a second width ratio (the ratio of the projected width of the exposed portion to the projected width of the semi-submerged portion). Its physical meaning is to explicitly characterize the floating state (i.e. the proportion exposed above the water surface) of the marine debris object in the form of a priori dimensions across the waterline, providing a priori geometric information across the water and air in addition to appearance features for subsequent marine debris identification models.

[0078] The marine debris identification model refers to a pre-trained multimodal classification model that takes exposed sub-images, partially submerged sub-images, and a prior vector of the partially submerged size as inputs, and outputs category identification results and confidence scores. In this embodiment, a dual-branch convolutional neural network backbone (one branch processes exposed sub-images, and the other branch processes partially submerged sub-images) combined with fully connected layers and the prior vector of the partially submerged size for feature concatenation before outputting through a classification head can be used. Alternatively, a structure with shared weights, twin convolutional branches, and attention-weighted fusion can be employed. The category identification result refers to the discrete label output by the marine debris identification model, indicating the category of the partially submerged marine debris. The debris categories include several preset categories such as plastic bottles, foam boards, fishing nets and gear, wooden floating objects, metal containers, and mixed waste. The confidence score refers to a continuous scalar output by the marine debris identification model, ranging from [0,1], representing the reliability of the category identification result. A pre-set confidence threshold T is used. P This refers to the critical value used to divide the confidence score into high and low confidence levels. In this embodiment, it can be set to 0.75. This value is a balance point where, under the premise of ensuring a controllable false negative rate, the proportion of candidate connected components triggering multi-frame temporal review operations does not exceed 30% of the total number of candidate connected components. The region to be reviewed refers to the region with a confidence score less than T. P The image region corresponding to the candidate connected component has ambiguity in the single-frame recognition dimension and needs to be subject to secondary decision in the temporal dimension.

[0079] Among them, the multi-frame temporal verification operation refers to the image processing operation that performs a second temporal dimension decision on the region to be verified. The temporal composition of frame images includes, but is not limited to, the following two methods: one is a backward sliding window, which consists of the current frame and the previous frame. The first is composed of -1 frames; the second is a symmetrical window, that is, with the current frame as the center, and the window is formed by the previous frame of the current frame. / 2⌋Frames and After⌈ The system consists of two frames. The former has low verification latency and is suitable for real-time scheduling scenarios, while the latter has good timing symmetry and is suitable for offline fine-grained verification scenarios. The value of should satisfy the following physical constraints: ,in, The frame rate of the image acquisition device. This represents the 95th percentile of the statistical distribution of the wave zero-crossing period under the current sea state level, expressed in seconds. The sea state level is determined according to the internationally accepted Dow scale and sea state level conversion table (or equivalent WMO sea state standard). In this embodiment, the value is taken as follows: sea state 1–2 is... =3.5 seconds, sea state 3-4 =5.0 seconds, sea state 5-6 =7.5 seconds, sea state 7 or above =10.0 seconds; the reason for using the 95th percentile instead of the theoretical maximum value is to avoid the occasional samples of extremely long-period waves being... The observation is magnified to the point of losing real-time performance, while ensuring that the observation range covers a complete crest-to-trough half-cycle for at least 95% of the time window. The physical significance of this constraint is to ensure that the observation window covers at least one crest-to-trough half-cycle of a typical wave cycle, thus providing the physical prerequisite that a low-water-cover phase frame must exist for the subsequent optimal exposure frame selection sub-step.

[0080] in, The value can also be dynamically and adaptively adjusted based on real-time sea state information released by wave height meters, surge sensors, or meteorological stations deployed synchronously with the image acquisition equipment. The temporal dimension secondary decision relies on the following three sea surface statistical characteristics that are approximately true within a short time window at the physical level: First, the short-term narrow-band quasi-periodicity of sea surface fluctuations, that is, within the observation window which is much smaller than the coherence time of the wave spectrum, the sea surface elevation can be approximated as a narrow-band random process with a slowly varying local dominant phase; second, the statistical stability of exposure differences, that is, the signal-to-noise ratio of target imaging at the wave trough phase is systematically higher than that at the wave crest phase; and third, the separability of motion scale, that is, the translational drift of floating targets on the sea surface is approximately a low-frequency component within the observation window, while the pixel-level disturbance caused by the waves is a high-frequency component, and the two can be separated on a time scale.

[0081] In the above embodiments, the specific implementation method for extracting the target image region sequence corresponding to the candidate connected component from the image sequence to be detected based on the candidate connected component position is as follows: taking the minimum bounding rectangle of the candidate connected component as the basis, expanding outward by a preset margin pixel along the row and column directions. (For example, take) =4 pixels), to obtain the target cropping rectangle; in each frame of the sea surface image of the image sequence to be detected, cropping is performed according to the pixel coordinate range of the target cropping rectangle to obtain a target image region sequence with the same number of frames as the image sequence to be detected. The technical significance of the outward preset margin pixels is that: candidate connected components may have slight translational drift at the sub-pixel level to several pixels level between frames due to the push of ocean waves. The outward margin can ensure that the target object is always located inside the target image region and does not go beyond the boundary within the entire observation window. Taking the relative position of the candidate waterline in the target image region as the dividing line, each frame of the target image region is divided into exposed part sub-images located on the side of the low-frequency fluctuation stable region and semi-submerged part sub-images located on the side of the high-frequency fluctuation stable region. The exposed part sub-image mainly carries the appearance features such as texture, color and outline of the above-water part of marine debris; the semi-submerged part sub-image mainly carries the appearance features of the underwater part of marine debris after absorption and scattering modulation by the water body. There are systematic differences in the visual representation between the two, so they are input into different branches of the marine debris recognition model for independent feature extraction.

[0082] In the above embodiment, the prior vector of the half-submerged size is constructed by calculating the projected length of the exposed portion. Projected length of the half-submerged portion First length ratio = and the exposed portion of the projection width Projected width of the half-submerged portion Second width ratio = Combine the two into a two-dimensional vector This serves as a prior vector for the semi-submerged size. The physical meaning of this vector lies in the significant inter-category differences in the floating state (i.e., the proportion above the water surface) of different types of marine debris at the water-air interface due to variations in material density and shape. For example, lower-density foam board debris has a larger proportion above the water surface. and All tend to be greater than 1; the proportion of wooden floating objects with a density close to that of water that are exposed above the water surface is relatively small. and The values ​​tend to be less than 1; however, plastic bottle waste, due to its internal air cavity, has a floating state between the two. This prior value of the cross-waterline size ratio is explicitly injected into the marine debris recognition model. This allows the model to utilize the floating state geometric prior to assist in category discrimination while fusing appearance features. Especially in cases where the appearance features of the partially submerged sub-image are degraded due to water occlusion, the partially submerged size prior vector can serve as a compensatory discrimination criterion to improve classification accuracy.

[0083] In the above embodiments, the network structure of the marine debris identification model adopts a fusion architecture of a dual-branch convolutional neural network backbone combined with prior vector injection. Specifically, the first branch is a lightweight convolutional network (e.g., using a Mobile NetV3-Small structure), which receives the exposed part sub-image as input, and outputs a first appearance feature vector after passing through multiple layers of depthwise separable convolutions and global average pooling. The first branch has 128 dimensions; the second branch uses the same network structure as the first branch but with independent weights, receives the partially submerged sub-image as input, and outputs a second appearance feature vector. The dimension is also 128. The first appearance feature vector... Second appearance feature vector With the prior vector of the half-submerged size The features are concatenated along the channel dimension to obtain the fused feature vector. =[ ; ; The dimension is 128 + 128 + 2 = 258. The fused feature vector F... fused After two fully connected layers (the first layer from 258 to 128 dimensions with ReLU activation; the second layer from 128 to C+1 dimensions, where C is the preset number of waste categories) and Softmax normalization, a C+1-dimensional probability vector is output. The first C dimensions correspond to the predicted probability of each waste category, and the C+1 dimension corresponds to the predicted probability of non-waste categories. The category recognition result is the category label corresponding to the maximum value of the first C dimensions in the probability vector, and the confidence score is the maximum probability value. The training dataset for the marine debris recognition model consists of manually labeled semi-submerged marine debris samples. Each training sample includes an exposed sub-image, a semi-submerged sub-image, a prior vector of the semi-submerged size, and the corresponding waste category label. During training, the cross-entropy loss function is used, and iterative optimization is performed using the Adam optimizer.

[0084] In the above embodiments, a confidence threshold T is preset. PThe value of T is 0.75. This value is determined as follows: On the validation dataset, the confidence scores are binary classified (high confidence level and low confidence level) with different thresholds. Precision-recall curves are plotted, and the threshold that achieves the highest precision while ensuring that the proportion of candidate connected components triggering multi-frame temporal review operations does not exceed 30% of the total number of candidate connected components is selected as T. P When the confidence score is greater than or equal to T P When the confidence score is high, it indicates that the marine debris identification model has a high degree of reliability in classifying the current candidate connected domains. The class identification results and confidence scores are directly added to the semi-submerged marine debris detection results as part of the final output for downstream cleanup scheduling.

[0085] In the above embodiment, when the confidence score is less than the preset confidence threshold T P At that time, candidate connected components are marked as regions to be reviewed, and multi-frame temporal review operations are performed on the regions to be reviewed. The specific implementation of the multi-frame temporal review operation is as follows: taking the current frame as the reference, and taking the previous frame... -1 frame, consisting of The timing verification window for the frame. In this embodiment, =25 frames / second, current sea state level is 3 ( =5.0 seconds), then ≥(25×5.0) / 2=62.5, take =65 frames. The above physical constraints ensure that the temporal verification window covers at least one complete half-cycle of a wave crest to trough, thus guaranteeing the existence of a phase frame within the window with the thinnest water cover (i.e., the best imaging quality for the half-submerged portion). Within the temporal verification window... Within a frame, the image sharpness index is calculated frame by frame for the sub-image of the half-submerged portion corresponding to the region to be reviewed. (In this embodiment, the variance of the Laplace operator response is used as the sharpness index, i.e.) =Var[Laplacian( )],in, (This refers to the partially submerged sub-image of frame t). A sharpness metric is selected. Top The frames are the set of best exposure frames. Pick 20% (for example) =65 times =13 frames). The physical basis for selecting the best exposure frame is that: at the wave trough phase, the water cover layer is the thinnest, and the texture and contour features of marine debris in the half-submerged sub-image are attenuated by water scattering at the least, resulting in the highest image clarity; while at the wave crest phase, the water cover layer is the thickest, resulting in the highest image blur. By selecting the frame with the highest clarity, it is equivalent to selecting frames near the wave trough phase for identification, thereby obtaining the optimal appearance features of the half-submerged part.

[0086] In the above embodiments, for each frame in the set of optimal exposure frames, its corresponding exposed portion sub-image, partially submerged portion sub-image, and prior vector of the partially submerged size are input into the marine debris recognition model to obtain... Group category identification results and confidence scores. The group output results are fused and judged using a multi-frame method: First, the values ​​of each garbage category are statistically analyzed. The frequency of occurrence in the group category identification results is used to determine the category with the highest frequency as the multi-frame fusion category; then, the arithmetic mean of the confidence scores of all frames corresponding to this multi-frame fusion category is calculated as the multi-frame fusion confidence score. . Fuse confidence scores from multiple frames Compared with the preset review confidence threshold Comparison, among which Values ​​lower than (For example =0.60), to reflect the reduction in the decision threshold brought about by the multi-frame fusion's effect on suppressing single-frame noise. In Greater than or equal to In this case, the multi-frame fused category is used as the final category recognition result. As the final confidence score, both were added to the semi-submerged marine debris detection results, along with a marker confirmed through multi-frame temporal verification to distinguish them from results directly accepted from a single frame. Less than In cases where the current candidate connected component still cannot obtain a reliable category classification in the temporal verification dimension, the status of the semi-submerged marine debris detection result for that candidate connected component is updated to "Questionable, awaiting manual review." This means that the location information, first and second size parameters of the candidate connected component are retained for manual inspection reference, but no category identification result or confidence score is added to the detection result, and the candidate connected component is not used as a trigger for automatic cleanup scheduling. It should be noted that this processing method does not negate the preliminary judgment result of the geometric morphology detection stage (the candidate connected component still meets the characteristic requirements of semi-submerged marine debris in the geometric morphology dimension), but rather indicates that a sufficiently reliable classification conclusion cannot be obtained in the category identification dimension at present, requiring manual intervention for final confirmation. This approach ensures the accuracy of automatic detection while avoiding the omission of real marine debris due to the limitations of the classification model.

[0087] In the above embodiments, the technical advantages of multi-frame temporal verification operation compared to single-frame recognition are as follows: First, by selecting the best exposure frame, it is equivalent to selecting the moment when the water cover layer is thinnest in the wave cycle for recognition, so that the appearance feature degradation of the half-submerged part of the sub-image is minimized, thereby improving the feature extraction quality of the marine debris recognition model for the half-submerged part; Second, by using multi-frame fusion decision, the statistical redundancy of multi-frame independent observation is used to suppress the influence of instantaneous interference in a single frame (such as occasional splashing waves, short-term light changes, camera noise pulses, etc.) on the recognition result, so that the final category judgment has temporal consistency guarantee; Third, by setting a verification confidence threshold lower than the single-frame threshold, Under the premise of ensuring that the false positive rate is controllable, some candidate connected components that are actually real marine debris but whose confidence is temporarily low due to instantaneous interference in a single frame are recovered, thereby reducing the false negative rate.

[0088] It should be noted that the examples of all the specific values ​​mentioned above are merely exemplary embodiments, and the specific values ​​are not limited to the examples mentioned above.

[0089] This application utilizes temporal features to divide the spatial location of the sea surface into a low-frequency fluctuation stable region, a high-frequency fluctuation stable region, and a dynamic interference region, achieving feature-level stripping of strong light interference components. Furthermore, by verifying the continuous transition of the cross-waterline grayscale curve and constraining the geometric shape of candidate connected regions, semi-submerged marine debris is jointly identified, effectively avoiding misjudging sea surface reflections as marine debris or missing real marine debris. Moreover, by injecting the semi-submerged size prior vector into the marine debris identification model and performing multi-frame temporal verification on low-confidence candidate connected regions, the probability of misidentification caused by instantaneous interference in a single frame is reduced, improving the accuracy and reliability of marine debris detection.

[0090] The marine debris detection device in the embodiments of this invention is described below from the perspective of hardware processing. (See attached document.) Figure 2 , Figure 2 This is a schematic diagram of the physical structure of a marine debris detection device in the embodiments of this application.

[0091] It should be noted that, Figure 2 The structure of the marine debris detection device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0092] like Figure 2As shown, the marine debris detection device includes a Central Processing Unit (CPU) 201, which can perform various appropriate actions and processes according to a program stored in Read-Only Memory (ROM) 202 or a program loaded from storage section 208 into Random Access Memory (RAM) 203, such as executing the methods described in the above embodiments. The RAM 203 also stores various programs and data required for platform operation. The CPU 201, ROM 202, and RAM 203 are interconnected via bus 204. An I / O interface 205 is also connected to bus 204. The following components are connected to the I / O interface 205: an input section 206 including audio input devices, push-button switches, etc.; an output section 207 including a Liquid Crystal Display (LCD) and audio output devices, indicator lights, etc.; a storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 209 performs communication processing via a network such as the Internet. The drive 210 is also connected to the I / O interface 205 as needed. Removable media 211, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive 210 as needed so that computer programs read from them can be installed into the storage section 208 as needed.

[0093] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 209, and / or installed from removable medium 211. When the computer program is executed by central processing unit (CPU) 201, it performs the various functions defined in the present invention.

[0094] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution platform, apparatus, or device.

[0095] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of platforms, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0096] Specifically, the marine debris detection device of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the semi-submerged marine debris detection method against light interference provided in the above embodiment.

[0097] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the marine debris detection device described in the above embodiments; or it may exist independently and not assembled into the marine debris detection device. The storage medium carries one or more computer programs, which, when executed by a processor of the marine debris detection device, cause the marine debris detection device to implement the light-interference-resistant semi-submerged marine debris detection method provided in the above embodiments.

[0098] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0099] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for detecting semi-submerged marine litter against light interference, characterized in that, include: The image acquisition device continuously acquires a sequence of images to be detected from the same target sea area under a stable field of view, and divides each frame of the sea surface image in the sequence of images to be detected into multiple image blocks to obtain multiple sets of image block sequences. Luminance temporal processing is performed on each of the multiple sets of image block sequences to obtain temporal fluctuation feature values; and color temporal processing is performed on each of the multiple sets of image block sequences to obtain temporal offset feature values. Based on the time-domain fluctuation feature value and the time-domain offset feature value, the sea surface spatial location corresponding to the multiple sets of image block sequences is divided into a low-frequency fluctuation stable region, a high-frequency fluctuation stable region, and a dynamic interference region. The adjacent common boundary within a preset neighborhood range between the low-frequency fluctuation stable region and the high-frequency fluctuation stable region, excluding the dynamic interference region, is determined as a candidate waterline. The first pixel grayscale value within a first preset length range is extracted along the normal direction of the candidate waterline towards the low-frequency fluctuation stable region, and the second pixel grayscale value within a second preset length range is extracted towards the high-frequency fluctuation stable region. A cross-waterline grayscale curve is generated based on the first pixel grayscale value and the second pixel grayscale value. If the grayscale curve across the waterline meets the preset continuous transition conditions, the low-frequency fluctuation stable region and the high-frequency fluctuation stable region are associated as suspected marine debris areas. Morphological feature detection was performed on the suspected marine debris area to obtain the detection results of semi-submerged marine debris.

2. The method of claim 1, wherein, The step of dividing the sea surface spatial location corresponding to the multiple sets of image patch sequences into a low-frequency fluctuation stable region, a high-frequency fluctuation stable region, and a dynamic interference region based on the time-domain fluctuation feature value and the time-domain offset feature value includes: Obtain the preset fluctuation detection threshold and the preset offset detection threshold; The temporal fluctuation feature value of each group of image block sequences is compared with the preset fluctuation discrimination threshold to obtain a first comparison result; and the temporal offset feature value of each group of image block sequences is compared with the preset offset discrimination threshold to obtain a second comparison result. If the first comparison result indicates that the temporal fluctuation feature value is less than the preset fluctuation discrimination threshold, and the second comparison result indicates that the temporal offset feature value is less than the preset offset discrimination threshold, the sea surface spatial position corresponding to each group of image block sequences is determined as the low-frequency fluctuation stable region, and the low-frequency fluctuation stable region includes the stable imaging part of marine debris exposed above the water surface; If the first comparison result indicates that the temporal fluctuation feature value is greater than or equal to the preset fluctuation discrimination threshold, and the second comparison result indicates that the temporal offset feature value is less than the preset offset discrimination threshold, the sea surface spatial location corresponding to each group of image block sequences is determined as the high-frequency fluctuation stable region, and the high-frequency fluctuation stable region characterizes the semi-submerged imaging part of the marine debris that is periodically covered by the water surface. If the first comparison result indicates that the temporal fluctuation feature value is greater than or equal to the fluctuation discrimination threshold, and the second comparison result indicates that the temporal offset feature value is greater than or equal to the offset discrimination threshold, the sea surface spatial position corresponding to each group of image block sequences is determined as the dynamic interference area, and the dynamic interference area represents the instantaneous specular reflection imaging part of the sea surface caused by strong light.

3. The method of claim 1, wherein, The step of associating the low-frequency fluctuation stable region and the high-frequency fluctuation stable region as suspected marine debris areas when the grayscale curve across the waterline meets the preset continuous transition conditions includes: Locate the waterline position point corresponding to the candidate waterline on the cross-waterline grayscale curve, and divide the cross-waterline grayscale curve into a first grayscale sub-curve located on one side of the low-frequency fluctuation stable region and a second grayscale sub-curve located on one side of the high-frequency fluctuation stable region according to the waterline position point. Determine the gray value of the first endpoint of the first gray sub-curve at the waterline location point and the gray value of the second endpoint of the second gray sub-curve at the waterline location point, and determine the gray value of the waterline location point based on the gray values ​​of the first endpoint and the second endpoint. The gray value of the gray value of the waterline location point characterizes the degree of gray value jump between the low-frequency fluctuation stable region and the high-frequency fluctuation stable region at the candidate waterline. The first gray value mean of all gray values ​​on the first gray value sub-curve and the second gray value mean of all gray values ​​on the second gray value sub-curve are determined, and the gray value transition difference between the two sides is determined based on the first gray value mean and the second gray value mean. The gray value transition difference between the two sides characterizes the degree of deviation between the overall gray value level of the low frequency fluctuation stable region and the overall gray value level of the high frequency fluctuation stable region. Based on the grayscale break value and the grayscale transition difference between the two sides, it is determined whether the grayscale curve of the waterline meets the preset continuous transition condition. If the grayscale curve of the waterline meets the preset continuous transition condition, the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are associated as the suspected marine debris area.

4. The method of claim 3, wherein, The step of determining whether the grayscale curve crossing the waterline meets the preset continuous transition condition based on the grayscale break value and the grayscale transition difference on both sides, and associating the low-frequency fluctuation stable region and the high-frequency fluctuation stable region as the suspected marine debris area when the grayscale curve crossing the waterline meets the preset continuous transition condition, includes: Obtain the preset fracture threshold and preset transition difference range; The grayscale fracture value is compared with the preset fracture threshold in the third comparison to obtain the third comparison result; and the grayscale transition difference between the two sides is compared with the preset transition difference range in the fourth comparison to obtain the fourth comparison result. If the third comparison result indicates that the grayscale break value is greater than or equal to the preset break threshold, or if the fourth comparison result indicates that the grayscale transition difference between the two sides is not within the preset transition difference range, it is determined that the cross-waterline grayscale curve does not meet the preset continuous transition condition, and the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are marked as non-associated sea surface areas. If the third comparison result indicates that the grayscale fracture value is less than the preset fracture threshold, and the fourth comparison result indicates that the grayscale transition difference between the two sides is within the preset transition difference range, then the grayscale curve across the waterline satisfies the preset continuous transition condition, and the low-frequency fluctuation stable area and the high-frequency fluctuation stable area are associated as the suspected marine debris area.

5. The method of claim 1, wherein, The step of performing luminance temporal processing on each of the multiple image block sequences to obtain temporal fluctuation feature values, and performing color temporal processing on each of the image block sequences to obtain temporal offset feature values, includes: Perform color space conversion on each frame image block in each group of image block sequences to obtain the luminance channel component and chrominance channel component of each frame image block in a preset color space. The first intra-block spatial mean is calculated for the luminance channel component of each frame image block to obtain the image block luminance mean; and the second intra-block spatial mean is calculated for the chrominance channel component of each frame image block to obtain the image block chrominance mean. The average brightness values ​​of the image blocks corresponding to each group of image block sequences are arranged in a first time sequence to obtain a time sequence of average brightness values. Determine the discrete statistics of the mean brightness time series in the time dimension, and define the discrete statistics as the time-domain fluctuation characteristic value; The average chromaticity values ​​of the image blocks corresponding to each group of image block sequences are arranged in a second time sequence to obtain a time sequence of average chromaticity values. Obtain preset sea surface reference chromaticity parameters, and determine the chromaticity distance sequence between each chromaticity mean in the chromaticity mean time series and the preset sea surface reference chromaticity parameters in the preset color space; The chromatic distances in the chromatic distance sequence that satisfy the preset maximum value condition are determined as the time-domain offset feature values.

6. The method of claim 1, wherein, The morphological feature detection of the suspected marine debris area to obtain the detection results of semi-submerged marine debris includes: Connectivity extraction is performed on the suspected marine debris area to obtain candidate connected components; Determine the principal axis direction where the candidate connected region intersects the candidate waterline, and use the principal axis direction as the waterline extension direction of the candidate waterline. Using the candidate waterline as the dividing line, the candidate connected region is divided into an exposed portion located on one side of the low-frequency fluctuation stable region and a semi-submerged portion located on one side of the high-frequency fluctuation stable region. A first dimension parameter of the exposed portion is determined along the extension direction of the waterline, and a second dimension parameter of the partially submerged portion is determined along the extension direction of the waterline. The first dimension parameter includes the projected length of the exposed portion along the extension direction of the waterline and the projected width of the exposed portion perpendicular to the extension direction of the waterline. The second dimension parameter includes the projected length of the partially submerged portion along the extension direction of the waterline and the projected width of the partially submerged portion perpendicular to the extension direction of the waterline. The waterline dimension continuity index and overall morphological proportion parameter of the candidate connected region are determined based on the first dimension parameter and the second dimension parameter. If the continuity index of the cross-waterline size is greater than or equal to a preset continuity threshold, and the overall morphological proportion parameter is within a preset morphological proportion range, the candidate connected region is determined to be the semi-submerged marine debris. The candidate connected component location, the first size parameter, and the second size parameter are used as the detection results of the semi-submerged marine debris.

7. The method of claim 6, wherein, After using the candidate connected component location, the first size parameter, and the second size parameter as the detection result of the semi-submerged marine debris, the method further includes: Based on the candidate connected component position, a target image region sequence corresponding to the candidate connected component is extracted from the image sequence to be detected. Using the candidate waterline as the dividing line, each frame of the target image region sequence is divided into an exposed part sub-image located on one side of the low-frequency fluctuation stable region and a half-submerged part sub-image located on one side of the high-frequency fluctuation stable region. Based on the first length ratio of the projected length of the exposed portion to the projected length of the semi-submerged portion and the second width ratio of the projected width of the exposed portion to the projected width of the semi-submerged portion, a prior vector for the semi-submerged size is constructed. The exposed portion sub-image, the partially submerged portion sub-image, and the prior vector of the partially submerged size are input into a pre-trained marine debris identification model to obtain the category identification result and confidence score output by the marine debris identification model. The category identification result represents the category of the partially submerged marine debris, and the confidence score represents the reliability of the category identification result. If the confidence score is greater than or equal to a preset confidence threshold, the category identification result and the confidence score are added to the semi-submerged marine debris detection result. If the confidence score is less than the preset confidence threshold, the candidate connected component is marked as a region to be reviewed, and a multi-frame temporal review operation is performed on the region to be reviewed. 8.A device for detecting marine litter, characterized by The marine debris detection device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the marine debris detection device to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising program instructions, characterized in that, The program instructions, when run on the sea litter detection device, cause the sea litter detection device to perform the method of any one of claims 1-7.

10. A computer program product, characterised in that, The computer program product, when run on the sea litter detection device, causes the sea litter detection device to perform the method of any one of claims 1-7.

Images