Method and device for detecting marine floaters in turbid seawater

Abstract

The invention provides a method and a device for detecting marine floaters in turbid seawater. The method provided by the invention comprises the following steps: collecting an ocean floater image of a target ocean area to obtain an ocean floater data set; training a target detection model based on the ocean floater data set; wherein the target detection model comprises a feature extraction network, a feature processing network and a detection network, the detection network comprises partial convolution of parameter sharing, a first convolution and a second convolution, the output of the partial convolution is connected with the first convolution and the second convolution, and the training of the target detection model comprises calculating bounding box loss based on the first convolution, calculating classification loss based on the second convolution, and calculating a loss function based on the bounding box loss and the classification loss; and inputting a to-be-detected image into the trained target detection model to obtain a marine floater detection result. The method and the device provided by the invention are used for realizing efficient ocean floating object detection and solving the problems of low detection precision and high omission ratio of an existing model.

Description

Technical Field

[0001] The present application relates to the technical field of marine floating object detection, and in particular to a method and device for marine floating object detection. Background Art

[0002] In recent years, artificial intelligence and deep learning technologies have been gradually applied to the monitoring and identification of marine plastic waste. In particular, unmanned ships and drones combined with high-precision cameras and image recognition technology have provided new solutions for real-time monitoring of marine debris. However, due to the irregular distribution and diverse shapes of floating objects on the water surface, traditional image recognition technology is difficult to monitor effectively, resulting in low detection accuracy. In real applications, especially in marine scenes with large scale changes, most floating objects have the characteristics of large scale changes and various shapes, and the marine environment is complex and changeable, resulting in poor generalization ability of existing models and unsatisfactory detection effects.

[0003] Therefore, there is an urgent need for a method to improve the existing model based on the characteristics of ocean floating objects and achieve efficient ocean floating object detection to solve the problems of low detection accuracy and high missed detection rate of the existing model. Summary of the invention

[0004] In view of this, the present application provides a method and device for detecting marine floating objects under turbid sea water, which are used to improve the existing model based on the characteristics of marine floating objects, achieve efficient marine floating object detection, and solve the problems of low detection accuracy and high missed detection rate of the existing model.

[0005] Specifically, the present application is implemented through the following technical solutions: The first aspect of the present application provides a method for detecting marine floating objects under turbid water, the method comprising: Collect images of marine floating objects in a target marine area to obtain a marine floating object data set; the target marine area is turbid seawater, and the turbidity of the seawater is greater than a preset threshold; in the marine floating object image, part of the floating objects are immersed in water or the distance between the surface color of part of the floating objects and the color of the marine floating object image is less than a preset value; Training a target detection model based on the ocean floating object dataset; The target detection model includes a feature extraction network, a feature processing network and a detection network, the detection network includes a parameter-sharing partial convolution, a first convolution and a second convolution, the output of the partial convolution is respectively connected to the first convolution and the second convolution, the training target detection model includes calculating a bounding box loss based on the first convolution, calculating a classification loss based on the second convolution, and calculating a loss function based on the bounding box loss and the classification loss; The image to be detected in the target ocean area is input into the trained target detection model to obtain the ocean floating object detection result.

[0006] The second aspect of the present application provides a device for detecting floating objects in turbid waters, the device comprising an acquisition module, a training module and a detection module; wherein: The acquisition module is used to collect images of marine floating objects in a target marine area to obtain a marine floating object data set; the target marine area is turbid seawater, and the turbidity of the seawater is greater than a preset threshold; part of the floating objects in the marine floating object image are immersed in water or the distance between the surface color of part of the floating objects and the color of the marine floating object image is less than a preset value; The training module is used to train a target detection model based on the ocean floating object dataset; The target detection model includes a feature extraction network, a feature processing network and a detection network, the detection network includes a parameter-sharing partial convolution, a first convolution and a second convolution, the output of the partial convolution is respectively connected to the first convolution and the second convolution, the training target detection model includes calculating a bounding box loss based on the first convolution, calculating a classification loss based on the second convolution, and calculating a loss function based on the bounding box loss and the classification loss; The detection module is used to input the image to be detected in the target ocean area into the trained target detection model to obtain the ocean floating object detection result.

[0007] The present application provides a method and device for detecting floating objects underwater in turbid water. In the first aspect, by using partial convolution instead of standard convolution, partial convolution can adjust the receptive field more flexibly by performing convolution calculation only on certain parts of the input features, and can capture floating objects of different scales and shapes. In addition, partial convolution can reduce the amount of calculation for each convolution operation, significantly improve the processing speed of the target detection model, and through partial convolution, the target detection model can focus more on important feature areas and reduce redundant calculations. This helps to improve the efficiency of feature extraction and thus improve detection accuracy. In the marine environment, the background is often complex and changeable. Partial convolution can enable the model to ignore irrelevant background more effectively, so as to better focus on the characteristics of the floating objects themselves. In the second aspect, the convolution kernel of the partial convolution shares a set of convolution parameters, which reduces the number of model parameters and reduces the computational complexity, making the target detection model easier to deploy on resource-constrained devices (such as edge computing devices), suitable for real-time monitoring of floating objects in the ocean, and improving the response speed to floating objects in the ocean. In addition, lightweight models can reduce the risk of overfitting, especially when training data is limited. By sharing weights, the generalization ability of the model can be enhanced, thereby improving the detection effect of different types of floating objects.

[0008] In summary, the present application achieves efficient detection of marine floating objects and solves the problems of low detection accuracy and high missed detection rate in existing models. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A flow chart of Example 1 of the method for detecting floating objects in turbid water provided by the present application; Figure 2 A schematic diagram of a marine floating object data set shown as an exemplary embodiment of the present application; Figure 3 A schematic diagram of the structure of a target detection model shown in an exemplary embodiment of the present application; Figure 4 A schematic diagram of the detection result of marine floating objects shown in an exemplary embodiment of the present application; Figure 5 This is a structural schematic diagram of Example 1 of the turbid seawater underwater marine floating object detection device provided in the present application. DETAILED DESCRIPTION

[0010] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0011] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items.

[0012] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0013] Specific embodiments are given below to introduce the technical solution of the present application in detail.

[0014] Figure 1 This is a flow chart of Example 1 of the method for detecting floating objects in turbid water provided by this application. Figure 1 , the method provided in this embodiment may include: S101. Collect images of marine floating objects in a target marine area to obtain a marine floating object dataset; the target marine area is turbid seawater, and the turbidity of the seawater is greater than a preset threshold; part of the floating objects in the marine floating object images are immersed in water or the distance between the surface color of part of the floating objects and the color of the marine floating object image is less than a preset value.

[0015] Specifically, the target ocean area is set according to actual needs, and this is not limited in this embodiment. It should be noted that the target ocean area is mostly turbid seawater, and its seawater turbidity is greater than the preset threshold, wherein the preset threshold is set according to actual needs. For example, in one embodiment, the target ocean area is Zhoushan sea area, and the environment of Zhoushan sea area is complex and diverse, with significant turbid water characteristics, and affected by the natural environment, the illumination conditions and water state change more drastically. It should be noted that under different illumination conditions, the pixel color of turbid seawater is different, and the water state is also different. For example, under noon illumination conditions, the illumination conditions are strong, and the pixel color of turbid seawater usually appears brighter and clearer. At the same time, suspended matter and plankton in the water body will affect the scattering and absorption of light, making the water color appear more turbid. Due to strong illumination, the shadow of marine floating objects may be more obvious and easier to identify. Under morning and evening illumination conditions, the illumination conditions are weak, and the pixel color of turbid seawater may present different tones, such as bluer or greener. The turbidity of the water may increase the scattering of light, making the colors richer. At this time, the effect of suspended matter in the water on light may be more complicated, and it is easy to produce reflection and refraction, resulting in uneven color and brightness, which increases the difficulty of identifying marine floating objects.

[0016] It should be noted that the distance between some floating objects in the marine floating object image and the color of some floating objects is less than the preset value, which makes it difficult to detect marine floating objects. On the one hand, when some floating objects are immersed in water, the water body will block part of the floating object surface, making the edge contour blurred, and it is difficult for the target detection model to accurately locate the edge, especially for monitoring based on edge information. In addition, the reflection and refraction of the water surface will interfere with the true color and shape of the floating objects, making the color, brightness and clarity of the floating objects in the image different from the original objects, thereby affecting the extraction and recognition of the floating object features by the target detection model. Due to the turbidity or transparency of the water, the floating objects immersed in the water often appear to have a color similar to the water background, especially in turbid water, the color features are more difficult to distinguish, which increases the difficulty of detection. On the other hand, when the color of the surface of some floating objects is similar to the color of the marine floating object image, the target detection model usually relies on the contrast between the floating objects and the background to identify the object. If the colors are similar and the contrast is low, the target detection model is difficult to distinguish the boundaries of the floating objects, making the edge detection and region segmentation inaccurate. Moreover, similar colors will mask the detailed features of floating objects, making the shape, texture and other information of floating objects visually "blurred". Even if there is boundary information, the target detection model is likely to mistake floating objects as background during segmentation. The target detection model relies on color thresholds or segmentation strategies based on pixel features. When the color of floating objects is close to the background color, the preset threshold cannot effectively distinguish between floating objects and background, resulting in missed detection. Therefore, it is necessary to collect images of marine floating objects in turbid seawater areas as much as possible.

[0017] In a specific implementation, the collecting of ocean floating object images of the target ocean area to obtain an ocean floating object data set includes: collecting a first ocean floating object image of the target ocean area on the spot; collecting a second ocean floating object image of the target ocean area on the Internet; performing data enhancement processing on the first ocean floating object image and the second ocean floating object image; the data enhancement processing at least includes flipping, rotating, scaling and color adjustment; labeling the ocean floating object images based on the type and position of the ocean floating objects in each ocean floating object image to obtain an ocean floating object data set; the ocean floating object data set contains ocean floating object images with different labels.

[0018] Specifically, a first image of ocean floating objects is photographed and collected in the target ocean area by using a drone, a ship or other acquisition equipment, and a second image of ocean floating objects in the target ocean area is obtained by using network resources (such as public databases, relevant research public data, etc.). Furthermore, different images of ocean floating objects are generated by horizontally or vertically flipping images, randomly rotating images, randomly scaling images, and changing image brightness, contrast, and saturation. Each ocean floating object image contains ocean floating objects of different types and at different positions in the image. The ocean floating object images are annotated based on the types and positions of the ocean floating objects in each ocean floating object image, and an ocean floating object dataset containing ocean floating object images with different labels is obtained. Figure 2 A schematic diagram of an ocean floating object dataset shown as an exemplary embodiment of the present application.

[0019] As an optional embodiment, the method for amplifying the sample image specifically includes: obtaining a first sample image, in which a first marine floating object is marked, and the image other than the first marine floating object in the first sample image is a first background; matching a first amplification operation of the first marine floating object, and matching a second amplification operation of the first background; changing the first marine floating object according to the first amplification operation, changing the first background according to the second amplification operation, merging the changed first marine floating object and the changed first background, and generating an amplified image of the sample image. Specifically, the first amplification operation can be no operation, horizontal or vertical flipping of the image, random rotation of the image, random scaling of the image, and changing the image brightness, contrast and saturation, and the amplitude of the change can be calculated based on a random number calculation method. The second amplification operation can be no operation, horizontal or vertical flipping of the image, random rotation of the image, random scaling of the image, and changing the image brightness, contrast and saturation, and in the same amplified image, the first amplification operation and the second amplification operation are different, or the first amplification operation and the second amplification operation are the same, but the operation direction and / or operation amplitude are different. For example, one is clockwise rotation and the other is counterclockwise rotation. Determine the annotation of the amplified image according to the annotation of the first sample image. Specifically, if the amplified image has only undergone the second amplification operation, the position and boundary of the first marine floating object marked in the first sample image are used as the position and boundary of the marine floating object in the amplified image. If the amplified image has undergone the first amplification operation, the position and boundary of the first marine floating object are adjusted according to the direction and amplitude of the first amplification operation to generate the annotation of the marine floating object in the amplified image.

[0020] The method provided in this embodiment, on the one hand, by collecting images of marine floating objects in the target ocean area, the constructed marine floating object data set is mostly turbid seawater and has significant turbid water characteristics. Since the target ocean area is affected by the natural environment, the lighting conditions and water state change more dramatically, which increases the difficulty of the marine floating object detection task. By collecting images of marine floating objects in such a complex environment, when the target detection model is subsequently trained based on the marine floating object data set, the generalization ability of the target detection model in different water scenes is improved, ensuring that it maintains a high detection accuracy under various environmental conditions. On the second hand, through field collection, network collection and data enhancement, a diversified marine floating object data set is constructed, which significantly expands the scale of the data set, and enhances the robustness and generalization ability of the target detection model, so that its detection effect in a complex marine environment has been effectively improved.

[0021] S102: training a target detection model based on the ocean floating object dataset.

[0022] The target detection model includes a feature extraction network, a feature processing network and a detection network, the detection network includes a parameter-sharing partial convolution, a first convolution and a second convolution, the output of the partial convolution is respectively connected to the first convolution and the second convolution, the training target detection model includes calculating a bounding box loss based on the first convolution, calculating a classification loss based on the second convolution, and calculating a loss function based on the bounding box loss and the classification loss; Specifically, Figure 3 This is a schematic diagram of the structure of the target detection model shown in an exemplary embodiment of the present application. Figure 3 The target detection model includes a feature extraction network, a feature processing network and a detection network. The feature extraction network is used to extract features from the input marine floating object image to obtain a feature map of the marine floating object image. The feature processing network is used to fuse the feature map of the marine floating object image to obtain a fused feature map. Based on the fused feature map, the detection network outputs the marine floating object detection results, including the type, location and weight of the marine floating object.

[0023] Optionally, the convolution kernels of the partial convolution share a set of convolution parameters, the partial convolution determines the ocean floating object pixels within the convolution kernel coverage area based on the mask of the input ocean floating object feature map, performs a convolution operation on the ocean floating object pixels based on the convolution parameters, and performs normalization processing based on the number of the ocean floating object pixels to obtain an output ocean floating object feature map.

[0024] It should be noted that since the computational amount and parameter amount of the detection network account for half of the total computational amount and total parameter amount of the entire target detection model, in order to reduce the number of parameters and computational amount of the target detection model, the idea of ​​weight sharing is introduced in the detection network. Multiple convolution kernels with a set of shared parameters are used instead of convolution kernels with independent parameters, that is, the convolution kernels of partial convolution share a set of convolution parameters. Specifically, if a standard convolution has N independent 3×3 convolution kernels, then each convolution kernel has 9 parameters, and the entire convolution layer will have 9N parameters. In partial convolution, these N convolution kernels will share the same set of 9 parameters, thereby reducing the total number of parameters to 9.

[0025] In specific implementation, the process of partially convolving and outputting the feature map of ocean floating objects includes: (1) Determine all the ocean floating object pixels in the ocean floating object feature map based on the mask of the input ocean floating object feature map.

[0026] Specifically, the mask value is related to the validity of the pixel. When the mask value of the pixel is 1, it indicates that the pixel is a valid pixel (not missing or transparent); when the mask value of the pixel is 0, it indicates that the pixel is an invalid pixel (missing value, blocked or transparent).

[0027] In a specific implementation, the mask based on the input ocean floating object feature map determines all ocean floating object pixels in the ocean floating object feature map, including: generating an initial mask feature map based on the characteristics of the ocean floating object feature map; the size of the initial mask feature map is the same as that of the ocean floating object feature map; in each convolution operation, when there are ocean floating object pixels in the coverage area of ​​the convolution kernel, the mask value of the coverage area is set to 1, and when there are no ocean floating object pixels in the coverage area of ​​the convolution kernel, the mask value of the coverage area is set to 0, to obtain an updated mask feature map; based on the mask feature map, pixels with a mask value of 1 in the coverage area of ​​the convolution kernel are determined as ocean floating object pixels.

[0028] Specifically, the size of the initial mask feature map is the same as the size of the input ocean floating object feature map. In this step, an initial mask feature map of the same size is created based on the size of the input ocean floating object feature map, and the initial mask feature map is initialized based on the validity of each ocean floating object pixel in the input ocean floating object feature map. Further, each time a convolution operation is performed, the initial mask feature map is updated based on whether there are ocean floating object pixels in the coverage area of ​​the convolution kernel (when there are ocean floating object pixels, the mask value of the corresponding coverage area is set to 1; when there are no ocean floating object pixel values, the mask value of the corresponding coverage area is set to 0). After all convolution operations are completed, the updated mask feature map is traversed, and all pixels with a mask value of 1 are determined as ocean floating object pixels.

[0029] (2) For each ocean floating object pixel, the product of the value of the ocean floating object pixel in the ocean floating object feature map and the convolution parameter is determined as the output ocean floating object feature map.

[0030] In specific implementation, the output ocean floating debris characteristic map can be calculated based on the following formula: ; Among them, the is the characteristic value of each pixel of ocean floating objects output; is the convolution parameter; For each ocean debris pixel, the value in the input ocean debris feature map.

[0031] For details, please continue to refer to Figure 3 , combined with the above description, the detection network is used to determine the marine floating object detection result based on the fused feature map output by the feature extraction network, and the detection network includes a partial convolution, a first convolution and a second convolution, wherein the output of the partial convolution is connected to the first convolution and the second convolution respectively.

[0032] Furthermore, when training a target detection model based on a marine floating object data set, the detection process of the detection network includes: the partial convolution determines the marine floating object pixels within the coverage area of ​​the convolution kernel based on the mask of the input marine floating object feature map, performs a convolution operation on the marine floating object pixels based on shared convolution parameters, performs normalization processing based on the number of marine floating object pixels, and passes the obtained marine floating object feature map to the first convolution and the second convolution respectively; the first convolution performs convolution processing on the marine floating object feature map based on the positioning task to obtain floating object position feature information, in which the proportion of floating object boundary information in the floating object position feature information is greater than that of other object information; the second convolution performs convolution processing on the marine floating object feature map based on the classification task to obtain floating object feature information, in which the proportion of turbid seawater feature information in the floating object feature information is lower than that of actual floating object feature information.

[0033] Specifically, the first convolution and the second convolution perform different convolution processing on the marine floating object feature map output by some convolutions based on different tasks. The first convolution is mainly used to detect the boundary information of marine floating objects and determine the position of marine floating objects in the output marine floating object feature map. The proportion of floating object boundary information in the floating object position feature information is greater than that of other object information. The second convolution is mainly used to detect the type of marine floating objects. The proportion of turbid seawater feature information in the floating object feature information is lower than that of the actual floating object feature information.

[0034] In specific implementation, combined with the above description, the partial convolution performs convolution operation on the marine floating object pixels within the convolution kernel coverage area during each convolution operation, and passes the obtained marine floating object feature map to the first convolution and the second convolution respectively. The first convolution performs convolution processing on the marine floating object feature map based on the positioning task, outputs the obtained floating object position feature information (usually including the marine floating object location), and calculates the corresponding bounding box loss. The second convolution performs convolution processing on the marine floating object feature map based on the classification task, outputs the obtained floating object feature information (usually including the marine floating object type and the corresponding weight), and calculates the corresponding classification loss. Furthermore, the bounding box loss and classification loss are calculated comprehensively to calculate the loss function, and the target detection model is adjusted by continuously comparing the loss function with the preset value.

[0035] Please refer to Figure 3 ,The target detection model includes multiple detection networks, each of which corresponds to feature maps of different scales to detect marine floating objects of different sizes.

[0036] Optionally, the feature extraction network includes a deformable convolution, which configures an offset and a weight for each sampling point, and the position and size of the convolution kernel of the deformable convolution dynamically change with the shape of the floating object; the feature processing network introduces a coordinate attention mechanism, performs global pooling on the input feature map in the horizontal and vertical directions, and obtains feature maps in the horizontal and vertical directions; the attention weights of the feature maps are calculated respectively, and the feature maps are multiplied and weighted based on the attention weights to obtain the processed feature maps.

[0037] For details, please continue to refer to Figure 3 The feature extraction network includes a deformable convolution. The position of each sampling point of the deformable convolution is not fixed. An offset is added to the original sampling point position. The offset is based on the characteristic changes of marine floating objects.

[0038] In specific implementation, the process of outputting the feature map of the deformable convolution includes: (1) Perform convolution operation on the input image to generate floating object features.

[0039] For details, please refer to Figure 3 ,The input image undergoes conventional convolution operations to generate preliminary float features.

[0040] (2) Generate an offset matrix based on the size of the input image and the floating object features; the offset matrix contains the offsets of the sampling points in different directions.

[0041] Specifically, the size of the offset matrix is ​​consistent with the size of the input image, and the offset matrix has 2N channels, which respectively represent the offsets of the sampling points in the X and Y directions of the two-dimensional plane, where N represents the number of sampling points.

[0042] In specific implementation, the offset field layer constructs an offset matrix based on the size of the input image and the floating image features, and determines the offset of the sampling point in different directions based on the offset matrix.

[0043] It should be noted that the calculated offset usually needs to be optimized through an interpolation algorithm and a back-propagation algorithm, and is usually a small value.

[0044] (3) The deformable convolution calculates the feature value of each sampling point based on the weight corresponding to the floating object feature and the position pixel of each sampling point; the position pixel of each sampling point is calculated based on the sampling point position, and the sampling point position is equal to the sum of the original sampling point position and the offset.

[0045] In specific implementation, the characteristic value of each sampling point can be calculated according to the following formula: ; Among them, the is the characteristic value of each sampling point; is the weight corresponding to the floating object feature; is the position pixel of each sampling point; is the sampling point; is the offset of the sampling point; is the original sampling point position.

[0046] in, Through bilinear interpolation calculation, it can be expressed as: ; Among them, the is the position pixel of each sampling point; is the location of each sampling point; is the sampling point position in the input image The nearest four integer points; For the input image The pixel value of is a two-dimensional bilinear interpolation kernel function, which can be expressed as: ; Among them, the , They are The components on the X-axis and Y-axis; , They are The components on the X-axis and Y-axis; , is a variable.

[0047] For details, please continue to refer to Figure 3 The target detection model includes a feature processing network. The feature processing network introduces a coordinate attention mechanism (convolution, fusion), and globally pools the feature map output by the feature extraction network in the horizontal and vertical directions to obtain the feature maps in the horizontal and vertical directions. The attention weights of the feature maps in the horizontal and vertical directions are calculated respectively, and the feature maps in the horizontal and vertical directions are multiplied and weighted based on the attention weights to obtain the processed feature maps.

[0048] In a specific implementation, the input feature map is globally pooled in the horizontal and vertical directions to obtain feature maps in the horizontal and vertical directions, including: for each horizontal position of the input feature map, the floating object features of the horizontal position in all vertical directions are aggregated to obtain a feature map in the horizontal direction; the width of the feature map in the horizontal direction is the same as the width of the input feature map, and the height of the feature map in the horizontal direction is 1; for each vertical position of the input feature map, the floating object features of the vertical position in all horizontal directions are aggregated to obtain a feature map in the vertical direction; the height of the feature map in the vertical direction is the same as the height of the input feature map, and the width of the feature map in the vertical direction is 1.

[0049] Specifically, the feature maps in the horizontal and vertical directions can be calculated respectively according to the following formulas: ; Among them, the The feature map in the vertical direction is and Channel The eigenvalue on is the width of the input feature map; The input feature map is at height and width The eigenvalues ​​on .

[0050] ; Among them, the is the feature map in the horizontal direction at height and Channel The eigenvalue on is the height of the input feature map; The input feature map is at height and width The eigenvalues ​​on .

[0051] Furthermore, in the spatial dimension, the feature maps in the vertical and horizontal directions are processed for feature dimensionality reduction using a convolution with a convolution kernel size of 1×1, and then batch normalization is performed. The feature maps in the vertical and horizontal directions are then processed by a nonlinear function, and the feature maps are further segmented using a convolution with a convolution kernel size of 1×1 to obtain segmented feature maps in the vertical and horizontal directions. The size of the segmented feature maps is consistent with the size of the input feature map. Furthermore, the segmented feature maps in the vertical and horizontal directions are processed by a Sigmoid activation function to obtain the vertical attention weight and the horizontal attention weight: ; ; Among them, the is the attention weight in the vertical direction; is the feature map in the vertical direction after segmentation; is the attention weight in the horizontal direction; It is the feature map in the horizontal direction after segmentation.

[0052] Furthermore, the feature maps in the vertical and horizontal directions after segmentation are product-weighted based on the attention weights in the vertical and horizontal directions to obtain the processed feature maps, that is, the product weighting is performed according to the following formula: ; Among them, the is the feature value of the processed feature map at width i and height j; is the characteristic value of the input feature map at width i and height j; is the attention weight of width i in the vertical direction; is the attention weight of height j in the horizontal direction.

[0053] S103: Input the image to be detected of the target ocean area into the trained target detection model to obtain the ocean floating object detection result.

[0054] Specifically, the ocean floating object detection results include the location, type and weight of the ocean floating objects.

[0055] In the specific implementation, the image to be detected in the target ocean area is input into the trained target detection model, and the target detection model outputs the ocean floating object detection result, including the location, type and weight of the ocean floating objects. Figure 4 This is a schematic diagram of the detection results of marine floating objects according to an exemplary embodiment of the present application. Figure 4 ,Different marine floating object detection results correspond to marine floating objects of different locations, types and weights.

[0056] The present embodiment provides a method for detecting floating objects in turbid waters. In the first aspect, by using partial convolution instead of standard convolution, partial convolution can adjust the receptive field more flexibly by performing convolution calculation only on certain parts of the input features, and can capture floating objects of different scales and shapes. In addition, partial convolution can reduce the amount of calculation for each convolution operation, significantly improve the processing speed of the target detection model, and through partial convolution, the target detection model can focus more on important feature areas and reduce redundant calculations. This helps to improve the efficiency of feature extraction and thus improve detection accuracy. In the marine environment, the background is often complex and changeable, and partial convolution can enable the model to more effectively ignore irrelevant backgrounds, thereby better focusing on the characteristics of the floating objects themselves. In the second aspect, the convolution kernels of the partial convolution share a set of convolution parameters, which reduces the number of model parameters and reduces the computational complexity, making the target detection model easier to deploy on resource-constrained devices (such as edge computing devices), suitable for real-time monitoring of floating objects in the ocean, and improving the response speed to floating objects in the ocean. In addition, lightweight models can reduce the risk of overfitting, especially when training data is limited. By sharing weights, the generalization ability of the model can be enhanced, thereby improving the detection effect of different types of floating objects.

[0057] In summary, the present application realizes efficient marine floating object detection and solves the problems of low detection accuracy and high missed detection rate in the existing model. On the third hand, the deformable convolution is introduced in the feature extraction network, which can improve the feature extraction ability of the model for floating objects of different scales and shapes. In this way, the convolution kernel can dynamically adjust the position and size according to the morphology of the marine floating objects, effectively improving the detection accuracy of irregular marine floating objects, especially in the marine floating object detection task with large scale changes and diverse morphology. On the fourth hand, a lightweight coordinate attention mechanism is integrated into the feature processing network, combining the spatial position information and channel information of the marine floating objects in the marine floating object image, thereby enhancing the target detection model's ability to focus on key feature areas. In an environment with turbid water and complex lighting, the robustness and accuracy of the model detection are effectively improved through this mechanism. In addition, by lightweight processing of the network structure, the demand for computing resources is significantly reduced, so that the model can be effectively deployed in an environment with limited computing resources to meet the needs of real-time detection. On the fifth hand, by collecting marine floating object images in the target marine area, the constructed marine floating object data set is mostly turbid seawater and has significant turbid water characteristics. Since the target ocean area is affected by the natural environment, the lighting conditions and water state change dramatically, which increases the difficulty of the marine floating object detection task. By collecting images of marine floating objects in such a complex environment, the generalization ability of the target detection model in different water scenes is improved when the target detection model is subsequently trained based on the marine floating object dataset, ensuring that it maintains a high detection accuracy under various environmental conditions. Sixthly, through field collection, network collection and data enhancement, a diverse marine floating object dataset is constructed, which significantly expands the scale of the dataset and enhances the robustness and generalization ability of the target detection model, so that its detection effect in complex marine environments has been effectively improved.

[0058] Corresponding to the aforementioned embodiment of a method for detecting marine floating objects under turbid sea water, the present application also provides an embodiment of a device for detecting marine floating objects under turbid sea water.

[0059] Figure 5 This is a schematic diagram of the structure of the first embodiment of the turbid seawater underwater marine floating object detection device provided by this application. Figure 3 The device provided in this embodiment includes an acquisition module 310, a training module 320 and a detection module 330; wherein, The acquisition module 310 is used to collect images of marine floating objects in a target marine area to obtain a marine floating object dataset; the target marine area is turbid seawater, and the seawater turbidity is greater than a preset threshold; part of the floating objects in the marine floating object image are immersed in water or the distance between the surface color of part of the floating objects and the color of the marine floating object image is less than a preset value; The training module 320 is used to train a target detection model based on the ocean floating object dataset; The target detection model includes a feature extraction network, a feature processing network and a detection network, the detection network includes a parameter-sharing partial convolution, a first convolution and a second convolution, the output of the partial convolution is respectively connected to the first convolution and the second convolution, the training target detection model includes calculating a bounding box loss based on the first convolution, calculating a classification loss based on the second convolution, and calculating a loss function based on the bounding box loss and the classification loss; The detection module 330 is used to input the image to be detected in the target ocean area into the trained target detection model to obtain the ocean floating object detection result.

[0060] The device of this embodiment can be used to perform Figure 1 The steps, specific implementation principles and implementation processes of the method embodiment shown are similar and will not be repeated here.

[0061] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0062] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0063] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for detecting floating objects in turbid sea water, characterized in that: The method comprises: Collect images of marine floating objects in a target marine area to obtain a marine floating object data set; the target marine area is turbid seawater, and the turbidity of the seawater is greater than a preset threshold; in the marine floating object image, part of the floating objects are immersed in water or the distance between the surface color of part of the floating objects and the color of the marine floating object image is less than a preset value; Training a target detection model based on the ocean floating object dataset; The target detection model includes a feature extraction network, a feature processing network and a detection network, the detection network includes a parameter-sharing partial convolution, a first convolution and a second convolution, the output of the partial convolution is respectively connected to the first convolution and the second convolution, the training target detection model includes calculating a bounding box loss based on the first convolution, calculating a classification loss based on the second convolution, and calculating a loss function based on the bounding box loss and the classification loss; The image to be detected in the target ocean area is input into the trained target detection model to obtain the ocean floating object detection result.

2. The method according to claim 1, characterized in that: The convolution kernels of the partial convolution share a set of convolution parameters. The partial convolution determines the ocean floating object pixels within the convolution kernel coverage area based on the mask of the input ocean floating object feature map, performs a convolution operation on the ocean floating object pixels based on the convolution parameters, and performs normalization processing based on the number of the ocean floating object pixels to obtain an output ocean floating object feature map.

3. The method according to claim 2, characterized in that The process of partially convolving and outputting the characteristic map of ocean floating objects includes: Determine all the ocean floating object pixels in the ocean floating object feature map based on the mask of the input ocean floating object feature map; For each ocean floating object pixel, the product of the value of the ocean floating object pixel in the ocean floating object feature map and the convolution parameter is determined as the output ocean floating object feature map.

4. The method according to claim 3, characterized in that The mask based on the input ocean floating object feature map determines all ocean floating object pixels in the ocean floating object feature map, including: Based on the characteristics of the ocean floating object feature map, an initial mask feature map is generated; the size of the initial mask feature map is the same as that of the ocean floating object feature map; In each convolution operation, when there are marine floating object pixels in the convolution kernel coverage area, the mask value of the coverage area is set to 1, and when there are no marine floating object pixels in the convolution kernel coverage area, the mask value of the coverage area is set to 0, to obtain an updated mask feature map; Based on the mask feature map, pixels with a mask value of 1 in the convolution kernel coverage area are determined as ocean floating object pixels.

5. The method according to claim 1, characterized in that The detection network is used to determine the marine floating object detection result based on the fused feature map output by the feature extraction network. The detection process of the detection network includes: The partial convolution determines the ocean floating object pixels within the convolution kernel coverage area based on the mask of the input ocean floating object feature map, performs a convolution operation on the ocean floating object pixels based on the shared convolution parameters, performs normalization processing based on the number of the ocean floating object pixels, and passes the obtained ocean floating object feature map to the first convolution and the second convolution respectively; The first convolution performs convolution processing on the marine floating object feature map based on the positioning task to obtain floating object position feature information, wherein the floating object boundary information accounts for a larger proportion than other object information in the floating object position feature information; The second convolution performs convolution processing on the ocean floating object feature map based on the classification task to obtain floating object feature information, in which the proportion of turbid seawater feature information in the floating object feature information is lower than that of actual floating object feature information.

6. The method according to claim 1, characterized in that The feature extraction network includes a deformable convolution, which configures an offset and a weight for each sampling point. The position and size of the convolution kernel of the deformable convolution dynamically change with the shape of the floating object; the feature processing network introduces a coordinate attention mechanism, performs global pooling on the input feature map in the horizontal and vertical directions, and obtains feature maps in the horizontal and vertical directions; the attention weights of the feature maps are calculated respectively, and the feature maps are multiplied and weighted based on the attention weights to obtain the processed feature maps.

7. The method according to claim 6, characterized in that The process of outputting the deformable convolution feature map includes: Perform convolution operation on the input image to generate floating object features; Generate an offset matrix based on the size of the input image and the floating object features; the offset matrix includes the offsets of the sampling points in different directions; The deformable convolution calculates the feature value of each sampling point based on the weight corresponding to the floating object feature and the position pixel of each sampling point; the position pixel of each sampling point is calculated based on the sampling point position, and the sampling point position is equal to the sum of the original sampling point position and the offset.

8. The method according to claim 6, characterized in that The input feature map is globally pooled in the horizontal and vertical directions to obtain feature maps in the horizontal and vertical directions, including: For each horizontal position of the input feature map, aggregate the floating object features of the horizontal position in all vertical directions to obtain a feature map in the horizontal direction; the width of the feature map in the horizontal direction is the same as the width of the input feature map, and the height of the feature map in the horizontal direction is 1; For each vertical position of the input feature map, the floating object features of the vertical position in all horizontal directions are aggregated to obtain a feature map in the vertical direction; the height of the feature map in the vertical direction is the same as the height of the input feature map, and the width of the feature map in the vertical direction is 1.

9. The method according to claim 1, characterized in that: The collecting of images of ocean floating objects in the target ocean area to obtain an ocean floating object dataset includes: On-site collection of the first marine floating object images in the target marine area; Collecting images of second marine floating objects in the target marine area on the Internet; Performing data enhancement processing on the first marine floating object image and the second marine floating object image; the data enhancement processing at least includes flipping, rotating, scaling and color adjustment; The ocean floating object images are labeled based on the types and positions of the ocean floating objects in each ocean floating object image to obtain an ocean floating object dataset; the ocean floating object dataset contains ocean floating object images with different labels.

10. A device for detecting floating objects in turbid sea water, characterized in that: The device comprises an acquisition module, a training module and a detection module; wherein, The acquisition module is used to collect images of marine floating objects in a target marine area to obtain a marine floating object data set; the target marine area is turbid seawater, and the turbidity of the seawater is greater than a preset threshold; part of the floating objects in the marine floating object image are immersed in water or the distance between the surface color of part of the floating objects and the color of the marine floating object image is less than a preset value; The training module is used to train a target detection model based on the ocean floating object dataset; The target detection model includes a feature extraction network, a feature processing network and a detection network, the detection network includes a parameter-sharing partial convolution, a first convolution and a second convolution, the output of the partial convolution is respectively connected to the first convolution and the second convolution, the training target detection model includes calculating a bounding box loss based on the first convolution, calculating a classification loss based on the second convolution, and calculating a loss function based on the bounding box loss and the classification loss; The detection module is used to input the image to be detected in the target ocean area into the trained target detection model to obtain the ocean floating object detection result.

Images