Water surface garbage classification and positioning model and water surface garbage classification and collection device and method

Abstract

The invention discloses a water surface garbage classification and positioning model and a water surface garbage classification and collection device and method. The model comprises a multi-scale feature extraction unit, a multi-scale feature fusion unit and a detection head unit. The model can be applied to a water surface garbage classification and collection device and comprises a ship body, a rotary object shifting device, a transmission device, a camera, a multi-bin storage device, a driving device, a rotating device and a controller. And a water surface garbage classification positioning model is preset in the controller. The model can efficiently and accurately identify the water surface garbage; the garbage classification and collection device applying the model can effectively classify and collect garbage on the water surface.

Description

Technical Field

[0001] The present invention relates to the field of garbage collection and classification, and more specifically, to a water surface garbage classification and positioning model, a water surface garbage classification and collection device and method. Background Art

[0002] In my country, the waters are vast, so the management of floating garbage on the water surface has become a major problem. Floating garbage on the water surface is not easy to degrade, and if it is not treated, it will cause serious damage to the ecological environment.

[0003] The traditional way of cleaning up water garbage mainly relies on manual salvage, which is difficult and inefficient. Existing surface cleaning machines mainly focus on garbage collection, but do not integrate garbage sorting, which is inefficient. The efficiency and accuracy of traditional garbage sorting are insufficient, which reduces the efficiency of garbage sorting and salvage.

[0004] The prior art discloses a surface garbage salvaging device and method based on image recognition, comprising an annular floating hull; a garbage collection device, the garbage collection device comprising a height monitoring component and a gauze arranged inside the annular floating hull, the gauze passes through the inner circle of the annular floating hull and is immersed in the water body, and the height monitoring component is used to monitor whether the height of the garbage in the gauze reaches the restricted height; a salvaging mechanism, the salvaging mechanism is arranged on the outside of the annular floating hull, the salvaging mechanism is used to salvage the garbage on the water surface and pour the salvaged garbage into the garbage collection device; an image acquisition mechanism, the image acquisition mechanism is arranged on the annular floating hull, and the image acquisition mechanism is used to identify the surface garbage and locate the position of the surface garbage. The garbage collection effect of the device and method is not good, and the garbage is soaked in water after collection, causing secondary pollution. At the same time, the efficiency and accuracy of image classification of the method are low. Summary of the invention

[0005] In view of the defects of the prior art in that the garbage classification and salvage efficiency is low and the efficiency and accuracy of garbage classification are insufficient, the present invention provides a water surface garbage classification and positioning model, a water surface garbage classification and collection device and method, which can effectively classify and collect water surface garbage and improve the accuracy and efficiency of garbage classification.

[0006] The primary purpose of the present invention is to solve the above technical problems. The technical solution of the present invention is as follows:

[0007] A water surface garbage classification and positioning model, comprising: a multi-scale feature extraction unit, a multi-scale feature fusion unit, and a detection head unit;

[0008] The image of the garbage on the water surface is input to the input end of the multi-scale feature extraction unit, the output end of the multi-scale feature extraction unit is connected to the input end of the multi-scale feature fusion unit, the output end of the multi-scale feature fusion unit is connected to the input end of the detection head unit, and the output end of the detection head unit outputs the classification result of the garbage on the water surface.

[0009] Further, the multi-scale feature extraction unit includes: a first coding layer, a first SGE unit, a second coding layer, a second SGE unit, a third coding layer, a third SGE unit, a fourth coding layer, and a fourth SGE unit;

[0010] The output end of the first coding layer is connected to the input end of the first SGE unit, the output end of the first SGE unit is connected to the input end of the second coding layer, the output end of the second coding layer is connected to the input end of the second SGE unit, the output end of the second SGE unit is connected to the input end of the third coding layer, the output end of the third coding layer is connected to the input end of the third SGE unit, the output end of the third SGE unit is connected to the input end of the fourth coding layer, and the output end of the fourth coding layer is connected to the input end of the fourth SGE unit;

[0011] The output end of the third coding layer, the output end of the fourth coding layer and the output end of the fourth SGE unit are all connected to the input end of the multi-scale feature fusion unit.

[0012] Further, the multi-scale feature fusion unit includes: a first MLCA unit, an SPPF layer, a first upsampling layer, a first connection layer, a first C3 unit, a second C3 unit, a second connection layer, a first convolutional layer, a second MLCA unit, a third MLCA unit, a third C3 unit, a third connection layer, a second convolutional layer, a fourth C3 unit, a fourth connection layer, a second upsampling layer, and a first addition point;

[0013] The output end of the fourth SGE unit is connected to the input end of the first MLCA unit, the output end of the first MLCA unit is connected to the input end of the SPPF layer, the output end of the SPPF layer is connected to the input end of the first upsampling layer, the output end of the first upsampling layer and the output end of the fourth coding layer are connected to the input end of the first connection layer, the output end of the first connection layer is connected to the input end of the first C3 unit, the output end of the first C3 unit is connected to the input end of the second MLCA unit, the output end of the second MLCA unit is connected to the input end of the second upsampling layer, the output end of the second upsampling layer and the output end of the third coding layer are connected to the input end of the second connection layer, the output end of the second connection layer is connected to the input end of the second C3 unit, and the output end of the second C3 unit is connected to the input end of the second C3 unit. The output end is connected to the input end of the first convolutional layer, the output end of the first convolutional layer and the output end of the second MLCA unit are connected to the input end of the third connection layer, the output end of the third connection layer is connected to the input end of the third C3 unit, the output end of the third C3 unit is connected to the input end of the third MLCA unit, the output end of the third MLCA unit is connected to the input end of the second convolutional layer, the output end of the second convolutional layer and the output end of the SPPF layer are connected to the input end of the fourth connection layer, the output end of the fourth connection layer is connected to the input end of the fourth C3 unit, the output end of the fourth C3 unit, the output end of the third MLCA unit, and the output end of the second C3 unit are connected to the input end of the first addition point, and the output end of the first addition point is connected to the input end of the detection head unit.

[0014] Further, the first C3 unit, the second C3 unit, the third C3 unit, and the fourth C3 unit have the same structure, and all include: a first CBS layer, a second CBS layer, a fifth connection layer, a third CBS layer, a SPAB module, and a faster-Block module;

[0015] The output end of the first CBS layer is connected to the input end of the faster-Block module, the output end of the faster-Block module and the output end of the second CBS layer are connected to the input end of the fifth connection layer, the output end of the fifth connection layer is connected to the input end of the third CBS layer, and the output end of the third CBS layer is connected to the input end of the SPAB module;

[0016] The faster-Block module includes: an eighth convolutional layer, a fourth CBS layer, a fifth CBS layer, and a third addition point;

[0017] The output end of the eighth convolutional layer is connected to the input end of the fourth CBS layer, the output end of the fourth CBS layer is connected to the input end of the fifth CBS layer, the output end of the fifth CBS layer and the input end of the eighth convolutional layer are connected to the input end of the third addition point.

[0018] Further, the detection head unit comprises: a sequence selection layer, a decoding layer;

[0019] The output end of the first adding point is connected to the input end of the sequence selection layer, and the output end of the sequence selection layer is connected to the input end of the decoding layer.

[0020] A water surface garbage classification and collection device, comprising: a hull and a controller, and also comprising a rotary object shifting device, a transmission device, a camera, a driving device and a rotating device arranged on the hull, wherein a multi-bin storage device is arranged on the rotating device;

[0021] The camera takes a water surface picture and sends the water surface picture to the controller;

[0022] The controller is preset with a water surface garbage classification and positioning model; the water surface picture is received and input into the water surface garbage classification and positioning model, the category of the water surface garbage and the relative position in the picture are obtained, and the real position of the water surface garbage is calculated; the controller controls the start and stop of the rotating device according to the category of the water surface garbage, and controls the start and stop of the driving device, the rotary object shifting device and the transmission device according to the real position of the water surface garbage;

[0023] The rotary object-moving device is used to move the surface garbage into the transmission device;

[0024] The transmission device is used to transmit the surface garbage to the multi-bin storage device;

[0025] The multi-bin storage device is provided with a plurality of storage bins for storing different types of garbage, and the rotating device drives the multi-bin storage device to rotate so that the surface garbage is transferred into the corresponding storage bin of the multi-bin storage device according to the type;

[0026] The driving device drives the hull to operate.

[0027] Furthermore, the transmission device includes: a bracket, a transmission belt, a first motor, a roller, and a plurality of limiting steps;

[0028] The bracket is vertically arranged on the hull, the top end of the bracket is connected to one end of the conveyor belt, and the other end of the conveyor belt extends into the water surface through the hull;

[0029] The inner surface of the conveyor belt is connected to the roller, the axis of the roller is connected to the rotor of the first motor, and the control end of the first motor is connected to the output end of the controller;

[0030] A plurality of limiting steps are evenly arranged on the outer surface of the conveyor belt.

[0031] Furthermore, the rotary object-shifting device comprises: a rotary shifting net provided with a rotating shaft, a second motor, and a support rod;

[0032] One end of the support rod is fixedly connected to the hull, and the other end of the support rod is connected to the rotating net. The rotating shaft of the rotating net is connected to the rotor of the second motor, and the control end of the second motor is connected to the output end of the controller.

[0033] Furthermore, the rotating device includes a rotating motor, a fixed tray, and a planetary gear;

[0034] The rotor end of the rotating motor is fixedly connected to the sun gear of the planetary gear, and the fixed tray is fixedly connected to the ring gear of the planetary gear; the multi-bin storage device is arranged on the fixed tray.

[0035] A method for collecting water surface garbage by classification, comprising:

[0036] S1: The camera acquires the water surface image and transmits it to the controller;

[0037] S2: The controller inputs the water surface image into a water surface garbage classification and positioning model to obtain the category of water surface garbage and the relative position of the water surface garbage in the image;

[0038] S3: The controller calculates the real position of the garbage on the water surface by combining the position of the camera and the relative position of the garbage in the picture;

[0039] S4: The controller controls the rotating device to drive the multi-bin storage device to rotate according to the type of the surface garbage, so that the corresponding storage bin rotates to the bottom of one end of the conveyor belt;

[0040] S5: The controller controls the driving device to drive the hull to the actual position of the surface garbage according to the position of the surface garbage;

[0041] S6: After reaching the actual position of the surface garbage, the controller controls the transmission device and the rotary object-shifting device to start, and the rotary object-shifting device shifts the surface garbage into the transmission device and then transports it to the corresponding storage bin of the multi-bin storage device.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention classifies and locates garbage through a surface garbage classification and positioning model, which speeds up the reasoning speed while effectively suppressing the interference of redundant noise, is more adaptable to the complex environment of the water area, and has high recognition accuracy. The device can complete the classification of surface garbage while realizing the automatic salvage of surface garbage, effectively reducing labor costs and improving the efficiency of salvaging and classification of surface garbage. The model is applied to the salvage device, the camera takes a picture of the water surface, and transmits it to the preset surface garbage classification and positioning model in the controller to identify the category of surface garbage and the relative position in the surface picture, and then calculate the real position of the surface garbage; then the controller controls the rotating device to drive the multi-bin storage device to rotate to the corresponding storage bin, controls the driving device to drive the hull to the real position of the garbage, controls the rotary object shifting device to start the surface garbage into the transmission device, and then transmits it to the corresponding storage bin of the multi-bin storage device, and the device improves the efficiency of garbage salvage and classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a structural diagram of the water surface garbage classification and positioning model provided in Example 1.

[0045] Figure 2 This is a structural diagram of the multi-scale feature extraction unit provided in Example 1.

[0046] Figure 3 This is a structural diagram of the multi-scale feature fusion unit provided in Example 1.

[0047] Figure 4 This is a structural diagram of the detection head unit provided in Example 1.

[0048] Figure 5 This is a structural diagram of the water surface garbage classification and positioning model provided in Example 1.

[0049] Figure 6 This is a schematic diagram of the principle of the water surface garbage classification and positioning model provided in Example 1.

[0050] Figure 7 This is a structural diagram of the MLCA unit provided in Example 1.

[0051] Figure 8 This is a schematic diagram of the principle of the MLCA unit provided in Example 1.

[0052] Fig. 9 This is a structural diagram of the SGE unit provided in Example 1.

[0053] Fig.10 This is a schematic diagram of the principle of the SGE unit provided in Example 1.

[0054] Fig.11 Structural diagram of the SGE module provided for Example 1

[0055] Fig.12 This is a structural diagram of the C3 unit provided in Example 1.

[0056] Fig.13 This is a schematic diagram of the principle of the C3 unit provided in Example 1.

[0057] Fig.14 This is a structural diagram of the faster-Block module provided in Example 1.

[0058] Fig.15 This is a structural diagram of the SPAB module provided in Example 1.

[0059] Fig.16 This is a schematic diagram of the principle of the SPAB module provided in Example 1.

[0060] Fig.17 This is a structural diagram of the SPAB submodule provided in Example 1.

[0061] Fig.18 The test result of the number of SPAB submodules provided in Example 1.

[0062] Fig.19 A structural diagram of a water surface garbage classification and collection device provided in Example 1.

[0063] Fig. 20 This is a structural front view of a water surface garbage classification and collection device provided in Example 1.

[0064] Fig.21 This is a diagram of the connection structure between the first motor and the drum provided in Example 1.

[0065] Fig. 22 This is a structural diagram of the rotating device provided in Example 1.

[0066] Fig.23 This is a structural diagram of the driving device provided in Example 1.

[0067] Fig.24 A flow chart of a method for collecting and classifying water surface garbage provided in Example 1.

[0068] Fig.25 A schematic diagram of garbage collection for a water surface garbage classification and collection device provided in Example 1.

[0069] Fig.26 This is the PR curve diagram of the water surface garbage classification and positioning model provided in Example 1.

[0070] Fig. 27 This is a normalized confusion matrix diagram of the water surface garbage classification and positioning model provided in Example 1.

[0071] Fig.28 This is a schematic diagram of the classification and positioning of the water surface garbage classification and positioning model provided in Example 1.

[0072] Fig.29 This is a comparison chart of model performance of the ablation experiment provided in Example 1.

[0073] Fig.30 This is a performance comparison chart of different algorithms provided in Example 1. DETAILED DESCRIPTION

[0074] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0075] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0076] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0077] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0078] Example 1

[0079] like Figure 1 As shown, a water surface garbage classification and positioning model includes: a multi-scale feature extraction unit, a multi-scale feature fusion unit, and a detection head unit;

[0080] The image of the garbage on the water surface is input to the input end of the multi-scale feature extraction unit, the output end of the multi-scale feature extraction unit is connected to the input end of the multi-scale feature fusion unit, the output end of the multi-scale feature fusion unit is connected to the input end of the detection head unit, and the output end of the detection head unit outputs the classification result of the garbage on the water surface.

[0081] Furthermore, if Figure 2 As shown, the multi-scale feature extraction unit includes: a first coding layer, a first SGE unit, a second coding layer, a second SGE unit, a third coding layer, a third SGE unit, a fourth coding layer, and a fourth SGE unit;

[0082] The output end of the first coding layer is connected to the input end of the first SGE unit, the output end of the first SGE unit is connected to the input end of the second coding layer, the output end of the second coding layer is connected to the input end of the second SGE unit, the output end of the second SGE unit is connected to the input end of the third coding layer, the output end of the third coding layer is connected to the input end of the third SGE unit, the output end of the third SGE unit is connected to the input end of the fourth coding layer, and the output end of the fourth coding layer is connected to the input end of the fourth SGE unit;

[0083] The output end of the third coding layer, the output end of the fourth coding layer and the output end of the fourth SGE unit are all connected to the input end of the multi-scale feature fusion unit.

[0084] Furthermore, if Figure 3 As shown, the multi-scale feature fusion unit includes: a first MLCA unit, an SPPF layer, a first upsampling layer, a first connection layer, a first C3 unit, a second C3 unit, a second connection layer, a first convolutional layer, a second MLCA unit, a third MLCA unit, a third C3 unit, a third connection layer, a second convolutional layer, a fourth C3 unit, a fourth connection layer, a second upsampling layer, and a first addition point;

[0085] The output end of the fourth SGE unit is connected to the input end of the first MLCA unit, the output end of the first MLCA unit is connected to the input end of the SPPF layer, the output end of the SPPF layer is connected to the input end of the first upsampling layer, the output end of the first upsampling layer and the output end of the fourth coding layer are connected to the input end of the first connection layer, the output end of the first connection layer is connected to the input end of the first C3 unit, the output end of the first C3 unit is connected to the input end of the second MLCA unit, the output end of the second MLCA unit is connected to the input end of the second upsampling layer, the output end of the second upsampling layer and the output end of the third coding layer are connected to the input end of the second connection layer, the output end of the second connection layer is connected to the input end of the second C3 unit, and the output end of the second C3 unit is connected to the input end of the second C3 unit. The output end is connected to the input end of the first convolutional layer, the output end of the first convolutional layer and the output end of the second MLCA unit are connected to the input end of the third connection layer, the output end of the third connection layer is connected to the input end of the third C3 unit, the output end of the third C3 unit is connected to the input end of the third MLCA unit, the output end of the third MLCA unit is connected to the input end of the second convolutional layer, the output end of the second convolutional layer and the output end of the SPPF layer are connected to the input end of the fourth connection layer, the output end of the fourth connection layer is connected to the input end of the fourth C3 unit, the output end of the fourth C3 unit, the output end of the third MLCA unit, and the output end of the second C3 unit are connected to the input end of the first addition point, and the output end of the first addition point is connected to the input end of the detection head unit.

[0086] Furthermore, if Figure 4 As shown, the detection head unit includes: a sequence selection layer and a decoding layer;

[0087] The output end of the first adding point is connected to the input end of the sequence selection layer, and the output end of the sequence selection layer is connected to the input end of the decoding layer.

[0088] It should be noted that if Figure 5 , Figure 6As shown, by combining the scale feature extraction unit, the multi-scale feature fusion unit, and the detection head unit, the water surface garbage classification and positioning model can be obtained, which includes: a first coding layer, a first SGE unit, a second coding layer, a second SGE unit, a third coding layer, a third SGE unit, a fourth coding layer, a fourth SGE unit, a first MLCA unit, an SPPF layer, a first upsampling layer, a first connection layer, a first C3 unit, a second C3 unit, a second connection layer, a first convolutional layer, a second MLCA unit, a third MLCA unit, a third C3 unit, a third connection layer, a second convolutional layer, a fourth C3 unit, a fourth connection layer, a second upsampling layer, a first addition point, a sequence selection layer, and a decoding layer;

[0089] Figure 6 The numbers contained in each level represent the data dimensions of the corresponding level.

[0090] The image of garbage on the water surface is input to the input end of the first coding layer, the output end of the first coding layer is connected to the input end of the first SGE unit, the output end of the first SGE unit is connected to the input end of the second coding layer, the output end of the second coding layer is connected to the input end of the second SGE unit, the output end of the second SGE unit is connected to the input end of the third coding layer, the output end of the third coding layer is connected to the input end of the third SGE unit, the output end of the third SGE unit is connected to the input end of the fourth coding layer, the output end of the fourth coding layer is connected to the input end of the fourth SGE unit, the output end of the fourth SGE unit is connected to the input end of the first MLCA unit, the output end of the first MLCA unit is connected to the input end of the SPPF layer, the output end of the SPPF layer is connected to the input end of the first upsampling layer, the output end of the first upsampling layer and the output end of the fourth coding layer are connected to the input end of the first connection layer, the output end of the first connection layer is connected to the input end of the first C3 unit, the output end of the first C3 unit is connected to the input end of the second MLCA unit, and the output end of the second MLCA unit is connected to the input end of the second upsampling layer. The input end of the sampling layer is connected, the output end of the second upsampling layer and the output end of the third coding layer are connected to the input end of the second connection layer, the output end of the second connection layer is connected to the input end of the second C3 unit, the output end of the second C3 unit is connected to the input end of the first convolutional layer, the output end of the first convolutional layer and the output end of the second MLCA unit are connected to the input end of the third connection layer, the output end of the third connection layer is connected to the input end of the third C3 unit, the output end of the third C3 unit is connected to the input end of the third MLCA unit, the output end of the third MLCA unit is connected to the input end of the second convolutional layer, the output end of the second convolutional layer and the output end of the SPPF layer are connected to the input end of the fourth connection layer, the output end of the fourth connection layer is connected to the input end of the fourth C3 unit, the output end of the fourth C3 unit, the output end of the third MLCA unit, and the output end of the second C3 unit are connected to the input end of the first addition point, the output end of the first addition point is connected to the input end of the sequence selection layer, the output end of the sequence selection layer is connected to the input end of the decoding layer, and the output end of the decoding layer outputs the water surface garbage classification result.

[0091] Furthermore, if Figure 7 , Figure 8 As shown, the first MLCA unit, the second MLCA unit, and the third MLCA unit have the same structure, and all include: a local average pooling layer, a first global average pooling layer, a first dimensional transformation layer, a second dimensional transformation layer, a third dimensional transformation layer, a third convolutional layer, a fourth convolutional layer, a second addition point, an anti-average pooling layer, and a mixed attention layer;

[0092] The output end of the local average pooling layer is connected to the input end of the first global average pooling layer and the input end of the first dimensional transformation layer, the output end of the first global average pooling layer is connected to the input end of the second dimensional transformation layer, the output end of the second dimensional transformation layer is connected to the input end of the third convolutional layer, the output end of the first dimensional transformation layer is connected to the input end of the fourth convolutional layer, the output end of the fourth convolutional layer is connected to the input end of the third dimensional transformation layer, the output end of the third dimensional transformation layer and the output end of the third convolutional layer are connected to the input end of the second addition point, the output end of the second addition point is connected to the input end of the inverse average pooling layer, and the output end of the inverse average pooling layer is connected to the input end of the mixed attention layer.

[0093] It should be noted that the present invention designs MLCA units, and adding them at different positions of the network backbone and head can utilize the channel attention mechanism to enhance the expression of key features. The deployment at key feature fusion nodes can simultaneously consider global and local information, and dynamically adjust the response strength of each channel by weighting feature channels, so that the water surface garbage classification and positioning model can better integrate the global information and local information of the image to make classification judgments. The output end of the local average pooling layer is connected to the input end of the first global average pooling layer and the input end of the first dimensional transformation layer, which helps to capture important local feature information. The first global average pooling layer is applied to include local spatial information; the first dimensional transformation layer and the third dimensional transformation layer are applied to include global information. The anti-average pooling layer restores the original resolution of the two vectors.

[0094] In a specific embodiment, the window size of the local average pooling layer is adaptively adjusted according to the depth of different layers of the image pyramid of the input water surface image and the size of the feature map of the overall water surface garbage classification and positioning model.

[0095] Furthermore, if Fig. 9 , Fig.10 As shown, the first SGE unit, the second SGE unit, the third SGE unit, and the fourth SGE unit have the same structure, and all include: a fifth convolutional layer, a channel partitioning layer, a first SGE module, a second SGE module, a third SGE module, a channel fusion layer, a sixth convolutional layer, a batch normalization layer, a second activation layer, and a seventh convolutional layer;

[0096] The output end of the fifth convolutional layer is connected to the input end of the channel partitioning layer, the output end of the channel partitioning layer is connected to the input end of the first SGE module, the input end of the second SGE module, and the input end of the third SGE module, the output end of the first SGE module, the output end of the second SGE module, and the output end of the third SGE module are connected to the input end of the channel fusion layer, the output end of the channel fusion layer is connected to the input end of the sixth convolutional layer, the output end of the sixth convolutional layer is connected to the input end of the batch normalization layer, the output end of the batch normalization layer is connected to the input end of the second activation layer, and the output end of the second activation layer is connected to the output end of the seventh convolutional layer;

[0097] It should be noted that the present invention designs the SGE unit so that the water surface garbage classification and positioning model can enhance target feature recognition.

[0098] In a specific embodiment, the fifth convolutional layer uses a Pconv3x3 convolutional layer.

[0099] like Fig.11 As shown, the structures of the first SGE module, the second SGE module, and the third SGE module are the same, and all include: a second global average pooling layer, a first multiplication point, a normalization layer, a first activation layer, and a second multiplication point;

[0100] The output end of the second global average pooling layer and the input end of the second global average pooling layer are connected to the first multiplication point, the output end of the first multiplication point is connected to the input end of the normalization layer, the output end of the normalization layer is connected to the input end of the first activation layer, and the output end of the first activation layer and the input end of the second global average pooling layer are connected to the input end of the second multiplication point.

[0101] It should be noted that the SGE module implements context enhancement in the spatial dimension based on the local convolution operation, so that the model can better capture the global information at different locations. The normalization layer prevents the large differences between different samples from affecting feature extraction.

[0102] In a specific embodiment, the first activation layer adopts a sigmoid activation function.

[0103] The specific formula for the first multiplication point is as follows:

[0104] c i =g·x i

[0105] i represents the serial number, g represents the global feature of the second global average pooling layer, x i represents the feature vector of the i-th channel, c i Represents the output of the first multiplication point of the i-th channel.

[0106]

[0107] i represents the serial number, g represents the global feature of the second global average pooling layer, x i represents the feature vector of the i-th channel, and m represents the total number of channels.

[0108] Furthermore, if Fig.12 , Fig.13 As shown, the first C3 unit, the second C3 unit, the third C3 unit, and the fourth C3 unit have the same structure, and all include: a first CBS layer, a second CBS layer, a fifth connection layer, a third CBS layer, a SPAB module, and a faster-Block module;

[0109] The output end of the first CBS layer is connected to the input end of the faster-Block module, the output end of the faster-Block module and the output end of the second CBS layer are connected to the input end of the fifth connection layer, the output end of the fifth connection layer is connected to the input end of the third CBS layer, and the output end of the third CBS layer is connected to the input end of the SPAB module;

[0110] It should be noted that the C3 unit designed in the present invention can speed up the reasoning speed of the water surface garbage classification and positioning model compared to the common C2f module, and can also effectively suppress the interference of redundant noise; it can quickly and effectively extract features from shallow to deep layers without additional parameter learning, and can also capture local important information and obtain higher-level feature information.

[0111] like Fig.14 As shown, the faster-Block module includes: an eighth convolutional layer, a fourth CBS layer, a fifth CBS layer, and a third addition point;

[0112] The output end of the eighth convolutional layer is connected to the input end of the fourth CBS layer, the output end of the fourth CBS layer is connected to the input end of the fifth CBS layer, the output end of the fifth CBS layer and the input end of the eighth convolutional layer are connected to the input end of the third addition point.

[0113] In a specific embodiment, the eighth convolution layer adopts the PConv convolution layer. After calculation, the floating-point calculation amount and memory usage are greatly reduced.

[0114] It should be noted that the use of the fourth CBS layer and the fifth CBS layer can maintain the diversity of features while effectively reducing computational redundancy and memory usage, thereby achieving lightweight feature extraction.

[0115] Furthermore, if Fig.15 , Fig.16 As shown, the SPAB module includes a first SPAB submodule, a second SPAB submodule, and a fourth adding point;

[0116] The output end of the first SPAB submodule is connected to the input end of the second SPAB submodule, and the output end of the first SPAB submodule and the output end of the second SPAB submodule are connected to the input end of the fourth adding point;

[0117] like Fig.17 As shown, the structures of the first SPAB submodule and the second SPAB submodule are the same, both including: a ninth convolutional layer, a third activation layer, a feature extraction layer, and a third multiplication point;

[0118] The output end of the ninth convolutional layer is connected to the input end of the third activation layer, the input end of the ninth convolutional layer and the output end of the third activation layer are connected to the input end of the feature extraction layer, and the output end of the third activation layer and the output end of the feature extraction layer are connected to the input end of the third multiplication point.

[0119] In a specific embodiment, the SPAB module can be implemented by cascading multiple SPAB sub-modules. The following is a formula used when cascading four SPAB sub-modules:

[0120] The formula for the ninth convolutional layer is:

[0121]

[0122] i and j represent the sequence number, W i (j) represents the jth convolution kernel of the i-th SPAB submodule, σ represents the activation function, represents the convolution operation, O i-1 represents the output of the i-1th SPAB submodule;

[0123] The formula for the third activation layer is:

[0124] V i =σ a (H i )

[0125] i represents the serial number, H i represents the output of the ninth convolutional layer of the i-th SPAB submodule, σ a represents the activation function;

[0126] The formula for the feature extraction layer is:

[0127]

[0128] i represents the sequence number, Indicates O i-1 With H i The residual connection of O i-1 represents the output of the i-1th SPAB submodule, H i represents the output of the ninth convolutional layer of the i-th SPAB submodule;

[0129] The formula for the third multiplication point is:

[0130] O i =U i ⊙V i

[0131] i represents the serial number, U i represents the output of the feature extraction layer of the i-th SPAB submodule, V i represents the output of the third activation layer of the i SPAB submodules.

[0132] The formula for the fourth addition point is as follows:

[0133]

[0134] O represents the final output feature, O1, O3, O4 are the outputs of the first SPAB submodule, the third SPAB submodule, and the fourth SPAB submodule, W f1 represents a 3x3 convolution kernel, O0 represents the output cascade of the first SPAB submodule, the third SPAB submodule, and the fourth SPAB submodule; Represents a convolution operation.

[0135] It should be noted that the present invention uses different numbers of SPAB submodules and calculates their impact on model floating point operations, parameter counts and average precision. Fig.18 As shown, stacking SPAB submodules effectively improves the performance of the model without the need for additional parameter learning, thereby avoiding the increase in parameter count. However, excessive stacking of SPAB submodules may lead to overfitting of the model. Therefore, through experimental verification, it is determined that the model achieves the highest classification average accuracy when using two SPAB submodules.

[0136] It should be noted that the sequence selection layer and the decoding layer constitute the RTDETR detection head, which is used to add position encoding to the multi-scale feature map, such as retaining spatial position information. This part processes these features through the self-attention mechanism, allowing the features at each position to pay attention to the features at other positions in the image, so as to better capture the relationship between objects and contextual information. The multi-scale fused features and their position encoding are introduced into the Transformer decoder, and the query vector and the feature map output by the encoder are interacted using the self-attention mechanism to generate target predictions for each query. The output of each query vector contains bounding box coordinates, category predictions, and confidence. To improve the performance of the model's target detection.

[0137] In a specific embodiment, the formula of the self-attention mechanism is as follows:

[0138]

[0139] pos is the position index, i is the dimension index, and d is the dimension of the position encoding; Q, K, and V are query, key, and value matrices, respectively. k is the dimension of the key vector.

[0140] like Fig.19 , Fig. 20 The device for collecting and classifying garbage on a water surface comprises: a hull 1 and a controller, and further comprises a rotary object shifting device 2, a transmission device 3, a camera 4, a driving device 7 and a rotating device 6 arranged on the hull 1, wherein a multi-bin storage device 5 is arranged on the rotating device 6;

[0141] The camera 4 takes a picture of the water surface and sends the picture of the water surface to the controller;

[0142] The controller is preset with the water surface garbage classification and positioning model; the water surface picture is received and input into any one of the water surface garbage classification and positioning models, the category of the water surface garbage and the relative position in the picture are obtained, and the real position of the water surface garbage is calculated; the controller controls the start and stop of the rotating device 6 according to the category of the water surface garbage, and controls the start and stop of the driving device 7, the rotary object shifting device 2 and the transmission device 3 according to the real position of the water surface garbage;

[0143] The rotary object-moving device 2 is used to move the surface garbage into the transmission device 3;

[0144] The transmission device 3 is used to transmit the surface garbage to the multi-bin storage device 5;

[0145] The multi-bin storage device 5 is provided with multiple storage bins for storing different types of garbage, and the rotating device 6 drives the multi-bin storage device 5 to rotate, so that the surface garbage is transferred into the corresponding storage bin of the multi-bin storage device 5 according to the type;

[0146] The driving device 7 drives the hull 1 to operate.

[0147] It should be noted that Fig.19 , Fig. 20 The number 8 in the middle represents garbage on the water surface.

[0148] It should be noted that, because the ship floats on the water, the distance from the camera to the water surface is known, and the angle at which the camera shoots the water surface is also known. Therefore, the real position of the water surface garbage can be simply calculated based on the relative position of the water surface garbage in the water surface picture using the geometric properties of triangles.

[0149] Furthermore, if Fig.21 As shown, the transmission device 3 includes: a bracket 304, a transmission belt 305, a first motor 301, a roller 302, and a plurality of limiting steps 303;

[0150] The bracket 304 is vertically arranged on the hull 1, the top end of the bracket 304 is connected to one end of the conveyor belt 305, and the other end of the conveyor belt 305 extends into the water surface through the hull 1;

[0151] The inner surface of the conveyor belt 305 is connected to the roller 302, the axis of the roller 302 is connected to the rotor of the first motor 301, and the control end of the first motor 301 is connected to the output end of the controller;

[0152] A plurality of limiting steps 303 are evenly arranged on the outer surface of the conveyor belt 305 .

[0153] Furthermore, the rotary object-shifting device 2 comprises: a rotary shifting net 203 provided with a rotating shaft, a second motor 201, and a support rod 202;

[0154] One end of the support rod 202 is fixedly connected to the hull 1, and the other end of the support rod 202 is connected to the rotating net 203. The rotating shaft of the rotating net 203 is connected to the rotor of the second motor 201, and the control end of the second motor 201 is connected to the output end of the controller.

[0155] Furthermore, if Fig. 22 As shown, the rotating device 6 includes a rotating motor 603, a fixed tray 602, and a planetary gear 601;

[0156] The rotor end of the rotating motor 603 is fixedly connected to the sun gear of the planetary gear 601 , and the fixed tray 602 is fixedly connected to the ring gear of the planetary gear 601 ; the multi-bin storage device 5 is arranged on the fixed tray 601 .

[0157] In a specific embodiment, Fig.23 As shown, the driving device 7 includes a propeller 701 and a driving motor 702 . The driving motor 702 is fixed to the bottom of the hull 1 , and the propeller 701 is fixedly connected to the rotor end of the driving motor 702 .

[0158] like Fig.24 As shown, a method for collecting water surface garbage by classification includes:

[0159] S1: The camera acquires the water surface image and transmits it to the controller;

[0160] S2: The controller inputs the water surface image into a water surface garbage classification and positioning model to obtain the category of water surface garbage and the relative position of the water surface garbage in the image;

[0161] S3: The controller calculates the real position of the garbage on the water surface by combining the position of the camera and the relative position of the garbage in the picture;

[0162] S4: The controller controls the rotating device to drive the multi-bin storage device to rotate according to the type of the surface garbage, so that the corresponding storage bin rotates to the bottom of one end of the conveyor belt;

[0163] S5: The controller controls the driving device to drive the hull to the actual position of the surface garbage according to the position of the surface garbage;

[0164] S6: After reaching the actual position of the surface garbage, the controller controls the transmission device and the rotary object-shifting device to start, and the rotary object-shifting device shifts the surface garbage into the transmission device and then transports it to the corresponding storage bin of the multi-bin storage device.

[0165] In a specific embodiment, Fig.25 As shown, the controller controls the driving device to drive the hull to move to the water area where the garbage is located according to the location of the surface garbage; the rotary object-shifting device shifts the surface garbage into the transmission device; and the transmission device transports the surface garbage to the multi-bin storage device.

[0166] It should be noted that Fig.25 The 8 in the figure represents surface garbage.

[0167] The images of six types of garbage, namely cans, plastic bottles, plastic bags, plastic boxes, milk boxes and leaves, were taken under different lighting conditions, and were flipped, rotated and expanded to 6000 images as a data set. The present invention verifies the results of the water surface garbage classification and positioning model, wherein the data set is divided into a training set, a validation set and a test set in a ratio of 7:2:1. The water surface garbage classification and positioning model is trained and tested under the framework of Python3.8 and pytorch2.2.1, with a batch size of 16, an epoch of 200, an initial learning rate of 0.0001 and a data image size of 640*640.

[0168] At the same time, the average precision (mAP), floating point operations (FLOPs), and frames per second (FPS) are used as evaluation indicators to evaluate the performance of the surface garbage classification and positioning model. mAP represents the average precision of all categories. By calculating the average precision of different categories and then taking the average; it provides a comprehensive evaluation of the performance of multi-class target detection tasks. By calculating the average precision of each category and averaging it, mAP can comprehensively evaluate the performance of the surface garbage classification and positioning model in detecting multiple categories of garbage.

[0169]

[0170] C represents the number of categories, j represents the sequence number, and AP j Represents the classification accuracy of the jth class.

[0171] like Fig.26As shown, the water surface garbage classification and positioning model of the present invention has an accuracy rate of more than 90% when tested on six types of data, and the mAP reaches 94.3%.

[0172] like Fig. 27 As shown, it can be seen intuitively that there is interference in the complex scene of garbage on the water surface. The diagonal value represents the proportion of consistent image predicted labels and actual labels. There is confusion in a small amount of data, and most of the data is classified accurately.

[0173] Identification and positioning results are as follows Fig.28 shown.

[0174] like Fig.29 , Fig.30 As shown in the figure, the algorithm of the present invention verifies the effectiveness of the algorithm model through ablation experiments. By introducing the RTDETR detection head and combining it with the SGE-Fasternet backbone network, the model performance is significantly improved. In addition, adaptive MLCA modules are added to the backbone and head of the network, which effectively realizes the accurate detection of multi-scale surface garbage targets, while the amount of calculation parameters and the computational complexity remain basically unchanged. We also designed a lightweight C3Faster-SPAB module, which has improved mAP compared to the improved model D, while the amount of parameters and computational complexity are low. Although the amount of parameters has doubled compared to model A, the computational complexity has only increased slightly, and the mAP has been effectively improved by 5.1%. These results show that the proposed algorithm significantly enhances the detection performance of the model while maintaining low complexity. As shown in Table 3, the algorithm model proposed in the present invention is only one-third of the RTDETR model, but the mAP has been improved, indicating that the lightweight improvement not only reduces the model size, but also enhances its generalization performance. Although the size of the YOLOv5s-MobileNetV3 model has increased compared to YOLOv8n, the mAP accuracy has increased by 5.1% and 2.7% respectively, while the FPS is only 1.7ms. These results verify that the model accuracy has been significantly improved while maintaining efficient detection speed.

[0175] The same or similar reference numerals correspond to the same or similar components;

[0176] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0177] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A water surface garbage classification and positioning model, comprising: Multi-scale feature extraction unit, multi-scale feature fusion unit, detection head unit; The image of the garbage on the water surface is input to the input end of the multi-scale feature extraction unit, the output end of the multi-scale feature extraction unit is connected to the input end of the multi-scale feature fusion unit, the output end of the multi-scale feature fusion unit is connected to the input end of the detection head unit, and the output end of the detection head unit outputs the classification result of the garbage on the water surface.

2. According to claim 1, a water surface garbage classification and positioning model is characterized in that: The multi-scale feature extraction unit includes: a first coding layer, a first SGE unit, a second coding layer, a second SGE unit, a third coding layer, a third SGE unit, a fourth coding layer, and a fourth SGE unit; The output end of the first coding layer is connected to the input end of the first SGE unit, the output end of the first SGE unit is connected to the input end of the second coding layer, the output end of the second coding layer is connected to the input end of the second SGE unit, the output end of the second SGE unit is connected to the input end of the third coding layer, the output end of the third coding layer is connected to the input end of the third SGE unit, the output end of the third SGE unit is connected to the input end of the fourth coding layer, and the output end of the fourth coding layer is connected to the input end of the fourth SGE unit; The output end of the third coding layer, the output end of the fourth coding layer and the output end of the fourth SGE unit are all connected to the input end of the multi-scale feature fusion unit.

3. According to claim 2, a water surface garbage classification and positioning model is characterized in that: The multi-scale feature fusion unit includes: a first MLCA unit, an SPPF layer, a first upsampling layer, a first connection layer, a first C3 unit, a second C3 unit, a second connection layer, a first convolutional layer, a second MLCA unit, a third MLCA unit, a third C3 unit, a third connection layer, a second convolutional layer, a fourth C3 unit, a fourth connection layer, a second upsampling layer, and a first addition point; The output end of the fourth SGE unit is connected to the input end of the first MLCA unit, the output end of the first MLCA unit is connected to the input end of the SPPF layer, the output end of the SPPF layer is connected to the input end of the first upsampling layer, the output end of the first upsampling layer and the output end of the fourth coding layer are connected to the input end of the first connection layer, the output end of the first connection layer is connected to the input end of the first C3 unit, the output end of the first C3 unit is connected to the input end of the second MLCA unit, the output end of the second MLCA unit is connected to the input end of the second upsampling layer, the output end of the second upsampling layer and the output end of the third coding layer are connected to the input end of the second connection layer, the output end of the second connection layer is connected to the input end of the second C3 unit, and the output end of the second C3 unit is connected to the input end of the second C3 unit. The output end is connected to the input end of the first convolutional layer, the output end of the first convolutional layer and the output end of the second MLCA unit are connected to the input end of the third connection layer, the output end of the third connection layer is connected to the input end of the third C3 unit, the output end of the third C3 unit is connected to the input end of the third MLCA unit, the output end of the third MLCA unit is connected to the input end of the second convolutional layer, the output end of the second convolutional layer and the output end of the SPPF layer are connected to the input end of the fourth connection layer, the output end of the fourth connection layer is connected to the input end of the fourth C3 unit, the output end of the fourth C3 unit, the output end of the third MLCA unit, and the output end of the second C3 unit are connected to the input end of the first addition point, and the output end of the first addition point is connected to the input end of the detection head unit.

4. According to claim 3, a water surface garbage classification and positioning model is characterized in that: The first C3 unit, the second C3 unit, the third C3 unit, and the fourth C3 unit have the same structure, and all include: a first CBS layer, a second CBS layer, a fifth connection layer, a third CBS layer, a SPAB module, and a faster-Block module; The output end of the first CBS layer is connected to the input end of the faster-Block module, the output end of the faster-Block module and the output end of the second CBS layer are connected to the input end of the fifth connection layer, the output end of the fifth connection layer is connected to the input end of the third CBS layer, and the output end of the third CBS layer is connected to the input end of the SPAB module; The faster-Block module includes: an eighth convolutional layer, a fourth CBS layer, a fifth CBS layer, and a third addition point; The output end of the eighth convolutional layer is connected to the input end of the fourth CBS layer, the output end of the fourth CBS layer is connected to the input end of the fifth CBS layer, the output end of the fifth CBS layer and the input end of the eighth convolutional layer are connected to the input end of the third addition point.

5. According to claim 3, a water surface garbage classification and positioning model is characterized in that: The detection head unit comprises: a sequence selection layer and a decoding layer; The output end of the first adding point is connected to the input end of the sequence selection layer, and the output end of the sequence selection layer is connected to the input end of the decoding layer.

6. A water surface garbage classification and collection device, characterized in that: include: The hull (1) and the controller also include a rotary object shifting device (2), a transmission device (3), a camera (4), a driving device (7) and a rotating device (6) arranged on the hull (1), wherein the rotating device (6) is provided with a multi-bin storage device (5); The camera (4) takes a picture of the water surface and sends the picture of the water surface to the controller; The controller is preset with a water surface garbage classification and positioning model as claimed in any one of claims 1 to 5; the water surface image is received as input into the water surface garbage classification and positioning model as claimed in any one of claims 1 to 5, the category of the water surface garbage and the relative position in the image are obtained, and the real position of the water surface garbage is calculated; The controller controls the start and stop of the rotating device (6) according to the type of the surface garbage, and controls the start and stop of the driving device (7), the rotary object shifting device (2) and the transmission device (3) according to the actual position of the surface garbage; The rotary object-moving device (2) is used to move the surface garbage into the transmission device (3); The transmission device (3) is used to transmit the surface garbage to the multi-bin storage device (5); The multi-bin storage device (5) is provided with a plurality of storage bins for storing different types of garbage, and the rotating device (6) drives the multi-bin storage device (5) to rotate so that the surface garbage is transferred into the corresponding storage bins of the multi-bin storage device (5) according to the type; The driving device (7) drives the hull (1) to operate.

7. A water surface garbage classification and collection device according to claim 6, characterized in that: The transmission device (3) comprises: a bracket (304), a transmission belt (305), a first motor (301), a roller (302), and a plurality of position-limiting steps (303); The support (304) is vertically arranged on the hull (1), the top end of the support (304) is connected to one end of a conveyor belt (305), and the other end of the conveyor belt (305) extends into the water surface through the hull (1); The inner surface of the transmission belt (305) is connected to the roller (302), the axis of the roller (302) is connected to the rotor of the first motor (301), and the control end of the first motor (301) is connected to the output end of the controller; A plurality of limiting steps (303) are evenly arranged on the outer surface of the conveyor belt (305).

8. A water surface garbage classification and collection device according to claim 6, characterized in that: The rotary object-moving device (2) comprises: a rotary moving net (203) provided with a rotating shaft, a second motor (201), and a support rod (202); One end of the support rod (202) is fixedly connected to the hull (1), the other end of the support rod (202) is connected to the rotary dial net (203), the rotating shaft of the rotary dial net (203) is connected to the rotor of the second motor (201), and the control end of the second motor (201) is connected to the output end of the controller.

9. A water surface garbage classification and collection device according to claim 6, characterized in that: The rotating device (6) comprises a rotating motor (603), a fixed tray (602), and a planetary gear (601); The rotor end of the rotating motor (603) is fixedly connected to the sun gear of the planetary gear (601), and the fixed tray (602) is fixedly connected to the ring gear of the planetary gear (601); the multi-bin storage device (5) is arranged on the fixed tray (601).

10. A method for collecting water surface garbage by classification, applied to the collection device according to any one of claims 6 to 9, characterized in that: include: S1: The camera acquires the water surface image and transmits it to the controller; S2: The controller inputs the water surface image into a water surface garbage classification and positioning model to obtain the category of water surface garbage and the relative position of the water surface garbage in the image; S3: The controller calculates the real position of the garbage on the water surface by combining the position of the camera and the relative position of the garbage in the picture; S4: The controller controls the rotating device to drive the multi-bin storage device to rotate according to the type of the surface garbage, so that the corresponding storage bin rotates to the bottom of one end of the conveyor belt; S5: The controller controls the driving device to drive the hull to the actual position of the surface garbage according to the position of the surface garbage; S6: After reaching the actual position of the surface garbage, the controller controls the transmission device and the rotary object-shifting device to start, and the rotary object-shifting device shifts the surface garbage into the transmission device and then transports it to the corresponding storage bin of the multi-bin storage device.

Images