Rapid image recognition method for unstable floating platform on water

Abstract

The invention provides a rapid image recognition method for an overwater unstable floating platform, and belongs to the field of computer vision and intelligent environment monitoring. The rapid image recognition method comprises the following steps: S1, collecting image data of at least one type of river water area floating objects, and marking river water area floating object data containing a specific semantic category in the collected image data of the river water area floating objects, constructing the watercourse water area floating object data containing the specific semantic category into a reference data set for floating object detection; s2, constructing a multi-scale attention mechanism network model suitable for river water area floating object detection; s3, in the multi-scale attention mechanism network model established in the step S2, training the river water area floating object data collected in the step S1 and the marked specific semantic category to obtain a trained weight; and S4, collecting image data of the floaters in the real river water area, inputting the image data into the multi-scale attention mechanism network model, and detecting the semantic category of the floaters in the real river water area by using the weight obtained in the step S3.

Description

Technical Field

[0001] The invention belongs to the field of computer vision and intelligent environment monitoring, and relates to a method for rapid image recognition of an unstable floating platform on water. Background Art

[0002] In recent years, with the rapid development of computer vision technology, intelligent garbage detection and cleaning systems based on image recognition have gradually become the key to solving this problem. However, due to the instability of the water surface environment (such as waves, wind and other factors), traditional image recognition technology faces challenges and is prone to reduced recognition accuracy due to platform shaking.

[0003] To this end, it is particularly important to develop an image recognition technology that can adapt to unstable water environments and combine attention mechanisms with target detection algorithms to improve the accuracy and efficiency of surface garbage detection. Summary of the invention

[0004] The purpose of the present invention is to at least partially solve the above-mentioned technical problems, and to provide a method for rapid image recognition of an unstable floating platform on water. The method for rapid image recognition of an unstable floating platform on water provided by the present invention particularly relates to garbage detection and processing of floating platforms on water based on image recognition technology. Specifically, the method for rapid image recognition is applied to unstable environments on water, and through the combination of real-time image acquisition, attention mechanism and target detection algorithm, rapid recognition and cleaning of surface garbage is achieved.

[0005] In one aspect of the present invention, a method for rapid image recognition of an unstable floating platform on water is provided, the method comprising the following steps:

[0006] S1 collects image data of at least one type of floating objects in river waters, annotates the collected image data of floating objects in river waters with floating objects data in river waters containing specific semantic categories, and constructs the floating objects data in river waters containing specific semantic categories as a benchmark data set for floating object detection;

[0007] S2 builds a multi-scale attention mechanism network model suitable for floating object detection in river waters;

[0008] S3 trains the river water floating object data collected in step S1 and the annotated specific semantic categories in the multi-scale attention mechanism network model established in step S2 to obtain trained weights;

[0009] S4 collects image data of floating objects in real river waters and inputs them into the multi-scale attention mechanism network model, and uses the weights obtained in step S3 to detect the semantic categories of floating objects in real river waters.

[0010] In some embodiments, in step S1, at least one type of river water area platform is used to collect image data of floating objects in the water area, and the images cover floating objects in the river water area in different time periods and / or different climates.

[0011] In some embodiments, in step S1, when the collected image data of floating objects in river waters are annotated with floating object data containing specific semantic categories, an annotation tool is used to annotate the collected image data of floating objects in river waters, select the floating objects by a rectangular frame and annotate the specific categories of the floating objects.

[0012] In some embodiments, in step S1, the semantic categories of floating objects in the river water area include: empty bottles, fallen leaves, plastic bags, and branches;

[0013] In order to collect images, a high-definition camera is installed on an unstable floating platform on water, and the high-definition camera has an anti-shake function.

[0014] In some embodiments, in step S2, the multi-scale attention mechanism network model uses the YOLOv5 model for target detection.

[0015] In some embodiments, the YOLOv5 model includes a YOLOv5s version model, and the YOLOv5s version model includes a feature extraction module, an attention mechanism module, and a detection module;

[0016] Among them, the feature extraction module is used to extract the features of the input image data to generate a feature map; the attention mechanism module can use the features extracted by the feature extraction module to highlight the monitored objects in the image; the detection module is used to detect the processing results of the attention module and obtain the information of floating objects in the water area in the input image data.

[0017] In some embodiments, the feature extraction module is used to extract features of input image data to generate a feature map, including:

[0018] Preprocess the image and adaptively scale the image to a uniform standard size;

[0019] Focus, CBL, CSP and SPP modules are used as the backbone network to extract initial features from the preprocessed image data, and higher-level features are extracted through continuous convolution, finally obtaining 4 feature maps.

[0020] In some embodiments, the process in which the attention mechanism module can highlight the monitored object in the image using the features extracted by the feature extraction module includes:

[0021] CBAM is used to enhance the expressiveness of features by strengthening the first feature and suppressing the second feature, wherein the importance of the first feature is higher than that of the second feature, thereby distinguishing normal areas from defective areas.

[0022] In some embodiments, the detection module is used to detect the processing result of the attention module, and the process of obtaining the information of floating objects in the water area in the input image data includes:

[0023] The FPN-PAN network structure is adopted. FPN fuses the deep semantic information with the underlying target position information through upsampling, and then integrates the PANT structure on this basis.

[0024] The method for rapid image recognition of an unstable floating platform on water according to an embodiment of the present invention has at least one of the following advantages:

[0025] 1. Efficient recognition capability: By integrating the YOLOv5 target detection algorithm, the image rapid recognition method achieves efficient and accurate recognition of surface garbage; the improved attention mechanism module significantly improves the image rapid recognition method's attention to key areas and reduces the interference of background noise, especially in dynamic water environments.

[0026] 2. Adapt to unstable water environment: The image rapid recognition method is designed to adapt to unstable water surface environment. Through anti-shake image acquisition and attention mechanism optimization, it can maintain stable detection accuracy under complex conditions such as water surface fluctuations and wind influence, ensuring the accuracy and reliability of the recognition results.

[0027] 3. Real-time processing and feedback: Through the modular image processing system, surface garbage can be detected and identified in real time and responded to immediately, which improves the efficiency of garbage cleaning and reduces delays and false detections.

[0028] 4. Lightweight and low-power design: Based on the efficient computing of the Jetson Nano platform and the lightweight optimization of the YOLOv5 algorithm, the image rapid recognition method has low power consumption while ensuring detection accuracy, extending the system's operating time, and is particularly suitable for long-term automatic operation of unmanned platforms; combined with multi-sensor fusion technology, it can realize real-time detection and processing of surface garbage, and can also monitor water quality at the same time.

[0029] 5. Strong scalability: The image rapid recognition method adopts a modular design, which can not only identify water surface garbage, but also be easily combined with other functional modules, such as water quality monitoring modules, to provide a comprehensive solution for water environmental protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] These and / or other aspects and advantages of the present invention will become apparent and readily understood from the following description of the preferred embodiments in conjunction with the accompanying drawings, in which:

[0031] Figure 1 It is a flow chart of a method for rapid image recognition of an unstable floating platform on water according to one embodiment of the present invention;

[0032] Figure 2A for Figure 1 A schematic diagram of the structure of the improved YOLOv5 network used in the method for rapid recognition of unstable floating platforms on water images;

[0033] Figure 2B for Figure 1 Schematic diagram of the Focus slicing operation used in target feature extraction in the image fast recognition method;

[0034] Figure 3 for Figure 1 A schematic diagram of the structure of FPN-PAN used in the target detection process of the image fast recognition method;

[0035] Figure 4 for Figure 1 Schematic diagram of the structure of CBAM, the attention mechanism of the image fast recognition method in ;

[0036] Figure 5 for Figure 4 Schematic diagram of the structure of the channel attention module CAM of CBAM;

[0037] Figure 6 for Figure 4 Schematic diagram of the structure of the spatial attention module SAM of CBAM;

[0038] Figure 7 for Figure 1 A schematic diagram of the structure of the PConv partial convolution used in the target feature extraction in the image fast recognition method;

[0039] Figure 8 for Figure 1 Schematic diagram of the NWD loss function used in the optimization of detection loss in the image fast recognition method. DETAILED DESCRIPTION

[0040] The technical solution of the present invention is further specifically described below by examples and in conjunction with the accompanying drawings. In the specification, the same or similar reference numerals indicate the same or similar components. The following description of the embodiments of the present invention with reference to the accompanying drawings is intended to explain the overall inventive concept of the present invention and should not be construed as a limitation of the present invention.

[0041] One of the main inventive concepts of the present invention is to combine the respective advantages of the image acquisition module, the attention mechanism module and the target detection module, so that the image rapid recognition method for unstable floating platforms on the water of the present invention can be applicable to garbage detection and cleaning tasks of unstable floating platforms on the water.

[0042] See also Figure 1 The main steps of the image rapid recognition method provided by the embodiment of the present invention are as follows:

[0043] S1 collects image data of at least one type (e.g., two types or each type) of floating objects in river waters, annotates the collected image data of floating objects in river waters with floating objects data in river waters containing specific semantic categories, and constructs the floating objects data in river waters containing specific semantic categories as a benchmark data set for floating object detection;

[0044] S2 builds a multi-scale attention mechanism network model suitable for floating object detection in river waters;

[0045] S3 trains the various types of river water floating object data collected in step S1 and the annotated specific semantic categories in the multi-scale attention mechanism network model established in step S2, and obtains the trained weights, the weights of convolution layer 1 [[[0.21, -0.32, 0.15], [0.45, -0.11, 0.33], [0.12, -0.44, 0.27]]], the weights of convolution layer 2 [[[0.55, -0.21, 0.34], [-0.13, 0.67, -0.19], [0.22, -0.36, 0.41]]], the channel attention weight (CAM) of the CBAM module [0.92, 0.73, 0.85, 0.67], and the spatial attention weight (SAM) of the CBAM module [[0.34, 0.56], [0.77, 0.81]];

[0046] S4 collects image data of floating objects in real river waters and inputs them into the multi-scale attention mechanism network model, and uses the weights obtained in step S3 to detect the semantic categories of floating objects in real river waters.

[0047] In step S1, at least one (e.g., two, three or more) types of river water platform are used to collect image data of, for example, one type, two types or various types of floating objects in waters, and the images cover, for example, multiple different types of floating objects in river waters in different time periods and / or different climates. Here, different time periods can be a certain time period of 8-9 o'clock in the morning during the day, such as 2-3 o'clock in the afternoon, or a certain time period in the evening or at night; similarly, different climates can also be various possible climatic conditions such as sunny, cloudy, light rain, heavy rain, etc.

[0048] It can be understood that, regarding the expression of at least one category or at least one type described herein, those skilled in the art can select their specific quantity according to actual needs, such as 1, 2, 3 or more.

[0049] In step S1, when the collected image data of various types of floating objects in river waters are annotated with floating object data containing specific semantic categories, an annotation tool is used to annotate the collected image data of various types of floating objects in river waters, select the floating objects by a rectangular frame and annotate the specific category of the floating objects.

[0050] In the actual step S1, the semantic categories of floating objects in the river water area include: empty bottles, fallen leaves, plastic bags, branches, etc.

[0051] Specifically, in step S1, the image acquisition process specifically includes:

[0052] In order to perform the image acquisition, corresponding equipment is configured: a high-definition camera is installed on the unstable floating platform on the water to capture the water surface image in real time. The camera has an anti-shake function, which can maintain the stability of the image when the unstable floating platform on the water shakes, ensuring that the system obtains a clear image.

[0053] Furthermore, refer to Figure 2A ,In steps (2) and (3), the YOLOv5 model is used for target detection, ,especially the YOLOv5s version, because of its excellent performance in ,detection speed and accuracy, which is suitable for the identification of ,water surface garbage.

[0054] In one embodiment, the YOLOv5s version model includes a feature extraction module, an attention mechanism module, and a detection module.

[0055] Among them, the feature extraction module is used to extract the features of the input image data to generate a feature map; the attention mechanism module can use the features extracted by the feature extraction module to highlight the monitored objects in the image; the detection module is used to detect the processing results of the attention module and obtain the information of floating objects in the water area in the input image data.

[0056] Specifically, the feature extraction module is used to extract features of input image data to generate a feature map, including:

[0057] The image is preprocessed and adaptively scaled to a standard size. Four modules, Focus, CBL, CSP and SPP, are used as the backbone network to extract initial features from the preprocessed image data, and higher-level features are extracted through continuous convolution to finally obtain four feature maps.

[0058] In some embodiments, the process in which the attention mechanism module can highlight the monitored object in the image using the features extracted by the feature extraction module includes:

[0059] CBAM enhances the expressiveness of features by strengthening the first feature (e.g., important feature) and suppressing the second feature (e.g., less important feature, that is, feature that is less important or more important than the first feature). It can easily distinguish between normal areas and defective areas, especially when the defects are not obvious or there is a lot of background noise, thereby improving the accuracy of detection and highlighting the detection object in the image.

[0060] In some embodiments, the detection module is used to detect the processing result of the attention module, and the process of obtaining the information of floating objects in the water area in the input image data includes:

[0061] The FPN-PAN network structure is adopted. FPN fuses deep semantic information with underlying target position information through upsampling to enhance the network's feature learning ability. On this basis, the PANT structure is integrated to generate a feature pyramid to enhance the network model's detection ability for objects of different scales, realize the recognition of different sizes and scales of the same object, and finally obtain the detection result frame.

[0062] The YOLOv5 model consists of an input end, a skeleton network, a neck network, and a detection head. The input end scales and enhances the image, and the skeleton network improves the feature extraction capability through multiple modules. The neck network combines the FPN and PAN structures to detect targets of different sizes. The detection head is responsible for processing the features extracted by the skeleton network and the neck network to generate the final detection results. The output layer of the detection head generates feature vectors and bounding boxes through downsampling. The improved YOLOv5s model improves the detection accuracy of surface garbage and ensures real-time performance and efficiency.

[0063] In a preferred embodiment of the present invention, the image processing process includes basic filtering, denoising and image enhancement operations to provide high-quality input data for subsequent recognition processes. During each training process, the algorithm adaptively calculates the value of the optimal anchor frame in different training sets. This method uses the Mosaic data enhancement method. During the training process, 4 pictures are randomly selected for rotation and scaling, and randomly cropped and spliced ​​to form new training data. This method enriches the data set, enhances the robustness of the network, performs better in small target detection, and calculates 4 picture data at the same time, which improves the model training speed and detection capability.

[0064] In the image rapid recognition method described in the present invention, an image acquisition module, an attention mechanism module and a detection module are mainly included or adopted. The water surface image is collected in real time by a high-definition camera, and the attention mechanism module is used to increase the attention to important areas and reduce the interference of environmental noise. Combined with the YOLOv5 target detection algorithm, efficient and accurate water surface garbage recognition can be achieved in unstable water environments. This technology optimizes the detection effect through an improved attention mechanism, especially in complex backgrounds or dynamic environments, and can still output results stably, significantly improving the efficiency and accuracy of water garbage cleaning.

[0065] In addition, refer to Figure 2B The target feature extraction process consists of four modules, namely Focus, CBL, CSP and SPP, which are used to extract target features in the image. The CSP module draws on the CrossStage Partial Network (CSPNet) to solve the problem of gradient information duplication in network optimization during reasoning from the perspective of network structure design, which enhances the network's learning ability and operating cost. The Focus structure is added to implement the slicing operation. Before the image enters the backbone network, it is downsampled by 2 times to obtain 4 feature maps. The W and H information of the image are concentrated in the spatial channel. This process does not cause the loss of image feature information, and can enable the network to extract more sufficient feature information. In YOLOv5s, the original 608×608×3 image is input into the Focus structure, and a 304×304×12 feature map is formed through the slicing operation. Then, 32 convolution kernels are convolved to finally construct a 320×320×32 feature map.

[0066] Reference Figure 3 ,The process of target detection is as follows: adopting the FPN-PAN network structure, FPN fuses deep semantic information with underlying target position information through upsampling, enhancing the network's feature learning ability, and on this basis, fusion of the PANT structure, conveying strong positioning features from bottom to top, and comprehensively improving the network's learning performance of semantic information and positioning information. The three detection heads of the output layer (Head) of the FPN-PAN structure downsample the original image by 8 times, 16 times, and 32 times respectively, and generate three feature vectors of different sizes to predict image features, generate bounding boxes, and predict the target category and confidence in the image.

[0067] Reference Figure 4, the attention mechanism consists of the following parts: CBAM can help the model focus on these key areas. It enhances the expressiveness of features by strengthening important features and suppressing less important features. It can easily distinguish normal areas from defective areas, especially when the defects are not obvious or there is a lot of background noise, thereby improving the accuracy of detection. This module captures global statistical information and extreme features through global average pooling and global maximum pooling, respectively, and then processes these global averages through two fully connected layers to obtain a vector representing the channel weight. Finally, this vector is normalized by the Sigmoid activation function to obtain the weight coefficient of each channel.

[0068] Reference Figure 5 and Figure 6 CBAM consists of two independent submodules, namely the Channel Attention Module (CAM) and the Spatial Attention Module (SAM), which perform channel and spatial attention respectively. This not only saves parameters and computing power, but also ensures that it can be integrated into the existing network architecture as a plug-and-play module.

[0069] Reference Figure 7 , the lightweight design of PConv (partial convolution) helps reduce computational complexity and FLOPs. This lightweight operation improves overall efficiency by reducing unnecessary calculations, especially when using CBAM, ensuring that the model can reduce computing resource consumption while maintaining detection accuracy. The lightweight process is as follows: Figure 7 As shown in (a), it is assumed that the dimensions of the input feature map are h height, w width, c number of channels and k convolution kernel size. In this case, the number of floating point operations per second (FLOPs) can be calculated by the following formula to quantify the computational complexity of the convolution operation. The formula is:

[0070] FLOPs = h × w × k 2 ×c 2

[0071] T-shaped convolution and PConv (partial convolution) are two common convolution structures, each with different advantages and characteristics. Figure 7As shown in (b) in the figure, although T-shaped convolution can achieve efficient operation when performing feature screening, it consumes more computation and memory access than PConv, resulting in computational redundancy and low time efficiency. In order to quantify the performance difference of these convolution structures, the FLOPs calculations of the two are respectively calculated by the following two formulas.

[0072] The calculation formula for FLOPs through T-shaped convolution is:

[0073] FLOPs = h × w × (k 2 ×c p ×c+c×(cc p ))

[0074] The calculation formula for FLOPs by PConv is:

[0075]

[0076] where c p It is the first or last continuous channel number of the input feature map, which represents the number of channels involved in performing the convolution operation. These two formulas show the difference in computational complexity between T-convolution and PConv. PConv can reduce FLOPs by reducing the convolution operation of the channel, thereby achieving lightweight model.

[0077] Reference Figure 8 , NWD loss function optimization improves the model's detection accuracy for small objects, enhances the spatial accuracy of the bounding box, better handles overlapping objects, ensures the stability of the training process, and thus improves the detection effect and robustness of the entire method. The process includes: the present invention combines the NWD loss function, which improves the model's detection ability for small objects, improves spatial accuracy, better handles overlapping objects, and provides a more stable and balanced training process by modeling the bounding box as a Gaussian distribution and using the Wasserstein distance to more accurately measure the difference between the predicted and true bounding boxes, and finally calculates the loss through normalization.

[0078] The method for rapid image recognition of an unstable floating platform on water according to an embodiment of the present invention has at least one of the following advantages:

[0079] 1. Efficient recognition capability: By integrating the YOLOv5 target detection algorithm, the image rapid recognition method achieves efficient and accurate recognition of surface garbage; the improved attention mechanism module significantly improves the image rapid recognition method's attention to key areas and reduces the interference of background noise, especially in dynamic water environments.

[0080] 2. Adapt to unstable water environment: The image rapid recognition method is designed to adapt to unstable water surface environment. Through anti-shake image acquisition and attention mechanism optimization, it can maintain stable detection accuracy under complex conditions such as water surface fluctuations and wind influence, ensuring the accuracy and reliability of the recognition results.

[0081] 3. Real-time processing and feedback: Through the modular image processing system, surface garbage can be detected and identified in real time and responded to immediately, which improves the efficiency of garbage cleaning and reduces delays and false detections.

[0082] 4. Lightweight and low-power design: Based on the efficient computing of the Jetson Nano platform and the lightweight optimization of the YOLOv5 algorithm, the image rapid recognition method has low power consumption while ensuring detection accuracy, extending the system's operating time, and is particularly suitable for long-term automatic operations on unmanned platforms.

[0083] 5. Strong scalability: The image rapid recognition method adopts a modular design, which can not only identify water surface garbage, but also be easily combined with other functional modules, such as water quality monitoring modules, to provide a comprehensive solution for water environmental protection.

[0084] Although some embodiments of the present general inventive concept have been shown and described, it will be appreciated by those skilled in the art that changes may be made to these embodiments without departing from the principles and spirit of the present general inventive concept, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for rapid image recognition of an unstable floating platform on water, characterized in that: The image rapid recognition method comprises the following steps: S1 collects image data of at least one type of floating objects in river waters, annotates the collected image data of floating objects in river waters with floating objects data in river waters containing specific semantic categories, and constructs the floating objects data in river waters containing specific semantic categories as a benchmark data set for floating object detection; S2 builds a multi-scale attention mechanism network model suitable for floating object detection in river waters; S3 trains the river water floating object data collected in step S1 and the annotated specific semantic categories in the multi-scale attention mechanism network model established in step S2 to obtain trained weights; S4 collects image data of floating objects in real river waters and inputs them into the multi-scale attention mechanism network model, and uses the weights obtained in step S3 to detect the semantic categories of floating objects in real river waters.

2. The method for rapid image recognition of an unstable floating platform on water according to claim 1, characterized in that: In step S1, at least one type of river water area platform is used to collect image data of floating objects in the water area, and the images cover floating objects in the river water area in different time periods and / or different climates.

3. The method for rapid image recognition of an unstable floating platform on water according to claim 2, characterized in that: In step S1, when the collected image data of floating objects in river waters are annotated with floating object data containing specific semantic categories, an annotation tool is used to annotate the collected image data of floating objects in river waters, select floating objects by rectangular frames and annotate the specific categories of floating objects.

4. The method for rapid image recognition of an unstable floating platform on water according to claim 3, characterized in that: In step S1, the semantic categories of floating objects in the river waters include: empty bottles, fallen leaves, plastic bags, and branches; In order to collect images, a high-definition camera is installed on an unstable floating platform on water, and the high-definition camera has an anti-shake function.

5. The method for rapid image recognition of an unstable floating platform on water according to any one of claims 1 to 4, characterized in that: In step S2, the multi-scale attention mechanism network model uses the YOLOv5 model for target detection.

6. The method for rapid image recognition of an unstable floating platform on water according to claim 5, characterized in that: The YOLOv5 model includes a YOLOv5s version model, and the YOLOv5s version model includes a feature extraction module, an attention mechanism module, and a detection module; Among them, the feature extraction module is used to extract the features of the input image data to generate a feature map; the attention mechanism module can use the features extracted by the feature extraction module to highlight the monitored objects in the image; the detection module is used to detect the processing results of the attention module and obtain the information of floating objects in the water area in the input image data.

7. The method for rapid image recognition of an unstable floating platform on water according to claim 6, characterized in that: The feature extraction module is used to extract features of input image data to generate a feature map, and the process includes: Preprocess the image and adaptively scale the image to a uniform standard size; Focus, CBL, CSP and SPP modules are used as the backbone network to extract initial features from the preprocessed image data, and higher-level features are extracted through continuous convolution, finally obtaining 4 feature maps.

8. The method for rapid image recognition of an unstable floating platform on water according to claim 7, characterized in that: The process in which the attention mechanism module can highlight the monitored object in the image using the features extracted by the feature extraction module includes: CBAM is used to enhance the expressiveness of features by strengthening the first feature and suppressing the second feature, wherein the importance of the first feature is higher than that of the second feature, thereby distinguishing normal areas from defective areas.

9. The method for rapid image recognition of an unstable floating platform on water according to claim 8, characterized in that: The detection module is used to detect the processing results of the attention module. The process of obtaining the information of floating objects in the water area in the input image data includes: The FPN-PAN network structure is adopted. FPN fuses the deep semantic information with the underlying target position information through upsampling, and then integrates the PANT structure on this basis.

Images