Fine detection method for floating objects in riverway

Abstract

The invention relates to a riverway floating object refined detection method, which comprises the following steps of: firstly, segmenting a riverway water body by adopting a DeepLabv3 + model semantic segmentation method, removing background interference around the riverway, only paying attention to the foreground of the riverway water body, and then detecting floating objects on a segmented riverway water body image by adopting a detic object detection method. According to the method, the categories of the river floaters are expanded from dozens of categories to tens of thousands of categories by designing the user-defined vocabulary, so that the categories of the floaters are finely distinguished, then the categories of the floaters concerned are configured through the black list and the white list, the capability of finely detecting the river floaters is improved, and the fine detection of the river floaters is realized.

Description

Technical Field

[0001] The present disclosure relates to the technical field of computer vision and image processing, which belongs to the field of computer science and technology, and further relates to a method, system, electronic device and readable storage medium for refined detection of floating objects in river channels. Background Art

[0002] Water resources are a critical component of the ecological environment. Over 71% of the Earth's surface is covered by oceans, yet freshwater resources account for only 2.5% of the total water volume, and less than 1% is directly available for human consumption. China has one of the largest number of rivers in the world, and these rivers are crucial to the country's economic and social development. However, with the advancement of industrialization and urbanization, waste enters water bodies through various pathways, leading to a significant increase in floating debris in rivers. This not only poses a serious threat to water bodies and the ecological environment, but also affects people's lives and health.

[0003] There are many types of floating debris in rivers, including plants, animals, domestic garbage, and industrial waste, and each category requires different treatment methods. Plant floating debris, such as aquatic plants and branches, is usually treated through biodegradation, using microorganisms to accelerate decomposition; animal floating debris, such as aquatic animal carcasses, requires physical treatment to remove moisture and reduce the impact on water quality; domestic garbage, including plastic bags and bottles, is often managed through interception, salvage, and physical treatment; and industrial waste, such as waste oil and chemical reagents, requires chemical treatment methods to convert toxic substances into low-toxic or non-toxic substances. Correctly identifying the type of floating debris is crucial to selecting the appropriate treatment method, which not only improves treatment efficiency but also effectively protects water bodies and the ecological environment.

[0004] With advances in computer vision and deep learning technologies, researchers have begun using deep learning algorithms to detect floating debris in rivers. While existing methods have achieved some success in detecting floating debris in rivers, several challenges remain. For example, when extracting water areas from images, the diverse structure of river channels and the large variations in surface shape can interfere with image segmentation and feature extraction due to reflections and shadows. Furthermore, the types of floating debris are influenced by environmental factors, human activity, and climate. In urban areas, common floating debris include plastic bags, bottles, and paper; in industrial areas, chemical residues and industrial waste may occur; and in rural areas, agricultural supplies and pesticide bottles may occur. Dynamic background noise in real river environments, such as ripples in the water, flickering lights and shadows, and swaying leaves, can easily be misidentified as floating debris by algorithms. Furthermore, due to the large variety and complex shapes of floating debris, and the difficulty in obtaining samples, traditional algorithms require a large number of labeled samples to detect multiple types of floating debris, which is costly and limits the number of detection categories. Therefore, addressing these issues and enabling the detection of various floating debris categories for refined river floating debris detection has become a pressing technical challenge. Summary of the Invention

[0005] In response to the above technical problems, the present disclosure provides a refined method for detecting floating objects in rivers. First, the semantic segmentation method of the DeepLabv3+ model is used to segment the river water body, remove the background interference around the river channel, and only focus on the foreground of the river water body. Then, the detic all-things detection method is used on the segmented river water body image to detect floating objects. By designing a custom vocabulary, the categories of river floating objects are expanded from dozens to tens of thousands, and the detection categories are expanded, thereby finely distinguishing the categories of floating objects. Then, the categories of floating objects of concern are configured through black and white lists, which improves the ability of refined detection of river floating objects, significantly improves the robustness and generalization ability of the algorithm, and realizes refined detection of river floating objects.

[0006] Specifically, the present disclosure provides a refined river floating object detection method, system, electronic device and storage medium.

[0007] According to one aspect of the present disclosure, a refined method for detecting floating objects in a river channel is provided, comprising the following steps:

[0008] (1) River data acquisition: using image acquisition devices to collect river monitoring image data;

[0009] (2) First-level detection: river segmentation is used to separate the river water from the background, generating a mask image representing the river water area. The mask image of the river water area is superimposed on the original river monitoring image, removing all background areas outside the river water, and retaining only the river water part to form a river water image;

[0010] (3) The second level detection is to detect floating objects in the river using the detic object detection model on the segmented river water image;

[0011] (4) Result mapping, mapping the output of detic all things detection back to the original river monitoring image;

[0012] Among them, the categories of floating objects in rivers are expanded through custom vocabulary, so as to finely distinguish the categories of floating objects; the floating object categories of concern are configured through whitelist and blacklist, the floating object targets in rivers are filtered, and the floating objects in rivers are finely detected.

[0013] Preferably, the river channel segmentation uses the DeepLabv3+ model to segment the river channel image to obtain a segmentation result rich in spatial details.

[0014] Preferably, the detic all things detection alternately uses sample sets of river target detection and sample sets of image classification to train the network model.

[0015] Preferably, the river channel image segmentation is based on a deep convolutional neural network architecture, which automatically learns the semantic features in the river channel image through multi-layer convolution operations, and extracts low-level features such as edges and textures and high-level features such as object categories; the high-level semantic features have strong semantic information, and the low-level features contain rich spatial detail information; then the encoder-decoder structure is used to fuse the high-level semantic feature information and the low-level feature information, so that the segmentation result of the river channel image can better capture the detail information in the image while maintaining semantic accuracy, thereby obtaining a segmentation result rich in spatial details.

[0016] Preferably, in the encoder stage, a deep convolutional neural network is first used to process the input image through the backbone network Resnet101 to obtain a preliminary effective feature layer, thereby realizing the extraction of low-level features; then, the hole convolution pyramid module ASPP is used to enhance the feature extraction capability; the ASPP module is composed of multiple operations including 1x1 convolution, hole convolution, and maximum pooling layer, which effectively captures semantic information at different scales, thereby realizing the extraction of high-level features; the high-level features extracted by the ASPP module are fused with the low-level features extracted by the backbone network to obtain a feature map compatible with the low-level and high-level semantic information of the image; the fused features are further extracted by 3x3 convolution, and the feature map is restored to the original image size by four times upsampling; the expansion rate of the hole convolution is 6, 12, and 18.

[0017] Preferably, in the decoder stage, first, the feature maps output by the encoder are upsampled to four times their original size; secondly, these upsampled feature maps are element-wise added to the feature maps output by the adjusted backbone feature extraction network, and the adjustment process is to match the number of channels of the feature maps through a 1×1 convolution operation to achieve effective feature fusion; finally, the final output result is obtained by performing bilinear interpolation upsampling on the upsampled feature maps and the fused feature maps to generate a high-resolution segmentation feature map.

[0018] Preferably, the detic all things detection first trains the model using the detected sample set, then freezes the detection head, and continues to train the image classification samples using the trained weights;

[0019] The loss function is:

[0020] L max-size =BCE(Wf j ,c),j=arg max j (size(b j )) (1)

[0021]

[0022] Among them, BCE is the binary cross entropy loss function, j is the index that maximizes the bounding box size, and b j is the bounding box, argmax j is the index of the largest bounding box, size is the size of the bounding box, I is the input image, and D det Refers to the detection dataset, D cls Refers to the classification dataset, L rpn Propose network loss for region, L reg is the bounding box regression loss, L cls is the target classification loss, L max-size is the maximum size loss, f j Represents the region of interest feature ROIfeature corresponding to the detected prediction box, c is the category corresponding to the largest prediction box, that is, the category label of the entire image, and W is the weight of the classifier; the loss function L shown in formula (1) max-size The loss function L used in joint traditional target detection rpn +L reg +L cls , which constitutes the final loss function L(I) of detic, where λ is the loss weight, which is set to 0.1 in this disclosure.

[0023] Preferably, (1) the river data acquisition unit acquires river monitoring image data through an image acquisition device;

[0024] (2) The first-level detection unit uses river segmentation to separate the river water body from the background and generate a mask image representing the river water area. The mask image of the river water area is superimposed on the original river monitoring image to remove all background areas outside the river water body and retain only the river water part to form a river water image;

[0025] (3) The second-level detection unit uses the detic object detection model to detect floating objects in the river on the segmented river water image;

[0026] (4) Result mapping unit, which maps the output of detic all things detection back to the original river monitoring image;

[0027] Among them, the categories of floating objects in rivers are expanded through custom vocabulary, so as to finely distinguish the categories of floating objects; the floating object categories of concern are configured through whitelist and blacklist, the floating object targets in rivers are filtered, and the floating objects in rivers are finely detected.

[0028] According to another aspect of the present disclosure, the present disclosure provides an electronic device, including:

[0029] a memory storing execution instructions; and

[0030] A processor executes the execution instructions stored in the memory, so that the processor performs the above method.

[0031] According to another aspect of the present disclosure, the present disclosure provides a readable storage medium, wherein the readable storage medium stores execution instructions, and the execution instructions are used to implement the above method when executed by a processor.

[0032] Beneficial effects of the present disclosure:

[0033] 1. This disclosure uses a deep convolutional neural network architecture to segment river images. Through multi-layer convolution operations, it automatically learns semantic features in river images, extracting both low-level features such as edges and textures and high-level features such as object categories. Simultaneously, an encoder-decoder architecture is employed to fuse high-level semantic and low-level feature information. This ensures that the river image segmentation results retain semantic accuracy while better capturing detailed information within the image, resulting in object category segmentation results rich in spatial detail.

[0034] 2. It effectively solves the problem of interference to image segmentation and feature extraction caused by factors such as diverse river structures, large changes in light environment, and a wide variety of floating objects in complex river monitoring.

[0035] 3. In response to the complex problems in the real river environment, by setting a custom vocabulary, the detection categories are made more accurate, further improving the accuracy of detecting floating objects in the river.

[0036] 4. By setting up black and white lists, the detected objects can be distinguished, thereby achieving rapid classification of river detection results.

[0037] 5. A river segmentation model based on the DeepLabv3+ model is used as the first-level detector, and a detic all-object detection model is used as the second-level detector. This cascade approach leverages the strengths of both models, enhancing adaptability to complex backgrounds and changing lighting. By applying object detection to the segmentation results, floating objects can be more accurately located, reducing false and missed detections, and achieving refined detection of floating objects in rivers. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of this specification. The above and other objects, features, and advantages of the present disclosure will become more apparent through a detailed description of the embodiments of the present disclosure in conjunction with the accompanying drawings.

[0039] Figure 1Shows the DeepLabv3+ model network structure diagram

[0040] Figure 2 Shows the comparison results before and after the vocabulary is added

[0041] Figure 3 Shows the refined detection process of river floating objects

[0042] Figure 4 Shows an example of a river sample set

[0043] Figure 5 shows the AP calculation

[0044] Figure 6 Shows the training loss graph

[0045] Figure 7 The Mask-PR curve is shown

[0046] Figure 8 Shows the comparison of different vocabulary detection output results

[0047] Figure 9 Shows the detic detection results

[0048] Figure 10 The results of each stage are shown in the figure DETAILED DESCRIPTION

[0049] The present disclosure will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to explain the relevant content and are not intended to limit the present disclosure. It should also be noted that, for ease of description, only the portions relevant to the present disclosure are shown in the accompanying drawings.

[0050] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0051] Unless otherwise stated, the exemplary embodiments / examples shown are to be understood as providing exemplary features of various details of some ways in which the technical concepts of the present disclosure can be implemented in practice. Therefore, unless otherwise stated, the features of the various embodiments / examples may be further combined, separated, interchanged, and / or rearranged without departing from the technical concepts of the present disclosure.

[0052] The terms used in this disclosure are for the purpose of describing specific embodiments and are not intended to be restrictive. As used herein, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, the features, integral bodies, steps, operations, parts, assemblies and / or their groups stated are indicated, but one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups are not excluded from existence or addition. It should also be noted that, as used herein, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, and as such, they are used to explain the inherent deviations of the measured values, calculated values and / or values provided that will be recognized by those of ordinary skill in the art.

[0053] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present application. In this specification, the schematic representations of the above terms are not necessarily the same embodiment / method or example. Moreover, the specific features, structures, materials or characteristics described may be combined in a suitable manner in any one or more embodiments / methods or examples. In addition, those skilled in the art may combine and combine the different embodiments / methods or examples described in this specification and the features of the different embodiments / methods or examples, unless they are mutually inconsistent.

[0054] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0055] The present invention discloses a method for fine-grained detection of floating objects in rivers.

[0056] Due to the complex and changeable conditions of real river monitoring scenes, in the process of extracting water body areas in images, first, the river structure and shape are diverse. Secondly, the flowing river water is affected by the environment, and the water surface shape varies greatly. Thirdly, it is affected by seasonal, temperature, and weather changes. Even the image of the same river section will appear different at different times. Reflections and shadows also interfere with image segmentation and feature extraction. In addition, the types of floating objects are also affected by factors such as the environment, human activities, and climate: in urban areas, floating objects are mostly plastic bags, bottles, and paper; in industrial areas, there may be chemical residues and industrial waste; in rural areas, they may include agricultural products and pesticide bottles. Therefore, in order to quickly and effectively detect floating objects in rivers, and prevent false detections caused by dynamic background noise in the environment, such as water fluctuations, flickering light and shadows, and swaying leaves, as well as to overcome the disadvantage of traditional methods that require a large number of sample annotations when detecting multiple floating objects, this disclosure proposes a two-stage detector cascade for refined detection of floating objects in rivers for this complex scene.

[0057] This method uses the river segmentation model based on the DeepLabv3+ model as the first-level detector and the detic all-things detection model as the second-level detector to perform refined detection of floating objects in the river.

[0058] The detection flow chart is as follows Figure 1 As shown in the figure. In the figure, in the first level of detection, the river segmentation model accurately separates the river water body and the background, and generates a mask map representing the river water body area (i.e., the river water body segmentation mask map). The mask map of the river water body area is superimposed with the original river monitoring image to remove all background areas outside the river water body, retaining only the river water body part to form a river water body image. In the second level of detection, the second-level detector detic all things detection model detects floating objects in the processed river water body image. By designing a custom vocabulary, the categories of river floating objects are expanded from dozens to tens of thousands, thereby finely distinguishing the categories of floating objects. Then, the whitelist and blacklist mechanisms are applied to filter the detected candidate floating object targets, and non-floating object targets are excluded. Targets that meet the characteristics of floating objects are formally determined to be floating objects, which improves the ability to fine-tune the detection of river floating objects. Subsequently, the final result output by the detic detection is mapped back to the original image. At this point, the entire floating object detection process is completed. The specific detection process is as follows:

[0059] 1. River segmentation

[0060] To achieve refined river floating object detection, this paper uses the DeepLabv3+ model for river image segmentation to address the issues of diverse river channel structures and large variations in water surface shape, as well as the interference of reflections and shadows on image segmentation and feature extraction. The DeepLabv3+ model is a semantic segmentation method using an encoder-decoder network structure.

[0061] Among them, in the encoder stage, the input image is first processed by the deep convolutional neural network through the backbone network Resnet101 to obtain a preliminary effective feature layer, thereby realizing the extraction of low-level features. Subsequently, the atrous convolution pyramid module ASPP (Atrous Spatial Pyramid Pooling) is used to enhance the feature extraction capability. The ASPP module consists of multiple operations including 1x1 convolution, atrous convolution (expansion rates of 6, 12, 18), and maximum pooling layer, which can effectively capture semantic information at different scales, thereby realizing the extraction of high-level features. The high-level features extracted by the ASPP module are fused with the low-level features extracted by the backbone network to obtain a feature map that is compatible with the low-level and high-level semantic information of the image. After the fused features are further extracted by 3x3 convolution, the feature map is restored to the original image size by four times upsampling.

[0062] In the decoder stage, the main function of the decoder is to solve the problem of feature information loss caused by multiple downsampling, so as to better restore edge information. First, the feature map output by the encoder is upsampled to four times its original size. Secondly, these upsampled feature maps are added element by element with the feature maps output by the adjusted backbone feature extraction network. The adjustment process is to match the number of channels of the feature map through a 1×1 convolution operation to achieve effective feature fusion. Finally, the final output result is obtained by performing bilinear interpolation upsampling on the upsampled feature map and the fused feature map to generate high-resolution segmentation feature maps, which are used for the final pixel-level segmentation task. The network structure diagram of the DeepLabv3+ model disclosed in this disclosure is shown in the figure. Figure 2 shown.

[0063] It can be seen that the river image segmentation is based on a deep convolutional neural network architecture, which automatically learns the semantic features in the river image through multi-layer convolution operations, and extracts both low-level features such as edges and textures and high-level features such as object categories; among them, the high-level semantic features have strong semantic information, while the low-level features contain rich spatial detail information.

[0064] Then, the encoder-decoder structure is used to fuse high-level semantic feature information and low-level feature information, so that the segmentation results of the river image can maintain semantic accuracy while better capturing the detailed information in the image, thereby obtaining a segmentation result rich in spatial details; it effectively solves the problem of interference in image segmentation and feature extraction caused by factors such as diverse river structures, large changes in lighting environment, and a wide variety of floating objects in complex river monitoring.

[0065] 2. Detic detection of all things

[0066] This paper uses the detic object detection method for river channel object detection, alternating between river channel object detection and image classification sample sets to train the network model. First, the model is trained using the detection sample set, then the detection head is frozen and the trained weights are used to train the image classification samples.

[0067] The loss function used in this disclosure is:

[0068] L max-size =BCE(Wf j ,c),j=argmax j (size(b j )) (1)

[0069]

[0070] For complex scenarios such as river monitoring, in order to use image-level supervision signals, the loss function L shown in formula (1) is adopted max-size , where BCE is the binary cross entropy loss function, j is the index that maximizes the bounding box size, and b j is the bounding box, argmax j is the index of the largest bounding box, size is the size of the bounding box, I is the input image, and D det Refers to the detection dataset, D cls Refers to the classification dataset, L rpn Propose network loss for region, L reg is the bounding box regression loss, L cls is the target classification loss, L max-size is the maximum size loss, f j Represents the region of interest feature ROIfeature corresponding to the detected prediction box, c is the category corresponding to the largest prediction box, that is, the category label of the entire image, and W is the weight of the classifier; the loss function L shown in formula (1) max-size The loss function L used in joint traditional target detection rpn +L reg +L cls , which constitutes the final loss function L(I) of detic, where λ is the loss weight, set to 0.1 in this disclosure. The largest prediction box in the entire image is selected and the class label of the image is assigned to it. This eliminates the label and bounding box assignment errors that may occur in traditional methods. This simplifies the training process and helps improve object detection performance, effectively detecting various targets in complex river scenes.

[0071] 3. Custom vocabulary

[0072] In order to solve the problem that the types of floating objects in river monitoring are complex and changeable due to factors such as environment, human activities and climate, and are difficult to detect, this paper sets a specific custom vocabulary to make the detection categories more accurate.

[0073] In river monitoring scenarios, without a custom vocabulary, common floating objects might be detected as general categories, such as box, bottle, and ball. However, with a custom vocabulary, floating object categories can be more refined, such as plastic bottle, plastic bag, carton, plastic box, polystyrene box, and balloon.

[0074] like Figure 3 As shown in the figure, the left figure shows the detection results of detic without setting a custom vocabulary. The floating foam box was missed, and the detected target categories were relatively broad, with various box-like floating objects detected as boxes. The right figure shows the detection results with a custom vocabulary. In this case, the foam box was accurately detected, and the box-like floating objects were distinguished into plastic boxes and polystyrene boxes, rather than all being detected as boxes. This shows that setting a custom vocabulary increases the refinement of river floating object detection and improves the accuracy of river floating object detection.

[0075] 4. Set up blacklist and whitelist

[0076] The present disclosure sets up a black and white list to determine whether the selected floating objects belong to river floating objects. The blacklist includes those targets that are determined to be river floating objects, such as garbage bags, plastic bottles, cans, etc. These targets are regarded as floating objects that need to be processed in the river scene. The whitelist contains targets that are determined to be non-river floating objects, such as boats, people, reflections of trees, the sky, etc. These objects should be filtered out in the detection of river floating objects. In actual applications, the blacklist of the river scene contains more than 20,000 types of floating objects, such as plastic bottles, paper cups, snack bags and foam boards, while the whitelist includes people, boats, fish and trees. The whitelist is shown in Table 1 below. The blacklist table is established corresponding to it, but there are more types.

[0077] Table 1 Whitelist

[0078]

[0079]

[0080] In actual river monitoring, the present disclosure distinguishes detected objects by setting up black and white lists, thereby achieving rapid classification of river detection results.

[0081] 5. Experimental data and experimental platform

[0082] 5.1 Experimental Data

[0083] Due to the scarcity of public sample sets of rivers, this paper uses data collected from a river near a technology industrial park through simulated camera angles as the experimental dataset. In terms of weather and lighting, the dataset covers river images under various weather conditions, including rainy, sunny, cloudy, overcast, and foggy weather. In order to fully consider different lighting conditions, the dataset also includes images taken in the morning, noon, evening, and dusk, covering different lighting environments such as strong light, soft light, and shadows. In terms of river perspective, the dataset includes images from multiple perspectives, such as top view, side view, and oblique view, ensuring a comprehensive observation of the river. Images from different perspectives can capture different characteristics of the river, such as water surface reflections, bank details, and the distribution of floating objects in the river. In terms of river width, the dataset covers rivers of different widths, from narrow streams to wide rivers. These changes help the model have better adaptability and accuracy when dealing with rivers of different sizes.

[0084] The river data set collected includes 18,000 images in the training set and 2,000 images in the validation set. The river sample set includes some common floating objects, such as plastic bottles, plastic boxes, foam boxes, cardboard boxes, balloons, and plastic bags. Some examples of river sample sets are as follows Figure 4 As shown:

[0085] 5.2 Experimental Platform

[0086] The experimental platform was built on the Ubuntu 22.04 operating system and based on the PyTorch deep learning framework. The hardware and software configurations of the experiment are shown in Table 2.

[0087] Table 2 Experimental platform configuration

[0088]

[0089]

[0090] 5.3 Experimental evaluation indicators

[0091] Precision, recall, and AP50 (average precision @ IOU = 0.5) are used to evaluate the performance of the model.

[0092] Precision: It indicates the proportion of true positive examples among all samples predicted to be positive examples.

[0093] Recall: It indicates the proportion of all actual positive examples that are correctly predicted as positive examples.

[0094] Taking river segmentation as an example, TP is the number of river areas correctly predicted as river areas, FP is the number of background areas incorrectly predicted as river areas, and FN is the number of river areas incorrectly predicted as background areas. The following formula is used:

[0095]

[0096] Among them, P is the precision rate (Precision), R is the recall rate (Recall) Average Precision (AP, AveragePrecision): obtained by calculating the area enclosed by the PR curve (Precision-Recall Curve) and the horizontal and vertical coordinates, such as Figure 5 As shown in the PR curve, recall (R) is plotted on the horizontal axis, and precision (P) is plotted on the vertical axis. The AP value represents the area under the PR curve. A larger AP value indicates that the model performs better in both precision and recall, and is able to maintain high precision and recall simultaneously.

[0097] AP50 is the average precision when the Intersection over Union (IOU) is set to 0.5. The IOU value measures the overlap between the predicted box and the ground-truth box. AP50 describes the performance of the model at a specific IOU threshold. A higher AP50 value indicates better detection at that IOU threshold and stronger overall model performance.

[0098] 6. Analysis of experimental results

[0099] 6.1. DeepLabv3+ model segmentation results

[0100] The experimental parameters of the DeepLabv3+ model in the river floating object fine detection method disclosed in this disclosure are set as follows: the training iteration round epoch is 64, the batch size batch_size is set to 200, the image input size is 640*640, and the initial learning rate lr is 0.01. The model training collects the labeled river segmentation samples as follows Figure 6 、 Figure 7 As shown in the figure, the loss is finally in a converged state and AP50 reaches 0.995. Figure 10In the figure, sub-images (a) to (d) show sample river images under different scenes and lighting conditions. These images include reflections of the river, buildings, trees, sky, and clouds. The complex shoreline and significant color variation of the water body can affect the segmentation effect. Sub-images (e) to (h) in the same section show the results of river segmentation. These images demonstrate that the river segmentation model performs well and has strong anti-interference capabilities. It can accurately segment the river outline in complex scenes, maintaining high accuracy even in the presence of interference factors such as reflections and lighting changes.

[0101] 6.2, detic test results

[0102] The detic all things detection in the river channel floating object refined detection method is used as a secondary detector to detect the processed river water image. By setting the category of floating objects through a custom vocabulary and a black and white list, refined detection can be achieved.

[0103] Figure 8 Figure 1 compares the detection output results of different vocabularies. Figures (a1) to (a3) show the detection results of a model trained using the COCO sample set, a multi-task computer vision benchmark dataset. Figures (b1) to (b3) show the detection results of a model trained using the LVIS sample set, a large-scale vocabulary instance segmentation dataset. Figures (c1) to (c3) show the detection results of a custom vocabulary added to the two vocabularies specifically for the river floating object detection task. The custom vocabulary includes words like plastic_bottle, plastic_bag, carton, boat, plastic_box, and polystyrene_box.

[0104] observe Figure 8Figures (a1) to (a3) and (b1) to (b3) show varying degrees of missed detection and false detection. In figure (a1), all floating objects are not detected. In figure (b1), plastic_box and polystyrene_box are missed. In figures (a2) and (b2), the model mistakenly detects box as boat. In figures (a3) and (b3), the model mistakenly detects ballon as ball. In contrast, in figures (c1) to (c3), all objects are detected, and their categories are generally accurate. Furthermore, compared to (c1) to (c3), the categories of floating object detection in figures (a1) to (a3) and (b1) to (b3) are relatively general and single. All box-like floating objects are detected as box, and all bottle-like floating objects are detected as bottle, failing to achieve refined detection. In summary, the introduction of a custom vocabulary can significantly improve the detection performance of the model, reduce missed detections and false detections, and enable refined detection of floating objects in rivers, thereby helping river management personnel to more accurately identify and analyze the types, quantities, distribution, and other related information of floating objects in rivers.

[0105] Figure 9 As shown in the figure, in a complex river scene, the detic model can accurately detect floating objects in the river, such as balloons, plastic bags, plastic bottles, plastic boxes, foam boxes, etc., with a high confidence level.

[0106] 6.3. DeepLabv3+ model and detic cascade detection results

[0107] The refined river floating object detection disclosed in this paper adopts the DeepLabv3+ model and the detic cascade detection method. The results of each stage are shown in the figure below. Figure 10 As shown in the figure, Figures (e) to (h) show the results of the DeepLabv3+ model on the river segmentation, Figures (i) to (j) are the mask images of the river, which represent the river area in binary form, Figures (m) to (p) are the river water body images without the riverbank background, Figures (q) to (t) show the detection effect of detic on the river water body image, and Figures (u) to (x) map the detection results of detic on the river water body image back to the original river image. The results show that the cascade method can take advantage of the advantages of both models and enhance the adaptability to complex backgrounds and lighting changes. By applying target detection on the segmentation result image, the position of floating objects can be more accurately located, reducing false detections and missed detections.

[0108] In summary, the proposed refined detection method for floating debris in rivers addresses the practical needs of river pollution control. First, the semantic segmentation algorithm DeepLabv3+ model is used to segment the river, separating the water body. Then, detic all-object detection is used as a secondary detector to detect floating debris in the river water. The segmentation effect is significant, with an AP50 index of 0.995. Furthermore, refined detection is achieved by setting the categories of floating debris through a custom vocabulary and blacklists. Experimental results also demonstrate that this method is capable of achieving refined detection of floating debris.

[0109] This method addresses the difficulty in detecting floating debris in rivers due to its diverse variety. To a certain extent, it overcomes the problem of misidentifying reflections in the river as floating debris due to environmental factors such as weather and lighting. The custom vocabulary used encompasses a rich set of floating object categories, enabling more refined detection of floating debris in rivers and providing more precise technical support for river environmental management. It has proven effective in practice and is readily applicable.

[0110] Any process or method description in the flowchart of the present disclosure or described in other ways herein can be understood as representing a module, fragment or portion of code including one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes other implementations, in which the functions may not be performed in the order shown or discussed, including performing the functions in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong. The processor performs the various methods and processes described above. For example, the method embodiments in the present disclosure can be implemented as a software program that is tangibly contained in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via a memory and / or a communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the method described above can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods in any other appropriate manner (e.g., by means of firmware).

[0111] The logic and / or steps represented in the flowchart or otherwise described herein may be embodied in any readable storage medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device).

[0112] For the purposes of this specification, a "readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use with or in conjunction with an instruction execution system, device or apparatus. More specific examples (a non-exhaustive list) of readable storage media include the following: an electrical connection having one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, a device on the Internet, and a portable read-only memory (CDROM). In addition, the readable storage medium can even be paper or other suitable medium on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner as necessary, and then storing it in a memory.

[0113] It should be understood that various parts of the present disclosure can be implemented using hardware, software, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0114] A person skilled in the art in the field of the present disclosure may understand that all or part of the steps of the above-mentioned implementation method may be accomplished by instructing related hardware through a program, and the program may be stored in a readable storage medium, which, when executed, includes one or a combination of the steps of the method implementation method.

[0115] Furthermore, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules. If the integrated modules are implemented as software functional modules and sold or used as independent products, they may also be stored in a readable storage medium. The storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc.

[0116] The present disclosure also provides an electronic device, including: a memory, the memory storing execution instructions; and a processor or other hardware module, the processor or other hardware module executing the execution instructions stored in the memory, so that the processor or other hardware module performs the above method.

[0117] The present disclosure also provides a readable storage medium, in which execution instructions are stored. When the execution instructions are executed by a processor, they are used to implement the above method.

[0118] In summary, the method disclosed herein has unique advantages and is easy to promote.

[0119] It can be seen that the present disclosure has achieved good results in actual use, and can realize the timely, effective and accurate allocation of hot issues involved in the hotline case text data information to the corresponding social function management departments, and realize the accurate and refined allocation of these special text data to the social function management departments.

[0120] Thus far, the technical solutions of the present disclosure have been described in conjunction with the preferred implementation methods shown in the accompanying drawings. However, those skilled in the art should understand that the above-mentioned implementation methods are merely for the purpose of clearly illustrating the present disclosure and are not intended to limit the scope of the present disclosure. The scope of protection of the present disclosure is obviously not limited to these specific implementation methods. Without departing from the principles of the present disclosure, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will still fall within the scope of the present disclosure.

Claims

1. A refined detection method for floating objects in a river, characterized in that: The steps include: (1) River data acquisition: using image acquisition devices to collect river monitoring image data; (2) First-level detection: river segmentation is used to separate the river water from the background, generating a mask image representing the river water area. The mask image of the river water area is superimposed on the original river monitoring image, removing all background areas outside the river water, and retaining only the river water part to form a river water image; (3) The second level detection is to detect floating objects in the river using the detic object detection model on the segmented river water image; (4) Result mapping, mapping the output of detic all things detection back to the original river monitoring image; Among them, the categories of floating objects in rivers are expanded through custom vocabulary, so as to finely distinguish the categories of floating objects; the floating object categories of concern are configured through whitelist and blacklist, the floating object targets in rivers are filtered, and the floating objects in rivers are finely detected.

2. A refined detection method for floating objects in a river according to claim 1, characterized in that: The river channel segmentation uses the DeepLabv3+ model to segment the river channel image and obtain a segmentation result rich in spatial details.

3. A refined detection method for floating objects in a river according to claim 1, characterized in that: The described detic object detection alternately uses sample sets of river target detection and sample sets of image classification to train the network model.

4. A refined detection method for floating objects in a river according to claim 2, characterized in that: The river channel image segmentation is based on a deep convolutional neural network architecture. It automatically learns the semantic features in river channel images through multi-layer convolution operations, and extracts both low-level features such as edges and textures and high-level features such as object categories. The high-level semantic features have strong semantic information, while the low-level features contain rich spatial detail information. The encoder-decoder structure is then used to fuse the high-level semantic feature information and the low-level feature information, so that the segmentation results of the river channel images can maintain semantic accuracy while better capturing the detail information in the image, thereby obtaining a segmentation result rich in spatial details.

5. A refined detection method for floating objects in a river according to claim 4, characterized in that: In the encoder stage, the input image is first processed by the backbone network Resnet101 using a deep convolutional neural network to obtain a preliminary effective feature layer, thereby realizing the extraction of low-level features; Subsequently, the atrous convolutional pyramid module (ASPP) is used to enhance feature extraction capabilities; The ASPP module consists of multiple operations including 1x1 convolution, dilated convolution, and maximum pooling layers, effectively capturing semantic information at different scales, thereby extracting high-level features. The high-level features extracted by the ASPP module are fused with the low-level features extracted by the backbone network to obtain a feature map compatible with both low-level and high-level semantic information of the image. The fused features are further extracted through 3x3 convolution, and the feature map is restored to the original image size through quadruple upsampling. The dilation rates of the dilated convolution are 6, 12, and 18.

6. A refined detection method for floating objects in a river according to claim 4, characterized in that: In the decoder stage, first, the feature maps output by the encoder are upsampled to four times their original size; second, these upsampled feature maps are element-wise added to the feature maps output by the adjusted backbone feature extraction network. The adjustment process is to match the number of channels of the feature maps through a 1×1 convolution operation to achieve effective feature fusion; finally, the final output result is obtained by bilinear interpolation upsampling the upsampled feature maps and the fused feature maps to generate a high-resolution segmentation feature map.

7. A refined detection method for floating objects in a river according to claim 1 or 3, characterized in that: The detic all things detection first uses the detected sample set to train the model, then freezes the detection head and continues to use the trained weights to train image classification samples; The loss function is: L max-size =BCE(Wf j ,c),j=argmax j (size(b j )) (1) Among them, BCE is the binary cross entropy loss function, j is the index that maximizes the bounding box size, and b j is the bounding box, argmax j is the index of the largest bounding box, size is the size of the bounding box, I is the input image, and D det Refers to the detection dataset, D cls Refers to the classification dataset, L rpn Propose network loss for region, L reg is the bounding box regression loss, L cls is the target classification loss, L max-size is the maximum size loss, f j Represents the region of interest feature ROIfeature corresponding to the detected prediction box, c is the category corresponding to the largest prediction box, that is, the category label of the entire image, and W is the weight of the classifier; the loss function L shown in formula (1) max-size The loss function L used in joint traditional target detection rpn +L reg +L cls , which constitutes the final loss function L(I) of detic, where λ is the loss weight, which is set to 0.1 in this disclosure.

8. A refined detection system for floating objects in a river, characterized by: include: (1) A river data acquisition unit, which collects river monitoring image data through an image acquisition device; (2) The first-level detection unit uses river segmentation to separate the river water body from the background and generate a mask image representing the river water area. The mask image of the river water area is superimposed on the original river monitoring image to remove all background areas outside the river water body and retain only the river water part to form a river water image; (3) The second-level detection unit uses the detic object detection model to detect floating objects in the river on the segmented river water image; (4) Result mapping unit, which maps the output of detic all things detection back to the original river monitoring image; Among them, the categories of floating objects in rivers are expanded through custom vocabulary, so as to finely distinguish the categories of floating objects; the floating object categories of concern are configured through whitelist and blacklist, the floating object targets in rivers are filtered, and the floating objects in rivers are finely detected.

9. An electronic device, characterized in that: include: a memory storing execution instructions; as well as A processor, wherein the processor executes the execution instruction stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images