A method, device and electronic device for detecting floating objects on water surface

Abstract

The present application provides a method, device and electronic device for detecting floating objects on the water surface, which are used to improve the accuracy of identifying floating objects on the water surface. The method comprises: detecting a first image set according to a target detection model in training, and obtaining a first detection result corresponding to the first image set; wherein the first detection result is used to mark floating objects on the water surface in the first image set; according to the first detection result and at least one size range, training the target detection model in training, and obtaining a trained target detection model; wherein at least one size range is used to determine the size grade of floating objects on the water surface.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and electronic device for detecting floating objects on a water surface. Background Art

[0002] With the rapid development of economy, environmental pollution is becoming more and more serious. A large number of floating objects appear on the surface of lakes and rivers, some of which even contain substances harmful to human body, directly threatening human survival and development. Monitoring of floating objects on the surface of water has become an urgent problem to be solved. Summary of the invention

[0003] The embodiments of the present application provide a method, device, and electronic device for detecting floating objects on the water surface, which can improve the accuracy of detecting floating objects on the water surface.

[0004] In a first aspect, an embodiment of the present application provides a method for detecting floating objects on the water surface, comprising: detecting a first image set according to a target detection model in training, and obtaining a first detection result corresponding to the first image set; wherein the first detection result is used to mark the floating objects on the water surface in the first image set; training the target detection model in training according to the first detection result and at least one size range, and obtaining a trained target detection model; wherein the at least one size range is used to determine the size grade of the floating objects on the water surface.

[0005] By combining the size range to enrich the background of training samples, the trained target detection model can detect floating objects of different sizes on the water surface, which improves the generalization performance of the model and is conducive to improving the accuracy of detecting floating objects on the water surface.

[0006] In one possible design, the target detection model in training is trained according to the first detection result and at least one size range to obtain a trained target detection model, including: performing data enhancement processing on the first image set according to the first detection result and at least one size range to obtain a second image set; and training the target detection model in training according to the second image set to obtain the trained target detection model.

[0007] In one possible design, the at least one size range includes N size ranges; wherein different size ranges in the N size ranges correspond to different size grades of the surface floating objects, and N is a positive integer greater than 1.

[0008] In a possible design, the size grades of the floating objects on the water surface corresponding to the N size ranges include small, medium and large.

[0009] In a possible design, the first image set is subjected to data enhancement processing according to the first detection result and at least one size range to obtain the second image set, including: when the number of large-sized floating objects in the first image set identified by the first detection result is lower than a set threshold, the floating objects in at least one image of the first image set are enlarged to obtain the second image set. Such a design can improve the detection model's ability to detect large targets, which is conducive to improving the model's generalization performance and the accuracy of detecting large floating objects on the water surface.

[0010] In a possible design, the first image set is subjected to data enhancement processing according to the first detection result and at least one size range to obtain a second image set, including: when the number of small-sized floating objects in the first image set identified by the first detection result is lower than a set threshold, multiple images in the first image set are subjected to data enhancement processing to obtain multiple enhanced images; wherein the data enhancement processing includes flipping, scaling, and color gamut conversion; and the multiple enhanced images are merged to obtain one image in the second image set. Through such a design, the detection model's ability to detect small targets can be improved, which is beneficial to improving the model's generalization performance and the accuracy of detecting small floating objects on the water surface.

[0011] In a possible design, the method also includes: detecting the image to be detected according to the trained target detection model to obtain a second detection result corresponding to the image to be detected, and the second detection result is used to mark water surface floating objects of different size grades in the image to be detected.

[0012] In a possible design, the method further includes: segmenting the water surface area and the non-water surface area in the image to be detected; and filtering the floating objects on the water surface in the image to be detected marked by the second detection result according to the segmentation result. By using image segmentation to filter out interference and delete the misdetected non-water surface objects, the accuracy of the detection of floating objects on the water surface can be further improved.

[0013] In one possible design, the water surface floating objects in the image to be detected marked by the second detection result are filtered according to the segmentation result, including: if the second detection result marks the first water surface floating object in the image to be detected, and the proportion of the first water surface floating object in the water surface area is lower than a set threshold, then the mark of the first water surface floating object in the second detection result is deleted; and / or, if the second detection result marks the second water surface floating object in the image to be detected, and the second water surface floating object is in a non-water surface area, then the mark of the second water surface floating object in the second detection result is deleted. Through such a design, for objects distributed on both the water surface and the non-water surface, it is possible to quickly determine whether to perform interference filtering, which can enhance the flexibility of interference filtering and improve the efficiency and accuracy of water surface floating object detection.

[0014] In a second aspect, an embodiment of the present application provides a device for detecting floating objects on a water surface, comprising:

[0015] An acquisition module, used for acquiring a first image set;

[0016] Processing modules for:

[0017] The first image set is detected according to the target detection model in training to obtain a first detection result corresponding to the first image set; wherein the first detection result is used to mark the floating objects on the water surface in the first image set; according to the first detection result and at least one size range, the target detection model in training is trained to obtain a trained target detection model; wherein the at least one size range is used to determine the size grade of the floating objects on the water surface.

[0018] By combining the size range to enrich the background of training samples, the trained target detection model can detect floating objects of different sizes on the water surface, which improves the generalization performance of the model and is conducive to improving the accuracy of detecting floating objects on the water surface.

[0019] In a possible design, the processing module is also used to: perform data enhancement processing on the first image set according to the first detection result and at least one size range to obtain a second image set; train the target detection model in training according to the second image set to obtain the trained target detection model.

[0020] In one possible design, the at least one size range includes N size ranges; wherein different size ranges in the N size ranges correspond to different size grades of the surface floating objects, and N is a positive integer greater than 1.

[0021] In a possible design, the size grades of the floating objects on the water surface corresponding to the N size ranges include small, medium and large.

[0022] In a possible design, when the processing module performs data enhancement processing on the first image set to obtain the second image set according to the first detection result and at least one size range, it is specifically used to: when the number of large-sized floating objects in the first image set identified by the first detection result is lower than a set threshold, the floating objects in at least one image of the first image set are enlarged to obtain the second image set. Through such a design, the detection model's ability to detect large targets can be improved, which is conducive to improving the model's pan-scaling performance and the accuracy of detecting large floating objects on the water surface.

[0023] In one possible design, when the processing module performs data enhancement processing on the first image set to obtain the second image set based on the first detection result and at least one size range, the processing module is specifically used to: when the number of small-sized floating objects in the first image set identified by the first detection result is lower than a set threshold, perform data enhancement processing on multiple images in the first image set to obtain multiple enhanced images; wherein the data enhancement processing includes flipping, scaling, and color gamut conversion; and merge the multiple enhanced images to obtain one image in the second image set. Through such a design, the detection model's ability to detect small targets can be improved, which is beneficial to improving the model's pan-performance and the accuracy of detecting small floating objects on the water surface.

[0024] In a possible design, the processing module is also used to: detect the image to be detected according to the trained target detection model to obtain a second detection result corresponding to the image to be detected, and the second detection result is used to mark water surface floating objects of different size grades in the image to be detected.

[0025] In a possible design, the processing module is further used to: segment the water surface area and the non-water surface area in the image to be detected; and filter the floating objects on the water surface in the image to be detected marked by the second detection result according to the segmentation result. By using image segmentation to filter out interference and delete the misdetected non-water surface objects, the accuracy of the detection of floating objects on the water surface can be further improved.

[0026] In one possible design, when filtering the floating objects on the water surface in the image to be detected marked by the second detection result according to the segmentation result, the processing module is specifically used to: if the second detection result marks the first floating object on the water surface in the image to be detected, and the proportion of the first floating object on the water surface area is lower than the set threshold, then the mark of the first floating object on the water surface in the second detection result is deleted; and / or, if the second detection result marks the second floating object on the water surface in the image to be detected, and the second floating object on the water surface is in the non-water surface area, then the mark of the second floating object on the water surface in the second detection result is deleted. Through such a design, for objects distributed on both the water surface and the non-water surface, it is possible to quickly determine whether to perform interference filtering, which can enhance the flexibility of interference filtering and improve the efficiency and accuracy of surface floating object detection.

[0027] In a third aspect, an embodiment of the present application provides an electronic device, which includes at least a processor and a memory, and the processor is used to execute the steps included in any optional implementation method in the first aspect when executing a computer program stored in the memory.

[0028] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the steps included in any one of the implementations in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application.

[0030] Figure 1 A schematic diagram of a flow chart of a method for detecting floating objects on a water surface provided in an embodiment of the present application;

[0031] Figure 2 A schematic diagram of a floating object on the water surface provided in an embodiment of the present application;

[0032] Figure 3 A schematic diagram of the effect of data enhancement processing provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of another data enhancement processing effect provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of a target detection process provided in an embodiment of the present application;

[0035] Figure 6A schematic diagram of a false detection filtering process provided in an embodiment of the present application;

[0036] Figure 7 A schematic diagram of the structure of a device for detecting floating objects on a water surface provided in an embodiment of the present application;

[0037] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0039] The terms "first" and "second" in the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variation thereof are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0040] In the embodiment of the present application, "at least one" may mean at least two, for example, two, three or more, and the embodiment of the present application does not limit this.

[0041] In addition, the term "and / or" in the embodiments of the present application is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally indicates that the associated objects before and after are in an "or" relationship.

[0042] With the rapid development of economy, environmental pollution is becoming more and more serious. A large number of floating objects appear on the surface of lakes and rivers, some of which even contain substances harmful to human body, directly threatening human survival and development. Monitoring of floating objects on the surface of water has become an urgent problem to be solved.

[0043] Due to the particularity of water scenes, the traditional method of relying on management personnel to monitor is inefficient and has prominent safety risks. In response to this, cameras are used for real-time monitoring in some scenes, but managers on the monitoring equipment side cannot take corresponding measures in real time and accurately.

[0044] In response to this, the related art provides a solution for detecting floating objects on the water surface using computer vision, such as Solution 1 and Solution 2 described below.

[0045] Solution 1: Collect surface drifting object data in advance, enhance the collected data, and mark the surface drifting objects in the data to obtain a surface drifting object data set, which is divided into a training set and a test set; use the surface drifting object training set to train the improved YOLOv3 network model; construct a surface drifting object test set based on the surface drifting object data image, and use the trained improved YOLOv3 network model to detect and identify the surface drifting object test set.

[0046] Solution 2: Real-time acquisition of the current water surface image, performing a differential operation between the current water surface image and the background image to obtain a differential image, which is composed of floating object pixels representing floating objects on the water surface; then the number of floating object pixels on the water surface and the actual area of ​​floating objects corresponding to each floating object pixel on the water surface are calculated. Search processing is performed based on the determined area and position of floating objects on the water surface.

[0047] As can be seen from the above, the performance of the YOLOv3 network model in Solution 1 is limited by the collected floating objects data, and it is highly dependent on the data. When detecting floating objects of different sizes in actual scenes, there may be a large probability of false detection and poor accuracy. Although Solution 2 uses image difference, it reduces the dependence on data compared to Solution 1, but the difference process involves the processing of multiple thresholds, the operation is complicated, and the detection efficiency is low.

[0048] Based on this, the embodiments of the present application provide a method, device and electronic device for detecting floating objects on the water surface, which can improve the efficiency and accuracy of detecting floating objects on the water surface.

[0049] For better understanding, the technical solution of the present application is described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments may be combined with each other.

[0050] See also Figure 1 A method for detecting floating objects on a water surface is shown in FIG. 1 , and the method comprises the following steps:

[0051] S101, detecting a first image set according to a target detection model in training to obtain a first detection result corresponding to the first image set, wherein the first detection result is used to mark floating objects on the water surface in the first image set.

[0052] Specifically, the first image set may include one or more images taken by a camera device, such as one or more images captured from a water surface monitoring video taken by a camera device. Optionally, in order to reduce memory and resource consumption, the interval between captured frames may be enlarged during capture, such as capturing a water surface image every 10 frames. Optionally, the aforementioned camera device is a camera for monitoring the water surface. Alternatively, the first image set may also be pre-stored water surface scene data, which includes one or more images related to water surface scenes. The images in the embodiments of the present application mainly refer to images containing water surface scenes, or are described as water surface images.

[0053] Optionally, part or all of the images included in the first image set contain at least one floating object on the water surface. Figure 2 A schematic diagram of a floating object on the water surface is given. Figure 2 In the figure, a rectangular block represents floating objects on the water surface, the area enclosed by the dotted line and the edge of the image represents the water surface, and the other areas are non-water surfaces, such as the ground, grass, etc.

[0054] Specifically, by inputting a certain image in the first image set into the target detection model being trained, a first detection result corresponding to the image can be obtained, and the first detection result is used to mark the floating object on the water surface in the image. For example, the first detection result includes an image with a marking frame, or it can also be understood as marking the position where the floating object on the water surface appears in the image using the marking frame. Optionally, the marking frame can be a rectangular frame, a circular frame or a frame of other shapes, which is not limited in the embodiments of the present application.

[0055] Optionally, the target detection model in training may be a target detection model based on deep learning, such as YOLOv3, YOLOv4, etc. This embodiment of the present application is not limited to this.

[0056] S102, training the object detection model in training according to the first detection result and at least one size range to obtain a trained object detection model

[0057] Wherein, the at least one size range is used to determine the size grade of the floating object on the water surface. Specifically, the at least one size range includes N size ranges; wherein different size ranges in the N size ranges correspond to different size grades of the floating object on the water surface, and N is a positive integer greater than 1. For example, the size grades of the floating object on the water surface corresponding to the N size ranges may include small, medium, and large; or it can also be described as: the size grade of the floating object on the water surface corresponding to one size range in the N size ranges is small, medium, or large.

[0058] N size ranges can be divided by one or more size thresholds, and the size threshold can also be described as an area threshold. Exemplarily, N is 3, and the N size ranges can be divided by a first size threshold (t1) and a second size threshold (t2). It can be understood that the first size threshold is smaller than the second size threshold. The N size ranges include a first size range, a second size range, and a third size range. The first size range corresponds to a range smaller than the first size threshold, such as expressed as (0, t1), and the size grade of the surface floating objects corresponding to the first size range is small; the second size range corresponds to the range between the first size threshold and the second size threshold, such as expressed as [t1, t2], and the size grade of the surface floating objects corresponding to the second size range is medium; the third size range corresponds to a range larger than the second size threshold, and the size grade of the surface floating objects corresponding to the third size range is large.

[0059] Based on this, if the size (or area) of a floating object on the water surface is smaller than the first size threshold, that is, the size of the floating object on the water surface is within the first size range, then the size grade of the floating object on the water surface is small, or the floating object on the water surface can be described as a small target. If the size of a floating object on the water surface is greater than or equal to the first size threshold, and less than or equal to the second size threshold, that is, the size of the floating object on the water surface is within the second size range, then the size grade of the floating object on the water surface is medium, or the floating object on the water surface can be described as a normal target; if the size of the floating object on the water surface is greater than the second size threshold, that is, the size of the floating object on the water surface is within the third size range, then the size grade of the floating object on the water surface is large, or the floating object on the water surface can be described as a large target.

[0060] Specifically, S102 may include the following steps 1 and 2.

[0061] Step 1: According to the first detection result and at least one size range, data enhancement processing is performed on the first image set to obtain a second image set.

[0062] First, the size level of the surface floating object marked in the first detection result can be determined according to at least one size range. Exemplarily, corresponding to the first size threshold and the second size threshold described in S101, it can be determined that the surface floating object marked in the first detection result includes one or more of the following: small target, normal target, and large target.

[0063] Then, the first image set may be subjected to relevant data enhancement processing according to the distribution of small targets, normal targets, and large targets in the first detection result. For example, during the training process of the target detection model, the cumulative loss (loss) of small targets, normal targets, and large targets marked in the first detection result up to the current number of iterations may be counted to obtain the cumulative loss loss_small corresponding to the small target, the cumulative loss loss_normal corresponding to the normal target, and the cumulative loss loss_big corresponding to the small target. Further, according to the cumulative losses of different targets, the first image set may be subjected to relevant data enhancement processing to obtain the second image set.

[0064] Optionally, the cumulative loss corresponding to small targets is used to indicate the difference in quantity between small targets marked in the first detection result and small targets actually contained in the surface image, the cumulative loss corresponding to normal targets is used to indicate the difference in quantity between normal targets marked in the first detection result and normal targets actually contained in the surface image, and the cumulative loss corresponding to large targets is used to indicate the difference in quantity between large targets marked in the first detection result and large targets actually contained in the surface image.

[0065] Specifically, the relevant data enhancement processing is performed on the first image set according to the accumulated losses of different targets, which can be performed with reference to the following optional implementation mode 1 or implementation mode 2.

[0066] Implementation Method 1

[0067] Case A1: When the cumulative loss of large targets identified by the first detection result is greater than a set loss threshold, or when the number of large floating objects in the first image set identified by the first detection result is lower than a set threshold, it indicates that the target detection model in training has poor detection performance for large floating objects. In this case, the floating objects in at least one image of the first image set can be enlarged to obtain the second image set.

[0068] As an example, Figure 3 A schematic diagram showing the effect of data enhancement processing. Figure 3 (a) is magnified to obtain Figure 3 (b) in Figure 3 (b) in FIG. 1 can be used as an image in the second image set. It can be understood that Figure 3In (a) or (b), a rectangular block is used to represent floating objects on the water surface, the area enclosed by the dotted line and the edge of the image represents the water surface, and other areas in the image represent non-water surfaces, such as grass, ground, etc.

[0069] Case A2: When the cumulative loss of small targets identified by the first detection result is greater than a set loss threshold, or when the number of small-sized floating objects in the first image set identified by the first detection result is lower than a set threshold, it indicates that the target detection model in training has poor detection performance for small floating objects. In such a case, data enhancement processing may be performed on multiple images in the first image set to obtain multiple enhanced images; and then the multiple enhanced images may be merged to obtain one image in the second image set. The data enhancement processing includes geometric transformation and color gamut transformation, and the geometric transformation here may include flipping, scaling or other geometric transformations. This embodiment of the present application is not limited to this.

[0070] As an example, Figure 4 Schematic diagram of another data enhancement processing effect. Figure 4 After flipping, scaling, and color gamut transformation, the four images (a), (b), (c), and (d) are merged to obtain Figure 4 (e) in which the upper left corner of (e) corresponds to (a) before processing, the upper right corner corresponds to (b) before processing, the lower left corner corresponds to (c) before processing, and the lower right corner corresponds to (d) before processing. Figure 4 (e) in can be used as an image in the second image set. It can be understood that Figure 4 In (a), (b), (c), (d) or (e), a rectangular block is used to represent floating objects on the water surface, the area enclosed by the dotted line and the edge of the image represents the water surface, and other areas in the image represent non-water surfaces, such as grass, ground, etc.

[0071] Case A3, when the cumulative loss of normal targets identified by the first detection result is greater than the set loss threshold, or when the number of floating objects with a size class of medium in the first image set identified by the first detection result is lower than the set threshold, it indicates that the target detection model in training has poor detection performance for floating objects with a size class of medium. In such a case, data enhancement processing may not be performed on the images in the first image set, and the second image set may include some or all of the images in the first image set.

[0072] Implementation Method 2

[0073] Compare the cumulative loss of large targets identified by the first detection result, the cumulative loss of small targets identified by the first detection result, and the cumulative loss of normal targets identified by the first detection result, and perform relevant data enhancement processing on the first image set according to the comparison result.

[0074] In case B1, when the cumulative loss of the large target identified by the first detection result is the largest, it indicates that the target detection model in training has poor detection performance for large floating objects on the water surface. In such a case, the floating objects on the water surface in at least one image of the first image set may be enlarged to obtain the second image set.

[0075] Case B2, when the cumulative loss of the small target identified by the first detection result is the largest, it means that the target detection model in training has poor detection performance for small floating objects on the water surface. In such a case, data enhancement processing can be performed on multiple images in the first image set respectively to obtain multiple enhanced images; and then the multiple enhanced images are merged to obtain an image in the second image set. Among them, the data enhancement processing includes geometric transformation and color gamut transformation, and the geometric transformation here may include flipping, scaling or other geometric transformations. This embodiment of the present application is not limited to this.

[0076] Case B3, when the cumulative loss of normal targets identified by the first detection result is the largest, it indicates that the target detection model in training has poor detection performance for floating objects on the water surface. In such a case, data enhancement processing may not be performed on the images in the first image set, and the second image set may include some or all of the images in the first image set.

[0077] Step 2: Train the target detection model in training according to the second image set to obtain the trained target detection model.

[0078] Specifically, the process of training the target detection model in training can be understood as the process of updating the model parameters of the target detection model in training. The trained target detection model has the same model structure as the target detection model in training, for example, both can be YOLOv3, YOLOv4 and other models, but the model parameters of the trained target detection model are different from those of the target detection model in training.

[0079] Furthermore, the following describes the process of using the trained target detection model. Figure 1 As an example, after executing S101 to S102, S103 may be optionally executed. As an optional step, Figure 1 Step S103 is indicated by a dotted line.

[0080] S103, detecting the image to be detected according to the trained target detection model to obtain a second detection result corresponding to the image to be detected, wherein the second detection result is used to mark water surface floating objects of different size grades in the image to be detected.

[0081] Specifically, the image to be detected may be an image obtained by photographing a water surface scene with a camera, or may be an image in the aforementioned first training set. This embodiment of the present application is not limited to this.

[0082] As an example, see Figure 5 A schematic diagram of a target detection process is shown, where an image to be detected is input into a trained target detection model, and the output of the trained target detection model is a detection result corresponding to the image to be detected, that is, the image to be detected with a marked box. Figure 5 It is shown that the image to be detected includes floating objects on the water surface with sizes of small, medium and large.

[0083] The above training method combines the size range to enrich the training sample background, so that the trained target detection model can take into account the detection of floating objects of different sizes on the water surface, improves the generalization performance of the model, and is conducive to improving the accuracy of detection of floating objects on the water surface.

[0084] Further optionally, considering that the target detection model may mark non-surface objects, that is, misdetect non-surface objects as floating objects on the water surface, the output of the trained target detection model may be verified after executing S101 to S103. Exemplarily, the above method may also include the following steps S104 and S105. Figure 1 Optional steps S104 and S105 are indicated by dashed boxes.

[0085] S104: Segmenting the water surface area and the non-water surface area in the image to be detected.

[0086] Specifically, the output result of the trained target detection model corresponding to the image to be detected can be obtained, that is, the image with a marked box, and then the output result can be input into the segmentation model to perform two-category segmentation of water surface and non-water surface, marking the water surface area and the non-water surface area.

[0087] S105: According to the segmentation result, filtering the floating objects on the water surface in the image to be detected marked by the second detection result.

[0088] Specifically, after determining the water surface area and the non-water surface area according to S104, it is determined whether there is a water surface area for the mark of floating objects on the water surface in the image to be detected using the trained target detection model, and the mark box for the area where the water surface exists completely or most of the area exists is retained, while the mark that is not in the water surface area or most of the area is not in the water surface area is filtered out.

[0089] Specifically, one or more of the following two examples may be used for implementation:

[0090] Example 1: If the second detection result marks the first floating object on the water surface in the image to be detected, and the proportion of the first floating object on the water surface area is lower than a set threshold, then the mark of the first floating object on the water surface in the second detection result is deleted; optionally, the set threshold can be set to 50% or other values, which is not limited to this in the embodiment of the present application.

[0091] Example 2: If the second detection result marks a second floating object on the water surface in the image to be detected, and the second floating object on the water surface is in a non-water surface area, then the mark of the second floating object on the water surface in the second detection result is deleted.

[0092] In addition, refer to Figure 6 A flowchart of false positive detection filtering is shown below. As an example, assume that the image to be detected is input into the trained target detection model, and the second detection result is an image with marked boxes 1, 2, and 3. False positive detection filtering is performed on the second detection result, that is, steps S104 to S105 are performed to filter out the marked boxes in the non-water surface area, such as Figure 6 It shows that the marking box 3 in the aforementioned second detection result is deleted after false detection filtering.

[0093] The above verification method uses image segmentation to filter out interference and delete misdetected non-water surface objects, which can further improve the accuracy of water surface floating object detection.

[0094] Based on the same concept, the embodiment of the present application provides a device for detecting floating objects on the water surface, which can realize the functions corresponding to the aforementioned method for detecting floating objects on the water surface. The device for detecting floating objects on the water surface can be a hardware structure, a software module, or a hardware structure plus a software module. The device for detecting floating objects on the water surface can be implemented by a chip system, which can be composed of a chip or include a chip and other discrete devices. Please refer to Figure 7 , is a schematic diagram of the structure of a device for detecting floating objects on the water surface provided in an embodiment of the present application, and the device for detecting floating objects on the water surface includes an acquisition module 701 and a processing module 702.

[0095] Specifically: an acquisition module 701 is used to acquire a first image set;

[0096] The processing module 702 is used to:

[0097] The first image set is detected according to the target detection model in training to obtain a first detection result corresponding to the first image set; wherein the first detection result is used to mark the floating objects on the water surface in the first image set; according to the first detection result and at least one size range, the target detection model in training is trained to obtain a trained target detection model; wherein the at least one size range is used to determine the size grade of the floating objects on the water surface.

[0098] In a possible design, the processing module 702 is also used to: detect the image to be detected according to the trained target detection model to obtain a second detection result corresponding to the image to be detected, and the second detection result is used to mark water surface floating objects of different size grades in the image to be detected.

[0099] In a possible design, the processing module 702 is also used to: perform data enhancement processing on the first image set according to the first detection result and at least one size range to obtain a second image set; train the target detection model in training according to the second image set to obtain the trained target detection model.

[0100] In one possible design, the at least one size range includes N size ranges; wherein different size ranges in the N size ranges correspond to different size grades of the surface floating objects, and N is a positive integer greater than 1.

[0101] In a possible design, the size grades of the floating objects on the water surface corresponding to the N size ranges include small, medium and large.

[0102] In one possible design, the processing module 702, when performing data enhancement processing on the first image set to obtain the second image set based on the first detection result and at least one size range, is specifically used to: when the number of large-size floating objects in the first image set identified by the first detection result is lower than a set threshold, enlarge the floating objects in at least one image of the first image set to obtain the second image set.

[0103] In one possible design, the processing module 702, when performing data enhancement processing on the first image set to obtain the second image set based on the first detection result and at least one size range, is specifically used to: when the number of small-size floating objects in the first image set identified by the first detection result is lower than a set threshold, perform data enhancement processing on multiple images in the first image set to obtain multiple enhanced images; wherein the data enhancement processing includes flipping, scaling, and color gamut transformation; and merge the multiple enhanced images to obtain one image in the second image set.

[0104] In a possible design, the processing module 702 is further used to: segment the water surface area and the non-water surface area in the image to be detected; and filter the floating objects on the water surface in the image to be detected marked by the second detection result according to the segmentation result.

[0105] In one possible design, the processing module 702, when filtering the floating objects on the water surface in the image to be detected marked by the second detection result according to the segmentation result, is specifically used to: if the second detection result marks a first floating object on the water surface in the image to be detected, and the proportion of the first floating object on the water surface area is lower than a set threshold, then delete the mark of the first floating object on the water surface in the second detection result; and / or, if the second detection result marks a second floating object on the water surface in the image to be detected, and the second floating object on the water surface is in a non-water surface area, then delete the mark of the second floating object on the water surface in the second detection result.

[0106] Based on the same inventive concept, the present application embodiment also provides an electronic device, see Figure 8 , is a structural diagram of an electronic device provided in an embodiment of the present application, the electronic device includes at least one processor 802, and a memory 801 connected or coupled to the at least one processor 802. In addition, the electronic device may further include a communication interface 803. The electronic device can exchange information with other devices through the communication interface 803. Exemplarily, the communication interface 803 may be a transceiver, circuit, bus, module, pin or other type of communication interface. When the electronic device is a chip-type device or circuit, the communication interface 803 in the electronic device may also be an input-output circuit, which can input information (or receive information) and output information (or send information), and the processor is an integrated processor or microprocessor or integrated circuit or logic circuit, and the processor can determine the output information based on the input information.

[0107] The coupling in the embodiment of the present application is an indirect coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules. The processor 802 may cooperate with the memory 801 and the communication interface 803. The specific connection medium between the above-mentioned processor 802, memory 801 and communication interface 803 is not limited in this disclosure.

[0108] Optional, see Figure 8, the processor 802, the memory 801 and the communication interface 803 are connected to each other via a bus 840. The bus 800 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0109] In the embodiment of the present application, the memory 801 is a non-volatile computer-readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 801 may include at least one type of storage medium, for example, it may include flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (ReadOnly Memory, ROM), electrically erasable programmable read-only memory (Electrically ErasableProgrammable Read-Only Memory, EEPROM), magnetic memory, disk, optical disk, etc. The memory 801 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 801 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing instructions, computer programs and / or data.

[0110] In the embodiment of the present application, the processor 802 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method for detecting floating objects on the water surface disclosed in the embodiment of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.

[0111] By designing and programming the processor 802, the code corresponding to the method for detecting floating objects on the water surface introduced in the aforementioned embodiment can be solidified into the chip, so that the chip can execute the steps of the aforementioned method for detecting floating objects on the water surface when running. How to design and program the processor 802 is a technology well known to those skilled in the art and will not be repeated here.

[0112] Specifically, in an embodiment of the present application, the memory 801 stores instructions that can be executed by at least one processor 802, and at least one processor 802 can execute the steps included in the aforementioned method for detecting floating objects on the water surface by calling the instructions or computer programs stored in the memory 801. Exemplarily, the processor 802 is used to obtain a first image set using the communication interface 803; the processor 802 is also used to: detect the first image set according to the target detection model in training to obtain a first detection result corresponding to the first image set; wherein the first detection result is used to mark the floating objects on the water surface in the first image set; according to the first detection result and at least one size range, the target detection model in training is trained to obtain a trained target detection model; wherein the at least one size range is used to determine the size grade of the floating objects on the water surface.

[0113] Furthermore, the processor 802 is also used to detect the image to be detected according to the trained target detection model to obtain a second detection result corresponding to the image to be detected, and the second detection result is used to mark water surface floating objects of different size grades in the image to be detected.

[0114] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer, the computer executes the steps of the aforementioned method for detecting floating objects on the water surface.

[0115] In some possible embodiments, various aspects of the method for detecting floating objects on the water surface provided by the present application can also be implemented in the form of a computer program product, which includes program code. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the method for detecting floating objects on the water surface according to various exemplary embodiments of the present application described above in this specification.

[0116] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0120] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the embodiments of the present application and their equivalents, the embodiments of the present application are also intended to include these modifications and variations.

Claims

1. A method for detecting floating objects on a water surface, characterized in that: include: Detect the first image set according to the target detection model being trained to obtain a first detection result corresponding to the first image set; wherein the first detection result is used to mark the floating objects on the water surface in the first image set; Determine, based on the first detection result and at least one size range, the number of floating objects on the water surface of each size class in the first image set predicted by the target detection model being trained; wherein the at least one size range is used to determine the size class of the floating objects on the water surface; Based on the comparison result of each number and the set threshold value corresponding to each number, the first image set is subjected to data enhancement processing to obtain a second image set; The object detection model in training is trained according to the second image set to obtain the trained object detection model.

2. The method according to claim 1, characterized in that The at least one size range includes N size ranges; wherein different size ranges in the N size ranges correspond to different size grades of the surface floating objects, and N is a positive integer greater than 1.

3. The method according to claim 2, characterized in that The size grades of the floating objects on the water surface corresponding to the N size ranges include small, medium and large.

4. The method according to claim 3, characterized in that The step of performing data enhancement processing on the first image set based on the comparison result of each number with the set threshold value corresponding to each number to obtain the second image set includes: When the number of large-size floating objects in the first image set identified by the first detection result is lower than a set threshold, the floating objects in at least one image of the first image set are enlarged to obtain the second image set.

5. The method according to claim 3, characterized in that The step of performing data enhancement processing on the first image set based on the comparison result of each number with the set threshold value corresponding to each number to obtain the second image set includes: When the number of small-sized floating objects in the first image set identified by the first detection result is lower than a set threshold, data enhancement processing is performed on multiple images in the first image set to obtain multiple enhanced images; wherein the data enhancement processing includes flipping, scaling, and color gamut conversion; The multiple enhanced images are merged to obtain an image in the second image set.

6. The method according to any one of claims 1 to 5, characterized in that: Also includes: The image to be detected is detected according to the trained target detection model to obtain a second detection result corresponding to the image to be detected, and the second detection result is used to mark water surface floating objects of different size grades in the image to be detected.

7. The method according to claim 6, characterized in that The method further comprises: Segmenting the water surface area and the non-water surface area in the image to be detected; According to the segmentation result, the floating objects on the water surface in the image to be detected marked by the second detection result are filtered.

8. The method according to claim 7, characterized in that The filtering of the floating objects on the water surface in the image to be detected marked by the second detection result according to the segmentation result includes: If the second detection result marks the first floating object on the water surface in the image to be detected, and the proportion of the first floating object on the water surface area is lower than a set threshold, then deleting the mark of the first floating object on the water surface in the second detection result; and / or, If the second detection result marks a second floating object on the water surface in the image to be detected, and the second floating object on the water surface is in a non-water surface area, then the mark for the second floating object on the water surface in the second detection result is deleted.

9. A device for detecting floating objects on a water surface, characterized in that: include: An acquisition module, used for acquiring a first image set; Processing modules for: Detect the first image set according to the target detection model being trained to obtain a first detection result corresponding to the first image set; wherein the first detection result is used to mark the floating objects on the water surface in the first image set; Determine, based on the first detection result and at least one size range, the number of floating objects on the water surface of each size class in the first image set predicted by the target detection model being trained; wherein the at least one size range is used to determine the size class of the floating objects on the water surface; Based on the comparison result of each number and the set threshold value corresponding to each number, the first image set is subjected to data enhancement processing to obtain a second image set; The object detection model in training is trained according to the second image set to obtain the trained object detection model.

10. An electronic device, characterized in that: The electronic device comprises at least a processor and a memory, and the processor is used to execute the steps of the method for detecting floating objects on the water surface described in any one of claims 1 to 8 when executing the computer program stored in the memory.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the steps included in the method according to any one of claims 1 to 8.

Images