A target detection method capable of reducing water surface light and shadow interference

Abstract

The present application relates to a kind of target detection method capable of reducing water surface light and shade interference, comprising the following steps: obtaining the video data of target object affected by light and shade, and extracting a frame from every N frames from the video data;The extracted image frame is spliced into a new picture according to every M frame;Mark the target object in the extracted image frame and the new picture after splicing, and obtain training data;Put training data into deep learning model for training, and obtain water surface floating object detection model;The sea surface floating target picture to be detected is put into the water surface floating object detection model, and the category, position and confidence of target are obtained.The present application can reduce the interference of water surface light and shade and other factors on target detection.

Description

Technical Field

[0001] This invention relates to the field of video detection technology for floating objects on water surfaces, and in particular to a target detection method that can reduce light and shadow interference on the water surface. Background Technology

[0002] As marine development deepens, the application of artificial intelligence to the ocean is becoming increasingly widespread. Therefore, by applying deep learning models to the detection of floating debris on the water surface, long-term monitoring of environmental changes in target areas can be achieved.

[0003] Because water waves and the irregular reflections they produce under sunlight can cause light and shadow interference when photographing objects on the water surface, resulting in wave patterns and light and shadow images, these waves and light and shadow can seriously affect the accuracy of target detection when using AI target detection. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a target detection method that can reduce the interference of water surface light and shadow, and can reduce the interference of water surface light and shadow and other factors on target detection.

[0005] The technical solution adopted by this invention to solve its technical problem is: to provide a target detection method that can reduce the interference of light and shadow on the water surface, comprising the following steps:

[0006] Acquire video data that is affected by light and shadow and contains the target object, and extract one frame every N frames from the video data;

[0007] The extracted image frames are stitched together into a new image, with each M frames representing a different image.

[0008] The target object is marked in the extracted image frames and the stitched new image to obtain training data;

[0009] The training data is fed into a deep learning model for training, resulting in a water surface floating object detection model.

[0010] The image of the floating target on the sea surface to be detected is put into the floating object detection model to obtain the target's category, location, and confidence level.

[0011] When the extracted image frames are stitched together into a new image in M ​​frames per frame, the stitching is performed in the order of acquisition.

[0012] The M satisfies M = m 2 , where m is a natural number greater than 1.

[0013] When marking the target object in the extracted image frame and the stitched new image, the labelImg tool is used for marking, and the marking content includes the location and category of the target object.

[0014] The deep learning model adopts the optimized Yolov4-Mobilenetv3 model. The optimized Yolov4-Mobilenetv3 model replaces the CSPDarknet53 feature extraction network in the original Yolov4 model with the MobileNetv3 network, introduces the SENet module in the 8th block of the MobileNetv3 network residual block, and adds the HS nonlinear activation function in the 3rd and 5th blocks.

[0015] Beneficial effects

[0016] Due to the adoption of the above technical solution, this invention has the following advantages and positive effects compared with the prior art: This invention stitches multiple spaced frames into one image by selecting video frames and stitching them together. Then, the original individual frames and the stitched image are both labeled and fed into a deep learning model for training, making the deep learning model more able to distinguish between light and shadow and the target, thereby achieving the purpose of eliminating light and shadow interference. At the same time, compared with common 3D models, this method uses a model recognition method based on 2D images, which requires less hardware computing power and memory, and can be applied to low computing power platforms. Attached Figure Description

[0017] Figure 1 This is a flowchart of an embodiment of the present invention;

[0018] Figure 2 This is a diagram showing the effect of extracting and stitching frames in this invention. Detailed Implementation

[0019] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0020] Embodiments of the present invention relate to a target detection method that can reduce interference from light and shadow on the water surface, such as... Figure 1 As shown, the process includes the following steps: acquiring video data that is affected by light and shadow and contains target objects, and extracting one frame every N frames from the video data; stitching the extracted image frames into a new image every M frames; marking the target objects in the extracted image frames and the stitched new image to obtain training data; feeding the training data into a deep learning model for training to obtain a floating object detection model; and feeding the image of the floating target to be detected into the floating object detection model to obtain the target's category, location, and confidence level.

[0021] This implementation method eliminates the influence of light and shadow by extracting video frames at intervals and then stitching them together into a single image. The principle is as follows: Within a given time period, the floating target is consistently present across all video frames, exhibiting continuity between frames. In contrast, waves and light / shadow interference appear in the frame at varying frequencies, displaying an irregular distribution across frames spaced apart by a certain time interval. Therefore, the image stitched together from appropriately spaced frames contains both continuous information about the target and irregular information about the changing waves and light / shadow over time. Training the model with images stitched together from multiple intervals improves the model's ability to learn light and shadow features. Furthermore, the simultaneous presence of light / shadow and the target enhances the model's ability to distinguish between the two.

[0022] The present invention will be further illustrated by a specific embodiment below.

[0023] First, Python is used to read the video containing the target object and affected by light and shadow, obtaining the video's frame count. Then, one frame out of every five frames is taken and stored in an array. After all the video has been read, Python is used to stitch the frames stored in the array. In this embodiment, stitching is performed every nine frames, meaning every nine elements in the array are combined. Specifically, Python creates a new RGB layer three times the size of a single frame image, and then pastes the nine frames onto this layer in the order they were acquired. The stitched result is shown below. Figure 2 As shown.

[0024] Based on the stitched image and the extracted original frame, the target objects in the extracted original frame and the stitched new image are marked using the labelImg tool. That is, an XML file is generated for each image to record the position, category and other information of the target object in the image, thus obtaining training data.

[0025] Based on the labeled training data, it is fed into a deep learning model for training, and the model with the best training results is saved. This model is the water surface floating object detection model. The deep learning model used in this embodiment is an optimized YOLOv4-Mobilenetv3 model. The optimization steps include replacing the CSPDarknet53 feature extraction network in the original YOLOv4 with the MobileNetv3 network, introducing the SENet module in the 8th block of the MobileNetv3 network residual block, and adding the HS nonlinear activation function in the 3rd and 5th blocks, thereby improving the accuracy of feature extraction and reducing the computational cost of the network.

[0026] Randomly acquired images or videos of floating targets on the water surface are put into the floating object detection model for testing. In this embodiment, the floating object detection model is tested using commonly used evaluation indicators for target detection models. The test results are shown in Tables 1 and 2.

[0027] Table 1 Overall Evaluation Indicators of the Model

[0028] Model mAP FPS Modelsize (M) Yolov4-Mobilenetv3(pre) 86.14％ 53 53.8 Yolov4-Mobilenetv3(lat) 91.75％ 52 53.8

[0029] Table 2 Detailed evaluation results of the model

[0030]

[0031] As shown in the comparison table of the two model evaluation results above, after applying the interval frame taking and stitching method described in this embodiment, the optimized model achieved an mAP value of 91.75% on the test dataset, with an FPS of 52 and a model size of only 53.8M. Furthermore, in terms of specific object detection performance, the model using the interval frame taking and stitching method of this embodiment showed higher accuracy, recall, and F1 score in specific classifications compared to the original method. As shown in Table 2, all evaluation metrics were higher after using this method than before.

[0032] It is easy to see that this invention extracts one frame every five frames from a video containing both the target and the light and shadow effects, and then stitches the extracted frames together into a new image every nine frames. This method can effectively eliminate the interference of light and shadow on the water surface, making the deep learning model more able to distinguish between light and shadow and the target.

Claims

1.A target detection method capable of reducing water surface light and shadow interference, characterized in that, The method comprises the following steps: Obtaining video data affected by light and shadow and containing a target object, and extracting a frame from every N frames of the video data; Splicing the extracted image frames into a new picture every M frames; Marking the target object in the extracted image frames and the spliced new picture to obtain training data; Putting the training data into a deep learning model for training to obtain a water surface floating object detection model; the deep learning model adopts an optimized Yolov4-Mobilenetv3 model, the optimized Yolov4-Mobilenetv3 model replaces a CSPDarknet53 feature extraction network in an original Yolov4 model with a MobileNetv3 network, introduces an SENet module in an eighth block of a residual block of the MobileNetv3 network, and adds HS nonlinear activation functions in the third and fifth blocks; Putting a picture of a sea surface floating target to be detected into the water surface floating object detection model to obtain a category, a position, and a confidence of the target. 2.The target detection method capable of reducing water surface light and shadow interference according to claim 1, characterized in that, When the extracted image frames are spliced into a new picture every M frames, the splicing is performed according to the acquisition sequence. 3.The target detection method capable of reducing water surface light and shadow interference according to claim 1, characterized in that, The M satisfies M = m 2 where m is a natural number greater than 1. 4.The target detection method capable of reducing water surface light and shadow interference according to claim 1, characterized in that, When the target object is marked in the extracted image frames and the spliced new picture, a labelImg tool is used for marking, and the marked content includes a position and a category of the target object.

Images