3-2D based target detection method capable of eliminating water surface light and shadow interference

Abstract

The present application relates to a kind of 3-2D-based target detection method capable of eliminating water surface light and shadow interference, comprising the following steps: obtaining the video data affected by light and shadow and containing target object, the video data is composed of several frame video frames;The video data is put into the trained 3D target recognition model to carry out target detection;When the trained 3D target recognition model identifies target, the last frame of the video data is sent into the trained 2D target recognition model to detect, and the category, position and confidence of the target are obtained.The present application can eliminate water surface light and shadow interference to improve the accuracy of target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of water surface target detection, in particular to a target detection method capable of eliminating water surface light and shadow interference based on 3-2D. BACKGROUND

[0002] Thanks to the continuous improvement of deep learning-based video detection technology, the application of deep learning models in the marine field is also increasingly widespread. Therefore, by applying a deep learning model to the detection of water surface floating garbage, long-term monitoring of changes in the target area environment can be achieved.

[0003] Since the deep learning target detection method applied to the ocean surface has the problem of being disturbed by water surface light and shadow, which seriously affects the detection accuracy of the deep learning model. Therefore, when water surface floating object detection is needed in the target water area, a suitable deep learning model needs to be established to reduce the interference of water surface light and shadow on target detection. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a target detection method based on 3-2D capable of eliminating water surface light and shadow interference, which can eliminate water surface light and shadow interference to improve the accuracy of target detection.

[0005] The technical solution adopted by the present application to solve its technical problem is to provide a target detection method based on 3-2D capable of eliminating water surface light and shadow interference, comprising the following steps:

[0006] Obtain video data affected by light and shadow and containing target objects, the video data is composed of a plurality of video frames;

[0007] Put the video data into the trained 3D target recognition model for target detection;

[0008] When the trained 3D target recognition model identifies the target, the last frame of the video data is sent to the trained 2D target recognition model for detection to obtain the category, position and confidence of the target.

[0009] The 3D target recognition model comprises eight Convolution3d layers, one Conv3d layer, two PReLU layers, one FC layer and one BatchNorm1d layer connected in turn; wherein the Conv3d is used to extract the time sequence information of the video data, the PReLU layer is used to improve the overfitting problem of the 3D target recognition model, the BatchNorm1d layer is used to accelerate the convergence speed and stability of the 3D target recognition model, and the FC layer is used to classify the target.

[0010] The Convolution3d layer comprises a Conv3d layer, a BatchNorm3d layer and a PreLU layer connected in sequence.

[0011] The 3D target recognition model comprises the following steps during training:

[0012] The obtained video is subjected to a frame segmentation operation to obtain K video frames, and then the K video frames are spliced according to a preset frame number N to obtain K / N videos in sequence.

[0013] The state of the K / N videos is marked to obtain training data.

[0014] The training data is put into the 3D target recognition model for training to obtain a trained 3D target recognition model.

[0015] When the trained 3D target recognition model fails to recognize a target, the detection task is ended.

[0016] The 2D target recognition model is constructed by using a YOLO network structure.

[0017] Advantages

[0018] Compared with the prior art, the present application has the following advantages and positive effects: the present application first determines whether there is a target by using a 3D target detection model to judge the frame to be detected, and if there is a target, the target is detected by using a 2D target model. This method avoids the situation that light and shadow are misdetected as targets when a 2D convolution network is directly applied. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a flowchart of the embodiment of the present application;

[0020] Figure 2 is a structural schematic diagram of the 3D target recognition model in the embodiment of the present application;

[0021] Figure 3 is a test result diagram of the 3D target recognition model in a single scene in the embodiment of the present application;

[0022] Figure 4 is a test result diagram of the 3D target recognition model in a complex scene in the embodiment of the present application. DETAILED DESCRIPTION

[0023] The application will be further described in connection with specific embodiments. It should be understood that these embodiments are only used to illustrate the application and not used to limit the scope of the application. Furthermore, it should be understood that after reading the content of the application, those skilled in the art can make various modifications or changes to the application, and these equivalent forms also fall within the scope defined by the appended claims.

[0024] The embodiment of the application relates to a 3-2D-based target detection method capable of eliminating water surface light and shadow interference, as shown in the formula (I), comprising the following steps: Figure 1 As shown in the formula (II), comprising the following steps:

[0025] Step 1, acquiring video data affected by light and shadow and containing target objects, wherein the video data is composed of a plurality of video frames.

[0026] Step 2, putting the video data into a trained 3D target recognition model for target detection. In this step, the 3D target recognition model, as shown in the formula (III), comprises eight Convolution3d layers, one Conv3d layer, two PReLU layers, one FC layer and one BatchNorm1d layer connected in sequence. The Convolution3d layer comprises a Conv3d layer, a BatchNorm3d layer and a PreLU layer connected in sequence; the Conv3d is used for extracting time sequence information of the video data; the PReLU layer is used for improving overfitting of the 3D target recognition model; the BatchNorm1d layer is used for accelerating convergence speed and stability of the 3D target recognition model; and the FC layer is used for classifying targets. Figure 2

[0027] The training method of the 3D target recognition model in the embodiment is as follows:

[0028] Python is used to read the field-collected video, frame the read video to obtain 3000 video frames, and then splice the 3000 video frames according to a preset frame number 15 in sequence to obtain 200 videos;

[0029] Marking the state of the 200 videos, marking 1 if there is a target in the video, and marking 0 otherwise to obtain training data;

[0030] Putting the training data into the 3D target recognition model for training to obtain the trained 3D target recognition model.

[0031] ​In the training of the model, the original video is spliced into a small video every N frames using a video frame splicing method, and then it is determined by artificial judgment whether the target exists in the video and recorded. The principle is that in the video time of continuous N frames, the water surface has only two states, i.e. with target or without target. Due to the problem of light and shadow interference often occurring on the water surface, the detection accuracy of the target detection model is seriously interfered. The embodiment utilizes the continuous characteristics of the video before and after the frame, and through the video training method, it can be determined in advance whether the target exists in the detection area, i.e. whether the detected target is light and shadow or real target, so as to avoid the problem of false detection caused by direct detection of the target detection model.

[0032] After the training is completed, the untrained data is substituted for testing, and the test results are shown in Figure 3 and Figure 4 , wherein Figure 3 represents the precision (Precision), recall (Recall), and balance index (F1) of the 3D target recognition model in a single scene, Figure 4 represents the precision (Precision), recall (Recall), and balance index (F1) of the 3D target recognition model in a complex scene.

[0033] According to the result graph of the above evaluation index, in the case of a single scene, the Precision, Recall and F1 index of the 3D target recognition model even reach 100% effect, which shows that the 3D target recognition model of the embodiment has strong discrimination ability and can accurately determine whether the target exists in the detected video, solving the problem of mistaking light and shadow as target objects. In the case of a complex background, Figure 4 , it can be known that the Precision of the 3D target recognition model of the embodiment is as high as 96.2%, the Recall is 95%, and the F1 is 95.2%. It shows that the 3D target recognition model of the embodiment has high applicability and can adapt to complex and variable environments.

[0034] Step 3: When the trained 3D target recognition model recognizes the target, i.e. the output of the 3D target recognition model is "1", the last frame of the video data is sent into the trained 2D target recognition model for detection, and the category, position and confidence of the target are obtained. The 2D target recognition model in this step can be a target detection model of YOLO network structure. When the trained 3D target recognition model does not recognize the target, i.e. the output of the 3D target recognition model is "0", the whole detection process is ended.

[0035] It is not difficult to find that the application firstly judges the frame to be detected by the 3D target detection model to determine whether there is a target, and if there is a target, enters the 2D target model to detect the target. This method avoids the situation that light and shadow are misdetected as targets when the 2D convolution network is directly applied.

Claims

1. A 3-2D based target detection method capable of eliminating water surface light and shadow interference, characterized in that, The method comprises the following steps: Obtaining video data affected by light and shadow and containing a target object, the video data being composed of a plurality of frames; Putting the video data into a trained 3D target recognition model for target detection; When the trained 3D target recognition model identifies a target, sending the last frame of the video data into a trained 2D target recognition model for detection to obtain the category, position and confidence of the target; The 3D target recognition model comprises the following steps during training: Frame operation is performed on the obtained video to obtain K video frames, and then K video frames are spliced according to a preset frame number N in sequence to obtain K / N videos; The state of the K / N videos is marked to obtain training data; The training data is put into the 3D target recognition model for training to obtain a trained 3D target recognition model. 2.The 3-2D based target detection method capable of eliminating water surface light and shadow interference according to claim 1, wherein, The 3D target recognition model comprises eight Convolution3d layers, one Conv3d layer, two PReLU layers, one FC layer and one BatchNorm1d layer connected in sequence; wherein the Conv3d is used for extracting time sequence information of the video data, the PReLU layer is used for improving overfitting of the 3D target recognition model, the BatchNorm1d layer is used for accelerating convergence speed and stability of the 3D target recognition model, and the FC layer is used for classifying the target. 3.The 3-2D based target detection method capable of eliminating water surface light and shadow interference according to claim 2, characterized in that, The Convolution3d layer comprises a Conv3d layer, a BatchNorm3d layer and a PreLU layer connected in sequence.

4. The 3-2D based target detection method capable of eliminating water surface light and shadow interference according to claim 1, characterized in that, When the trained 3D target recognition model does not identify a target, the detection task is ended.

5. The 3-2D based target detection method capable of eliminating water surface light and shadow interference according to claim 1, characterized in that, The 2D target recognition model is constructed by using a YOLO network structure.

Images