Water surface target detection method based on semi-supervision

Abstract

The invention discloses a water surface target detection method based on semi-supervision. The method comprises the steps that training set images containing a water surface target object are acquired, and the training set images comprise label images and label-free images; performing depth estimation on each image in the training set images by using a depth estimation model to obtain a depth map of each image, the depth map of each image including a depth value of each pixel; extracting a height value of a target frame of each target object of each category in the target labeled image according to the category of the target object in the training set image; and when the training set image is subjected to target round processing, target detection is performed on a target label-free image based on a teacher model, a pseudo label of each target object in the label-free image is generated, and the pseudo label comprises a detection frame and a category of each target object. According to the invention, the technical problem of low target detection efficiency when the existing semi-supervised target detection technology is applied to the water surface is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a method for detecting a surface target based on semi-supervision. Background Art

[0002] Surface target detection is a core task in the field of surface vision. It aims to accurately identify and locate surface targets by performing high-precision target-level detection on images captured by cameras. The accuracy of this technology is crucial for the decision-making of unmanned surface vessels (USVs) because it determines the quality of USV's acquisition of target information, including key data such as the number of targets, their categories, sizes, shape characteristics, and accurate locations. Through surface target detection technology, USVs can more efficiently complete a series of key tasks, such as autonomous navigation, water quality testing, and water rescue. Therefore, continuous and in-depth research on surface target detection technology is of irreplaceable importance for the technological advancement and widespread application of surface unmanned vessels.

[0003] In recent years, deep learning has made significant progress in the field of target detection with its powerful fitting ability and has gradually become the mainstream method in this field. However, detection models based on deep learning usually contain a large number of trainable parameters, which means that training models in a fully supervised manner requires relying on a large amount of labeled data. Due to the diversity of target categories and the complexity of scenes, manual target labeling is both time-consuming and costly. Therefore, semi-supervised methods, as an effective means to significantly reduce the cost of manual labeling, have attracted the attention of more and more scholars. Although semi-supervised target detection methods have achieved remarkable results in land scenarios, their application in surface scenarios still faces the following challenges and limitations.

[0004] Based on the above factors, the current research on semi-supervised object detection is mainly focused on the land environment. In contrast, the research and application of semi-supervised object detection for the surface environment are particularly slow. In view of the uniqueness and complexity of the surface environment and the broad application prospects of USV, we conduct a more in-depth and systematic study on surface semi-supervised object detection to promote the continuous development and practical application of USV technology. Summary of the invention

[0005] The embodiment of the present invention provides a semi-supervised surface target detection method, which at least solves the technical problem of low efficiency of target detection on the water surface when the existing semi-supervised target detection technology is applied.

[0006] According to one aspect of an embodiment of the present invention, a semi-supervised surface target detection method is provided. The method may include: obtaining a training set image containing a surface target object, wherein the training set image includes a labeled image and an unlabeled image; using a depth estimation model to perform depth estimation on each image in the training set image to obtain a depth map of each image, wherein the depth map of each image includes a depth value of each pixel; extracting the height value of the target frame of each target object of each category in the target labeled image according to the category of the target object in the training set image; when the training set image is subjected to target round processing, target detection is performed on the target unlabeled image based on the teacher model to generate a pseudo label for each target object in the unlabeled image, wherein the pseudo label includes a detection frame and category of each target object; fitting the height value and depth value of the target frame of each target object of each category in the labeled image through an initial two-dimensional Gaussian distribution model to obtain a target two-dimensional Gaussian distribution model, wherein the target two-dimensional Gaussian distribution model includes joint distribution parameters of each category of target objects, and the joint distribution parameters include the mean vector of the height value and depth value of each category of target objects. and a covariance matrix describing the relationship between the height value and the depth value of each category of target objects; calculating the Mahalanobis distance between the height value and the depth value corresponding to the pseudo-label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object, and determining the target result of the pseudo-label of each target object in each unlabeled image based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object; based on the target result, fusing any unlabeled image of the target round with any unlabeled image of the previous round to obtain a fused new image; inputting the new image and the labeled image into the student model to obtain the category and position of the target object of the new image and the category and position of the target object of the labeled image, calculating the classification loss and regression loss corresponding to the new image, and the classification loss and regression loss corresponding to the labeled image; updating the weights of the teacher model based on the parameters of the student model of the target round, repeating the iteration, and obtaining the category and position of the target object of the new image in the last round and the category and position of the target object of the labeled image.

[0007] Optionally, the height value and depth value of the target frame of each target object of each class of the labeled image are fitted by the initial two-dimensional Gaussian distribution model to obtain an expression of the target two-dimensional Gaussian distribution model:

[0008] in, For the i The target two-dimensional Gaussian distribution model of the category, For the i The height value of the target box of the category, For the iThe median depth value on the lower edge of the target box of the category, is the mean vector of height and depth of each type of target object, is the covariance matrix of the relationship between the height and depth variables for each class of target objects.

[0009] Optionally, the expression for calculating the Mahalanobis distance between the height value and the depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object is:

[0010] in, For each unlabeled image j The Mahalanobis distance between the height and depth values ​​corresponding to the pseudo labels of the target objects and the target two-dimensional Gaussian distribution model of the category of each target object, For each unlabeled image j The height and depth values ​​corresponding to the pseudo labels of the target objects, represents transpose, Represents the inverse matrix.

[0011] Optionally, determining a target result of the pseudo-label of each target object in each unlabeled image based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object includes: if the Mahalanobis distance is greater than the square of the category screening threshold corresponding to each target object in each unlabeled image, then removing the pseudo-label of each target object in each unlabeled image; if the Mahalanobis distance is less than or equal to the square of the category screening threshold corresponding to each target object in each unlabeled image, then retaining the pseudo-label of each target object in each unlabeled image.

[0012] Optionally, the expression of fusing any unlabeled image of the target round with any unlabeled image of the previous round to obtain a fused new image is: +(1 ) (0.5 +0.5 )+

[0013] in, After fusion New images, is the mask of each target object in any unlabeled image of the target wheel, is the mask of each target object in any unlabeled image in the previous round, is the mth unlabeled image in the nth round, is the mth unlabeled image in the n-1th round, is the dynamic fusion ratio, is the number of unlabeled images in the target round and any unlabeled image in the previous round in the mask M1 area. , The proportion of fusion, is the number of unlabeled images in the target round and any unlabeled image in the previous round in the mask M2 area. The proportion of fusion, (1 ) (0.5 +0.5 ) is the area outside the mask M1 and M2 area. Any unlabeled image of the target round and any unlabeled image of the previous round are fused at a ratio of 0.5. In the overlapping area of ​​masks M1 and M2, any unlabeled image of the target round is randomly fused in the area of ​​M1, or any unlabeled image of the previous round is fused in the area of ​​M2.

[0014] Optionally, the expression of the dynamic fusion ratio is:

[0015] in, is the dynamic fusion ratio, is the initial mixing ratio, Final mixing ratio, t is the number of iterations corresponding to the target round, T is the total number of training iterations.

[0016] Beneficial effects of the present invention: (1) The present invention adopts a depth-aware pseudo-label filtering strategy. By utilizing the depth and scale information of the target, a joint distribution model is established to judge the reliability of the pseudo-labels and filter out the pseudo-labels that deviate from the distribution center, thereby reducing the interference of false detections in the pseudo-labels, improving the overall quality of the pseudo-labels, and providing better training data for the student model.

[0017] (2) The dynamic region mixing enhancement method proposed in the present invention is based on the region-level mixing operation, which adaptively adjusts the mixing ratio at the image region level to maintain the integrity and semantic consistency of the target. It is particularly suitable for maritime environments with complex backgrounds. This method dynamically adjusts the mixing ratio during the training process, making data enhancement gradually more difficult, thereby improving the generalization ability of the model under different backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flow chart of a semi-supervised surface target detection method according to an embodiment of the present invention; Figure 2 is a framework diagram of a semi-supervised surface target detection method according to an embodiment of the present invention; Figure 3 is a schematic diagram of qualitative analysis results on Seaships and SMD datasets according to an embodiment of the present invention; Figure 4 is a statistical graph of false positive rate and true positive rate of a depth-aware pseudo-label filtering strategy according to an embodiment of the present invention; Figure 5 1 is a heat map corresponding to different image enhancement methods according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] Example 1 According to an embodiment of the present invention, a semi-supervised surface target detection method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system comprising at least one set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0022] Figure 1 is a flow chart of a method for detecting a surface target based on semi-supervision according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps: Step S101, obtaining a training set of images containing surface target objects, wherein the training set images include labeled images and unlabeled images.

[0023] In this embodiment, a training data set is obtained, and the training set is an image containing a surface target object. The target object can be a general cargo ship, a bulk carrier, an ore carrier, a fishing boat, a container ship, a passenger ship, etc. This is only an example and is not specifically limited. The training set images include labeled images and unlabeled images. The labeled images are images in which the target objects in the images are marked with detection boxes and categories, and the unlabeled images are images in which the target objects in the images are not marked. Figure 2 is a framework diagram of a semi-supervised surface target detection method according to an embodiment of the present invention, such as Figure 2 As shown, Figure 2 The unlabeled data in the training dataset are unlabeled images, and the labeled data are labeled images. All images in the training dataset are resized to a fixed size required by the model, such as 1333×800, to ensure input consistency and facilitate subsequent processing and computational efficiency.

[0024] Step S102: Use the depth estimation model to perform depth estimation on each image in the training set of images to obtain a depth map of each image, wherein the depth map of each image includes a depth value of each pixel.

[0025] In this embodiment, a depth estimation model is used to perform depth estimation on labeled images and unlabeled images in a training set to obtain a depth value of each pixel of each image, that is, a depth map of each image.

[0026] Step S103 , extracting the height value of the target box of each target object of each category in the target labeled image according to the category of the target object in the training set image.

[0027] In this embodiment, the height value of the target box of each target object of each category in the target label image is extracted according to the category of the target object in the training set image.

[0028] Step S104, when the training set image is subjected to target round processing, target detection is performed on the target unlabeled image based on the teacher model to generate a pseudo label for each target object in the unlabeled image, wherein the pseudo label includes a detection box and a category of each target object.

[0029] In this embodiment, when the training set images are processed, the number of training rounds of the target detection network is 3 rounds, which is only used as an example and is not specifically limited. It can also be the 5th round, the 6th round, etc. Figure 2 As shown, target detection is performed on the target unlabeled images according to the teacher model to generate pseudo labels for each target object in the unlabeled images, where the pseudo labels include the detection box and category of each target object. It should be noted that there are 4 target unlabeled images, the detection box of each target object is the length and width of the target object, and the category of each target object is which category the target object belongs to, for example, the label of a general cargo ship is 1.

[0030] Step S105, fitting the height value and depth value of the target frame of each target object of each class in the labeled image through the initial two-dimensional Gaussian distribution model to obtain a target two-dimensional Gaussian distribution model, wherein the target two-dimensional Gaussian distribution model includes joint distribution parameters of each class of target objects, and the joint distribution parameters include the mean vector of the height value and depth value of each class of target objects and a covariance matrix describing the relationship between the height value and the depth value of each class of target objects.

[0031] In this embodiment, the height value and depth value of the target frame of each target object of each class in the labeled image are fit to the initial two-dimensional Gaussian distribution model of the unfused data to obtain a fitted target two-dimensional Gaussian distribution model, wherein the target two-dimensional Gaussian distribution model includes joint distribution parameters of each class of target objects, and the joint distribution parameters include the mean vector of the height value and the depth value of each class of target objects and a covariance matrix describing the relationship between the height value and the depth value of each class of target objects.

[0032] Step S106, calculate the Mahalanobis distance between the height value and the depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object, and determine the target result of the pseudo label of each target object in each unlabeled image based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object.

[0033] In this embodiment, the height value and depth value corresponding to the pseudo-label of each target object in each unlabeled image are calculated, and the Mahalanobis distance between the target two-dimensional Gaussian distribution model of the category of each target object is calculated, and the size of the Mahalanobis distance and the square of the category screening threshold corresponding to each target object is determined to determine whether the pseudo-label of each target object in each unlabeled image is retained or eliminated, wherein the Mahalanobis distance measures the similarity between the pseudo-label and the known distribution, that is, the degree of deviation: if the scale-depth coordinate of the pseudo-label falls near the distribution center of the target two-dimensional Gaussian distribution model, it is considered that its reliability is high and is retained for training; if the scale-depth coordinate of the pseudo-label deviates from the distribution center of the target two-dimensional Gaussian distribution model (exceeding the range of two times the standard deviation), it is considered that there is a risk of false detection and is eliminated.

[0034] Step S107: Based on the target result, any unlabeled image of the target round is fused with any unlabeled image of the previous round to obtain a fused new image.

[0035] In this embodiment, according to the target result, when the target round is the third round, any one of the four unlabeled images of the third round is fused with any one of the four unlabeled images of the previous round to obtain a fused new image, such as Figure 2 Dynamic area mixing in .

[0036] Step S108, input the new image and the labeled image into the student model, obtain the category and position of the target object of the new image and the category and position of the target object of the labeled image, calculate the classification loss and regression loss corresponding to the new image, and the classification loss and regression loss corresponding to the labeled image.

[0037] In this embodiment, if Figure 2 As shown, the new image and the labeled image are input into the student model to obtain the category and position of the target object of the new image and the category and position of the target object of the labeled image, and the classification loss and regression loss corresponding to the new image and the classification loss and regression loss corresponding to the labeled image are calculated.

[0038] Step S109, update the weights of the teacher model based on the parameters of the student model of the target round, repeat the iteration, and obtain the category and position of the target object of the new image of the last round and the category and position of the target object of the labeled image.

[0039] In this embodiment, after the student model completes one round of training, its weights are updated to the teacher model through exponential moving average to ensure that the teacher model can dynamically adapt to the new pseudo-label distribution, and the iteration is repeated until the category and position of the target object of the new image and the category and position of the target object of the labeled image in the last round.

[0040] The above method of this embodiment is further introduced below.

[0041] As an optional implementation mode, step S105, the height value and depth value of the target frame of each target object of each class of the labeled image are fitted by the initial two-dimensional Gaussian distribution model, and the expression of the target two-dimensional Gaussian distribution model is obtained as follows:

[0042] in, For the i The target two-dimensional Gaussian distribution model of the category, For the i The height value of the target box of the category, For the i The median depth value on the lower edge of the target box of the category, is the mean vector of height and depth of each type of target object, is the covariance matrix of the relationship between the height and depth variables for each class of target objects.

[0043] In this embodiment, the height value and the depth value of the target frame of each target object of each category of the labeled image are put into the initial two-dimensional Gaussian distribution model for fitting to obtain a target two-dimensional Gaussian distribution model.

[0044] As an optional embodiment, in step S106, the expression of the Mahalanobis distance between the height value and the depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object is calculated as follows:

[0045] in, For each unlabeled image j The Mahalanobis distance between the height and depth values ​​corresponding to the pseudo labels of the target objects and the target two-dimensional Gaussian distribution model of the category of each target object, For each unlabeled image j The height and depth values ​​corresponding to the pseudo labels of the target objects, represents transpose, Represents the inverse matrix.

[0046] In this embodiment, the Mahalanobis distance between the height value and the depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object is calculated by the above formula.

[0047] As an optional implementation manner, step S106, the target result of the pseudo-label of each target object in each unlabeled image is determined based on the Mahalanobis distance and the square of the category filtering threshold corresponding to each target object, including: if the Mahalanobis distance is greater than the square of the category filtering threshold corresponding to each target object in each unlabeled image, then each target object pseudo-label in each unlabeled image is eliminated; if the Mahalanobis distance is less than or equal to the square of the category filtering threshold corresponding to each target object in each unlabeled image, then each target object pseudo-label in each unlabeled image is retained.

[0048] In this embodiment, the size of the Mahalanobis distance and the square of the category screening threshold corresponding to each target object is determined. When the Mahalanobis distance is greater than the square of the category screening threshold corresponding to each target object in each unlabeled image, the pseudo-label of each target object in each unlabeled image is eliminated. When the Mahalanobis distance is less than or equal to the square of the category screening threshold corresponding to each target object in each unlabeled image, the pseudo-label of each target object in each unlabeled image is retained.

[0049] As an optional implementation manner, step S107, the expression of fusing any unlabeled image of the target round with any unlabeled image of the previous round to obtain a fused new image is: +(1 ) (0.5 +0.5 )+

[0050] in, After fusion New images, is the mask of each target object in any unlabeled image of the target wheel, is the mask of each target object in any unlabeled image in the previous round, is the mth unlabeled image in the nth round, is the mth unlabeled image in the n-1th round, is the dynamic fusion ratio, is the number of unlabeled images in the target round and any unlabeled image in the previous round in the mask M1 area. , The proportion of fusion, is the number of unlabeled images in the target round and any unlabeled image in the previous round in the mask M2 area. The proportion of fusion, (1 ) (0.5 +0.5 ) is the area outside the mask M1 and M2 area. Any unlabeled image of the target round and any unlabeled image of the previous round are fused at a ratio of 0.5. In the overlapping area of ​​masks M1 and M2, any unlabeled image of the target round is randomly fused in the area of ​​M1, or any unlabeled image of the previous round is fused in the area of ​​M2.

[0051] In this embodiment, through the above formula, any unlabeled image of the target round is fused with any unlabeled image of the previous round to obtain a new image. The number of new images is 4. For example, the first unlabeled image of the third round is fused with the third unlabeled image of the second round, as shown in 2. The dynamic area mixing enhancement module, for example, when a ship (positive sample) is mixed with a complex nearshore background (negative sample), too much interfering semantic information may be introduced, blurring the ship's features and confusing the model in the early training process. This module selectively applies Mixup to specific areas through a dynamic area-level enhancement strategy. The obtained mixed image retains the characteristics of the target object and reduces background interference, which is adapted to the specific needs of the surface target detection task.

[0052] As an optional implementation mode, the expression of the dynamic fusion ratio is:

[0053] in, is the dynamic fusion ratio, is the initial mixing ratio, Final mixing ratio, t is the number of iterations corresponding to the target round, T is the total number of training iterations.

[0054] In this embodiment, a dynamic fusion ratio is obtained by the above formula, and the fusion ratio is gradually adjusted as the training progresses to gradually increase the difficulty of data enhancement.

[0055] Experimental Section The embodiments of the present invention use two datasets, namely the SeaShips and SMD datasets; the SeaShips dataset is a resource widely used for maritime target detection, including 7,000 images with a resolution of 1920×1080, covering six categories: general cargo ships, bulk carriers, ore carriers, fishing vessels, container ships and passenger ships; the dataset is divided into a training set (5,500 images) and a test set (750 images).

[0056] The categories of the Singapore Maritime Dataset (SMD) include ferry, buoy, vessel / ship, speedboat, boat, kayak, sailboat, bird / aircraft, other and swimmer (the swimmer category does not appear in the dataset); the training set contains 5455 images and the test set contains 1346 images; in the experiment, 10% of the training images are randomly selected as labeled data, and the remaining images are removed from the labels as unlabeled data.

[0057] The evaluation index of the present invention is to use the mean average precision (mAP) as the evaluation index. mAP quantifies the network performance by measuring the area under the precision-recall curve, combining the precision (P) and recall rate (R). A higher mAP value indicates a higher detection accuracy. The mean average precision (AP) calculation formula is:

[0058] in, is the accuracy on the precision-recall curve. The AP of all categories is averaged to obtain the calculation formula of mAP as follows:

[0059] in, is the number of categories, It is The average precision of the class.

[0060] The experiments conducted on the two data sets of the embodiments of the present invention were carried out on 4 NVIDIA GeForce RTX 3090 graphics cards. Six images (2 labeled images and 4 unlabeled images) were loaded on each GPU. The training was performed for 50,000 iterations, and the model used an SGD optimizer with a constant learning rate of 0.01, a momentum of 0.9, and a weight decay of 0.0001. In particular, the standard GaussianMixture model in the sklearn library was used for Gaussian mixture model (GMM) fitting, the threshold c was set to 2, the initial mixing ratio was 0.9, and the final mixing ratio was set to 0.5.

[0061] The experimental results on two maritime target detection datasets, SeaShips and SMD, are shown in Tables 1 and 2. It can be seen that the method proposed in the embodiment of the present invention is significantly better than the existing advanced methods; on the SeaShips dataset, the proposed method is 5.4% higher than the baseline model mAP and 3.4% higher than the advanced method; on the SMD dataset, the proposed method is 2% higher than the baseline model mAP and 1.5% higher than the advanced method.

[0062] Table 1 Experimental results of different methods on the Seaships dataset

[0063] Table 2 Experimental results of different methods on SMD dataset

[0064] Figure 3 is a schematic diagram of qualitative analysis results on Seaships and SMD datasets according to an embodiment of the present invention, Figure 3 The qualitative analysis results on the Seaships and SMD datasets are shown. The first two rows show the results of the SMD dataset, and the last two rows are the results of the SeaShips dataset. Boxes of different colors are used to represent the detection results. The visualization diagrams of the detection results of different semi-supervised methods are compared. In the comparison models (such as ARSL), there are problems to varying degrees, including false detection in the background, overlap between targets, inaccurate positioning of the bounding box, and low confidence scores. Relatively speaking, the method proposed in the embodiment of the present invention can generate a more accurate target bounding box with a higher confidence score, while significantly reducing the false detection rate, and achieving significant improvements in the accuracy and reliability of target detection.

[0065] In order to further verify the independent contribution of each module in the model of the embodiment of the present invention, the embodiment of the present invention carried out an ablation experiment on the Seaships dataset, and the results are detailed in Table 3; the experimental results show that the DAPF module achieves a significant improvement of 5.1% in the mAP indicator compared with the baseline model, and the DRmix module also improves the mAP by 3.9% compared with the baseline model; these data fully demonstrate the effectiveness of the depth-aware pseudo-label filtering (DAPF) strategy and dynamic region mixing (DRMix) data augmentation technology proposed in the embodiment of the present invention in the model.

[0066] Table 3 Ablation experiment results

[0067] The effectiveness of DAPF and DRmix is ​​verified respectively. In order to intuitively demonstrate the effect of DAPF, the embodiment of the present invention uses different classification thresholds to compare false positives (FP) and true positives (TP) with the baseline model, and draws a statistical result graph. Figure 4 is a statistical graph of false positive rate and true positive rate of the depth perception pseudo label filtering strategy according to an embodiment of the present invention, such as Figure 4 As shown in the figure, the DAPF method achieved a significant reduction in FP compared to the baseline model, with the maximum reduction reaching 36.5%; Figure 5 are heat maps corresponding to different image enhancement methods according to embodiments of the present invention, such as Figure 5As shown, the visual comparison of dynamic region mixing (DRMix) augmentation (first row) and Mixup (second row), by comparing the results of detection reasoning with the Grad-CAM heatmaps of the two enhanced images, it can be clearly seen that the heatmap produced by the DRmix method is more accurate and can more effectively highlight the target area.

[0068] In an embodiment of the present invention, a training set of images containing surface target objects is obtained, wherein the training set images include labeled images and unlabeled images; a depth estimation model is used to perform depth estimation on each image in the training set images to obtain a depth map of each image, wherein the depth map of each image includes a depth value of each pixel; a height value of a target frame of each target object of each category in a target labeled image is extracted according to the category of the target object in the training set image; when the training set images are subjected to target round processing, target detection is performed on the target unlabeled image based on the teacher model to generate a pseudo label for each target object in the unlabeled image. The pseudo-label includes the detection frame and category of each target object; the height value and depth value of the target frame of each target object of each category in the labeled image are fitted by the initial two-dimensional Gaussian distribution model to obtain a target two-dimensional Gaussian distribution model, wherein the target two-dimensional Gaussian distribution model includes the joint distribution parameters of each category of target objects, and the joint distribution parameters include the mean vector of the height value and the depth value of each category of target objects and the covariance matrix describing the relationship between the height value and the depth value of each category of target objects; the height value and the depth value corresponding to the pseudo-label of each target object in each unlabeled image are calculated, and the height value and the depth value corresponding to the pseudo-label of each target object ... The Mahalanobis distance between the target two-dimensional Gaussian distribution models of the categories is used to determine the target result of the pseudo label of each target object in each unlabeled image based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object; based on the target result, any unlabeled image of the target round is fused with any unlabeled image of the previous round to obtain a fused new image; the new image and the labeled image are input into the student model to obtain the category and position of the target object of the new image and the category and position of the target object of the labeled image, and the classification loss and regression loss corresponding to the new image and the classification loss and regression loss corresponding to the labeled image are calculated; the weights of the teacher model are updated based on the parameters of the student model of the target round, and the iteration is repeated to obtain the category and position of the target object of the new image in the last round and the category and position of the target object of the labeled image, which solves the technical problem of low efficiency of the existing semi-supervised target detection technology applied to target detection on the water surface, and achieves the technical effect of using only a small amount of annotated water surface data set and a large number of unlabeled real water surface images, through deep perception pseudo label filtering and dynamic region hybrid augmentation strategy, to train the semi-supervised network based on the teacher-student architecture, effectively improving the detection performance of the model in complex maritime environments and its generalization ability.

[0069] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0070] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0072] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0073] In addition, each functional unit in each embodiment of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0074] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A semi-supervised surface target detection method, characterized in that: include: Acquire a training set of images containing surface target objects, wherein the training set images include labeled images and unlabeled images; Using the depth estimation model to perform depth estimation on each image in the training set of images to obtain a depth map of each image, wherein the depth map of each image includes a depth value of each pixel; According to the category of the target object in the training set image, the height value of the target box of each target object of each category in the target labeled image is extracted; When the training set images are processed for the target round, target detection is performed on the target unlabeled images based on the teacher model to generate pseudo labels for each target object in the unlabeled images, where the pseudo labels include the detection box and category of each target object; Fitting the height value and depth value of the target frame of each target object of each class in the labeled image through the initial two-dimensional Gaussian distribution model to obtain a target two-dimensional Gaussian distribution model, wherein the target two-dimensional Gaussian distribution model includes joint distribution parameters of each class of target objects, and the joint distribution parameters include the mean vector of the height value and the depth value of each class of target objects and the covariance matrix describing the relationship between the height value and the depth value of each class of target objects; Calculate the Mahalanobis distance between the height value and the depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object, and determine the target result of the pseudo label of each target object in each unlabeled image based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object; Based on the target result, any unlabeled image of the target round is fused with any unlabeled image of the previous round to obtain a fused new image; Input the new image and the labeled image into the student model, obtain the category and position of the target object of the new image and the category and position of the target object of the labeled image, calculate the classification loss and regression loss corresponding to the new image, and the classification loss and regression loss corresponding to the labeled image; The weights of the teacher model are updated based on the parameters of the student model of the target round, and the iterations are repeated to obtain the categories and positions of the target objects in the new images of the final round and the categories and positions of the target objects in the labeled images.

2. The method according to claim 1, characterized in that The height value and depth value of the target frame of each target object of each class of the labeled image are fitted by the initial two-dimensional Gaussian distribution model, and the expression of the target two-dimensional Gaussian distribution model is obtained as follows: in, For the i The target two-dimensional Gaussian distribution model of the category, For the i The height value of the target box of the category, For the i The median depth value on the lower edge of the target box of the category, is the mean vector of height and depth of each type of target object, is the covariance matrix of the relationship between the height and depth variables for each class of target objects.

3. The method according to claim 2, characterized in that The expression for calculating the Mahalanobis distance between the height value and the depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object is: in, For each unlabeled image j The Mahalanobis distance between the height and depth values ​​corresponding to the pseudo labels of the target objects and the target two-dimensional Gaussian distribution model of the category of each target object, For each unlabeled image j The height and depth values ​​corresponding to the pseudo labels of the target objects, represents transpose, Represents the inverse matrix.

4. The method according to claim 3, characterized in that The method of determining a target result of a pseudo label for each target object in each unlabeled image based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object includes: If the Mahalanobis distance is greater than the square of the category screening threshold corresponding to each target object in each unlabeled image, the pseudo label of each target object in each unlabeled image is removed; If the Mahalanobis distance is less than or equal to the square of the category screening threshold corresponding to each target object in each unlabeled image, the pseudo label of each target object in each unlabeled image is retained.

5. The method according to claim 4, characterized in that The expression of fusing any unlabeled image of the target round with any unlabeled image of the previous round to obtain a fused new image is: +(1 ) (0.5 +0.5 )+ in, After fusion New images, is the mask of each target object in any unlabeled image of the target wheel, is the mask of each target object in any unlabeled image in the previous round, is the mth unlabeled image in the nth round, is the mth unlabeled image in the n-1th round, is the dynamic fusion ratio, is the number of unlabeled images in the target round and any unlabeled image in the previous round in the mask M1 area. , The proportion of fusion, is the number of unlabeled images in the target round and any unlabeled image in the previous round in the mask M2 area. The proportion of fusion, (1 ) (0.5 +0.5 ) is the area outside the mask M1 and M2 area. Any unlabeled image of the target round and any unlabeled image of the previous round are fused at a ratio of 0.

5. In the overlapping area of ​​masks M1 and M2, any unlabeled image of the target round is randomly fused in the area of ​​M1, or any unlabeled image of the previous round is fused in the area of ​​M2.

6. The method according to claim 5, characterized in that The expression of the dynamic fusion ratio is: in, is the dynamic fusion ratio, is the initial mixing ratio, Final mixing ratio, t is the number of iterations corresponding to the target round, T is the total number of training iterations.

7. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

8. A computer-readable storage medium, characterized in that Computer executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

9. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the method of claim 1 when being executed.

Images