Semi-supervised method for surface target detection

Abstract

The present invention discloses a method for detecting surface targets based on semi-supervision. The method comprises: obtaining a training set of images containing surface target objects, wherein the training set images comprise labeled images and unlabeled images; performing depth estimation on each image in the training set images using a depth estimation model to obtain a depth map of each image, wherein the depth map of each image comprises a depth value for each pixel; extracting the height value of the target frame of each target object of each category in the target labeled image according to the category of the target object in the training set image; when the training set images undergo target round processing, performing target detection on the target unlabeled image based on the teacher model to generate a pseudo label for each target object in the unlabeled image, wherein the pseudo label comprises a detection frame and category for each target object. The present invention solves the technical problem of low efficiency of target detection on the surface of water when the existing semi-supervised target detection technology is applied.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a semi-supervised water surface target detection method. Background Art

[0002] Surface object detection is a core task in the field of surface vision. It aims to accurately identify and locate surface targets by performing high-precision, object-level detection on camera-captured images. The accuracy of this technology is crucial for decision-making for unmanned surface vessels (USVs), as it determines the quality of target information acquired by USVs, including key data such as the number of targets, their categories, sizes, shapes, and precise locations. Using surface object detection technology, USVs can more efficiently complete a range of critical tasks, such as autonomous navigation, water quality testing, and water rescue. Therefore, continued and in-depth research in surface object detection technology is of irreplaceable importance for the technological advancement and widespread application of USVs.

[0003] In recent years, deep learning, with its powerful fitting capabilities, has made significant progress in the field of object detection and has gradually become the mainstream method in this field. However, deep learning-based detection models typically contain a large number of trainable parameters, which means that training the models in a fully supervised manner requires a large amount of labeled data. Due to the diversity of object categories and the complexity of scenes, manual object labeling is both time-consuming and costly. Therefore, semi-supervised methods, as an effective means to significantly reduce the cost of manual labeling, have attracted increasing attention from scholars. Although semi-supervised object detection methods have achieved remarkable results in terrestrial scenarios, their application in surface scenarios still faces the following challenges and limitations.

[0004] Due to these factors, current research on semi-supervised object detection primarily focuses on terrestrial environments. In contrast, research and application of semi-supervised object detection in aquatic environments have been particularly slow. Given the uniqueness and complexity of the aquatic environment, as well as the broad application prospects of USVs, we are conducting a more in-depth and systematic study of semi-supervised object detection on aquatic environments to promote the continued development and practical application of USV technology. Summary of the Invention

[0005] The embodiment of the present invention provides a semi-supervised surface target detection method, which at least solves the technical problem of low efficiency of target detection on the water surface when the existing semi-supervised target detection technology is applied.

[0006] According to one aspect of an embodiment of the present invention, a semi-supervised surface target detection method is provided. The method may include: obtaining a training set image containing a surface target object, wherein the training set image includes a labeled image and an unlabeled image; using a depth estimation model to perform depth estimation on each image in the training set image to obtain a depth map of each image, wherein the depth map of each image includes a depth value of each pixel; extracting the height value of the target box of each target object of each category in the target labeled image according to the category of the target object in the training set image; when the training set image is subjected to target round processing, target detection is performed on the target unlabeled image based on the teacher model to generate a pseudo label for each target object in the unlabeled image, wherein the pseudo label includes a detection box and category of each target object; fitting the height value and depth value of the target box of each target object of each category in the labeled image through an initial two-dimensional Gaussian distribution model to obtain a target two-dimensional Gaussian distribution model, wherein the target two-dimensional Gaussian distribution model includes a joint distribution parameter of each category of target objects, and the joint distribution parameter includes a mean vector of the height value and depth value of each category of target objects. And a covariance matrix describing the relationship between the height value and depth value of each category of target objects; calculating the Mahalanobis distance between the height value and depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object, and determining the target result of the pseudo label of each target object in each unlabeled image based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object; based on the target result, fusing any unlabeled image of the target round with any unlabeled image of the previous round to obtain a fused new image; inputting the new image and the labeled image into the student model to obtain the category and position of the target object of the new image and the category and position of the target object of the labeled image, calculating the classification loss and regression loss corresponding to the new image, and the classification loss and regression loss corresponding to the labeled image; updating the weights of the teacher model based on the parameters of the student model of the target round, repeating the iteration, and obtaining the category and position of the target object of the new image of the last round and the category and position of the target object of the labeled image.

[0007] Optionally, the height value and depth value of the target box of each target object of each class in the labeled image are fitted by the initial two-dimensional Gaussian distribution model, and the expression of the target two-dimensional Gaussian distribution model is obtained as follows:

[0008]

[0009] in, For the i The target two-dimensional Gaussian distribution model of the category, For the i The height value of the target box of the category, For the iThe median depth value on the lower edge of the target box of the category, is the mean vector of height and depth of each type of target object, is the covariance matrix of the relationship between the height and depth variables of each class of target objects.

[0010] Optionally, the expression for calculating the Mahalanobis distance between the height value and the depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object is:

[0011]

[0012] in, For each unlabeled image j The Mahalanobis distance between the height and depth values ​​corresponding to the pseudo labels of the target objects and the target two-dimensional Gaussian distribution model of the category of each target object, For each unlabeled image j The height and depth values ​​corresponding to the pseudo labels of the target objects, represents transpose, Indicates taking the inverse matrix.

[0013] Optionally, determining a target result of the pseudo-label of each target object in each unlabeled image based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object includes: if the Mahalanobis distance is greater than the square of the category screening threshold corresponding to each target object in each unlabeled image, then removing the pseudo-label of each target object in each unlabeled image; if the Mahalanobis distance is less than or equal to the square of the category screening threshold corresponding to each target object in each unlabeled image, then retaining the pseudo-label of each target object in each unlabeled image.

[0014] Optionally, the expression for obtaining a fused new image by fusing any unlabeled image of the target round with any unlabeled image of the previous round is:

[0015] +(1 ) (0.5 +0.5 )+

[0016] in, After fusion New images, is the mask of each target object in any unlabeled image of the target wheel, is the mask of each target object in any unlabeled image in the previous round, is the mth unlabeled image in the nth round, is the mth unlabeled image in the n-1th round, is the dynamic fusion ratio, is the area of ​​the target round in the mask M1 where any unlabeled image and any unlabeled image in the previous round occupy 、 The proportion of fusion, The number of unlabeled images in the target round and any unlabeled image in the previous round in the mask M2 area is The proportion of fusion, (1 ) (0.5 +0.5 ) is the area outside the mask M1 and M2 area. Any unlabeled image of the target round and any unlabeled image of the previous round are fused at a ratio of 0.5. In the overlapping area of ​​masks M1 and M2, any unlabeled image of the target round is randomly fused in the area of ​​M1 or any unlabeled image of the previous round is fused in the area of ​​M2.

[0017] Optionally, the expression of the dynamic fusion ratio is:

[0018]

[0019] in, is the dynamic fusion ratio, is the initial mixing ratio, Final mixing ratio, t is the number of iterations corresponding to the target round, T is the total number of training iterations.

[0020] Beneficial effects of the present invention:

[0021] (1) The present invention adopts a depth-aware pseudo-label filtering strategy. By utilizing the depth and scale information of the target, a joint distribution model is established to judge the reliability of the pseudo-labels and filter out the pseudo-labels that deviate from the distribution center, thereby reducing the interference of false detection in the pseudo-labels, improving the overall quality of the pseudo-labels, and providing better training data for the student model.

[0022] (2) The dynamic region mixing enhancement method proposed in this invention is based on the region-level mixing operation, which adaptively adjusts the mixing ratio at the image region level to maintain the integrity and semantic consistency of the target. It is particularly suitable for maritime environments with complex backgrounds. This method dynamically adjusts the mixing ratio during the training process, making the data enhancement gradually more difficult, thereby improving the generalization ability of the model in different backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0024] Figure 1 is a flow chart of a semi-supervised surface target detection method according to an embodiment of the present invention;

[0025] Figure 2 is a framework diagram of a semi-supervised surface target detection method according to an embodiment of the present invention;

[0026] Figure 3 is a schematic diagram of qualitative analysis results on Seaships and SMD datasets according to an embodiment of the present invention;

[0027] Figure 4 is a statistical graph of false positive rate and true positive rate of the depth-aware pseudo-label filtering strategy according to an embodiment of the present invention;

[0028] Figure 5 1 is a heat map corresponding to different image enhancement methods according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or devices.

[0031] Example 1

[0032] According to an embodiment of the present invention, a semi-supervised surface target detection method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system comprising at least one set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] Figure 1 is a flow chart of a semi-supervised surface target detection method according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0034] Step S101 : obtaining a training set of images containing surface target objects, wherein the training set of images includes labeled images and unlabeled images.

[0035] In this embodiment, a training data set is obtained. The training set is an image containing a target object on the water surface. The target object can be a general cargo ship, a bulk carrier, an ore carrier, a fishing boat, a container ship, a passenger ship, etc. This is only an example and is not limited to a specific example. The training set images include labeled images and unlabeled images. The labeled images are images in which the target objects in the images are marked with detection boxes and categories, and the unlabeled images are images in which the target objects in the images are not marked. Figure 2 is a framework diagram of a semi-supervised surface target detection method according to an embodiment of the present invention, such as Figure 2 As shown, Figure 2 The unlabeled data in the training dataset are unlabeled images, and the labeled data are labeled images. All images in the training dataset are resized to a fixed size required by the model, such as 1333×800, to ensure input consistency and facilitate subsequent processing and computational efficiency.

[0036] Step S102: Use a depth estimation model to perform depth estimation on each image in the training set of images to obtain a depth map of each image, wherein the depth map of each image includes a depth value of each pixel.

[0037] In this embodiment, a depth estimation model is used to perform depth estimation on labeled images and unlabeled images in a training set to obtain a depth value of each pixel in each image, that is, a depth map of each image.

[0038] Step S103 , extracting the height value of the target box of each target object of each category in the target labeled image according to the category of the target object in the training set image.

[0039] In this embodiment, the height value of the target box of each target object of each category in the target labeled image is extracted according to the category of the target object in the training set image.

[0040] Step S104: When the training set images are subjected to target round processing, target detection is performed on the target unlabeled images based on the teacher model to generate pseudo labels for each target object in the unlabeled images, wherein the pseudo labels include the detection box and category of each target object.

[0041] In this embodiment, when the training set images are processed, the number of training rounds of the target detection network is 3 rounds, which is only used as an example and is not limited to a specific number. It can also be the 5th round, the 6th round, etc. Figure 2 As shown in the figure, target detection is performed on the target unlabeled images according to the teacher model to generate pseudo labels for each target object in the unlabeled images, where the pseudo labels include the detection box and category of each target object. It should be noted that there are 4 target unlabeled images, the detection box of each target object is the length and width of the target object, and the category of each target object is which category the target object belongs to. For example, the label of a general cargo ship is 1.

[0042] Step S105, fitting the height value and depth value of the target box of each target object of each class in the labeled image through the initial two-dimensional Gaussian distribution model to obtain a target two-dimensional Gaussian distribution model, wherein the target two-dimensional Gaussian distribution model includes the joint distribution parameters of the target objects of each class, and the joint distribution parameters include the mean vector of the height value and the depth value of the target object of each class and the covariance matrix describing the relationship between the height value and the depth value of the target object of each class.

[0043] In this embodiment, the height value and depth value of the target box of each target object of each category in the labeled image are fitted to the initial two-dimensional Gaussian distribution model of the unfused data to obtain a fitted target two-dimensional Gaussian distribution model, wherein the target two-dimensional Gaussian distribution model includes the joint distribution parameters of the target objects of each category, and the joint distribution parameters include the mean vector of the height value and the depth value of the target objects of each category and the covariance matrix describing the relationship between the height value and the depth value of the target objects of each category.

[0044] Step S106, calculate the Mahalanobis distance between the height value and depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object, and determine the target result of the pseudo label of each target object in each unlabeled image based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object.

[0045] In this embodiment, the Mahalanobis distance between the height value and depth value corresponding to the pseudo-label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object is calculated, and the size of the Mahalanobis distance and the square of the category screening threshold corresponding to each target object is determined to determine whether the pseudo-label of each target object in each unlabeled image is retained or eliminated. The Mahalanobis distance measures the similarity between the pseudo-label and the known distribution, that is, the degree of deviation: if the scale-depth coordinate of the pseudo-label falls near the distribution center of the target two-dimensional Gaussian distribution model, it is considered to have high reliability and is retained for training. If the scale-depth coordinate of the pseudo-label deviates from the distribution center of the target two-dimensional Gaussian distribution model (beyond the range of two standard deviations), it is considered to have a risk of false detection and is eliminated.

[0046] Step S107: Based on the target result, any unlabeled image of the target round is fused with any unlabeled image of the previous round to obtain a fused new image.

[0047] In this embodiment, according to the target result, when the target round is the third round, any one of the four unlabeled images in the third round is fused with any one of the four unlabeled images in the previous round to obtain a fused new image, such as Figure 2 Dynamic area mixing in .

[0048] Step S108: Input the new image and the labeled image into the student model to obtain the category and position of the target object in the new image and the category and position of the target object in the labeled image, and calculate the classification loss and regression loss corresponding to the new image, as well as the classification loss and regression loss corresponding to the labeled image.

[0049] In this embodiment, Figure 2 As shown, the new image and the labeled image are input into the student model to obtain the category and position of the target object of the new image and the category and position of the target object of the labeled image, and the classification loss and regression loss corresponding to the new image and the classification loss and regression loss corresponding to the labeled image are calculated.

[0050] Step S109: update the weights of the teacher model based on the parameters of the student model of the target round, repeat the iteration, and obtain the category and position of the target object of the new image and the category and position of the target object of the labeled image in the last round.

[0051] In this embodiment, after the student model completes one round of training, its weights are updated to the teacher model through exponential moving average to ensure that the teacher model can dynamically adapt to the new pseudo-label distribution, and the iteration is repeated until the category and position of the target object of the new image in the last round and the category and position of the target object of the labeled image.

[0052] The above method of this embodiment is further introduced below.

[0053] As an optional embodiment, in step S105, the height value and depth value of the target box of each target object of each class in the labeled image are fitted by the initial two-dimensional Gaussian distribution model, and the expression of the target two-dimensional Gaussian distribution model is obtained as follows:

[0054]

[0055] in, For the i The target two-dimensional Gaussian distribution model of the category, For the i The height value of the target box of the category, For the i The median depth value on the lower edge of the target box of the category, is the mean vector of height and depth of each type of target object, is the covariance matrix of the relationship between the height and depth variables of each class of target objects.

[0056] In this embodiment, the height value and depth value of the target frame of each target object of each category in the labeled image are put into the initial two-dimensional Gaussian distribution model for fitting to obtain a target two-dimensional Gaussian distribution model.

[0057] As an optional embodiment, in step S106, the expression for the Mahalanobis distance between the height value and depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object is calculated as follows:

[0058]

[0059] in, For each unlabeled image j The Mahalanobis distance between the height and depth values ​​corresponding to the pseudo labels of the target objects and the target two-dimensional Gaussian distribution model of the category of each target object, For each unlabeled image j The height and depth values ​​corresponding to the pseudo labels of the target objects, represents transpose, Indicates taking the inverse matrix.

[0060] In this embodiment, the Mahalanobis distance between the height value and depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object is calculated by the above formula.

[0061] As an optional embodiment, step S106, determining the target result of the pseudo-label of each target object in each unlabeled image based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object, includes: if the Mahalanobis distance is greater than the square of the category screening threshold corresponding to each target object in each unlabeled image, then eliminating the pseudo-label of each target object in each unlabeled image; if the Mahalanobis distance is less than or equal to the square of the category screening threshold corresponding to each target object in each unlabeled image, then retaining the pseudo-label of each target object in each unlabeled image.

[0062] In this embodiment, the size of the Mahalanobis distance and the square of the category screening threshold corresponding to each target object is determined. When the Mahalanobis distance is greater than the square of the category screening threshold corresponding to each target object in each unlabeled image, the pseudo-label of each target object in each unlabeled image is eliminated. When the Mahalanobis distance is less than or equal to the square of the category screening threshold corresponding to each target object in each unlabeled image, the pseudo-label of each target object in each unlabeled image is retained.

[0063] As an optional embodiment, in step S107, any unlabeled image of the target round is fused with any unlabeled image of the previous round to obtain the expression of the fused new image:

[0064] +(1 ) (0.5 +0.5 )+

[0065] in, After fusion New images, is the mask of each target object in any unlabeled image of the target wheel, is the mask of each target object in any unlabeled image in the previous round, is the mth unlabeled image in the nth round, is the mth unlabeled image in the n-1th round, is the dynamic fusion ratio, is the area of ​​the target round in the mask M1 where any unlabeled image and any unlabeled image in the previous round occupy 、 The proportion of fusion, The number of unlabeled images in the target round and any unlabeled image in the previous round in the mask M2 area is The proportion of fusion, (1 ) (0.5 +0.5 ) is the area outside the mask M1 and M2 area. Any unlabeled image of the target round and any unlabeled image of the previous round are fused at a ratio of 0.5. In the overlapping area of ​​masks M1 and M2, any unlabeled image of the target round is randomly fused in the area of ​​M1 or any unlabeled image of the previous round is fused in the area of ​​M2.

[0066] In this embodiment, through the above formula, any unlabeled image of the target round is fused with any unlabeled image of the previous round to obtain a new image. The number of new images is 4. For example, the first unlabeled image of the third round is fused with the third unlabeled image of the second round, as shown in 2. The dynamic area mixing enhancement module, for example, when a ship (positive sample) is mixed with a complex nearshore background (negative sample), too much interfering semantic information may be introduced, blurring the ship features, and thus confusing the model in the early training process. This module selectively applies Mixup to specific areas through a dynamic area-level enhancement strategy. The obtained mixed image retains the features of the target object and reduces background interference, adapting to the specific requirements of the surface target detection task.

[0067] As an optional embodiment, the expression of the dynamic fusion ratio is:

[0068]

[0069] in, is the dynamic fusion ratio, is the initial mixing ratio, Final mixing ratio, t is the number of iterations corresponding to the target round, T is the total number of training iterations.

[0070] In this embodiment, a dynamic fusion ratio is obtained by the above formula, and the fusion ratio is gradually adjusted as the training progresses to gradually increase the difficulty of data enhancement.

[0071] Experimental part

[0072] This embodiment of the present invention uses two datasets, namely the SeaShips and SMD datasets. The SeaShips dataset is a resource widely used for maritime target detection, including 7,000 images with a resolution of 1920×1080, covering six categories: general cargo ships, bulk carriers, ore carriers, fishing vessels, container ships, and passenger ships. The dataset is divided into a training set (5,500 images) and a test set (750 images).

[0073] The categories of the Singapore Maritime Dataset (SMD) include ferry, buoy, vessel / ship, speedboat, boat, kayak, sailboat, bird / aircraft, other, and swimmer (the swimmer category does not appear in the dataset); the training set contains 5455 images and the test set contains 1346 images; in the experiment, 10% of the training images are randomly selected as labeled data, and the remaining images are removed from the labels as unlabeled data.

[0074] The evaluation metric of this invention is average precision (mAP). mAP quantifies network performance by measuring the area under the precision-recall curve. It combines precision (P) and recall (R). A higher mAP value indicates higher detection accuracy. The formula for calculating average precision (AP) is:

[0075]

[0076] in, is the accuracy on the precision-recall curve. Taking the average AP of all categories, the calculation formula for mAP is as follows:

[0077]

[0078] in, is the number of categories, It is The average precision of the class.

[0079] The experiments conducted on the two data sets in the embodiments of the present invention were performed on 4 NVIDIA GeForce RTX 3090 graphics cards. Six images (2 labeled images and 4 unlabeled images) were loaded on each GPU, and training was performed for 50,000 iterations. The model used an SGD optimizer with a constant learning rate of 0.01, a momentum of 0.9, and a weight decay of 0.0001. In particular, the standard GaussianMixture model in the sklearn library was used for Gaussian mixture model (GMM) fitting, the threshold c was set to 2, the initial mixing ratio was 0.9, and the final mixing ratio was set to 0.5.

[0080] Experimental results on two maritime object detection datasets, SeaShips and SMD, are shown in Tables 1 and 2. It can be seen that the method proposed in this embodiment significantly outperforms existing advanced methods. On the SeaShips dataset, the proposed method improves mAP by 5.4% over the baseline model and by 3.4% over the state-of-the-art methods. On the SMD dataset, the proposed method improves mAP by 2% over the baseline model and by 1.5% over the state-of-the-art methods.

[0081] Table 1 Experimental results of different methods on the Seaships dataset

[0082]

[0083] Table 2 Experimental results of different methods on the SMD dataset

[0084]

[0085] Figure 3 is a schematic diagram of the qualitative analysis results on the Seaships and SMD datasets according to an embodiment of the present invention, Figure 3 Qualitative analysis results on the Seaships and SMD datasets are shown. The first two rows show the results of the SMD dataset, and the last two rows are the results of the SeaShips dataset. Different colored boxes are used to represent the detection results. A visualization of the detection results of different semi-supervised methods is compared. In the comparison model (such as ARSL), there are problems to varying degrees, including false detection in the background, overlap between targets, inaccurate positioning of the bounding box, and low confidence scores. In contrast, the method proposed in the embodiment of the present invention can generate more accurate target bounding boxes with higher confidence scores, while significantly reducing the false detection rate, and achieving significant improvements in the accuracy and reliability of target detection.

[0086] To further verify the independent contributions of each module in the model of the present invention, an ablation experiment was conducted on the Seaships dataset. The results are detailed in Table 3. The experimental results show that the DAPF module achieves a significant improvement of 5.1% in the mAP indicator compared to the baseline model, while the DRmix module also improves mAP by 3.9% compared to the baseline model. These data fully demonstrate the effectiveness of the depth-aware pseudo-label filtering (DAPF) strategy and dynamic region mixing (DRMix) data augmentation technology proposed in the present invention in the model.

[0087] Table 3 Ablation experiment results

[0088]

[0089] The effectiveness of DAPF and DRmix is ​​verified respectively. In order to intuitively demonstrate the effect of DAPF, the embodiment of the present invention uses different classification thresholds to compare false positives (FP) and true positives (TP) with the baseline model, and draws a statistical result graph. Figure 4 is a statistical diagram of the false positive rate and true positive rate of the depth-aware pseudo-label filtering strategy according to an embodiment of the present invention, as shown in FIG. Figure 4 As shown in Figure 2, the DAPF method achieved a significant reduction in FP compared to the baseline model, with the maximum reduction reaching 36.5%; Figure 5is a heat map corresponding to different image enhancement methods according to an embodiment of the present invention, such as Figure 5 As shown, the visual comparison of dynamic region mixing (DRMix) augmentation (first row) and Mixup (second row), by comparing the results of detection reasoning with the Grad-CAM heatmaps of the two enhanced images, it can be clearly seen that the heatmap generated by the DRmix method is more accurate and can more effectively highlight the target area.

[0090] In an embodiment of the present invention, a training set of images containing surface target objects is obtained, wherein the training set images include labeled images and unlabeled images; a depth estimation model is used to perform depth estimation on each image in the training set images to obtain a depth map of each image, wherein the depth map of each image includes a depth value of each pixel; the height value of the target frame of each target object of each category in the target labeled image is extracted according to the category of the target object in the training set image; when the training set images are subjected to target round processing, target detection is performed on the target unlabeled image based on the teacher model to generate a pseudo label for each target object in the unlabeled image. Among them, the pseudo label includes the detection frame and category of each target object; the height value and depth value of the target frame of each target object of each category in the labeled image are fitted by the initial two-dimensional Gaussian distribution model to obtain the target two-dimensional Gaussian distribution model, wherein the target two-dimensional Gaussian distribution model includes the joint distribution parameters of each category of target objects, and the joint distribution parameters include the mean vector of the height value and depth value of each category of target objects and the covariance matrix describing the relationship between the height value and depth value of each category of target objects; the height value and depth value corresponding to the pseudo label of each target object in each unlabeled image are calculated, and the height value and depth value corresponding to the pseudo label of each target object are compared with the height value and depth value of each target object. The Mahalanobis distance between the target two-dimensional Gaussian distribution models of the categories is calculated. Based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object, the target result of the pseudo label of each target object in each unlabeled image is determined. Based on the target result, any unlabeled image of the target round is fused with any unlabeled image of the previous round to obtain a fused new image. The new image and the labeled image are input into the student model to obtain the category and position of the target object in the new image and the category and position of the target object in the labeled image, and the classification loss and regression loss corresponding to the new image and the labeled image are calculated. The weights of the teacher model are updated based on the parameters of the student model of the target round. The iteration is repeated to obtain the category and position of the target object in the new image and the category and position of the target object in the labeled image in the final round. This solves the technical problem of low efficiency of existing semi-supervised target detection technology when applied to target detection on the water surface. It achieves the technical effect of training the semi-supervised network based on the teacher-student architecture using only a small amount of annotated water surface dataset and a large number of real unlabeled water surface images through depth-aware pseudo-label filtering and dynamic region hybrid augmentation strategy, effectively improving the detection performance and generalization ability of the model in complex maritime environments.

[0091] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0092] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0094] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0095] In addition, the functional units in various embodiments of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0096] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A semi-supervised surface target detection method, characterized in that: include: Acquire a training set of images containing water surface target objects, wherein the training set images include labeled images and unlabeled images; Using the depth estimation model to perform depth estimation on each image in the training set of images to obtain a depth map for each image, wherein the depth map of each image includes the depth value of each pixel; According to the category of the target object in the training set image, the height value of the target box of each target object of each category in the target labeled image is extracted; When the training set images are processed for the target round, target detection is performed on the target unlabeled images based on the teacher model to generate pseudo labels for each target object in the unlabeled images, where the pseudo labels include the detection box and category of each target object; Fitting the height and depth values ​​of the target box of each target object of each class in the labeled image through the initial two-dimensional Gaussian distribution model to obtain a target two-dimensional Gaussian distribution model, wherein the target two-dimensional Gaussian distribution model includes joint distribution parameters of the target objects of each class, and the joint distribution parameters include the mean vector of the height and depth values ​​of the target objects of each class and the covariance matrix describing the relationship between the height and depth values ​​of the target objects of each class; Calculate the Mahalanobis distance between the height and depth values ​​corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object, and determine the target result of the pseudo label of each target object in each unlabeled image based on the Mahalanobis distance and the square of the category screening threshold corresponding to each target object; Based on the target result, any unlabeled image of the target round is fused with any unlabeled image of the previous round to obtain a fused new image; Input the new image and the labeled image into the student model, obtain the category and position of the target object in the new image and the category and position of the target object in the labeled image, and calculate the classification loss and regression loss corresponding to the new image, as well as the classification loss and regression loss corresponding to the labeled image; The weights of the teacher model are updated based on the parameters of the student model of the target round, and the iterations are repeated to obtain the categories and positions of the target objects in the new images of the final round and the categories and positions of the target objects in the labeled images.

2. The method according to claim 1, characterized in that The initial two-dimensional Gaussian distribution model is used to fit the height and depth values ​​of the target frame of each target object of each class in the labeled image, and the expression of the target two-dimensional Gaussian distribution model is obtained as follows: in, For the i The target two-dimensional Gaussian distribution model of the category, For the i The height value of the target box of the category, For the i The median depth value on the lower edge of the target box of the category, is the mean vector of height and depth of each type of target object, is the covariance matrix of the relationship between the height and depth variables of each class of target objects.

3. The method according to claim 2, characterized in that The expression for calculating the Mahalanobis distance between the height value and depth value corresponding to the pseudo label of each target object in each unlabeled image and the target two-dimensional Gaussian distribution model of the category of each target object is: in, For each unlabeled image j The Mahalanobis distance between the height and depth values ​​corresponding to the pseudo labels of the target objects and the target two-dimensional Gaussian distribution model of the category of each target object, For each unlabeled image j The height and depth values ​​corresponding to the pseudo labels of the target objects, represents transpose, Indicates taking the inverse matrix.

4. The method according to claim 3, characterized in that The step of determining a target result of a pseudo label for each target object in each unlabeled image based on the Mahalanobis distance and the square of a category screening threshold corresponding to each target object includes: If the Mahalanobis distance is greater than the square of the category screening threshold corresponding to each target object in each unlabeled image, the pseudo label of each target object in each unlabeled image is removed; If the Mahalanobis distance is less than or equal to the square of the category screening threshold corresponding to each target object in each unlabeled image, the pseudo label of each target object in each unlabeled image is retained.

5. The method according to claim 4, characterized in that The expression for fusing any unlabeled image of the target round with any unlabeled image of the previous round to obtain a fused new image is: +(1 ) (0.5 +0.5 )+ in, After fusion New images, is the mask of each target object in any unlabeled image of the target wheel, is the mask of each target object in any unlabeled image in the previous round, is the mth unlabeled image in the nth round, is the mth unlabeled image in the n-1th round, is the dynamic fusion ratio, is the area of ​​the target round in the mask M1 where any unlabeled image and any unlabeled image in the previous round occupy 、 The proportion of fusion, The number of unlabeled images in the target round and any unlabeled image in the previous round in the mask M2 area is The proportion of fusion, (1 ) (0.5 +0.5 ) is the area outside the mask M1 and M2 area. Any unlabeled image of the target round and any unlabeled image of the previous round are fused at a ratio of 0.

5. In the overlapping area of ​​masks M1 and M2, any unlabeled image of the target round is randomly fused in the area of ​​M1 or any unlabeled image of the previous round is fused in the area of ​​M2.

6. The method according to claim 5, characterized in that The expression of the dynamic fusion ratio is: in, is the dynamic fusion ratio, is the initial mixing ratio, Final mixing ratio, t is the number of iterations corresponding to the target round, T is the total number of training iterations.

7. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

8. A computer-readable storage medium, characterized in that Computer-executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

9. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the method of claim 1 when the instructions are executed.

Images