Semi-supervised rotation target detection method based on local spatial consistency prior information
By constructing local spatial consistency prior information to filter and correct pseudo-labels, the problem of low detection accuracy in remote sensing images is solved, and the model can be trained quickly and detected efficiently.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NORTHWESTERN POLYTECHNICAL UNIV
- Filing Date
- 2023-12-28
- Publication Date
- 2026-06-19
AI Technical Summary
Existing semi-supervised rotating target detection models fail to fully utilize the local spatial consistency information of remote sensing images, resulting in low detection accuracy.
By constructing local spatial consistency prior information, filtering high-confidence pseudo-labels and low-confidence pseudo-labels, merging pseudo-labels using local spatial consistency prior conditions, performing angle correction, iteratively training the student model, and updating the teacher model.
It improved the quality and quantity of pseudo-labels, enhanced the detection accuracy and speed of the model, expanded the training base data, and enabled the student model to converge quickly.
Smart Images
Figure CN117893916B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of rotating target detection technology, and in particular to a semi-supervised rotating target detection method based on local spatial consistency prior information. Background Technology
[0002] With the development of remote sensing technology, more and more remote sensing images are being applied to various industries. When processing large-scale remote sensing images, manual target identification and localization is a time-consuming and labor-intensive task. Using rotating target detection algorithms can automate this task, improving processing efficiency, which is very useful for monitoring targets over a wide area. However, training fully supervised rotating target detection models involves time-consuming, labor-intensive, and expensive data labeling. Recently, semi-supervised object detection (SSOD) has attracted increasing attention. Models can learn from labeled data as well as readily available unlabeled data, greatly alleviating the cost problem of data labeling.
[0003] In semi-supervised object detection, the selection and utilization of pseudo-labels is a crucial issue. One approach is to replace cross-entropy loss with focus loss to address class imbalance in pseudo-labeled boxes, filtering them through a fixed high threshold. However, in practice, selecting this fixed high threshold is challenging; thresholds that are too high or too low can significantly impact the model. Another approach adaptively weights the loss of each pseudo-labeled box using classification scores and proposes using the variance of multiple box-jitter calculations to select reliable pseudo-labels. The problem with this method is that multiple box-jitter calculations increase computation time, slowing down model training and reducing efficiency.
[0004] Both of the above methods are based on semi-supervised object detection in natural scenes, and do not fully consider the characteristics of rotating objects in remote sensing scenes. Therefore, a new semi-supervised rotating object detection model based on the mainstream pseudo-labeling framework was designed. For oriented objects in remote sensing scenes, this model uses adaptive weights to regularize the consistency between each pseudo-label prediction pair based on their orientation differences, and applies global consistency constraints to the many-to-many relationship explicitly established between the pseudo-label set and the prediction set on the target layout of the entire image.
[0005] However, this semi-supervised rotating target detection model, which is based on the mainstream pseudo-labeling framework, only uses the angle and target distribution information unique to remote sensing images to constrain the prediction consistency between the teacher model and the student model. It fails to make full use of the local spatial consistency information on remote sensing images, which results in low detection accuracy of the trained model. Summary of the Invention
[0006] The purpose of this application is to provide a semi-supervised rotating target detection method based on local spatial consistency prior information. This method makes full use of local spatial consistency prior information in remote sensing images, provides more high-quality pseudo-labels for the rotating target detection model to train, and significantly improves the detection accuracy of the trained model.
[0007] To address the aforementioned technical problems, embodiments of this application provide a semi-supervised rotating target detection method based on local spatial consistency prior information, comprising the following steps: using a teacher model to predict an unlabeled image to obtain pseudo-labels corresponding to each target in the unlabeled image; retaining pseudo-labels with a confidence level greater than a first preset threshold to obtain a pseudo-label set; obtaining a low-confidence group based on pseudo-labels with a confidence level less than a second preset threshold in the pseudo-label set, and obtaining a high-confidence group based on pseudo-labels with a confidence level greater than or equal to the second preset threshold in the pseudo-label set; wherein, the second preset threshold is greater than the first preset threshold; constructing local spatial consistency prior conditions based on the consistency relationship between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group, and merging the pseudo-labels in the high-confidence group with the pseudo-labels in the low-confidence group that meet the local spatial consistency prior conditions to obtain a candidate pseudo-label set; performing angle correction on the candidate pseudo-labels in the candidate pseudo-label set; iteratively training a student model based on the unlabeled image and the angle-corrected candidate pseudo-label set; and performing EMA (Exponential Moving Average) calculation based on the trained student model. The teacher model is updated using the average (exponential mean) index; the image to be detected is input into the updated teacher model to determine each target in the image to be detected.
[0008] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described semi-supervised rotating target detection method based on local spatial consistency prior information.
[0009] Embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described semi-supervised rotating target detection method based on local spatial consistency prior information.
[0010] The semi-supervised rotation target detection method based on local spatial consistency prior information provided in this application uses a teacher model to predict unlabeled images, obtaining pseudo-labels corresponding to each target in the unlabeled image. Based on the confidence level of each pseudo-label, pseudo-labels with too low confidence are filtered out through two thresholds, resulting in usable high-confidence pseudo-label groups and low-confidence pseudo-label groups. Using the high-confidence group as a benchmark, local spatial consistency prior conditions are constructed using the local spatial consistency prior information in the image. More high-quality pseudo-labels are mined from the low-confidence group and merged with the high-confidence group to obtain a candidate pseudo-label set, significantly improving the completeness of the pseudo-labels. Subsequently, angle correction is performed on the candidate pseudo-labels in the candidate pseudo-label set, effectively improving the quality of the pseudo-labels. This allows for iterative training of the student model, and the teacher model is updated based on the EMA of the trained student model. The updated teacher model is then used as the detection model for rotating target detection in the image to be detected. The use of local spatial consistency prior information greatly improves the quantity and quality of pseudo-labels, enabling some correctly predicted but low-confidence targets to be added to the training process of the student model, expanding the basic training data of the student model, enabling the student model to converge quickly during training, improving the quality and speed of model training, and significantly improving the detection accuracy and detection speed of the final detection model.
[0011] In some optional embodiments, the angle correction of the candidate pseudo-labels in the candidate pseudo-label set includes: traversing each candidate pseudo-label in the candidate pseudo-label set, determining the target candidate pseudo-label that is closest to the current candidate pseudo-label in the candidate pseudo-label set; if the confidence of the target candidate pseudo-label is greater than the confidence of the current candidate pseudo-label, then replacing the angle corresponding to the current candidate pseudo-label with the angle corresponding to the target candidate pseudo-label; if the confidence of the target candidate pseudo-label is less than or equal to the confidence of the current candidate pseudo-label, then keeping the angle corresponding to the current candidate pseudo-label unchanged. Besides mining more low-confidence pseudo-labels to add to the model training, this application also corrects the angles corresponding to these mined pseudo-labels, making the angles represented by the pseudo-labels more accurate, thereby further improving the accuracy of the angle prediction of the final detection model.
[0012] In some optional embodiments, the local spatial consistency prior condition is specifically a local spatial consistency prior matrix. The step of constructing the local spatial consistency prior condition based on the consistency relationship between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group includes: calculating the distance between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group; determining that two pseudo-labels whose distance is less than or equal to a third preset threshold represent adjacent target spaces; and obtaining a position consistency matrix based on the distance determination result; determining whether the categories corresponding to each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group are the same; and obtaining a category consistency matrix based on the category determination result; calculating the angle difference between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group; determining that two pseudo-labels whose absolute value of the angle difference is less than or equal to a fourth preset threshold represent consistent target angles; and obtaining an angle consistency matrix based on the angle determination result; and obtaining the local spatial consistency prior matrix based on the position consistency matrix, the category consistency matrix, and the angle consistency matrix. This application uses three aspects—spatial location, category, and representation angle—to represent local spatial consistency prior information. It constructs a location consistency matrix, a category consistency matrix, and an angle consistency matrix in sequence to form a local spatial consistency prior matrix. The pseudo-labels of the low-confidence group that can be successfully mined are adjacent to the target space represented by the pseudo-labels in the high-confidence group, have the same category, and have the same target angle, ensuring that the quality of the mined pseudo-labels in the low-confidence group meets the training requirements.
[0013] In some optional embodiments, the distance between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group is calculated using the following formula. Two pseudo-labels whose distance is less than or equal to a third preset threshold are determined to be spatially adjacent. A positional consistency matrix is then obtained based on the distance determination results.
[0014] ;
[0015] ;
[0016] in, Indicates the first in the high confidence group i The x-coordinate of each pseudo-label Indicates the first in the high confidence group i The ordinate of each pseudo-label. Indicates the first in the low confidence group j The x-coordinate of each pseudo-label Indicates the first in the low confidence group j The ordinate of each pseudo-label. Indicates the first in the high confidence group iThe pseudo-label and the first pseudo-label in the low-confidence group j The distance between pseudo-tags The third preset threshold, The position consistency matrix is the first... i Line number j The value of the column, Indicates the first in the high confidence group i The pseudo-label and the first pseudo-label in the low-confidence group j The target spaces represented by the pseudo-labels are adjacent. Indicates the first in the high confidence group i The pseudo-label and the first pseudo-label in the low-confidence group j The target spaces represented by the pseudo-labels are not adjacent.
[0017] In some optional embodiments, the following formula is used to determine whether the category corresponding to each pseudo-label in the high-confidence group is the same as that corresponding to each pseudo-label in the low-confidence group, and a category consistency matrix is obtained based on the category determination result:
[0018] ;
[0019] in, Indicates the first in the high confidence group i The corresponding category of each pseudo-tag Indicates the first in the low confidence group j The category corresponding to each pseudo-label, the The first in the category consistency matrix i Line number j The value of the column, Indicates the first in the high confidence group i The category corresponding to the pseudo-label is the same as the first pseudo-label in the low-confidence group. j The pseudo-tags correspond to the same category. Indicates the first in the high confidence group i The category corresponding to the pseudo-label is the same as the first pseudo-label in the low-confidence group. j The pseudo-tags correspond to different categories.
[0020] In some optional embodiments, the angle difference between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group is calculated using the following formula. Two pseudo-labels whose absolute value of the angle difference is less than or equal to a fourth preset threshold are determined to have consistent target angles. An angle consistency matrix is then obtained based on the angle determination results.
[0021] ;
[0022] in, Indicates the first in the high confidence groupi The corresponding angle of each pseudo-tag Indicates the first in the low confidence group j The angle corresponding to each pseudo-label Indicates the first in the high confidence group i The pseudo-label and the first pseudo-label in the low-confidence group j The absolute value of the angle difference between the pseudo-tags. The fourth preset threshold, The first in the angle consistency matrix i Line number j The value of the column, Indicates the first in the high confidence group i The pseudo-label and the first pseudo-label in the low-confidence group j The target perspective represented by each pseudo-label is consistent. Indicates the first in the high confidence group i The pseudo-label and the first pseudo-label in the low-confidence group j The target perspectives represented by the pseudo-labels are inconsistent.
[0023] In some optional embodiments, the local spatial consistency prior matrix is obtained based on the position consistency matrix, the category consistency matrix, and the angle consistency matrix using the following formula:
[0024] ;
[0025] in, The position consistency matrix is... For the category consistency matrix, The angle consistency matrix is... This indicates element-wise multiplication. Let be the local spatial consistency prior matrix.
[0026] In some optional embodiments, the pseudo-labels in the high-confidence group and the pseudo-labels in the low-confidence group that meet the prior condition of local spatial consistency are merged using the following formula to obtain a candidate pseudo-label set:
[0027] ;
[0028] ;
[0029] in, L This refers to the low-confidence group. Indicates the first in the low confidence group j A pseudo-label, The first in the local spatial consistency prior matrix i Line number j The value of the column,E This represents the set of all pseudo-labels in the low-confidence group that satisfy the prior condition of local spatial consistency. H This refers to the high-confidence group. T This represents the set of candidate pseudo-labels. Attached Figure Description
[0030] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.
[0031] Figure 1 This is a flowchart of a semi-supervised rotating target detection method based on local spatial consistency prior information provided in one embodiment of this application;
[0032] Figure 2 This is a schematic diagram of a model training process provided in one embodiment of this application;
[0033] Figure 3 This is a flowchart of angle correction for candidate pseudo-labels in a candidate pseudo-label set, provided in one embodiment of this application;
[0034] Figure 4 This is a schematic diagram of an angle-corrected pseudo-label provided in one embodiment of this application;
[0035] Figure 5 This is a flowchart provided in one embodiment of the present application, which describes the construction of local spatial consistency prior conditions based on the consistency relationship between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group.
[0036] Figure 6 This is a comparison diagram of the processing of multiple pseudo-tags by different processing methods provided in one embodiment of this application;
[0037] Figure 7 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation
[0038] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.
[0039] In the field of semi-supervised object detection, how to select and utilize pseudo-labels has always been a crucial issue. Liu et al., in their paper "Yen-Cheng Liu, Chih-Yao Ma, Zijian He, Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, and Peter Vajda. Unbiased Teacher for Semi-Supervised Object Detection. in Proc. of International Conference on Learning Representations, 2021," proposed a technique using focus loss instead of cross-entropy loss to address the class imbalance problem of pseudo-labels, filtering them through a fixed high threshold. However, in practical applications, selecting this fixed high threshold is very difficult; setting the threshold too high or too low can significantly impact model training and performance. In the paper "Mengde Xu, Zheng Zhang, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, and Zicheng Liu. End-to-end Semi-Supervised Object Detection with Soft Teacher. in Porc. IEEEIntl. Conf. on Computer Vision, 2021, pp. 3060–3069", Xu et al. proposed using classification scores to adaptively weight the loss of each pseudo-label and proposed using the variance of the calculation results from multiple box jitters to select reliable pseudo-labels. The problem with this method is that multiple box jitters increase the computation time, making the model training process slow.
[0040] The methods proposed in the two aforementioned papers share a common problem: both are based on semi-supervised object detection in natural scenes, failing to fully consider the characteristics of rotating objects in remote sensing scenes. To address this, Hua et al., in their paper "Wei Hua, Dingkang Liang, Jingyu Li, Xiaolong Liu, Zhikang Zou, Xiaoqing Ye, and Xiang Bai. SOOD: Towards Semi-Supervised Oriented Object Detection. inPorc. Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pp. 15558-15567, 2023", proposed a novel semi-supervised rotating object detection model based on the mainstream pseudo-label framework. For oriented objects in remote sensing scenes, adaptive weights are used to regularize the consistency between each pseudo-label prediction pair based on their orientation differences. Furthermore, a many-to-many relationship is explicitly established between the pseudo-label set and the prediction set across the entire image's target layout for global consistency constraints. However, this method only uses the angle and target distribution information unique to remote sensing images to constrain the prediction consistency between teacher and student models, and fails to use the prior information of local spatial consistency on remote sensing images to improve the quality of pseudo-labels generated by the teacher model, which results in low detection accuracy of the trained model.
[0041] To address the issues of low detection accuracy in the aforementioned schemes due to failure to utilize prior information on local spatial consistency in remote sensing images, one embodiment of this application proposes a semi-supervised rotating target detection method based on prior information on local spatial consistency. This method is applied to an electronic device, which can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the semi-supervised rotating target detection method based on prior information on local spatial consistency proposed in this embodiment are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this solution.
[0042] The specific process of the semi-supervised rotating target detection method based on local spatial consistency prior information proposed in this embodiment can be described as follows: Figure 1 As shown, it includes:
[0043] Step 101: Use the teacher model to predict the unlabeled image to obtain the pseudo-labels corresponding to each target in the unlabeled image. Retain the pseudo-labels with confidence greater than the first preset threshold to obtain the pseudo-label set.
[0044] In the specific implementation, the server inputs the unlabeled image into the teacher model for target prediction (detection), obtaining pseudo-labels corresponding to each target in the unlabeled image output by the teacher model. The teacher model is a mature detection model capable of detecting the location of each target in the unlabeled image, marking the location boxes, and outputting them. These location boxes are predicted by the teacher model without human intervention, hence the term "pseudo-labels." For each target in the unlabeled image, the pseudo-label output by the teacher model can characterize the target's spatial location, category, and angle. The teacher model's prediction is not absolutely accurate; it is essentially a probability. Therefore, this probability can be used as the confidence level of the pseudo-label. The higher the probability, the higher the confidence level of the pseudo-label; the lower the probability, the lower the confidence level. The server uses a first preset threshold. All false labels predicted by the teacher model are filtered out, and those with a confidence level less than or equal to 1 are discarded. Pseudo-labels, only retain those with a confidence level greater than 1. The pseudo-labels are obtained, resulting in a set of pseudo-labels.
[0045] In one example, let the set of pseudo-labels predicted by the teacher model be . S Using the first preset threshold Filtering all pseudo-labels predicted by the teacher model can be achieved using the following formula:
[0046] ;
[0047] In the formula, Represents the first pseudo-label in the set. i A pseudo-label, The first in the set of pseudo-tags i Confidence of each pseudo-label This represents the set of pseudo-labels obtained after filtering.
[0048] In one example, unlabeled images can be selected from open-source datasets available on the internet.
[0049] Step 102: Obtain a low-confidence group based on pseudo-labels in the pseudo-label set whose confidence level is less than the second preset threshold, and obtain a high-confidence group based on pseudo-labels in the pseudo-label set whose confidence level is greater than or equal to the second preset threshold.
[0050] In the specific implementation, after the server obtains the set of pseudo-labels, it can use the second preset threshold. Group all pseudo-tags in the pseudo-tag set, and set a second preset threshold. Greater than the first preset threshold The server will select pseudo-labels with a confidence level less than [a certain value]. The pseudo-labels are designated as low-confidence pseudo-labels, and the pseudo-labels with confidence greater than or equal to [the specified confidence level] are selected. The pseudo-labels are used as high-confidence pseudo-labels, thus dividing the pseudo-label set into low-confidence groups. L and high confidence group H .
[0051] In one example, the low confidence group L and high confidence group H It can be represented as:
[0052] ;
[0053] ;
[0054] In the formula, Indicates the first in the low confidence group i A pseudo-label, Indicates the first in the low confidence group i Confidence of each pseudo-label L This indicates the low confidence group. Indicates the first in the high confidence group i A pseudo-label, Indicates the first in the high confidence group i Confidence of each pseudo-label H This indicates the high confidence level group. This represents a set of pseudo-labels.
[0055] It is worth noting that the high confidence group H The pseudo-labels in the data are considered reliable, accurate, and low-confidence groups. L Although the pseudo-labels in the model meet the minimum standards, there are still many inaccurate and incorrect pseudo-labels. At the same time, there are some high-quality pseudo-labels that are worth exploring and re-incorporating into the model training.
[0056] Step 103: Based on the consistency relationship between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group, construct the local spatial consistency prior condition, and merge the pseudo-labels in the high-confidence group with the pseudo-labels in the low-confidence group that meet the local spatial consistency prior condition to obtain a candidate pseudo-label set.
[0057] In practical implementation, remote sensing images generally possess local spatial consistency priors. This means that similar targets within local areas of a remote sensing image typically exhibit clustered distributions, and their angles are roughly the same. The server utilizes this local spatial consistency prior information in the remote sensing image to identify high-confidence groups. H Each pseudo-label and low-confidence group L The consistency relationship of each pseudo-label is analyzed, and local spatial consistency prior conditions are constructed. High-confidence groups are then grouped together.H Pseudo-labels and low-confidence groups L Pseudo-labels that meet the prior condition of local spatial consistency are merged to obtain a candidate pseudo-label set. T Low confidence group L Pseudo-labels that meet the prior condition of local spatial consistency are considered high-quality pseudo-labels. Such pseudo-labels are consistent with the high-confidence group in terms of spatial location, category, and perspective. H The pseudo-labels in the data are similar, making it worthwhile to select from the low-confidence group. L Extracted from the data and added to the model training.
[0058] Step 104: Perform angle correction on the candidate pseudo-labels in the candidate pseudo-label set, iteratively train the student model based on the unlabeled image and the angle-corrected candidate pseudo-label set, and update the teacher model based on the EMA of the trained student model.
[0059] In the specific implementation, the server obtains a set of candidate pseudo-tags. T Next, the candidate pseudo-labels in the candidate pseudo-label set can be angle-corrected to obtain the angle-corrected candidate pseudo-label set. T 2. Using unlabeled images as training samples, and then using the angle-corrected candidate pseudo-label set... T 2 is used as the training label. Unlabeled images are input into the student model, and the student model is iteratively trained until convergence, resulting in a trained student model. The EMA of the trained student model is then calculated, and the teacher model is updated based on the EMA of the trained student model. It is worth noting that the student model is a detection model with initialized parameters, and its model architecture is the same as the teacher model.
[0060] In one example, the model training process can be as follows: Figure 2 As shown, the target detector is first trained using available labeled data, resulting in a mature teacher model. Once the detector converges during the burn-in phase, the teacher-student interactive learning phase begins. This process consists of two steps: Student model learning: Pseudo-labels are generated using fixed weights of the teacher model to train the student model. The teacher and student models are given weak and strong augmented inputs, respectively. Teacher model updating: The knowledge learned by the student model is transferred to the slowly progressing teacher model via the EMA of the network weights.
[0061] In one example, the server performs angle correction on candidate pseudo-labels in the candidate pseudo-label set, which can be done through methods such as... Figure 3 The steps shown are implemented as follows:
[0062] Step 201: Traverse each candidate pseudo-label in the candidate pseudo-label set and determine the target candidate pseudo-label that is closest to the current candidate pseudo-label in the candidate pseudo-label set.
[0063] In the specific implementation, after obtaining the candidate pseudo-tag set, the server also needs to perform angle correction on the candidate pseudo-tags in the set. The server iterates through each candidate pseudo-tag in the set and determines the target candidate pseudo-tag that is closest to the current candidate pseudo-tag. It is worth noting that the server needs to determine whether the distance between the current candidate pseudo-tag and the target candidate pseudo-tag is less than a third preset threshold. If the distance between the current candidate pseudo-label and the target candidate pseudo-label is greater than or equal to This indicates that the difference between the current candidate pseudo-label and the target candidate pseudo-label is too large, and angle correction is not necessary. Only when the distance between the current candidate pseudo-label and the target candidate pseudo-label is less than... Only then will the server enter the angle correction process for the current candidate pseudo-label.
[0064] Step 202: Determine whether the confidence level of the target candidate pseudo-label is greater than the confidence level of the current candidate pseudo-label. If yes, proceed to step 203; otherwise, proceed to step 204.
[0065] Step 203: Replace the angle corresponding to the current candidate pseudo-label with the angle corresponding to the target candidate pseudo-label.
[0066] Step 204: Keep the angle corresponding to the current candidate pseudo-label unchanged.
[0067] In the specific implementation, after determining the target candidate pseudo-tag that is closest to the current candidate pseudo-tag, the server can check if the confidence level of the target candidate pseudo-tag is greater than that of the current candidate pseudo-tag. If the confidence level of the target candidate pseudo-tag is greater than that of the current candidate pseudo-tag, the server replaces the angle corresponding to the current candidate pseudo-tag with the angle corresponding to the target candidate pseudo-tag. If the confidence level of the target candidate pseudo-tag is less than or equal to that of the current candidate pseudo-tag, the server keeps the angle corresponding to the current candidate pseudo-tag unchanged. After traversing all candidate pseudo-tags, the server obtains the result of keeping the angle corresponding to the current candidate pseudo-tag unchanged.
[0068] In one example, the angle correction of the current candidate pseudo-label can be achieved using the following formula:
[0069] ;
[0070] In the formula, This indicates the confidence level of the current candidate pseudo-label. This indicates the confidence level of the target candidate pseudo-label. This indicates the angle corresponding to the current candidate pseudo-label. This indicates the angle corresponding to the target candidate pseudo-label.
[0071] In one example, the comparison of candidate pseudo-labels before and after angle correction can be shown as follows: Figure 4 As shown, the angle information of the candidate pseudo-labels is more accurate after angle correction.
[0072] Step 105: Input the image to be detected into the updated teacher model to determine each target in the image to be detected.
[0073] In the specific implementation, after the server updates the teacher model, it deploys the updated teacher model to the required application scenario to perform the rotating target detection task as the detection model. During rotating target detection, the image to be detected is input into the updated teacher model to determine each target in the image, which is then output as a bounding box for each target in the image. This bounding box contains spatial location information, category information, and angle information.
[0074] In this embodiment, a teacher model is used to predict unlabeled images, obtaining pseudo-labels corresponding to each target in the unlabeled image. Based on the confidence level of each pseudo-label, two thresholds are used to filter out pseudo-labels with too low confidence, resulting in a usable high-confidence pseudo-label group and a low-confidence pseudo-label group. Using the high-confidence group as a benchmark, local spatial consistency prior information in the image is used to construct local spatial consistency prior conditions. More high-quality pseudo-labels are extracted from the low-confidence group and merged with the high-confidence group to obtain a candidate pseudo-label set, significantly improving the completeness of the pseudo-labels. Subsequently, angle correction is performed on the candidate pseudo-labels in the candidate pseudo-label set, effectively improving the quality of the pseudo-labels. This allows for iterative training of the student model, and the teacher model is updated based on the EMA of the trained student model. The updated teacher model is then used as the detection model for rotating target detection in the image to be detected. The use of local spatial consistency prior information greatly improves the quantity and quality of pseudo-labels, enabling some correctly predicted but low-confidence targets to be added to the training process of the student model, expanding the basic training data of the student model, enabling the student model to converge quickly during training, improving the quality and speed of model training, and significantly improving the detection accuracy and detection speed of the final detection model.
[0075] In one embodiment, the local spatial consistency prior condition is specifically a local spatial consistency prior matrix. The server constructs the local spatial consistency prior condition based on the consistency relationship between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group. This can be achieved through methods such as... Figure 5 The steps shown are implemented as follows:
[0076] Step 301: Calculate the distance between each pseudo-label in the high confidence group and each pseudo-label in the low confidence group, determine that the target spaces represented by two pseudo-labels whose distance is less than or equal to a third preset threshold are adjacent, and obtain the position consistency matrix based on the distance determination result.
[0077] In a specific implementation, the server can calculate the distance between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group using the following formula, determine that two pseudo-labels whose distance is less than or equal to a third preset threshold represent adjacent target spaces, and obtain a position consistency matrix based on the distance determination result:
[0078] ;
[0079] ;
[0080] In the formula, Indicates high confidence group H The Middle i The x-coordinate of each pseudo-label Indicates high confidence group H The Middle i The ordinate of each pseudo-label. Indicates low confidence group L The Middle j The x-coordinate of each pseudo-label Indicates low confidence group L The Middle j The ordinate of each pseudo-label. Indicates high confidence group H The Middle i Pseudo-labels and low-confidence groups L The Middle j The distance between pseudo-tags The third preset threshold, Position Consistency Matrix D The Middle i Line number j The value of the column, Indicates high confidence group H The Middle i Pseudo-labels and low-confidence groups L The Middle j The target spaces represented by the pseudo-labels are adjacent. Indicates high confidence group H The Middle i Pseudo-labels and low-confidence groups L The Middle j The target spaces represented by the pseudo-labels are not adjacent.
[0081] Step 302: Determine whether the category corresponding to each pseudo-label in the high confidence group is the same as that corresponding to each pseudo-label in the low confidence group, and obtain the category consistency matrix based on the category determination result.
[0082] In practical implementation, the server can use the following formula to determine whether the category corresponding to each pseudo-label in the high-confidence group is the same as that corresponding to each pseudo-label in the low-confidence group, and obtain the category consistency matrix based on the category determination result:
[0083] ;
[0084] In the formula, Indicates high confidence group H The Middle i The corresponding category of each pseudo-tag Indicates low confidence group L The Middle j The category corresponding to each pseudo-tag Category Consistency Matrix C The Middle i Line number j The value of the column, Indicates high confidence group H The Middle i The corresponding category of each pseudo-label and the low confidence group L The Middle j The pseudo-tags correspond to the same category. Indicates high confidence group H The Middle i The corresponding category of each pseudo-label and the low confidence group L The Middle j The pseudo-tags correspond to different categories.
[0085] Step 303: Calculate the angle difference between each pseudo-label in the high confidence group and each pseudo-label in the low confidence group, determine that the target angle represented by two pseudo-labels whose absolute value of the angle difference is less than or equal to the fourth preset threshold is consistent, and obtain the angle consistency matrix based on the angle determination result.
[0086] In practical implementation, the server can calculate the angle difference between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group using the following formula. It then determines that two pseudo-labels whose absolute value of the angle difference is less than or equal to a fourth preset threshold represent consistent target angles. Based on the angle determination results, an angle consistency matrix is obtained.
[0087] ;
[0088] In the formula, Indicates high confidence group H The Middle i The corresponding angle of each pseudo-tag Indicates low confidence group L The Middle j The angle corresponding to each pseudo-label Indicates high confidence group H The Middle i Pseudo-labels and low-confidence groups L The Middle j The absolute value of the angle difference between the pseudo-tags. The fourth preset threshold, Angle Consistency Matrix A The Middle i Line number j The value of the column, Indicates high confidence group H The Middle i Pseudo-labels and low-confidence groups L The Middle j The target perspective represented by each pseudo-label is consistent. Indicates high confidence group H The Middle i Pseudo-labels and low-confidence groups L The Middle j The target perspectives represented by the pseudo-labels are inconsistent.
[0089] Step 304: Based on the position consistency matrix, category consistency matrix, and angle consistency matrix, obtain the local spatial consistency prior matrix.
[0090] In practical implementation, the server can obtain the local spatial consistency prior matrix based on the location consistency matrix, category consistency matrix, and angle consistency matrix using the following formula:
[0091] ;
[0092] In the formula, For the position consistency matrix, For the category consistency matrix, For the angle consistency matrix, This indicates element-wise multiplication. Let be the local spatial consistency prior matrix.
[0093] In one example, the server can use the following formula to merge pseudo-labels from the high-confidence group with pseudo-labels from the low-confidence group that meet the prior condition of local spatial consistency, to obtain a candidate pseudo-label set:
[0094] ;
[0095] ;
[0096] In the formula, LThis indicates the low confidence group. Indicates the first in the low confidence group j A pseudo-label, The first in the local space consistent prior matrix i Line number j The value of the column, E Let represent the set of all pseudo-labels in the low-confidence group that satisfy the prior condition of local spatial consistency. H This indicates the high confidence level group. T This represents the set of candidate pseudo-labels.
[0097] In this embodiment, three aspects—spatial location, category, and representation angle—are used to represent local spatial consistency prior information. A location consistency matrix, a category consistency matrix, and an angle consistency matrix are constructed sequentially to form a local spatial consistency prior matrix. The pseudo-labels of the low-confidence group that can be successfully mined are adjacent to the target space represented by the pseudo-labels in the high-confidence group, have the same category, and have the same target angle, ensuring that the quality of the mined pseudo-labels in the low-confidence group meets the training requirements.
[0098] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.
[0099] In one embodiment, the comparison of multiple pseudo-tag processing methods under different processing methods can be as follows: Figure 6 As shown, the semi-supervised rotating target detection method based on local spatial consistency prior information proposed in this application is denoted as "OURS". Figure 6 As can be seen, the "OURS" method can not only accurately obtain all usable pseudo-labels, but also correct the angle of inaccurate pseudo-labels. In contrast, the "fixed threshold method with a threshold of 0.25" retains a large number of incorrect pseudo-labels, while the "fixed threshold method with a threshold of 0.5" loses a large number of correct pseudo-labels. Furthermore, neither of these methods can solve the problem of pseudo-label angle deviation.
[0100] In one embodiment, to verify the superiority of the semi-supervised rotating target detection method based on local spatial consistency prior information proposed in this application, comparative experiments were conducted on the detection model trained based on the above embodiment (hereinafter referred to as OURS), Faster R-CNN, Ubiased Teacher, Soft Teacher and SOOD.
[0101] 1. Experimental Environment
[0102] The comparative experiment was conducted using the PyTorch framework on an Intel(R) Xeon(R) Silver 4214R CPU @ 2.40GHz, 16GB of memory, and a Linux operating system.
[0103] The dataset used in the experiment came from the Dotav 1.5 dataset, proposed by Xia et al. in the paper "Gui-Song Xia, Xiang Bai, Jian Ding, Zhen Zhu, Serge Belongie, Jiebo Luo, MihaiDatcu, Marcello Pelillo, and Liangpei Zhang. DOTA: A Large-Scale Dataset for Object Detection in Aerial Images. in Proc. IEEE conference on computer vision and pattern recognition, 2018, pp. 3974-3983". It contains 2806 remote sensing images and nearly 190,000 rotated bounding box annotation instances, covering 16 categories of common objects in remote sensing imagery. The RGB images were from Google Earth and CycloMedia, while the grayscale images were from the panchromatic bands of GF-2 and JL-1 satellite images. To study the semi-supervised learning method, we divided the training set into labeled and unlabeled training data at 10%, 20%, and 30% respectively.
[0104] 2. Experiment Content
[0105] The comparative experiments trained the model with 10%, 20%, and 30% of the labeled training data, respectively, and calculated the average accuracy (mAP50) on the validation set. The comparison results are shown in Table 1.
[0106] Table 1: Comparative Experiment Results
[0107]
[0108] As can be seen from Table 1, when training the model with 10%, 20%, and 30% of labeled training data, the average accuracy of OURS is better than the other three semi-supervised models, and it has an improvement of 5 to 6 percentage points compared with the fully supervised model.
[0109] Another embodiment of this application relates to an electronic device, such as... Figure 7As shown, it includes: at least one processor 401; and a memory 402 communicatively connected to the at least one processor 401; wherein the memory 402 stores instructions executable by the at least one processor 401, the instructions being executed by the at least one processor 401 to enable the at least one processor 401 to execute the semi-supervised rotating target detection method based on local spatial consistency prior information in the above embodiments.
[0110] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.
[0111] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.
[0112] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.
[0113] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0114] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.
Claims
1. A semi-supervised rotation target detection method based on local spatial consistency prior information, characterized in that, include: The teacher model is used to predict the unlabeled image to obtain the pseudo-labels corresponding to each target in the unlabeled image. Pseudo-labels with confidence greater than a first preset threshold are retained to obtain a pseudo-label set. A low-confidence group is obtained based on pseudo-labels in the pseudo-label set whose confidence level is less than a second preset threshold, and a high-confidence group is obtained based on pseudo-labels in the pseudo-label set whose confidence level is greater than or equal to the second preset threshold; wherein, the second preset threshold is greater than the first preset threshold; Based on the consistency relationship between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group, a local spatial consistency prior condition is constructed, and the pseudo-labels in the high-confidence group and the pseudo-labels in the low-confidence group that meet the local spatial consistency prior condition are merged to obtain a candidate pseudo-label set. The candidate pseudo-labels in the candidate pseudo-label set are angle-corrected, the student model is iteratively trained based on the unlabeled image and the angle-corrected candidate pseudo-label set, and the teacher model is updated based on the EMA of the trained student model. The image to be detected is input into the updated teacher model to determine each target in the image to be detected; The angle correction of the candidate pseudo-labels in the candidate pseudo-label set includes: Iterate through each candidate pseudo-label in the candidate pseudo-label set, and determine the target candidate pseudo-label that is closest to the current candidate pseudo-label in the candidate pseudo-label set; If the confidence level of the target candidate pseudo-label is greater than the confidence level of the current candidate pseudo-label, then the angle corresponding to the current candidate pseudo-label is replaced with the angle corresponding to the target candidate pseudo-label. If the confidence level of the target candidate pseudo-label is less than or equal to the confidence level of the current candidate pseudo-label, then the angle corresponding to the current candidate pseudo-label remains unchanged. The local spatial consistency prior condition is specifically a local spatial consistency prior matrix. The construction of the local spatial consistency prior condition based on the consistency relationship between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group includes: Calculate the distance between each pseudo-label in the high confidence group and each pseudo-label in the low confidence group, determine that the target spaces represented by two pseudo-labels whose distance is less than or equal to a third preset threshold are adjacent, and obtain the position consistency matrix based on the distance determination result; Determine whether the category corresponding to each pseudo-label in the high confidence group is the same as that corresponding to each pseudo-label in the low confidence group, and obtain the category consistency matrix based on the category determination result; Calculate the angle difference between each pseudo-label in the high confidence group and each pseudo-label in the low confidence group, determine that the target angle represented by two pseudo-labels whose absolute value of the angle difference is less than or equal to a fourth preset threshold is consistent, and obtain the angle consistency matrix based on the angle determination result; The local spatial consistency prior matrix is obtained based on the position consistency matrix, the category consistency matrix, and the angle consistency matrix.
2. The semi-supervised rotating object detection method based on local spatial consistency prior information according to claim 1, characterized in that, The distance between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group is calculated using the following formula. Two pseudo-labels whose distance is less than or equal to a third preset threshold are determined to be spatially adjacent. A positional consistency matrix is then obtained based on the distance determination results. ； ； in, Indicates the first in the high confidence group The x-coordinate of each pseudo-label Indicates the first in the high confidence group The ordinate of each pseudo-label. Indicates the first in the low confidence group The x-coordinate of each pseudo-label Indicates the first in the low confidence group The ordinate of each pseudo-label. Indicates the first in the high confidence group The pseudo-label and the first pseudo-label in the low-confidence group The distance between pseudo-tags The third preset threshold, The position consistency matrix is the first... Line number The value of the column, Indicates the first in the high confidence group i The pseudo-label and the first pseudo-label in the low-confidence group The target spaces represented by the pseudo-labels are adjacent. Indicates the first in the high confidence group The pseudo-label and the first pseudo-label in the low-confidence group The target spaces represented by the pseudo-labels are not adjacent.
3. The semi-supervised rotating object detection method based on local spatial consistency prior information according to claim 1, characterized in that, The following formula is used to determine whether the category corresponding to each pseudo-label in the high-confidence group is the same as that corresponding to each pseudo-label in the low-confidence group, and the category consistency matrix is obtained based on the category determination result: ； in, Indicates the first in the high confidence group The corresponding category of each pseudo-tag Indicates the first in the low confidence group The category corresponding to each pseudo-label, the The first in the category consistency matrix Line number The value of the column, Indicates the first in the high confidence group The category corresponding to the pseudo-label is the same as the first pseudo-label in the low-confidence group. The pseudo-tags correspond to the same category. Indicates the first in the high confidence group The category corresponding to the pseudo-label is the same as the first pseudo-label in the low-confidence group. The pseudo-tags correspond to different categories.
4. The semi-supervised rotating object detection method based on local spatial consistency prior information according to claim 1, characterized in that, The angle difference between each pseudo-label in the high-confidence group and each pseudo-label in the low-confidence group is calculated using the following formula. Pseudo-labels whose absolute value of the angle difference is less than or equal to a fourth preset threshold represent consistent target angles. An angle consistency matrix is obtained based on the angle determination results. ； in, Indicates the first in the high confidence group The corresponding angle of each pseudo-tag Indicates the first in the low confidence group The angle corresponding to each pseudo-label Indicates the first in the high confidence group The pseudo-label and the first pseudo-label in the low-confidence group The absolute value of the angle difference between the pseudo-tags. The fourth preset threshold, The first in the angle consistency matrix Line number The value of the column, Indicates the first in the high confidence group The pseudo-label and the first pseudo-label in the low-confidence group The target perspective represented by each pseudo-label is consistent. Indicates the first in the high confidence group The pseudo-label and the first pseudo-label in the low-confidence group The target perspectives represented by the pseudo-labels are inconsistent.
5. The semi-supervised rotating target detection method based on local spatial consistency prior information according to claim 1, characterized in that, The local spatial consistency prior matrix is obtained using the following formula, based on the position consistency matrix, the category consistency matrix, and the angle consistency matrix: ； wherein, is the position consistency matrix, is the category consistency matrix, is the angle consistency matrix, denotes element-wise multiplication, is the local spatial consistency prior matrix.
6. The semi-supervised rotating object detection method based on local spatial consistency prior information according to claim 5, characterized in that, The pseudo-labels in the high-confidence group and the pseudo-labels in the low-confidence group that meet the prior condition of local spatial consistency are merged using the following formula to obtain a candidate pseudo-label set: ； ； in, This refers to the low-confidence group. Indicates the first in the low confidence group A pseudo-label, The first in the local spatial consistency prior matrix Line number The value of the column, This represents the set of all pseudo-labels in the low-confidence group that satisfy the prior condition of local spatial consistency. This refers to the high-confidence group. This represents the set of candidate pseudo-labels.
7. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the semi-supervised rotating target detection method based on local spatial consistency prior information as described in any one of claims 1 to 6.
8. A computer readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the semi-supervised rotating target detection method based on local spatial consistency prior information as described in any one of claims 1 to 6.