Multilevel domain adaptive pedestrian detection method and system based on teacher model

Abstract

The invention provides a multi-level domain adaptive pedestrian detection method and system based on a teacher model, and the method comprises the steps: training a target detector through employing source domain perspective image data with a pedestrian label, and obtaining an initial target detector; constructing a target domain teacher model and a cross-domain student model; and performing multi-level domain adaptive adversarial training by using the cross-domain student model, updating parameters of the target domain teacher model through index moving average of the weight of the cross-domain student model, generating a pseudo label to supervise the multi-level domain adaptive adversarial training of the cross-domain student model, and optimizing the cross-domain student model in combination with a designed loss function. Obtaining a final cross-domain student model; and detecting pedestrians in the overlooking fisheye image to be detected based on the final cross-domain student model. According to the technical scheme of the invention, the requirements of people flow statistics, safety monitoring and people flow guidance on rapid and accurate detection of pedestrians in a scene can be met.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent surveillance technology, specifically relating to a multi-level domain adaptive pedestrian detection method and system based on a teacher model. Background Technology

[0002] In wide-area, high-density scenarios (such as transportation hubs, shopping malls, and stadiums), pedestrian detection is a key technology supporting intelligent security, traffic management, and smart city construction. These scenarios often employ top-down fisheye cameras for wide-area coverage monitoring. However, top-down fisheye image pedestrian datasets are severely lacking, making it difficult to support the training of high-precision detection models. In contrast, conventional perspective image datasets are large in scale and rich in annotation information, enabling conventional pedestrian detection tasks to achieve high accuracy. However, top-down fisheye images, due to geometric distortion, the arbitrary orientation of pedestrians, and the bird's-eye view characteristics, exhibit significant domain differences compared to perspective images. This leads to a substantial drop in detection accuracy when perspective-based detection models are directly transferred to top-down fisheye image scenarios. For example, when using top-down fisheye cameras to monitor pedestrians in transportation hubs, conventional models experience numerous missed and false detections due to the domain differences, failing to accurately count pedestrian flow distribution and impacting pedestrian flow management efficiency, thus failing to meet the real-world demand for rapid and accurate pedestrian detection.

[0003] To address the cross-domain adaptation problem, existing domain-adaptive object detection methods attempt to reduce the feature distribution differences between the source domain (perspective image) and the target domain (top-down fisheye image) through domain classifiers and adversarial learning. However, in pedestrian detection tasks in wide-area, highly clustered scenes, adversarial learning alone is insufficient to fully eliminate the domain gap between the two types of images, resulting in limited performance improvement. There is still a significant accuracy gap compared to the target domain fully supervised model. For example, in densely populated scenes such as shopping malls, these methods cannot effectively adapt to the distortion characteristics of fisheye images and the pedestrian distribution characteristics, leading to poor detection results for dense pedestrians and making it difficult to assist managers in carrying out efficient crowd control.

[0004] Some methods further employ a teacher-student self-training framework, extending semi-supervised learning to domain adaptation tasks. The teacher model generates pseudo-labels to guide the student model's training, thereby improving target domain detection performance. However, due to the significant difference between the source and target domains, the teacher model, trained on source domain data, exhibits a clear bias towards the source domain, accurately capturing only source domain features. The pseudo-labels generated in the target domain are of low quality (e.g., containing numerous false positives and pseudo-labels with large localization deviations). Directly applying this framework leads to poor training performance of the student model, failing to effectively adapt to the target domain. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a multi-level domain adaptive pedestrian detection method and system based on a teacher model. This method can effectively reduce the domain gap between the source domain (perspective image) and the target domain (top-down fisheye image), improve the pedestrian detection accuracy based on top-down fisheye images in wide-area, high-crowding scenes (such as transportation hubs, shopping malls, stadiums, etc.), and improve the problem of low quality of pseudo-labels in the target domain in the teacher-student self-training framework. This enables the model to accurately learn the pedestrian features in the target domain, meeting the needs of pedestrian flow statistics, security monitoring, and pedestrian flow management in the scene for fast and accurate pedestrian detection.

[0006] To achieve the above objectives, the present invention provides the following solution: A multi-level domain adaptive pedestrian detection method based on a teacher model includes: The target detector is trained using source domain perspective image data with pedestrian annotations to obtain the initial target detector; The initial target detector is replicated to construct a target domain teacher model and a cross-domain student model, respectively; wherein, the target domain teacher model only inputs a weakly enhanced target domain top-view fisheye image, and the cross-domain student model simultaneously inputs a strongly enhanced source domain perspective image and a target domain top-view fisheye image; Multi-level domain adaptive adversarial training is performed using the cross-domain student model, and the parameters of the target domain teacher model are updated by the exponential moving average of the weights of the cross-domain student model. Pseudo-labels are generated to supervise the multi-level domain adaptive adversarial training of the cross-domain student model. The cross-domain student model is optimized by combining the designed loss function to obtain the final cross-domain student model. Based on the final cross-domain student model, pedestrians are detected in the top-down fisheye image to be detected.

[0007] Preferably, the weak enhancement process includes random horizontal flipping and cropping of the image; The enhanced processing includes random color jittering, grayscale conversion, Gaussian blurring, and cropping patching of the image.

[0008] Preferably, the multi-level domain adaptive adversarial training includes image-level feature adversarial training, instance-level feature adversarial training, and domain-level feature adversarial training; wherein, the method for image-level feature adversarial training using the cross-domain student model includes: A weighted multi-layer feature alignment module is constructed using an adversarial domain classifier, and the loss of each adversarial domain classifier is calculated to quantify the transferability of the feature layer and calculate the weighting factor of the adversarial domain classifier; wherein, the transferability refers to the contribution of the knowledge contained in the image-level features to cross-domain adaptation; Based on the weighting factors, the objective function of the weighted multi-layer feature alignment module is calculated to complete image-level feature adversarial training.

[0009] Preferably, the method for instance-level feature adversarial training using the cross-domain student model includes: In Faster R-CNN, region proposals are generated through a region proposal network to describe the foreground and background of the input image; The adjacency matrix is ​​derived based on the intersection-union ratio of regional proposals, and a relationship graph is constructed based on the spatial correlation provided by the adjacency matrix. The feature embeddings and classification confidence of regional proposals are aggregated based on the relationship graph to obtain the aggregation result. Based on the aggregation results, a category prototype is constructed, and a supervised prototype alignment method is used to achieve cross-domain instance-level feature alignment.

[0010] Preferably, the method for performing domain-level feature adversarial training using the cross-domain student model includes: Geometric distortion is applied to the source domain perspective image using a fisheye camera model to generate a pseudo fisheye image as an auxiliary domain. Calculate the image-level triplet loss to make the distance between the target domain image-level features and the auxiliary domain features in the metric space smaller than the distance between them and the source domain features; at the same time, calculate the object-level triplet loss to make the distance between the target domain object-level features and the auxiliary domain object-level features smaller than the distance between them and the source domain object-level features. By jointly optimizing the image-level triplet loss and the object-level triplet loss, the model is driven to learn domain-invariant features, thus completing domain-level feature adversarial training.

[0011] Preferably, the loss function includes supervised loss, unsupervised pseudo-label loss, and multi-level adversarial loss.

[0012] This invention also provides a multi-level domain adaptive pedestrian detection system based on a teacher model, used to implement the method, comprising: The target detector initialization module is used to train the target detector using source domain perspective image data with pedestrian annotations to obtain the initial target detector; A dual-model construction module is used to replicate the initial target detector and construct a target domain teacher model and a cross-domain student model respectively; wherein, the target domain teacher model only inputs a weakly enhanced target domain top-view fisheye image, and the cross-domain student model simultaneously inputs a strongly enhanced source domain perspective image and a target domain top-view fisheye image; The student model training module is used to perform multi-level domain adaptive adversarial training using the cross-domain student model, and to update the parameters of the target domain teacher model by the exponential moving average of the weights of the cross-domain student model, generate pseudo-labels to supervise the multi-level domain adaptive adversarial training of the cross-domain student model, and optimize the cross-domain student model by combining the designed loss function to obtain the final cross-domain student model. The pedestrian detection module is used to detect pedestrians in the top-down fisheye image based on the final cross-domain student model.

[0013] Preferably, in the student model training module, the multi-level domain adaptive adversarial training includes image-level feature adversarial training, instance-level feature adversarial training, and domain-level feature adversarial training; wherein, the method for image-level feature adversarial training using the cross-domain student model includes: A weighted multi-layer feature alignment module is constructed using an adversarial domain classifier, and the loss of each adversarial domain classifier is calculated to quantify the transferability of the feature layer and calculate the weighting factor of the adversarial domain classifier; wherein, the transferability refers to the contribution of the knowledge contained in the image-level features to cross-domain adaptation; Based on the weighting factors, the objective function of the weighted multi-layer feature alignment module is calculated to complete image-level feature adversarial training.

[0014] Compared with existing technologies, the beneficial effects of this invention are as follows: The multi-level domain adaptive pedestrian detection algorithm (DAPD) based on a teacher model constructs a dual-module architecture of a target domain teacher model and a cross-domain student model. The teacher model only receives weakly enhanced images from the target domain to avoid source domain bias, generating more reliable pseudo-labels to supervise the student model. Simultaneously, the student model receives strongly enhanced images from both the source and target domains and utilizes the EMA mechanism to achieve mutual learning between the teacher and student, continuously optimizing pseudo-label quality and effectively solving the problem of low pseudo-label quality caused by domain differences in traditional teacher-student frameworks. Multi-level feature adversarial training is introduced into the student model. At the image level, coupling loss is used to map the negative log-likelihood loss of the discriminator to the influence factor of the gradient reversal layer, reducing the impact on the main network when the discriminator performance is poor, ensuring the stability of adversarial training, and avoiding detection accuracy loss caused by training fluctuations. At the instance level, a graph structure is used to construct the relationship graph between region proposals, and the adjacency matrix is ​​calculated using IoU. The algorithm integrates multimodal information from different instances into a category prototype through information transfer and feature aggregation in regional proposals. Combined with pseudo-label enhancement, it improves prototype-based adaptation, enhances instance-level feature alignment accuracy, and reduces detection errors caused by differences in instance feature distribution between the source and target domains. At the domain level, pseudo-fisheye data augmentation transforms perspective images into pseudo-fisheye images to generate an auxiliary domain. Image-level and object-level triplet losses constrain the feature distances between the source, auxiliary, and target domains, narrowing the domain gap between the source and target domains and solving the knowledge transfer problem in situations where fisheye datasets are scarce. Based on the above technical features, compared with baseline models trained only in the source domain and traditional domain adaptation methods, this algorithm can effectively alleviate domain bias. It shows significant improvements in mAP, AP@50, and AP@75 metrics for pedestrian detection in the target domain (top-down fisheye image). Especially in heterogeneous scenarios where the source domain is a perspective image and the target domain is a fisheye image, it can achieve more accurate pedestrian detection and meet the detection needs in wide-area, high-cluster scenarios. Attached Figure Description

[0015] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a multi-level domain adaptive pedestrian detection method based on a teacher model, as described in an embodiment of the present invention. Figure 2 This is a flowchart of a multi-level domain adaptive adversarial training method according to an embodiment of the present invention; Figure 3The diagram illustrates the effect of an embodiment of the present invention; where (a) is the Faster R-CNN baseline; (b) is the unbiased teacher model; (c) is the model of the present invention; and (d) is the ground truth annotation. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] Example 1 like Figure 1 As shown, a multi-level domain adaptive pedestrian detection method based on a teacher model includes: S1: Train the target detector using source domain perspective image data with pedestrian annotations to obtain the initial target detector. Specifically, initialize the feature encoder and detector components. As described in step S1, obtain the target detector in the source domain. An image with a label In the target domain Unlabeled image ,in Indicates bounding box annotation, Indicates the corresponding category label, Represents the source domain image. Target domain image. No comments. The ultimate goal of cross-domain object detection is to leverage... and Design a domain-invariant detector. For example, use LOAF as the target dataset and a custom subset of the COCO dataset, COCO-person, as the source domain, which contains 64k images with only pedestrian annotations.

[0020] S2: Duplicate the initial object detector to construct a target domain teacher model and a cross-domain student model. The target domain teacher model receives only a weakly enhanced target domain top-down fisheye image as input, while the cross-domain student model receives both a strongly enhanced source domain perspective image and a target domain top-down fisheye image as input. A further implementation involves the weak enhancement processing including random horizontal flipping and cropping of the image. The enhanced processing includes random color jittering, grayscale conversion, Gaussian blurring, and cropping patching of the image.

[0021] S3: Utilize a cross-domain student model for multi-level domain adaptive adversarial training, and update the parameters of the target domain teacher model using an exponential moving average of the cross-domain student model weights. Generate pseudo-labels to supervise the multi-level domain adaptive adversarial training of the cross-domain student model. Combined with the designed loss function, optimize the cross-domain student model to obtain the final cross-domain student model. For example... Figure 2 As shown.

[0022] A further implementation method is that the multi-level domain adaptive adversarial training includes image-level feature adversarial training, instance-level feature adversarial training, and domain-level feature adversarial training.

[0023] To learn image-level domain-invariant features, a weighted multi-layer feature alignment module (FPN) is proposed, which includes multiple adversarial domain classifiers. Cross-domain feature alignment is achieved by constructing a max-min game between the domain classifier D and the backbone network G. Specifically, the domain classifier D aims to distinguish features in the source / target domains, while the backbone network G's task is to confuse the domain classifiers. Adversarial learning is implemented by inserting a gradient inversion layer (GRL) between D and G. Since low-level features contain fine-grained details and are more susceptible to fisheye image distortion, alignment is performed only using high-level features. Here, the FPN backbone network... and Image-level style alignment was performed on the layers. Specifically, based on the fact that features extracted from different feature layers have specific attributes—that is, their adaptability to scenes (such as lighting, viewpoint, etc.) differs—a weighting mechanism was introduced to highlight the feature confusion capabilities of imbalanced classifiers from different domains. Methods for image-level feature adversarial training using cross-domain student models include: A weighted multi-layer feature alignment module is constructed using adversarial domain classifiers, and the loss of each adversarial domain classifier is calculated to quantify the transferability of the feature layers and calculate the weighting factors of the adversarial domain classifiers. Transferability refers to the contribution of knowledge contained in image-level features to cross-domain adaptation; specifically, a larger loss indicates poorer discriminatory power of the features against the source / target domain, implying high transferability, and vice versa. Quantifying the first... The transferability of each feature layer. Transferability can be understood as the contribution of the knowledge contained in the features to cross-domain adaptation. Features with high transferability make a significant contribution to improving the performance of domain adaptation. The definition is as follows: , in The weighting factor of the domain classifier indicates that the greater the loss of the adversarial classifier, the better. The larger the value, the more focused the feature alignment process will be on that feature layer. Representative at the The adversarial learning objective function loss is calculated on each feature layer. The indexed representation of this loss.

[0024] Based on the weighting factors, the objective function of the weighted multi-layer feature alignment module is calculated to complete image-level feature adversarial training. The objective function of the weighted multi-layer alignment module is as follows: .

[0025] A further implementation method involves using a cross-domain student model for instance-level feature adversarial training, including: In Faster R-CNN, a Region Proposal Network (RPN) is used to generate region proposals to describe the foreground and background of the input image; Adjacency matrices are derived based on the intersection-union ratio (IU) of region proposals. A relational graph is constructed based on the spatial relevance provided by the adjacency matrix. Feature embeddings and classification confidence scores of region proposals are then aggregated based on this graph to obtain the aggregation result. Specifically, candidate boxes generated by the RPN are constructed into a graph. ,in Indicates and The set of vertices corresponding to each candidate box. An adjacency matrix represents the set of edges, i.e., the relationships between candidate boxes. This is used to model this relationship. The Intersection over Union (IoU) ratio is a widely used metric that considers both the position and size of the proposals. The adjacency matrix is ​​derived using IoU as follows: , in, and They represent the first The and the first Regional proposals ( The establishment of the relationship graph lays the foundation for information transfer between regional proposals. This is achieved by utilizing the adjacency matrix. Provided spatial relevance, feature embedding of regional proposals ( (This refers to the embedding dimension) and classification confidence. , The number of categories is aggregated as follows: in, Indicates having The diagonal matrix of the entries. After region aggregation, and More precise instance-level information can be expressed by propagating signals between adjacent proposals.

[0026] Based on the aggregation results, category prototypes are constructed, and supervised prototype alignment is used to achieve cross-domain instance-level feature alignment. Specifically, the category prototype aggregates instance prototypes of k categories from N images. The category prototype can be represented as: , When there are category labels, implicitly assigning regions to categories becomes explicit. Therefore, a method called Supervised Prototype Alignment (SPA) is proposed. This involves aligning regions with any truly labeled regions. The overlap exceeds a certain threshold The region is considered a foreground sample; the region that overlaps with all ground truth labels by less than a certain threshold. The region is considered the background sample. A region With any real-world example The maximum overlap can be expressed as: , The weights used to calculate which features belong to the two classes: , For cases with more than two categories, the extension is obvious, but not applicable to the setup of this work. Category prototype: , In the formula, Representing the i The characteristics of the regional proposals belong to the following categories. The weight.

[0027] The N images corresponding to all region proposals are aggregated and aligned using contrastive loss.

[0028] A further implementation method involves using a cross-domain student model for domain-level feature adversarial training, including: A pseudo-fisheye image is generated as an auxiliary domain by geometrically distorting the source domain perspective image using a fisheye camera model. Specifically, by converting an image captured by a standard camera into an image with a fisheye effect, the fisheye camera first needs to be calibrated to obtain its intrinsic parameters and distortion coefficients (general parameters can be used, but for specific tasks, parameters of an actual fisheye camera are best). Then, through distortion correction and image reprojection, the standard image is converted into a pseudo-fisheye image. Finally, field-of-view cropping and necessary image processing operations are performed as needed.

[0029] Calculate the image-level triplet loss to ensure that the distance between the target domain image-level features and the auxiliary domain features in the metric space is smaller than their distance to the source domain features; simultaneously calculate the object-level triplet loss to ensure that the distance between the target domain object-level features and the auxiliary domain object-level features is smaller than their distance to the source domain object-level features; specifically, the first... The global image-level features of the training images are defined as follows: , and Expectations are decreasing. and After the domain difference between them, ensure and Feature metric distance ratio and The feature metric distance between them is closer, and it is defined as follows: , in, This represents the metric distance between corresponding features. This constraint can be implemented using a triplet structure, where... (Source domain characteristics) (Target domain features) and (Auxiliary domain features) can be viewed as anchor points, positive samples, and negative samples in a triplet structure. Therefore, as a neighborhood-level metric for regularization of image features, image-level constraints are equivalent to minimizing the following image-level triplet loss: , Where parameters Used as an edge constraint, set in the experiment .

[0030] Similarly, the first The first training image Object-level features , and Defined as , and As a domain-level metric regularization of object features, the following object-level triplet loss will also be minimized: .

[0031] By jointly optimizing image-level triplet loss and object-level triplet loss, the model learns domain-invariant features, thus completing domain-level feature adversarial training.

[0032] Strong pseudo-labels are obtained from the target domain image, and the teacher model is updated by temporarily replicating the weights of the student model using the Exponential Moving Average (EMA) method. The update formula can be defined as: , in, and These represent the parameters of the teacher network and the student network, respectively. The exponential moving average weighting smoothing coefficient parameter for the teacher model is set to 0.9996.

[0033] A further implementation method involves using a loss function that includes supervised loss, unsupervised pseudo-label loss, and multi-level adversarial loss.

[0034] The final training loss of the proposed network is the sum of the individual parts, which can be expressed as: , in, and These are hyperparameters used to control the corresponding loss weights, set to... and The confidence threshold is set to It is noted that and It was developed to learn the feature encoder and detector in the student model, and This is introduced to update the domain-adaptive loss function. During training, the proposed domain-adaptive object detection framework uses the standard stochastic gradient descent algorithm for end-to-end training, with a learning rate of 0.04, momentum of 0.9, and weight decay of 0.0001.

[0035] Define the supervised loss for training and initializing the student model using labeled source data: , Among them, RPN loss This is the loss used to learn the Region Proposal Network (RPN), which aims to generate candidate proposals, and the ROI (Region of Interest) loss. This is the prediction branch of ROI. Both RPN and ROI perform bounding box regression (reg) and classification (cls). and Using binary cross-entropy loss, for and use loss.

[0036] After obtaining the pseudo-labels generated by the teacher model from the target domain image, the student network is updated using the following loss: in, Pseudo-labels generated on the target domain by the teacher model.

[0037] In image-level feature adversarial training, to avoid using a fixed scheduling strategy, we propose coupling the influence in each iteration i with the discriminator's quality, using the domain classification loss as a surrogate. Coupling influence with loss reduces the impact on the main network when discriminator performance is poor, thus achieving more stable training. The proposed implementation method for this coupling is... , By using the negative log-likelihood loss of the discriminator Mapping to Impact Factor The exponential function, and the coupling scheme of this invention also simplifies the hyperparameters in the original formula. This simplifies the process of antagonistic adaptation.

[0038] In instance-level feature adversarial training, the SPA-bbox method is proposed, which assigns regions to their most overlapping ground truth labels. Only regions with an overlap IoU greater than a threshold t are considered. Unlike assigning regions to foreground and background prototypes, the offset along each bounding box offset dimension is calculated. For each dimension d, features along the positive offset +Δd and negative offset -Δd are aggregated into the offset prototype.

[0039] , Representative characteristics The contribution weight.

[0040] The four pairs of positive / negative offset prototypes obtained in different offset dimensions are not mutually exclusive (e.g. and (This can be applied to the same region). Therefore, contrastive losses between different categories should not be used to separate prototypes on different dimensions. Instead, a separate contrastive loss is defined for each dimension. This is similar to using four separate SPA-bbox heads, each for another offset dimension. The only difference is that all prototypes are generated from the same feature embeddings (all dimensions use the same initial fully connected layer). The impact factor of each pair of losses is reduced by 0.25 times.

[0041] During testing, the domain adaptation component was removed, and the trained cross-domain student model was used to detect pedestrians in top-down fisheye images. During testing, the original Faster R-CNN architecture with trained adaptive weights could be used for object detection, while the domain adaptation component was removed. Performance was evaluated using COCO-style mean accuracy (mAP, AP@75, AP@50), and qualitative results on the LOAF dataset are as follows. Figure 3 As shown.

[0042] S4: Based on the final cross-domain student model, detect pedestrians in the top-down fisheye image to be detected.

[0043] Example 2 This invention also provides a multi-level domain adaptive pedestrian detection system based on a teacher model, used to implement the method of Embodiment 1, comprising: The target detector initialization module is used to train the target detector using source domain perspective image data with pedestrian annotations to obtain the initial target detector; The dual-model construction module is used to replicate the initial object detector and construct a target domain teacher model and a cross-domain student model respectively. The target domain teacher model only takes a weakly enhanced target domain top-view fisheye image as input, while the cross-domain student model takes a strongly enhanced source domain perspective image and a target domain top-view fisheye image as input. The student model training module is used to conduct multi-level domain adaptive adversarial training using cross-domain student models. It updates the parameters of the target domain teacher model by the exponential moving average of the weights of the cross-domain student models, generates pseudo-labels to supervise the multi-level domain adaptive adversarial training of the cross-domain student models, and optimizes the cross-domain student models by combining the designed loss function to obtain the final cross-domain student model. The pedestrian detection module is used to detect pedestrians in the top-down fisheye image based on the final cross-domain student model.

[0044] A further implementation involves a student model training module where multi-level domain adaptive adversarial training includes image-level feature adversarial training, instance-level feature adversarial training, and domain-level feature adversarial training; wherein, the method for image-level feature adversarial training using a cross-domain student model includes: A weighted multi-layer feature alignment module is constructed using an adversarial domain classifier, and the loss of each adversarial domain classifier is calculated to quantify the transferability of the feature layer and calculate the weighting factor of the adversarial domain classifier; where transferability refers to the contribution of knowledge contained in image-level features to cross-domain adaptation. Based on the weighting factor, the objective function of the weighted multi-layer feature alignment module is calculated to complete image-level feature adversarial training.

[0045] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A multi-level domain adaptive pedestrian detection method based on a teacher model, characterized in that, include: The target detector is trained using source domain perspective image data with pedestrian annotations to obtain the initial target detector; The initial target detector is replicated to construct a target domain teacher model and a cross-domain student model, respectively; wherein, the target domain teacher model only inputs a weakly enhanced target domain top-view fisheye image, and the cross-domain student model simultaneously inputs a strongly enhanced source domain perspective image and a target domain top-view fisheye image; Multi-level domain adaptive adversarial training is performed using the cross-domain student model, and the parameters of the target domain teacher model are updated by the exponential moving average of the weights of the cross-domain student model. Pseudo-labels are generated to supervise the multi-level domain adaptive adversarial training of the cross-domain student model. The cross-domain student model is optimized by combining the designed loss function to obtain the final cross-domain student model. Based on the final cross-domain student model, pedestrians are detected in the top-down fisheye image to be detected.

2. The method according to claim 1, characterized in that, The weak enhancement process includes random horizontal flipping and cropping of the image; The enhanced processing includes random color jittering, grayscale conversion, Gaussian blurring, and cropping patching of the image.

3. The method according to claim 1, characterized in that, The multi-level domain adaptive adversarial training includes image-level feature adversarial training, instance-level feature adversarial training, and domain-level feature adversarial training; wherein, the method for image-level feature adversarial training using the cross-domain student model includes: A weighted multi-layer feature alignment module is constructed using an adversarial domain classifier, and the loss of each adversarial domain classifier is calculated to quantify the transferability of the feature layer and calculate the weighting factor of the adversarial domain classifier; wherein, the transferability refers to the contribution of the knowledge contained in the image-level features to cross-domain adaptation; Based on the weighting factors, the objective function of the weighted multi-layer feature alignment module is calculated to complete image-level feature adversarial training.

4. The method according to claim 2, characterized in that, The method for instance-level feature adversarial training using the cross-domain student model includes: In Faster R-CNN, region proposals are generated through a region proposal network to describe the foreground and background of the input image; The adjacency matrix is ​​derived based on the intersection-union ratio of regional proposals, and a relationship graph is constructed based on the spatial correlation provided by the adjacency matrix. The feature embeddings and classification confidence of regional proposals are aggregated based on the relationship graph to obtain the aggregation result. Based on the aggregation results, a category prototype is constructed, and a supervised prototype alignment method is used to achieve cross-domain instance-level feature alignment.

5. The method according to claim 2, characterized in that, The method for domain-level feature adversarial training using the aforementioned cross-domain student model includes: Geometric distortion is applied to the source domain perspective image using a fisheye camera model to generate a pseudo fisheye image as an auxiliary domain. Calculate the image-level triplet loss to make the distance between the target domain image-level features and the auxiliary domain features in the metric space smaller than the distance between them and the source domain features; at the same time, calculate the object-level triplet loss to make the distance between the target domain object-level features and the auxiliary domain object-level features smaller than the distance between them and the source domain object-level features. By jointly optimizing the image-level triplet loss and the object-level triplet loss, the model is driven to learn domain-invariant features, thus completing domain-level feature adversarial training.

6. The method according to claim 1, characterized in that, The loss function includes supervised loss, unsupervised pseudo-label loss, and multi-level adversarial loss.

7. A multi-level domain adaptive pedestrian detection system based on a teacher model, used to implement the method described in any one of claims 1-6, characterized in that, include: The target detector initialization module is used to train the target detector using source domain perspective image data with pedestrian annotations to obtain the initial target detector; A dual-model construction module is used to replicate the initial target detector and construct a target domain teacher model and a cross-domain student model respectively; wherein, the target domain teacher model only inputs a weakly enhanced target domain top-view fisheye image, and the cross-domain student model simultaneously inputs a strongly enhanced source domain perspective image and a target domain top-view fisheye image; The student model training module is used to perform multi-level domain adaptive adversarial training using the cross-domain student model, and to update the parameters of the target domain teacher model by the exponential moving average of the weights of the cross-domain student model, generate pseudo-labels to supervise the multi-level domain adaptive adversarial training of the cross-domain student model, and optimize the cross-domain student model by combining the designed loss function to obtain the final cross-domain student model. The pedestrian detection module is used to detect pedestrians in the top-down fisheye image based on the final cross-domain student model.

8. The system according to claim 7, characterized in that, In the student model training module, the multi-level domain adaptive adversarial training includes image-level feature adversarial training, instance-level feature adversarial training, and domain-level feature adversarial training; wherein, the method for image-level feature adversarial training using the cross-domain student model includes: A weighted multi-layer feature alignment module is constructed using an adversarial domain classifier, and the loss of each adversarial domain classifier is calculated to quantify the transferability of the feature layer and calculate the weighting factor of the adversarial domain classifier; wherein, the transferability refers to the contribution of the knowledge contained in the image-level features to cross-domain adaptation; Based on the weighting factors, the objective function of the weighted multi-layer feature alignment module is calculated to complete image-level feature adversarial training.

Images