Fusion models, fusion methods, training methods, devices, equipment and media

Abstract

This invention provides a fusion model, fusion method, training method, apparatus, device, and medium. The fusion model includes at least a multi-head detector and a fusion unit. The multi-head detector includes several detector heads. Each detector head is used to label several target datasets input to the fusion model to obtain target labels corresponding to the target datasets. The fusion unit is used to fuse the target datasets and their corresponding target labels, outputting a fused dataset for the several target datasets. This addresses or partially addresses the problem of low-quality fused datasets caused by directly merging datasets from different datasets due to their different label spaces, resulting in numerous unlabeled targets, mislabeled targets, and poor label accuracy.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a multi-dataset fusion model, a domain-adaptive data fusion method, a training method for a multi-dataset fusion model, a domain-adaptive data fusion device, a training device for a multi-dataset fusion model, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In the field of machine learning, data is often the key factor determining model performance. A high-precision detection model requires a large amount of labeled target data. However, the high cost of customized data collection and labeling greatly limits the application of models in various fields. But each field has a certain amount of existing data, namely labeled datasets accumulated in historical customized tasks. Although their corresponding label spaces may differ, they can contain the same type of targets. This allows different datasets to form supplementary datasets for their respective fields. By making reasonable use of this existing data, the cost of data collection and labeling can be greatly reduced.

[0003] The difference in label spaces makes it impossible to directly merge datasets for model training. For example, directly merging datasets can result in a large number of unlabeled targets in the training dataset, causing the model to miss detections. Common methods include: manually merging datasets and labeling and modifying all data, which is time-consuming, labor-intensive, and costly, and cannot meet the needs of practical applications; and automatically labeling the dataset, but this also suffers from missed or incorrect labels, and the labeled labels often have low accuracy, seriously affecting the overall quality of the dataset. Summary of the Invention

[0004] The embodiments of the present invention provide a fusion model, fusion method, training method, apparatus, device and medium to solve or partially solve the problem of low quality of fused datasets due to missing labels, mislabeling and poor label accuracy in the dataset annotation.

[0005] This invention discloses a domain-adaptive multi-dataset fusion model, which includes at least a multi-head detector and a fusion unit. The multi-head detector comprises a plurality of detector heads.

[0006] The detection head is used to label several target datasets input into the multi-dataset fusion model to obtain target labels corresponding to the target datasets;

[0007] The fusion unit is used to fuse the target datasets and the target labels corresponding to the target datasets, and output a fused dataset for the target datasets.

[0008] Optionally, each detection head corresponds to a preset dataset; wherein, the multi-dataset fusion model further includes a feature extractor;

[0009] The feature extractor is used to extract features from several target datasets input into the multi-dataset fusion model to obtain the first data features corresponding to the target datasets;

[0010] The detection head is also used to label the datasets based on the first data features corresponding to other target datasets besides the preset dataset corresponding to the detection head, and output pseudo-labels corresponding to the target datasets.

[0011] Optionally, each target dataset corresponds to an original label, the original label including the original detection bounding box in the target dataset, and the pseudo-label including the pseudo detection bounding box in the target dataset; the fusion unit includes a label fusion unit and a dataset fusion unit; wherein,

[0012] The label fusion processor is configured to: merge the original labels and pseudo labels to obtain the target label corresponding to the target dataset if the overlapping areas between the original detection boxes and the pseudo detection boxes, as well as the overlapping areas between each pseudo detection box, in the same target dataset are all less than a third preset threshold; merge the original labels and pseudo labels to obtain the target label corresponding to the target dataset if the overlapping areas between some pseudo detection boxes in the same target dataset are greater than or equal to the third preset threshold, and take the overlapping areas greater than or equal to the third preset threshold as the target overlapping areas, and take the pseudo detection boxes corresponding to the target overlapping areas as overlapping detection boxes; merge the original detection boxes and other pseudo detection boxes in the target overlapping areas (excluding the overlapping detection boxes) to obtain the target label corresponding to the target dataset if the original detection boxes are not present in the target overlapping areas; and merge the confidence scores of each overlapping detection box, and merge the original detection boxes and / or pseudo detection boxes corresponding to the remaining overlapping areas, as well as the overlapping detection box with the highest confidence score in the target overlapping areas, to obtain the target label corresponding to the target dataset; and fuse the target labels corresponding to each target dataset to obtain the fused labels corresponding to the plurality of target datasets.

[0013] The dataset fusion processor is used to merge the various target datasets to obtain a fused dataset corresponding to the plurality of target datasets.

[0014] Optionally, the multi-dataset fusion model includes a multi-head classifier; wherein,

[0015] The feature extractor is also used to extract features from several training datasets input to the multi-dataset fusion model to obtain second data features corresponding to the training datasets;

[0016] The detection head is also used to train a model based on the second data features corresponding to the training dataset corresponding to the detection head, and to obtain the first classification information corresponding to the training dataset;

[0017] The multi-head classifier is used to train the model based on the second data features corresponding to each training dataset to obtain the second classification information corresponding to the training dataset, and to perform adversarial training on the multi-dataset fusion model based on the first classification information and the second classification information.

[0018] Optionally, the multi-head classifier is further configured to obtain an image-level loss function, an instance-level loss function, and a consistency constraint loss corresponding to the multi-data fusion model, and to perform adversarial training on the multi-data fusion model based on the first classification information and the second classification information, and to calculate a first loss value corresponding to the image-level loss function, a second loss value corresponding to the instance-level loss function, and a third loss value corresponding to the consistency constraint loss. When the first loss value, the second loss value, and the third loss value reach the convergence condition, the model training is stopped, and the multi-dataset fusion model is obtained.

[0019] This invention also discloses a domain-adaptive data fusion method, comprising:

[0020] Acquire several target datasets and a multi-dataset fusion model, wherein the multi-dataset fusion model includes a multi-head detector, and the multi-head detector includes several detection heads;

[0021] Each of the target datasets is input into the multi-dataset fusion model, and the target datasets are labeled according to the several detection heads to obtain the target labels corresponding to the target datasets;

[0022] The target datasets and their corresponding target labels are fused together to output a fused dataset for the target datasets.

[0023] Optionally, it also includes:

[0024] Obtain the original labels corresponding to each of the target datasets, the set of detection boxes corresponding to each of the target labels, and the initial detection model. The original labels include the original detection boxes corresponding to the target datasets.

[0025] The original detection boxes corresponding to each of the target datasets are matched with the corresponding set of detection boxes to obtain the overlapping area between the original detection boxes and the set of detection boxes;

[0026] Based on the overlapping area between the original detection boxes and the set of detection boxes in each of the target datasets, a target loss function corresponding to the target dataset is determined. The target loss function includes the original loss function and the pseudo-label loss function.

[0027] The initial detection model is trained based on the original loss function and the pseudo-label loss function to obtain a first loss value corresponding to the original loss function and a second loss value corresponding to the pseudo-label loss function.

[0028] When both the first loss value and the second loss value reach the convergence condition, the model iteration of the initial detection model is stopped, and the target detection model is obtained.

[0029] Optionally, determining the target loss function corresponding to the target dataset based on the overlapping area between the original detection boxes and the set of detection boxes in each target dataset includes:

[0030] The set of detection boxes with overlapping regions greater than or equal to a first preset threshold is taken as the original set of detection boxes, and the original loss function corresponding to the original set of detection boxes is obtained.

[0031] The set of detection boxes with overlapping areas smaller than the first preset threshold is taken as the target detection box set, and the set of pseudo-label boxes corresponding to the target detection box set is obtained. The target detection box set and the set of pseudo-label boxes are used to filter the detection boxes to obtain the set of pseudo-label ground truth values ​​corresponding to the target detection box set, and the pseudo-label loss function corresponding to the set of pseudo-label ground truth values ​​is obtained.

[0032] Optionally, the target detection box set includes target detection boxes, the pseudo-label box set includes pseudo-label boxes, and the step of filtering detection boxes using the target detection box set and the pseudo-label box set to obtain a pseudo-label truth value set corresponding to the target detection box set includes:

[0033] Calculate the intersection-union ratio (IUR) between each target detection box and each pseudo-label box, and take the pseudo-label boxes with IUR greater than or equal to a second preset threshold as the ground truth labels of the corresponding target detection boxes to obtain the pseudo-label ground truth set corresponding to each target detection box.

[0034] Optionally, training the initial detection model based on the original loss function and the pseudo-label loss function to obtain a first loss value corresponding to the original loss function and a second loss value corresponding to the pseudo-label loss function includes:

[0035] The original set of detection boxes and the first target dataset corresponding to the original set of detection boxes are input into the initial detection model for model training, and the original loss function is used to calculate the first loss value corresponding to each detection box in the original set of detection boxes;

[0036] The initial detection model is trained by outputting the pseudo-label ground truth set of the non-empty set and the second target dataset corresponding to the pseudo-label ground truth set of the non-empty set, and the second loss value corresponding to each detection box in the pseudo-label ground truth set of the non-empty set is calculated by using the pseudo-label loss function.

[0037] Optionally, the multi-head detector further includes a feature extractor, each of the detector heads corresponding to a preset dataset, and the step of labeling the target dataset according to a plurality of the detector heads to obtain the target label corresponding to the target dataset includes:

[0038] The feature extractor is used to extract features from each of the target datasets to obtain the data features corresponding to the target datasets.

[0039] Each of the aforementioned detection heads is used to annotate the data features corresponding to other target datasets in the plurality of target datasets, excluding the preset dataset corresponding to the detection head, to obtain pseudo-labels corresponding to the target datasets output by each of the aforementioned detection heads. The pseudo-labels are included in the pseudo-detection boxes in their respective target datasets.

[0040] Obtain the original labels corresponding to each of the target datasets, wherein the original labels include the original detection boxes in the target datasets;

[0041] The labels are merged based on the positional relationship between the original detection boxes and the pseudo detection boxes to obtain the target labels corresponding to the target dataset.

[0042] Optionally, the positional relationship represents the overlapping region of the detection boxes in the target dataset, and the step of merging labels based on the positional relationship between the original detection boxes and the pseudo detection boxes to obtain the target labels corresponding to the target dataset includes:

[0043] If the overlapping area between the original detection box and the pseudo detection box and the overlapping area between each pseudo detection box in the same target dataset are both less than the third preset threshold, then the original label and the pseudo label are merged to obtain the target label corresponding to the target dataset.

[0044] If there is an overlapping region between some of the pseudo detection boxes in the same target dataset that is greater than or equal to the third preset threshold, then the overlapping region that is greater than or equal to the third preset threshold is taken as the target overlapping region, and the pseudo detection box corresponding to the target overlapping region is taken as the overlapping detection box.

[0045] If the target overlapping region contains an original detection box from the original label corresponding to the target dataset, then the original detection box and other pseudo detection boxes in the pseudo detection box except for the overlapping detection box are merged to obtain the target label corresponding to the target dataset.

[0046] If the original detection box does not exist in the target overlapping region, the confidence level of each overlapping detection box is obtained, and the original detection boxes and / or pseudo detection boxes corresponding to the remaining overlapping regions and the overlapping detection box with the highest confidence level in the target overlapping region are merged to obtain the target label corresponding to the target dataset.

[0047] This invention also discloses a model training method for a multi-dataset fusion model, comprising:

[0048] Acquire several training datasets and an initial multi-dataset fusion model. The initial multi-dataset fusion model includes a multi-domain classification head, a feature extractor, and a multi-head detector. The multi-head detector includes several detection heads, and each detection head corresponds to a training dataset.

[0049] The training datasets are input into the feature extractor respectively to obtain the data features corresponding to each training dataset;

[0050] The data features corresponding to each training dataset are input into the corresponding detection head to train the detection head and obtain the first classification information corresponding to each training dataset.

[0051] The data features corresponding to each of the training datasets are input into the multi-domain classification head for training to obtain the second classification information corresponding to each of the training datasets.

[0052] The initial multi-data fusion model is subjected to adversarial training based on the first classification information and the second classification information to generate the multi-data fusion model.

[0053] Optionally, the step of performing adversarial training on the initial multi-data fusion model based on the first classification information and the second classification information to generate the multi-data fusion model includes:

[0054] Obtain the adversarial classification loss function corresponding to the multi-domain classification head, wherein the adversarial classification loss function includes an image-level loss function and an instance-level loss function;

[0055] Obtain the consistency constraint loss corresponding to the initial multi-data fusion model;

[0056] The initial multi-data fusion model is subjected to adversarial training based on the first classification information and the second classification information. A first loss value corresponding to the image-level loss function, a second loss value corresponding to the instance-level loss function, and a third loss value corresponding to the consistency constraint loss are calculated. When the first loss value, the second loss value, and the third loss value reach the convergence condition, the model training is stopped, and the multi-dataset fusion model is obtained.

[0057] This invention also discloses a domain-adaptive data fusion apparatus, comprising:

[0058] The dataset acquisition module is used to acquire several target datasets and a multi-dataset fusion model, wherein the multi-dataset fusion model includes a multi-head detector, and the multi-head detector includes several detection heads;

[0059] The dataset annotation module is used to input each of the target datasets into the multi-dataset fusion model, annotate the target datasets according to a number of detection heads, and obtain the target labels corresponding to the target datasets;

[0060] The dataset fusion module is used to fuse the target datasets and their corresponding target labels to output a fused dataset for the target datasets.

[0061] Optionally, it also includes:

[0062] The first model acquisition module is used to acquire the original labels corresponding to each of the target datasets, the set of detection boxes corresponding to each of the target labels, and the initial detection model. The original labels include the original detection boxes corresponding to the target datasets.

[0063] The overlapping region determination module is used to match the original detection boxes corresponding to each of the target datasets with the corresponding set of detection boxes to obtain the overlapping region between the original detection boxes and the set of detection boxes.

[0064] The loss function determination module is used to determine the target loss function corresponding to the target dataset based on the overlapping area between the original detection box and the detection box set in each target dataset. The target loss function includes the original loss function and the pseudo-label loss function.

[0065] The loss value calculation module is used to train the initial detection model based on the original loss function and the pseudo-label loss function to obtain the first loss value corresponding to the original loss function and the second loss value corresponding to the pseudo-label loss function.

[0066] The model iteration module is used to stop iterating the initial detection model when both the first loss value and the second loss value reach the convergence condition, thereby obtaining the target detection model.

[0067] Optionally, the loss function determination module is specifically used for:

[0068] The set of detection boxes with overlapping regions greater than or equal to a first preset threshold is taken as the original set of detection boxes, and the original loss function corresponding to the original set of detection boxes is obtained.

[0069] The set of detection boxes with overlapping areas smaller than the first preset threshold is taken as the target detection box set, and the set of pseudo-label boxes corresponding to the target detection box set is obtained. The target detection box set and the set of pseudo-label boxes are used to filter the detection boxes to obtain the set of pseudo-label ground truth values ​​corresponding to the target detection box set, and the pseudo-label loss function corresponding to the set of pseudo-label ground truth values ​​is obtained.

[0070] Optionally, the target detection box set includes target detection boxes, the pseudo-label box set includes pseudo-label boxes, and the loss function determination module is specifically used for:

[0071] Calculate the intersection-union ratio (IUR) between each target detection box and each pseudo-label box, and take the pseudo-label boxes with IUR greater than or equal to a second preset threshold as the ground truth labels of the corresponding target detection boxes to obtain the pseudo-label ground truth set corresponding to each target detection box.

[0072] Optionally, the loss value calculation module is specifically used for:

[0073] The original set of detection boxes and the first target dataset corresponding to the original set of detection boxes are input into the initial detection model for model training, and the original loss function is used to calculate the first loss value corresponding to each detection box in the original set of detection boxes;

[0074] The initial detection model is trained by outputting the pseudo-label ground truth set of the non-empty set and the second target dataset corresponding to the pseudo-label ground truth set of the non-empty set, and the second loss value corresponding to each detection box in the pseudo-label ground truth set of the non-empty set is calculated by using the pseudo-label loss function.

[0075] Optionally, the multi-head detector further includes a feature extractor, where each detector head corresponds to a preset dataset, and the dataset annotation module is specifically used for:

[0076] The feature extractor and each of the detection heads are used to annotate other target datasets in the plurality of target datasets except for the preset dataset corresponding to the detection head, to obtain pseudo labels corresponding to the target datasets output by each of the detection heads, and the pseudo labels include pseudo detection boxes in their respective target datasets;

[0077] Obtain the original labels corresponding to each of the target datasets, wherein the original labels include the original detection boxes in the target datasets;

[0078] The labels are merged based on the positional relationship between the original detection boxes and the pseudo detection boxes to obtain the target labels corresponding to the target dataset.

[0079] Optionally, the positional relationship represents the overlapping region of the detection boxes in the target dataset, and the dataset annotation module is specifically used for:

[0080] If the overlapping area between the original detection box and the pseudo detection box and the overlapping area between each pseudo detection box in the same target dataset are all less than the third preset threshold, then the pseudo labels are merged to obtain the target label corresponding to the target dataset.

[0081] If there is an overlapping region between some of the pseudo detection boxes in the same target dataset that is greater than or equal to the third preset threshold, then the overlapping region that is greater than or equal to the third preset threshold is taken as the target overlapping region, and the pseudo detection box corresponding to the target overlapping region is taken as the overlapping detection box.

[0082] If the target overlapping region contains an original detection box from the original label corresponding to the target dataset, then the original detection box and other pseudo detection boxes in the pseudo detection box except for the overlapping detection box are merged to obtain the target label corresponding to the target dataset.

[0083] If the original detection box does not exist in the target overlapping region, the confidence level of each overlapping detection box is obtained, and the original detection boxes and / or pseudo detection boxes corresponding to the remaining overlapping regions and the overlapping detection box with the highest confidence level in the target overlapping region are merged to obtain the target label corresponding to the target dataset.

[0084] This invention also discloses a model training apparatus for a multi-dataset fusion model, comprising:

[0085] The second model acquisition module is used to acquire several training datasets and an initial multi-dataset fusion model. The initial multi-dataset fusion model includes a multi-domain classification head, a feature extractor, and a multi-head detector. The multi-head detector includes several detection heads, and each detection head corresponds to a training dataset.

[0086] The feature extraction module is used to input the training datasets into the feature extractor respectively to obtain the data features corresponding to each training dataset;

[0087] The first training module is used to input the data features corresponding to each training dataset into the corresponding detection head, train the detection head, and obtain the first classification information corresponding to each training dataset.

[0088] The second training module is used to input the data features corresponding to each training dataset into the multi-domain classification head for training, and to obtain the second classification information corresponding to each training dataset.

[0089] The model generation module is used to perform adversarial training on the initial multi-data fusion model based on the first classification information and the second classification information to generate the multi-data fusion model.

[0090] Optionally, the model generation module is specifically used for:

[0091] Obtain the adversarial classification loss function corresponding to the multi-domain classification head, wherein the adversarial classification loss function includes an image-level loss function and an instance-level loss function;

[0092] Obtain the consistency constraint loss corresponding to the initial multi-data fusion model;

[0093] The initial multi-data fusion model is subjected to adversarial training based on the first classification information and the second classification information. A first loss value corresponding to the image-level loss function, a second loss value corresponding to the instance-level loss function, and a third loss value corresponding to the consistency constraint loss are calculated. When the first loss value, the second loss value, and the third loss value reach the convergence condition, the model training is stopped, and the multi-dataset fusion model is obtained.

[0094] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0095] The memory is used to store computer programs;

[0096] When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0097] This invention also discloses a computer-readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0098] The embodiments of the present invention have the following advantages:

[0099] In this embodiment of the invention, during the fusion of different datasets, several target datasets to be fused and a multi-dataset fusion model can be obtained. The multi-dataset fusion model can include a multi-head detector, which can include several detection heads. Then, each target dataset can be input into the multi-dataset fusion model, and the detection heads in the multi-head detector can be used to label the target datasets to obtain the target labels corresponding to each target dataset. Labeling the datasets with different detection heads ensures that each dataset has a corresponding target label, guaranteeing the comprehensiveness of the dataset labeling. On the other hand, different detection heads can effectively improve the detection accuracy of the model in different scenarios and domains, effectively extract features from different datasets, reduce label noise, improve the accuracy of dataset labeling, and ensure data quality. Therefore, when fusing the various target datasets and their corresponding target labels to obtain the fused dataset, the quality of the fused dataset can be effectively guaranteed. Attached Figure Description

[0100] Figure 1 This is a flowchart illustrating the steps of a domain-adaptive data fusion method provided in this embodiment of the invention.

[0101] Figure 2 This is a schematic diagram of a multi-head detector provided in an embodiment of the present invention;

[0102] Figure 3 This is a schematic diagram of the multi-dataset fusion model provided in an embodiment of the present invention;

[0103] Figure 4 This is a schematic diagram of an image annotation provided in an embodiment of the present invention;

[0104] Figure 5 This is a schematic diagram of the detection frame provided in an embodiment of the present invention;

[0105] Figure 6 This is a schematic diagram of the detection frame provided in an embodiment of the present invention;

[0106] Figure 7 This is a schematic diagram of the detection frame provided in an embodiment of the present invention;

[0107] Figure 8 This is a schematic diagram of the detection frame provided in an embodiment of the present invention;

[0108] Figure 9 This is a structural block diagram of a domain-adaptive data fusion device provided in an embodiment of the present invention;

[0109] Figure 10This is a structural block diagram of a training device for a multi-data fusion model provided in an embodiment of the present invention.

[0110] Figure 11 This is a block diagram of an electronic device provided in an embodiment of the present invention;

[0111] Figure 12 This is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. Detailed Implementation

[0112] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0113] As an example, in the field of machine learning, data is often the key factor determining model performance. A high-precision experimental model requires a large amount of labeled target data. However, the high cost of customized data collection and labeling greatly limits the application of models in various fields. But each field has a certain amount of existing data, namely labeled datasets accumulated in historical customized tasks. Although their corresponding label spaces may differ, they can contain the same type of targets, allowing different datasets to form supplementary datasets for the corresponding fields. By making reasonable use of this existing data, the cost of data collection and labeling can be greatly reduced.

[0114] The difference in label spaces makes it impossible to directly merge datasets for model training. For example, directly merging datasets can lead to a large number of unlabeled targets in the training dataset, causing the model to miss detections. Common methods include: manually merging datasets and labeling and modifying all data, which is time-consuming, labor-intensive, and costly, and cannot meet the needs of practical applications; or automatically labeling the dataset, but during automatic labeling, different datasets may have different performance characteristics depending on the specific scenario they represent, resulting in lower labeling accuracy. Labeling datasets through the above process and then merging the labeled data is prone to problems such as missing labels, mislabeling, and low label accuracy, leading to poor quality of the merged dataset and consequently affecting the model quality trained using the merged dataset.

[0115] One of the core inventive points of this invention lies in training a domain-adaptive multi-dataset fusion model. This model can include a multi-head detector, which can be trained using several different detector heads, each trained on a specific dataset. During the fusion of the target datasets, the detector heads in the multi-head detector can be used to label the target datasets, obtaining target labels for each dataset. By labeling the datasets with different detector heads, on the one hand, each dataset has a corresponding target label, ensuring the comprehensiveness of the dataset labeling; on the other hand, different detector heads can effectively improve the detection accuracy of the model in different scenarios and domains, effectively extract features from different datasets, reduce label noise, improve the accuracy of dataset labeling, and ensure data quality. Therefore, when fusing the various target datasets and their corresponding target labels to obtain a fused dataset, the quality of the fused dataset can be effectively guaranteed.

[0116] To enable those skilled in the art to better understand the technical solutions of the embodiments of the present invention, some technical features involved in the embodiments are explained and described below:

[0117] A multi-dataset fusion model can be used to label different datasets and, after obtaining the target labels for each dataset, merge the datasets and their corresponding target labels to obtain a fused dataset. For example, if the datasets to be merged include dataset A, dataset B, and dataset C, the multi-dataset fusion model can be used to label dataset A, dataset B, and dataset C separately to obtain target label 'a' for dataset A, target label 'b' for dataset B, and target label 'c' for dataset C. Then, datasets A, B, and C, along with target labels 'a', 'b', and 'c', can be merged to obtain the fused dataset.

[0118] The detection model can be an image detection model, an audio detection model, a video detection model, or a text detection model, etc. For example, a detection model trained on a fused dataset can perform image detection in different domain scenarios and obtain corresponding detection results. For example, the detection model can perform face detection, industrial detection, etc. It is understood that when the fused dataset is an image dataset, the detection model trained on the fused dataset can be an image detection model; when the fused dataset is an audio dataset, the detection model trained on the fused dataset can be an audio detection model; when the fused dataset is a video dataset, the detection model trained on the fused dataset can be a video detection model; when the fused dataset is a text dataset, the detection model trained on the fused dataset can be a text detection model, or it can be a fusion model of the aforementioned detection types, etc., and this invention does not impose any limitations on this.

[0119] A dataset can include image datasets, audio datasets, video datasets, and text datasets, etc. The target dataset can be the dataset used to train a detection model; the training dataset can be the dataset used to train a multi-dataset fusion model. Both can be the same dataset or different datasets.

[0120] Labels can include original labels and pseudo-labels. Original labels can be labels manually assigned to the dataset; pseudo-labels can be labels assigned by the model. The confidence level of original labels should not be lower than that of pseudo-labels. Furthermore, the target label can include both original and pseudo-labels, or it can contain only pseudo-labels.

[0121] A detection box can be an image box that labels the target object in the label. For example, if the target image includes a pedestrian, a vehicle, etc., then the detection box for the target image can include an image box labeled for the pedestrian and an image box labeled for the vehicle, etc.

[0122] The label space can be the set of all labels contained in the dataset.

[0123] The first preset threshold targets the overlapping area between the detection boxes in the original label and the detection boxes in the pseudo label, emphasizing the overlap between labels; the third preset threshold targets the overlapping area between two detection boxes at the same location, emphasizing the overlap between detection boxes; the second preset threshold targets the intersection-union ratio between two detection boxes.

[0124] It should be noted that the embodiments of the present invention use an image dataset and an image detection model as examples for illustrative purposes. It is understood that the present invention does not impose any limitations on these examples.

[0125] Specifically, refer to Figure 1 The diagram illustrates a flowchart of a domain-adaptive data fusion method provided in an embodiment of the present invention, which may specifically include the following steps:

[0126] Step 101: Obtain several target datasets and a multi-dataset fusion model, wherein the multi-dataset fusion model includes a multi-head detector, and the multi-head detector includes several detection heads;

[0127] For a multi-dataset fusion model, it can include a multi-head detector, which can include several different detector heads. Each detector head can be trained using a corresponding dataset. For example, detector head A can be trained using dataset A, detector head B can be trained using dataset B, and detector head C can be trained using dataset C, etc.

[0128] In one optional embodiment, the multi-dataset fusion model includes at least a multi-head detector and a fusion unit. The multi-head detector includes several detection heads. The detection heads are used to label several target datasets input into the multi-dataset fusion model to obtain target labels corresponding to the target datasets. The fusion unit is used to fuse the target datasets and the target labels corresponding to the target datasets to output a fused dataset for the several target datasets.

[0129] In a specific implementation, each detection head corresponds to a preset dataset; wherein, the multi-dataset fusion model further includes a feature extractor, which is used to extract features from several target datasets input into the multi-dataset fusion model to obtain the first data features corresponding to the target datasets. Furthermore, the detection head is also used to label the datasets based on the first data features corresponding to other target datasets besides the preset dataset corresponding to the detection head, and output pseudo-labels corresponding to the target datasets.

[0130] Furthermore, each target dataset corresponds to an original label, which includes the original detection boxes in the target dataset, and the pseudo-label includes the pseudo-detection boxes in the target dataset. The fusion unit includes a label fusion unit and a dataset fusion unit. Specifically, the label fusion unit is used to merge the original label and the pseudo-label if the overlapping area between the original detection boxes and the pseudo-detection boxes in the same target dataset, as well as the overlapping area between each pseudo-detection box, is less than a third preset threshold, to obtain the target label corresponding to the target dataset. If the overlapping area between some pseudo-detection boxes in the same target dataset is greater than or equal to the third preset threshold, then the overlapping area greater than or equal to the third preset threshold is taken as the target overlapping area, and the pseudo-detection boxes corresponding to the target overlapping area are taken as overlapping detections. If an original detection box from the original label corresponding to the target dataset exists in the target overlapping region, then the original detection box and other pseudo detection boxes in the pseudo detection boxes, excluding the overlapping detection box, are merged to obtain the target label corresponding to the target dataset; if the original detection box does not exist in the target overlapping region, then the confidence scores corresponding to each overlapping detection box are obtained, and the original detection boxes and / or pseudo detection boxes corresponding to the remaining overlapping regions, as well as the overlapping detection box with the highest confidence score in the target overlapping region, are merged to obtain the target label corresponding to the target dataset; the target labels corresponding to each target dataset are fused to obtain the fused label corresponding to the plurality of target datasets; the dataset fusioner is used to fuse the various target datasets to obtain the fused dataset corresponding to the plurality of target datasets.

[0131] Optionally, during the training of the multi-dataset fusion model, a multi-head classifier can be added to the multi-dataset fusion model. During training, the feature extractor is further used to extract features from several training datasets input to the multi-dataset fusion model to obtain second data features corresponding to the training datasets. The detection head is further used to train the model based on the second data features corresponding to the training datasets corresponding to the detection head, to obtain first classification information corresponding to the training datasets. The multi-head classifier is used to train the model based on the second data features corresponding to each training dataset, to obtain second classification information corresponding to the training datasets, and to perform adversarial training on the multi-dataset fusion model based on the first classification information and the second classification information.

[0132] Specifically, the multi-head classifier is further used to obtain the image-level loss function, the instance-level loss function, and the consistency constraint loss corresponding to the multi-data fusion model. It then performs adversarial training on the multi-data fusion model based on the first classification information and the second classification information, and calculates the first loss value corresponding to the image-level loss function, the second loss value corresponding to the instance-level loss function, and the third loss value corresponding to the consistency constraint loss. When the first loss value, the second loss value, and the third loss value reach the convergence condition, the model training is stopped, and the multi-dataset fusion model is obtained.

[0133] In this embodiment of the invention, the training process for a multi-dataset fusion model may include:

[0134] Several training datasets and an initial multi-dataset fusion model are obtained. The initial multi-dataset fusion model includes a multi-domain classification head, a feature extractor, and a multi-head detector. The multi-head detector includes several detection heads, each corresponding to a training dataset. The training datasets are input into the feature extractor to obtain the data features corresponding to each training dataset. The data features corresponding to each training dataset are input into the corresponding detection head to train the detection head and obtain the first classification information corresponding to each training dataset. The data features corresponding to each training dataset are input into the multi-domain classification head to train and obtain the second classification information corresponding to each training dataset. The initial multi-dataset fusion model is then subjected to adversarial training based on the first and second classification information to generate a multi-dataset fusion model.

[0135] In the training process of the multi-dataset fusion model, multi-domain classification heads and feature extractors can be added to the model. The feature extractors and individual detection heads can form a multi-head model structure. In this structure, by using a feature extractor with shared parameters and constructing a corresponding detection head for each dataset, the label spaces between different datasets can be effectively isolated, ensuring the detection accuracy of the detection head within its corresponding dataset label system. Furthermore, through joint optimization learning across multiple heads, the model's feature capture capability for different scenarios can be improved, enhancing its robustness. This allows for the extraction of more representative features during dataset annotation. Moreover, by having the multi-domain classification head engage in adversarial training with the feature extractor during training, the feature extractor's ability to extract the essential features of different datasets is improved through competition, further overcoming the domain differences between datasets and enhancing the accuracy of model annotation.

[0136] In its implementation, the feature extractor extracts the data features of the training dataset itself and inputs these features into the corresponding detection head for training, obtaining the first classification information for each dataset. This first classification information can be the classification information corresponding to the current category of the dataset; different datasets can correspond to different first classification information. Simultaneously, the data features from each dataset can be input into a multi-domain classification head to extract common features, obtaining the second classification information for each training dataset. This second classification information can be the common feature information across all datasets. After obtaining the corresponding first classification information from each detection head and the corresponding second classification information from the multi-domain classification head, the model can be trained adversarially based on both. This adversarial relationship allows the feature extractor to map the originally separate feature spaces of different domains into a common feature space, reducing the impact of the dataset on the model's detection accuracy.

[0137] Specifically, the adversarial classification loss function corresponding to the multi-domain classification head can be obtained. The adversarial classification loss function includes an image-level loss function and an instance-level loss function. The consistency constraint loss corresponding to the initial multi-data fusion model can also be obtained. Then, the initial multi-data fusion model can be adversarially trained based on the first classification information and the second classification information. The first loss value corresponding to the image-level loss function, the second loss value corresponding to the instance-level loss function, and the third loss value corresponding to the consistency constraint loss can be calculated. When the first loss value, the second loss value, and the third loss value reach the convergence condition, the model training is stopped, and the multi-dataset fusion model is obtained.

[0138] For different dataset classification tasks, different adversarial granularities can be designed for model training, including image-level domain adversarial and instance-level domain adversarial. For image-level domain adversarial, the model network can be constrained using an image-level classification loss function, the underlying idea being that "given an image, regardless of which dataset it belongs to, its detection result is the same." For instance-level domain adversarial, the model network can be constrained using an instance-level classification loss function, the underlying idea being that "given an image region containing an object, regardless of which dataset the image comes from, the object's category label is the same." Furthermore, when using image-level and instance-level domain adaptation, there is a prerequisite: the target region remains unchanged across different domains for the same objective. However, when domain differences exist, this prerequisite cannot be strictly satisfied. Therefore, a consistency constraint loss is used to ensure the results have domain invariance.

[0139] In one example, refer to Figure 2This diagram illustrates a multi-head detector provided in an embodiment of the present invention. It includes a feature extractor, a region proposal network (RPN) connected to the feature extractor, and several detector heads connected to the RPN. By inputting different datasets into the model, the feature extractor extracts features to obtain data features, which are then transmitted to the RPN for region box selection, yielding features corresponding to each dataset. These features are then used to train the corresponding detector heads. Through this detection head training process, a multi-dataset fusion model can be obtained. This multi-dataset fusion model ensures that each dataset has a corresponding target label, guaranteeing comprehensive dataset annotation. Furthermore, different detector heads effectively improve the model's detection accuracy in different scenarios, effectively extracting features from different datasets, reducing label noise, improving dataset annotation accuracy, and ensuring data quality.

[0140] Furthermore, to further improve the detection accuracy of the multi-dataset fusion model, a multi-domain classification head can be added to the model during training to conduct adversarial training with the feature extractor. Specifically, during adversarial training, the domain classification task of the multi-domain classification head can be used to extract features from the source domain (such as dataset A) and the target domain (other datasets besides dataset A) extracted by the feature extractor. Meanwhile, the domain adaptive fusion task of the feature extractor can be used to address features that cannot be effectively classified between the source and target domains. During the adversarial training, a GRL layer (Gradient reversal layer) can be added to the model to create an adversarial relationship between the multi-domain classification head and the model's feature extractor. The multi-domain classification head can distinguish images from different domains, while the feature extractor, under the control of the GRL layer, extracts consistent features from images from different domains. This adversarial relationship allows the feature extractor to map the originally separate feature spaces of different domains into a common feature space, reducing the impact of domain offset on the model's detection accuracy.

[0141] Reference Figure 3The diagram illustrates a multi-dataset fusion model provided in an embodiment of the present invention. It shows a portion of the model's structure, including a multi-head detector and a multi-domain classification head. During model training, domain classification tasks can be designed according to specific tasks. For example, in object detection tasks, image-level and instance-level domain adaptive adversarial mechanisms and consistency constraints between image-level and instance-level mechanisms can be designed to find a better mapping space, fuse features from different domains, and enable the feature extractor to obtain better feature representations at both the image and instance levels while maintaining a certain degree of consistency. Specifically, unlabeled images (i.e., the training dataset) can be input into the model, and features can be extracted using a feature extractor to obtain image-level features. These image-level features can then be input into the GRL layer of the multi-domain classification head to calculate the corresponding loss value using the image-level neighborhood classification loss function. Simultaneously, image-level features can be input into the RPN layer for region selection, and instance-level features can be input into the GRL layer of the multi-domain classification head to calculate the corresponding loss value using the instance-level neighborhood classification loss function. The loss values ​​corresponding to the image level and the instance level are then constrained by a consistency regularization loss function to maintain a certain consistency. Thus, through the adversarial relationship during training, the feature extractor can map the originally separated feature spaces of different neighborhoods into a common feature space, reducing the impact of domain offset on the model's detection accuracy.

[0142] Step 102: Input each of the target datasets into the multi-dataset fusion model, and label the target datasets according to the several detection heads to obtain the target labels corresponding to the target datasets;

[0143] In a multi-head detector, each detector head can correspond to a pre-defined dataset, which can be the dataset used to train the detector head. During model execution, the detector head does not need to label its own pre-defined dataset but can label other datasets. In specific implementations, during the operation of the multi-dataset fusion model, the multi-domain classification head can be removed, and the datasets can be labeled and fused using a feature extractor and each detector head. Specifically, the feature extractor and each detector head can be used to label other target datasets (excluding the pre-defined dataset corresponding to the detector head) to obtain pseudo-labels for the target datasets output by each detector head. The pseudo-labels include pseudo-detection boxes in their respective target datasets. Then, the original labels for each target dataset are obtained, which include the original detection boxes in the target datasets. Finally, the labels can be merged based on the positional relationship between the original detection boxes and the pseudo-detection boxes to obtain the target labels for the target datasets.

[0144] For example, suppose a multi-dataset fusion model includes detector head A, detector head B, detector head C, etc., and the target dataset includes dataset A corresponding to detector head A, dataset B corresponding to detector head B, dataset C corresponding to detector head C, etc. Then, a feature extractor can be used with detector head A to extract features from datasets B and C, obtaining pseudo-labels for datasets B and C; a feature extractor can be used with detector head B to extract features from datasets A and C, obtaining pseudo-labels for datasets A and C; a feature extractor can be used with detector head C to extract features from datasets A and B, obtaining pseudo-labels for datasets A and B, and so on.

[0145] For each target dataset, there may be corresponding original labels, meaning the dataset has already been manually labeled. Therefore, the original labels and several pseudo-labels obtained after model labeling for a dataset need to be merged to remove labels with low confidence and reduce label noise. Specifically, when the dataset is images, the original labels and pseudo-labels can include bounding boxes labeled for the images, as shown below. Figure 4 The diagram illustrates an image annotation method provided in an embodiment of the present invention. The image may include several detection boxes, each of which can be used to annotate target objects (such as pedestrians, vehicles, chairs, etc.) appearing in the image. For the original label and the pseudo label, each may include at least one detection box. There may be overlapping or identical detection boxes between different labels. Therefore, the labels need to be optimized to reduce label noise.

[0146] In the specific implementation, the positional relationship between different detection boxes can characterize the overlapping area of ​​the detection boxes in their respective target datasets. Labels can then be merged based on the positional relationship between the original detection boxes and pseudo-detection boxes to obtain the target label corresponding to the target dataset. Specifically, if the overlapping area between all pseudo-detection boxes in the same target dataset is less than a third preset threshold, the original label and pseudo-label are merged to obtain the target label corresponding to the target dataset. If the overlapping area between some pseudo-detection boxes in the same target dataset is greater than or equal to the third preset threshold, the overlapping area greater than or equal to the third preset threshold is taken as the target overlapping area, and the pseudo-detection boxes corresponding to the target overlapping area are taken as overlapping detection boxes. If the target overlapping area contains original detection boxes from the original label corresponding to the target dataset, the original detection boxes and other pseudo-detection boxes (excluding overlapping detection boxes) are merged to obtain the target label corresponding to the target dataset. If no original detection boxes exist in the target overlapping area, the confidence scores of each overlapping detection box are obtained, and the pseudo-detection boxes corresponding to the remaining overlapping areas are merged with the overlapping detection box with the highest confidence score in the target overlapping area to obtain the target label corresponding to the target dataset.

[0147] In one example, refer to Figure 5 This diagram illustrates a detection box provided in an embodiment of the present invention. Assuming that a corresponding original label exists in an image within the target dataset, then... Figure 5 As shown, the image may include the detection boxes ① and ② of the original label, and the detection boxes ③ and ④ of the pseudo label. At this time, there is no overlap between the detection boxes, so all the detection boxes can be merged (that is, the original label and the pseudo label can be merged) to obtain the target label of the image, including detection boxes ①, ②, ③, and ④.

[0148] like Figure 6 As shown, the detection box ① of the original label overlaps with the detection box ② of the pseudo label, and the overlap area is less than the preset threshold, which indicates that the two detection boxes are labeled with different objects. Since there is no overlap between other detection boxes, all detection boxes can be merged (that is, the original label and the pseudo label are merged) to obtain the target label of the image, including detection box ①, detection box ②, detection box ③, and detection box ④.

[0149] like Figure 7 As shown, the detection box ① of the original label overlaps with the detection box ② of the pseudo label, and the overlapping area is greater than or equal to a preset threshold, which indicates that the two detection boxes label the same object. Since there is no overlap between other detection boxes, for detection boxes ① and ②, since the confidence of the original label is higher than that of the pseudo label, detection box ① is retained first and detection box ② is removed. Then, other detection boxes are merged to obtain the target label of the image, including detection box ①, detection box ③, and detection box ④.

[0150] like Figure 8 As shown, the detection boxes ② and ③ in the pseudo-label overlap, and the overlapping area is greater than or equal to a preset threshold, which indicates that the two detection boxes label the same object. Since there is no overlap between other detection boxes, the corresponding confidence scores can be obtained for detection boxes ② and ③, and the detection box with the highest confidence score can be retained. For example, if the confidence score of detection box ③ is high, other detection boxes are then merged to obtain the target label of the image, including detection boxes ①, ③, and ④.

[0151] It should be noted that the present invention includes, but is not limited to, the above examples. In practice, a target dataset may include multiple images. For each image, the labels can be merged by referring to the above process. Furthermore, for an image, it may or may not have a corresponding original label. The merging process for pseudo-labels can also refer to the above process. The present invention does not impose any limitations on this.

[0152] Step 103: Perform data fusion on each of the target datasets and the target labels corresponding to the target datasets, and output a fused dataset for the plurality of target datasets.

[0153] Optionally, the multi-dataset fusion model may further include a data fusion unit for fusing the datasets and their corresponding target labels. In a specific implementation, the multi-dataset fusion model can fuse each target dataset and its corresponding target labels separately to obtain a fused dataset. This allows for the labeling of the datasets using different detection heads within the multi-dataset fusion model. On one hand, this ensures that each dataset has a corresponding target label, guaranteeing the comprehensiveness of the dataset labeling. On the other hand, different detection heads can effectively improve the model's detection accuracy in different scenarios and domains, effectively extracting features from different datasets, reducing label noise, improving the accuracy of dataset labeling, and ensuring data quality. Therefore, when fusing the various target datasets and their corresponding target labels to obtain the fused dataset, the quality of the fused dataset can be effectively guaranteed.

[0154] In one optional embodiment, after obtaining the fused dataset, the detection model can be trained based on the fused dataset to obtain a model capable of detecting different scene domains. Specifically, the original labels corresponding to each target dataset, the set of detection boxes corresponding to each target label, and the initial detection model can be obtained first. The original labels include the original detection boxes corresponding to the target dataset. Then, the original detection boxes corresponding to each target dataset are matched with the corresponding set of detection boxes to obtain the overlapping area between the original detection boxes and the set of detection boxes. Then, based on the overlapping area between the original detection boxes and the set of detection boxes in each target dataset, the target loss function corresponding to the target dataset is determined. The target loss function includes the original loss function and the pseudo-label loss function. The initial detection model is then trained based on the original loss function and the pseudo-label loss function to obtain the first loss value corresponding to the original loss function and the second loss value corresponding to the pseudo-label loss function. When both the first loss value and the second loss value reach the convergence condition, the model iteration of the initial detection model is stopped to obtain the target detection model. Here, the set of detection boxes can be the set of all detection boxes contained in the target label in the target dataset, which can include the original detection boxes in the original label and the pseudo detection boxes in the pseudo label, or it can only include the pseudo detection boxes in the pseudo label.

[0155] During training, the set of detection boxes corresponding to the target dataset can be matched with the original detection boxes in the original labels to determine whether the original labels corresponding to the target dataset exist in the set of detection boxes. If they do, the normal training process can be carried out based on the original loss function of the detection model; if they do not exist, the training can be carried out based on the pseudo-label loss function to combat the uncertainty of pseudo-labels and average potential noise through multiple matching results.

[0156] Specifically, the set of detection boxes with overlapping regions greater than or equal to a first preset threshold can be used as the original detection box set, and the original loss function corresponding to the original detection box set can be obtained. In this case, it can represent that the similarity between the target label corresponding to the target dataset and the original label is high, and the model can be trained using the original loss function. The set of detection boxes with overlapping regions less than the first preset threshold can be used as the target detection box set, and the set of pseudo-label boxes corresponding to the target detection box set can be obtained. The detection boxes are filtered using the target detection box set and the pseudo-label box set to obtain the pseudo-label ground truth set corresponding to the target detection box set, and the pseudo-label loss function corresponding to the pseudo-label ground truth set can be obtained. In this case, it can ensure that the similarity between the target label corresponding to the target dataset and the original label is low. Therefore, in order to combat the uncertainty of the labeled pseudo-labels and the average potential noise, the model can be trained using the pseudo-label loss function. The pseudo-label box set can include the set of pseudo detection boxes in all pseudo-labels corresponding to the target dataset.

[0157] Optionally, for the screening of pseudo-labeled detection boxes, the intersection-union ratio (IUR) between each target detection box and each pseudo-labeled box is calculated. Pseudo-labeled boxes with an IUR greater than or equal to a second preset threshold are used as the ground truth labels for the corresponding target detection boxes, thus obtaining the pseudo-label ground truth set for each target detection box. Each detection box in the target detection box set can correspond to a pseudo-label ground truth set. This pseudo-label ground truth set can be empty or not. For an empty set, it can be determined that the detection box labels the background of the image. Model training can be performed based on such detection boxes to ensure the detection accuracy of the detection model.

[0158] During training, for the normal training process, the original set of detection boxes and the corresponding first target dataset can be input into the initial detection model for model training, and the original loss function is used to calculate the first loss value corresponding to each detection box in the original set of detection boxes. For the pseudo-label training process, the non-empty set of pseudo-label ground truth values ​​and the corresponding second target dataset can be output into the initial detection model for model training, and the pseudo-label loss function is used to calculate the second loss value corresponding to each detection box in the non-empty set of pseudo-label ground truth values. When both the first loss value and the second loss value reach the convergence condition, the model iteration of the initial detection model is stopped, and the target detection model is obtained. For the training of the detection model, on the one hand, the target dataset is labeled and fused through a multi-dataset fusion model to reduce label noise and effectively ensure the quality of the fused dataset. On the other hand, during the training process, a pseudo-label loss function is added to combat the uncertainty of pseudo-labels and average latent noise, reduce the impact of pseudo-label noise in the dataset on model training, and improve the final detection accuracy of the detection model.

[0159] In one example, for training the detection model, several target datasets can be labeled and fused using a multi-dataset fusion model to obtain a fused dataset. Then, the set of detection boxes D corresponding to the target labels of each target dataset in the fused dataset, as well as the original labels corresponding to the target datasets, are obtained. For each target dataset, the original labels can be matched with the set of detection boxes D. If a match is successful (the matching process can be referred to the previous description), the loss is calculated for each detection box in the set of detection boxes D and the detection boxes in the original labels according to the original loss function. If a match fails, the set of detection boxes D can be marked as the set of detection boxes D', and the set of pseudo-label boxes G corresponding to the set of detection boxes D' is obtained. Then, the intersection-union ratio I between each detection box d in the set of detection boxes D' and each pseudo-label box g in the set of pseudo-label boxes G can be calculated. If I is greater than or equal to a preset threshold, the pseudo-label box g is retained as the ground truth label of d to calculate the loss. Thus, for each detection box d, a set of pseudo-label ground truth Gm can be obtained. In calculating the loss using the pseudo-label loss function, if Gm is an empty set, meaning the detection box d does not match any pseudo-label, model training can be performed without using that detection box d. If Gm is not an empty set, then for the training process of each detection box, the corresponding loss value can be calculated based on the following pseudo-label loss function:

[0160]

[0161] Where p is the predicted class distribution probability, c(g) is the class of the pseudo-label box g (e.g., pedestrian, vehicle, chair, etc.), and τ(S det (g) is the weighting parameter, S det(g) represents the confidence level of the pseudo-label box, and the weighted equation τ can be taken as τ(x) = x, where Z is τ(S det (g)) and.

[0162] Through the above training process of the detection model, a pseudo-label loss function is added during training to combat the uncertainty of pseudo-labels and average potential noise, reduce the impact of pseudo-label noise in the dataset on model training, and improve the final detection accuracy of the detection model.

[0163] It should be noted that the embodiments of the present invention include, but are not limited to, the examples described above. It is understood that those skilled in the art can make further settings according to actual needs under the guidance of the ideas in the embodiments of the present invention, and the present invention does not limit such settings.

[0164] In this embodiment of the invention, during the fusion of different datasets, several target datasets to be fused and a multi-dataset fusion model can be obtained. The multi-dataset fusion model can include a multi-head detector, which can include several detection heads. Then, each target dataset can be input into the multi-dataset fusion model, and the detection heads in the multi-head detector can be used to label the target datasets to obtain the target labels corresponding to each target dataset. Labeling the datasets with different detection heads ensures that each dataset has a corresponding target label, guaranteeing the comprehensiveness of the dataset labeling. On the other hand, different detection heads can effectively improve the detection accuracy of the model in different scenarios and domains, effectively extract features from different datasets, reduce label noise, improve the accuracy of dataset labeling, and ensure data quality. Therefore, when fusing the various target datasets and their corresponding target labels to obtain the fused dataset, the quality of the fused dataset can be effectively guaranteed.

[0165] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0166] Reference Figure 9 The diagram illustrates a structural block diagram of a domain-adaptive data fusion device provided in an embodiment of the present invention, which may specifically include the following modules:

[0167] The dataset acquisition module 901 is used to acquire several target datasets and a multi-dataset fusion model, wherein the multi-dataset fusion model includes a multi-head detector, and the multi-head detector includes several detection heads;

[0168] The dataset annotation module 902 is used to input each of the target datasets into the multi-dataset fusion model, annotate the target datasets according to a plurality of detection heads, and obtain the target labels corresponding to the target datasets;

[0169] The dataset fusion module 903 is used to fuse the target datasets and the target labels corresponding to the target datasets, and output a fused dataset for the target datasets.

[0170] In one alternative embodiment, it further includes:

[0171] The first model acquisition module is used to acquire the original labels corresponding to each of the target datasets, the set of detection boxes corresponding to each of the target labels, and the initial detection model. The original labels include the original detection boxes corresponding to the target datasets.

[0172] The overlapping region determination module is used to match the original detection boxes corresponding to each of the target datasets with the corresponding set of detection boxes to obtain the overlapping region between the original detection boxes and the set of detection boxes.

[0173] The loss function determination module is used to determine the target loss function corresponding to the target dataset based on the overlapping area between the original detection box and the detection box set in each target dataset. The target loss function includes the original loss function and the pseudo-label loss function.

[0174] The loss value calculation module is used to train the initial detection model based on the original loss function and the pseudo-label loss function to obtain the first loss value corresponding to the original loss function and the second loss value corresponding to the pseudo-label loss function.

[0175] The model iteration module is used to stop iterating the initial detection model when both the first loss value and the second loss value reach the convergence condition, thereby obtaining the target detection model.

[0176] In one optional embodiment, the loss function determination module is specifically used for:

[0177] The set of detection boxes with overlapping regions greater than or equal to a first preset threshold is taken as the original set of detection boxes, and the original loss function corresponding to the original set of detection boxes is obtained.

[0178] The set of detection boxes with overlapping areas smaller than the first preset threshold is taken as the target detection box set, and the set of pseudo-label boxes corresponding to the target detection box set is obtained. The target detection box set and the set of pseudo-label boxes are used to filter the detection boxes to obtain the set of pseudo-label ground truth values ​​corresponding to the target detection box set, and the pseudo-label loss function corresponding to the set of pseudo-label ground truth values ​​is obtained.

[0179] In one optional embodiment, the target detection box set includes target detection boxes, the pseudo-label box set includes pseudo-label boxes, and the loss function determination module is specifically used for:

[0180] Calculate the intersection-union ratio (IUR) between each target detection box and each pseudo-label box, and take the pseudo-label boxes with IUR greater than or equal to a second preset threshold as the ground truth labels of the corresponding target detection boxes to obtain the pseudo-label ground truth set corresponding to each target detection box.

[0181] In one optional embodiment, the loss value calculation module is specifically used for:

[0182] The original set of detection boxes and the first target dataset corresponding to the original set of detection boxes are input into the initial detection model for model training, and the original loss function is used to calculate the first loss value corresponding to each detection box in the original set of detection boxes;

[0183] The initial detection model is trained by outputting the pseudo-label ground truth set of the non-empty set and the second target dataset corresponding to the pseudo-label ground truth set of the non-empty set, and the second loss value corresponding to each detection box in the pseudo-label ground truth set of the non-empty set is calculated by using the pseudo-label loss function.

[0184] In an optional embodiment, the multi-head detector further includes a feature extractor, with each detector head corresponding to a preset dataset, and the dataset annotation module 902 is specifically used for:

[0185] The feature extractor and each of the detection heads are used to annotate other target datasets in the plurality of target datasets except for the preset dataset corresponding to the detection head, to obtain pseudo labels corresponding to the target datasets output by each of the detection heads, and the pseudo labels include pseudo detection boxes in their respective target datasets;

[0186] Obtain the original labels corresponding to each of the target datasets, wherein the original labels include the original detection boxes in the target datasets;

[0187] The labels are merged based on the positional relationship between the original detection boxes and the pseudo detection boxes to obtain the target labels corresponding to the target dataset.

[0188] In one optional embodiment, the positional relationship characterizes the overlapping region of the detection box in the target dataset, and the dataset annotation module 902 is specifically used for:

[0189] If the overlapping area between the original detection box and the pseudo detection box and the overlapping area between each pseudo detection box in the same target dataset are all less than the third preset threshold, then the pseudo labels are merged to obtain the target label corresponding to the target dataset.

[0190] If there is an overlapping region between some of the pseudo detection boxes in the same target dataset that is greater than or equal to the third preset threshold, then the overlapping region that is greater than or equal to the third preset threshold is taken as the target overlapping region, and the pseudo detection box corresponding to the target overlapping region is taken as the overlapping detection box.

[0191] If the target overlapping region contains an original detection box from the original label corresponding to the target dataset, then the original detection box and other pseudo detection boxes in the pseudo detection box except for the overlapping detection box are merged to obtain the target label corresponding to the target dataset.

[0192] If the original detection box does not exist in the target overlapping region, the confidence level of each overlapping detection box is obtained, and the original detection boxes and / or pseudo detection boxes corresponding to the remaining overlapping regions and the overlapping detection box with the highest confidence level in the target overlapping region are merged to obtain the target label corresponding to the target dataset.

[0193] Reference Figure 10 The diagram shows a structural block diagram of a training device for a multi-dataset fusion model provided in an embodiment of the present invention, which may specifically include the following modules:

[0194] The second model acquisition module 1001 is used to acquire several training datasets and an initial multi-dataset fusion model. The initial multi-dataset fusion model includes a multi-domain classification head, a feature extractor, and a multi-head detector. The multi-head detector includes several detection heads, and each detection head corresponds to a training dataset.

[0195] Feature extraction module 1002 is used to input the training datasets into the feature extractor respectively to obtain data features corresponding to each training dataset;

[0196] The first training module 1003 is used to input the data features corresponding to each training dataset into the corresponding detection head, train the detection head, and obtain the first classification information corresponding to each training dataset.

[0197] The second training module 1004 is used to input the data features corresponding to each training dataset into the multi-domain classification head for training, and obtain the second classification information corresponding to each training dataset.

[0198] The model generation module 1005 is used to perform adversarial training on the initial multi-data fusion model based on the first classification information and the second classification information to generate the multi-data fusion model.

[0199] In one optional embodiment, the model generation module 1005 is specifically used for:

[0200] Obtain the adversarial classification loss function corresponding to the multi-domain classification head, wherein the adversarial classification loss function includes an image-level loss function and an instance-level loss function;

[0201] Obtain the consistency constraint loss corresponding to the initial multi-data fusion model;

[0202] The initial multi-data fusion model is subjected to adversarial training based on the first classification information and the second classification information. A first loss value corresponding to the image-level loss function, a second loss value corresponding to the instance-level loss function, and a third loss value corresponding to the consistency constraint loss are calculated. When the first loss value, the second loss value, and the third loss value reach the convergence condition, the model training is stopped, and the multi-dataset fusion model is obtained.

[0203] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0204] In addition, embodiments of the present invention also provide an electronic device, such as... Figure 11 As shown, it includes a processor 1101, a communication interface 1102, a memory 1103, and a communication bus 1104. The processor 1101, communication interface 1102, and memory 1103 communicate with each other via the communication bus 1104.

[0205] Memory 1103 is used to store computer programs;

[0206] When processor 1101 executes the program stored in memory 1103, it performs the following steps:

[0207] Acquire several target datasets and a multi-dataset fusion model, wherein the multi-dataset fusion model includes a multi-head detector, and the multi-head detector includes several detection heads;

[0208] Each of the target datasets is input into the multi-dataset fusion model, and the target datasets are labeled according to the several detection heads to obtain the target labels corresponding to the target datasets;

[0209] The target datasets and their corresponding target labels are fused together to output a fused dataset for the target datasets.

[0210] In one alternative embodiment, it further includes:

[0211] Obtain the original labels corresponding to each of the target datasets, the set of detection boxes corresponding to each of the target labels, and the initial detection model. The original labels include the original detection boxes corresponding to the target datasets.

[0212] The original detection boxes corresponding to each of the target datasets are matched with the corresponding set of detection boxes to obtain the overlapping area between the original detection boxes and the set of detection boxes;

[0213] Based on the overlapping area between the original detection boxes and the set of detection boxes in each of the target datasets, a target loss function corresponding to the target dataset is determined. The target loss function includes the original loss function and the pseudo-label loss function.

[0214] The initial detection model is trained based on the original loss function and the pseudo-label loss function to obtain a first loss value corresponding to the original loss function and a second loss value corresponding to the pseudo-label loss function.

[0215] When both the first loss value and the second loss value reach the convergence condition, the model iteration of the initial detection model is stopped, and the target detection model is obtained.

[0216] In one optional embodiment, determining the target loss function corresponding to the target dataset based on the overlapping area between the original detection boxes and the set of detection boxes in each target dataset includes:

[0217] The set of detection boxes with overlapping regions greater than or equal to a first preset threshold is taken as the original set of detection boxes, and the original loss function corresponding to the original set of detection boxes is obtained.

[0218] The set of detection boxes with overlapping areas smaller than the first preset threshold is taken as the target detection box set, and the set of pseudo-label boxes corresponding to the target detection box set is obtained. The target detection box set and the set of pseudo-label boxes are used to filter the detection boxes to obtain the set of pseudo-label ground truth values ​​corresponding to the target detection box set, and the pseudo-label loss function corresponding to the set of pseudo-label ground truth values ​​is obtained.

[0219] In one optional embodiment, the target detection box set includes target detection boxes, the pseudo-label box set includes pseudo-label boxes, and the step of filtering detection boxes using the target detection box set and the pseudo-label box set to obtain a pseudo-label truth value set corresponding to the target detection box set includes:

[0220] Calculate the intersection-union ratio (IUR) between each target detection box and each pseudo-label box, and take the pseudo-label boxes with IUR greater than or equal to a second preset threshold as the ground truth labels of the corresponding target detection boxes to obtain the pseudo-label ground truth set corresponding to each target detection box.

[0221] In one optional embodiment, training the initial detection model based on the original loss function and the pseudo-label loss function to obtain a first loss value corresponding to the original loss function and a second loss value corresponding to the pseudo-label loss function includes:

[0222] The original set of detection boxes and the first target dataset corresponding to the original set of detection boxes are input into the initial detection model for model training, and the original loss function is used to calculate the first loss value corresponding to each detection box in the original set of detection boxes;

[0223] The initial detection model is trained by outputting the pseudo-label ground truth set of the non-empty set and the second target dataset corresponding to the pseudo-label ground truth set of the non-empty set, and the second loss value corresponding to each detection box in the pseudo-label ground truth set of the non-empty set is calculated by using the pseudo-label loss function.

[0224] In one optional embodiment, the multi-head detector further includes a feature extractor, each detector head corresponding to a preset dataset, and the step of labeling the target dataset according to a plurality of the detector heads to obtain the target label corresponding to the target dataset includes:

[0225] The feature extractor and each of the detection heads are used to annotate other target datasets in the plurality of target datasets except for the preset dataset corresponding to the detection head, to obtain pseudo labels corresponding to the target datasets output by each of the detection heads, and the pseudo labels include pseudo detection boxes in their respective target datasets;

[0226] Obtain the original labels corresponding to each of the target datasets, wherein the original labels include the original detection boxes in the target datasets;

[0227] The labels are merged based on the positional relationship between the original detection boxes and the pseudo detection boxes to obtain the target labels corresponding to the target dataset.

[0228] In one optional embodiment, the positional relationship represents the overlapping region of the detection boxes in the target dataset, and the step of merging labels based on the positional relationship between the original detection boxes and the pseudo detection boxes to obtain the target labels corresponding to the target dataset includes:

[0229] If the overlapping area between the original detection box and the pseudo detection box and the overlapping area between each pseudo detection box in the same target dataset are all less than the third preset threshold, then the pseudo labels are merged to obtain the target label corresponding to the target dataset.

[0230] If there is an overlapping region between some of the pseudo detection boxes in the same target dataset that is greater than or equal to the third preset threshold, then the overlapping region that is greater than or equal to the third preset threshold is taken as the target overlapping region, and the pseudo detection box corresponding to the target overlapping region is taken as the overlapping detection box.

[0231] If the target overlapping region contains an original detection box from the original label corresponding to the target dataset, then the original detection box and other pseudo detection boxes in the pseudo detection box except for the overlapping detection box are merged to obtain the target label corresponding to the target dataset.

[0232] If the original detection box does not exist in the target overlapping region, the confidence level of each overlapping detection box is obtained, and the original detection boxes and / or pseudo detection boxes corresponding to the remaining overlapping regions and the overlapping detection box with the highest confidence level in the target overlapping region are merged to obtain the target label corresponding to the target dataset.

[0233] In one alternative embodiment, it further includes:

[0234] Acquire several training datasets and an initial multi-dataset fusion model. The initial multi-dataset fusion model includes a multi-domain classification head, a feature extractor, and a multi-head detector. The multi-head detector includes several detection heads, and each detection head corresponds to a training dataset.

[0235] The training datasets are input into the feature extractor respectively to obtain the data features corresponding to each training dataset;

[0236] The data features corresponding to each training dataset are input into the corresponding detection head to train the detection head and obtain the first classification information corresponding to each training dataset.

[0237] The data features corresponding to each of the training datasets are input into the multi-domain classification head for training to obtain the second classification information corresponding to each of the training datasets.

[0238] The initial multi-data fusion model is subjected to adversarial training based on the first classification information and the second classification information to generate the multi-data fusion model.

[0239] In one optional embodiment, the step of performing adversarial training on the initial multi-data fusion model based on the first classification information and the second classification information to generate the multi-data fusion model includes:

[0240] Obtain the adversarial classification loss function corresponding to the multi-domain classification head, wherein the adversarial classification loss function includes an image-level loss function and an instance-level loss function;

[0241] Obtain the consistency constraint loss corresponding to the initial multi-data fusion model;

[0242] The initial multi-data fusion model is subjected to adversarial training based on the first classification information and the second classification information. A first loss value corresponding to the image-level loss function, a second loss value corresponding to the instance-level loss function, and a third loss value corresponding to the consistency constraint loss are calculated. When the first loss value, the second loss value, and the third loss value reach the convergence condition, the model training is stopped, and the multi-dataset fusion model is obtained.

[0243] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0244] The communication interface is used for communication between the aforementioned terminal and other devices.

[0245] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0246] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0247] like Figure 12 As shown, in another embodiment of the present invention, a computer-readable storage medium 1301 is also provided, which stores instructions that, when executed on a computer, cause the computer to perform the domain-adaptive data fusion method described in the above embodiments.

[0248] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the domain-adaptive data fusion method described in the above embodiments.

[0249] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0250] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0251] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0252] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A domain adaptive data fusion method applied to the field of image detection, characterized in that, The method comprises the following steps: obtaining a plurality of target data sets and a multi-data set fusion model, the multi-data set fusion model comprising a multi-head detector, the multi-head detector comprising a plurality of detection heads; the target data set is an image data set; inputting each target data set into the multi-data set fusion model, labeling the target data set according to a plurality of detection heads, and obtaining a target label corresponding to the target data set; performing data fusion on each target data set and the target label corresponding to the target data set, and outputting a fusion data set for the plurality of target data sets; wherein the multi-head detector further comprises a feature extractor, each detection head corresponds to a preset data set, and the labeling of the target data set according to a plurality of detection heads to obtain a target label corresponding to the target data set comprises: extracting features of each target data set using the feature extractor to obtain data features corresponding to the target data set; labeling each target data set other than the preset data set corresponding to the detection head according to the data features corresponding to the other target data sets to obtain a pseudo label output by each detection head, the pseudo label comprising a pseudo detection box in the target data set; obtaining an original label corresponding to each target data set, the original label comprising an original detection box in the target data set; merging labels according to the positional relationship between the original detection box and the pseudo detection box to obtain a target label corresponding to the target data set.

2. The method of claim 1, wherein, Further comprising: obtaining an original label corresponding to each target data set, a detection box set corresponding to each target label, and an initial detection model, the original label comprising an original detection box corresponding to the target data set; matching the original detection box corresponding to each target data set with the corresponding detection box set to obtain an overlapping area between the original detection box and the detection box set; determining a target loss function corresponding to the target data set according to the overlapping area between the original detection box and the detection box set in each target data set, the target loss function comprising an original loss function and a pseudo label loss function; training the initial detection model according to the original loss function and the pseudo label loss function to obtain a first loss value corresponding to the original loss function and a second loss value corresponding to the pseudo label loss function; when the first loss value and the second loss value both reach a convergence condition, stop model iteration of the initial detection model to obtain a target detection model.

3. The method of claim 2, wherein, The determination of the target loss function corresponding to the target data set according to the overlapping area between the original detection box and the detection box set in each target data set comprises: detecting a detection box set with an overlapping area greater than or equal to a first preset threshold as an original detection box set, and obtaining an original loss function corresponding to the original detection box set; The detection frame set with an overlapping area less than the first preset threshold is taken as a target detection frame set, a pseudo-label frame set corresponding to the target detection frame set is obtained, the target detection frame set and the pseudo-label frame set are used for detection frame screening, a pseudo-label true value set corresponding to the target detection frame set is obtained, and a pseudo-label loss function corresponding to the pseudo-label true value set is obtained.

4. The method of claim 3, wherein, The target detection frame set includes target detection frames, the pseudo-label frame set includes pseudo-label frames, the target detection frame set and the pseudo-label frame set are used for detection frame screening to obtain a pseudo-label true value set corresponding to the target detection frame set, and the pseudo-label true value set corresponding to the pseudo-label true value set is obtained. The pseudo-label true value set corresponding to the target detection frame set is obtained by calculating the intersection over union between each target detection frame and each pseudo-label frame and taking a pseudo-label frame with an intersection over union greater than or equal to a second preset threshold as a true value label of the corresponding target detection frame.

5. The method according to claim 2 or 3, characterized in that, The initial detection model is trained according to the original loss function and the pseudo-label loss function to obtain a first loss value corresponding to the original loss function and a second loss value corresponding to the pseudo-label loss function, including: The initial detection model is trained by inputting the original detection frame set and a first target data set corresponding to the original detection frame set into the initial detection model, and the first loss value corresponding to each detection frame in the original detection frame set is calculated by using the original loss function; The initial detection model is trained by inputting the non-empty pseudo-label true value set and a second target data set corresponding to the non-empty pseudo-label true value set into the initial detection model, and the second loss value corresponding to each detection frame in the non-empty pseudo-label true value set is calculated by using the pseudo-label loss function.

6. The method of claim 1, wherein, The position relationship represents an overlapping area of a detection frame in a target data set, and the target label corresponding to the target data set is obtained by merging the original label and the pseudo-label according to the position relationship between the original detection frame and the pseudo detection frame, including: If the overlapping area between the original detection frame and the pseudo detection frame and the overlapping area between each pseudo detection frame in the same target data set are all less than a third preset threshold, the original label and the pseudo-label are merged to obtain the target label corresponding to the target data set; If there is an overlapping area between some pseudo detection frames in the same target data set that is greater than or equal to the third preset threshold, the overlapping area greater than or equal to the third preset threshold is taken as a target overlapping area, and the pseudo detection frame corresponding to the target overlapping area is taken as an overlapping detection frame; If there is an original detection frame in the target data set in the target overlapping area, the original detection frame and other pseudo detection frames except the overlapping detection frame in the pseudo detection frame are merged to obtain the target label corresponding to the target data set; If the original detection frame does not exist in the target overlapping region, confidence corresponding to each overlapping detection frame is obtained, and the original detection frame and / or the pseudo detection frame corresponding to the remaining overlapping region and the overlapping detection frame with the highest confidence in the target overlapping region are merged to obtain a target label corresponding to the target data set.

7. A domain adaptive data fusion device applied to the field of image detection, characterized in that, The method comprises the following steps: A data set acquisition module is configured to acquire a plurality of target data sets and a multi-data set fusion model, wherein the multi-data set fusion model comprises a multi-head detector, and the multi-head detector comprises a plurality of detection heads; the target data set is an image data set; A data set labeling module is configured to input each target data set into the multi-data set fusion model, label the target data set according to the plurality of detection heads, and obtain a target label corresponding to the target data set; A data set fusion module is configured to fuse each target data set and the target label corresponding to the target data set, and output a fusion data set for the plurality of target data sets; The multi-head detector further comprises a feature extractor, each detection head corresponds to a preset data set, and the data set labeling module is configured to: label other target data sets in the plurality of target data sets except the preset data set corresponding to the detection head by using the feature extractor and each detection head, respectively, to obtain a pseudo label corresponding to the target data set output by each detection head, wherein the pseudo label comprises a pseudo detection frame in the target data set; obtain an original label corresponding to each target data set, wherein the original label comprises an original detection frame in the target data set; merge the labels according to the positional relationship between the original detection frame and the pseudo detection frame to obtain a target label corresponding to the target data set.

8. An electronic device, comprising: A processor, a communication interface, a memory and a communication bus are included, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus; The memory is configured to store a computer program; The processor is configured to execute the program stored on the memory to implement the method of any one of claims 1-6.

9. A computer-readable storage medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method of any one of claims 1-6.

Images