Cross-domain target identification method and device, equipment and medium

Abstract

The invention provides a cross-domain target identification method and device, equipment and a medium, and relates to the field of target identification. Comprising the following steps: acquiring a source domain image carrying a label and an unmarked target domain image, and aligning the source domain image carrying the label to the unmarked target domain image in a frequency domain to obtain a source domain target image carrying the label; inputting the source domain target image into the source domain target detection model to obtain a target detection result of the source domain target image as a pseudo label of an unmarked target domain image, and marking the unmarked target domain image to obtain a target domain image carrying the pseudo label; and performing parameter updating on the source domain target detection model by using the target domain image carrying the pseudo tag to obtain the target detection model of the target domain so as to perform target detection on the to-be-detected target domain image and obtain a target detection result of the to-be-detected target domain image, thereby realizing domain migration of the target detection model, and improving detection accuracy. And the performance and generalization ability of cross-domain target identification are improved.

Description

Technical Field

[0001] The present invention relates to the field of target recognition, and in particular to a cross-domain target recognition method, device, equipment and medium. Background Art

[0002] In recent years, on the one hand, in target recognition tasks, differences in feature distribution have led to a decline in the cross-domain performance of target detection models: Since images in different fields have significant differences in feature distribution, including target morphology, background complexity, lighting conditions, and noise distribution, the features learned by the target detection model during source domain training are often unable to effectively adapt to the target domain. This distribution difference causes the model to show recognition bias or misdetection of specific categories in the target domain, especially when the target area features overlap with the background or other domain features, making it difficult to accurately locate and classify the target, which ultimately significantly reduces the cross-domain recognition performance and generalization ability of the target detection model.

[0003] On the other hand, in target recognition tasks, the method of generating pseudo-labels is widely used in cross-domain applications. However, the unreliability of pseudo-labels weakens the cross-domain performance of the model: due to the difference in feature distribution between the target domain and the source domain, pseudo-labels often contain incorrect target positioning or category annotations. This low reliability can easily cause the target detection model to overfit the noise features in the pseudo-labels, making it difficult to accurately capture the key features of the target domain, thereby significantly weakening the performance and generalization ability of cross-domain target recognition. Therefore, how to improve the performance and generalization ability of cross-domain target recognition is a technical problem to be solved in the present invention. Summary of the invention

[0004] Based on the above technical problems, the present invention provides a cross-domain target recognition method, device, equipment and medium, aiming to improve the performance and generalization ability of cross-domain target recognition.

[0005] A first aspect of the present invention provides a cross-domain target recognition method, the method comprising: Acquire a source domain image carrying a label and an unlabeled target domain image, wherein the label carried by the source domain image represents the category and position box of the target in the source domain image; Aligning the labeled source domain image to the unlabeled target domain image in the frequency domain to obtain a labeled source domain target image aligned with the feature distribution of the unlabeled target domain image; Inputting the source domain target image into a source domain target detection model to obtain a target detection result of the source domain target image, wherein the target detection result of the source domain target image represents a category and a position box of a target in the source domain target image; Using the target detection result of the source domain target image as the pseudo label of the unlabeled target domain image, labeling the unlabeled target domain image, and obtaining a target domain image carrying the pseudo label; Using the target domain image carrying the pseudo-label, updating the parameters of the source domain object detection model to obtain the target domain object detection model; By using the target detection model of the target domain, target detection is performed on the target domain image to be detected to obtain a target detection result of the target domain image to be detected, wherein the target detection result of the target domain image to be detected represents a category and a position frame of the target in the target domain image to be detected.

[0006] A second aspect of the present invention provides a cross-domain target recognition device, the device comprising: An image acquisition module is used to acquire a source domain image carrying a label and an unlabeled target domain image, wherein the label carried by the source domain image represents the category and position box of the target in the source domain image; A distribution alignment module, used to align the labeled source domain image to the unlabeled target domain image in the frequency domain, so as to obtain a labeled source domain target image that is aligned with the feature distribution of the unlabeled target domain image; An image processing module, used for inputting the source domain target image into a source domain target detection model to obtain a target detection result of the source domain target image, wherein the target detection result of the source domain target image represents a category and a position box of a target in the source domain target image; a pseudo-label marking module, configured to use the target detection result of the source domain target image as the pseudo-label of the unlabeled target domain image, mark the unlabeled target domain image, and obtain a target domain image carrying the pseudo-label; A model updating module, used to update the parameters of the source domain target detection model using the target domain image carrying the pseudo-label, so as to obtain the target domain target detection model; The target detection module is used to perform target detection on the target domain image to be detected through the target detection model of the target domain, and obtain the target detection result of the target domain image to be detected, wherein the target detection result of the target domain image to be detected represents the category and position box of the target in the target domain image to be detected.

[0007] A third aspect of an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the cross-domain target recognition method of the first aspect of the embodiment of the present invention is implemented.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the cross-domain target recognition method of the first aspect of the embodiment of the present invention is implemented.

[0009] Through the cross-domain target recognition method of this embodiment, the labeled source domain image is aligned to the unlabeled target domain image in the frequency domain to obtain the labeled source domain target image aligned with the feature distribution of the unlabeled target domain image. In this way, this embodiment significantly alleviates the problem of feature distribution differences between domains by aligning the feature distribution of source domain and target domain features; then, the source domain target image is input into the source domain target detection model to obtain the target detection result of the source domain target image as a pseudo-label for the unlabeled target domain image. In this way, this embodiment performs style transfer on the source domain data in the frequency domain, maps the features of the source domain data to a feature space close to the target domain distribution, thereby generating high-quality labeled data (i.e., pseudo-labels) matching the target domain, and further guiding model optimization; thereafter, the target domain image carrying the pseudo-label is used to update the parameters of the source domain target detection model to obtain the target detection model of the target domain to achieve domain migration of the target detection model. In this embodiment, target domain distribution data generation is achieved through feature distribution alignment, and unsupervised optimization is performed on the target domain data (that is, the target domain images are not labeled, but pseudo labels are generated through feature distribution alignment to achieve unsupervised optimization). This not only effectively avoids the performance loss caused by pseudo label generation errors, but also significantly improves the adaptability and robustness of the target detection model in multi-domain scenarios, and improves the target recognition performance and generalization ability in cross-domain complex tasks, thereby achieving efficient and robust target recognition in cross-domain scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0011] Figure 1 is a flowchart of a cross-domain target recognition method according to an embodiment of the present invention; Figure 2 is an overall schematic diagram of an object recognition method based on source domain memory and category-level alignment shown in one embodiment of the present invention; Figure 3 is a structural block diagram of a cross-domain target recognition device provided by an embodiment of the present invention; Figure 4 It is a schematic diagram of an electronic device shown in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] Please refer to Figure 1 , Figure 1 FIG. 1 is a flowchart of a cross-domain target recognition method according to an embodiment of the present invention. Figure 1 As shown, the cross-domain target recognition method provided by this embodiment includes at least the following steps: Step S11: obtaining a source domain image carrying a label and an unlabeled target domain image, wherein the label carried by the source domain image represents the category and position box of the target in the source domain image.

[0014] In this embodiment, source domain images carrying labels and unlabeled target domain images can be obtained. Unlabeled target domain images are image samples in the target domain used for training target detection models (target recognition models, used for target recognition of images), and the image samples do not carry labels. Source domain images are image samples for target detection (i.e., target recognition) in the source domain, and the source domain images carry labels, and the labels carried by the source domain images represent the category and position box of the target in the source domain image. Among them, the source domain is the field in which the target detection model has been pre-trained, and the target domain is the field to which the target detection model will be applied, that is, the target domain is the field to be migrated, and the source domain in this embodiment may include one or more different fields. In an optional embodiment, the lighting conditions (brightness), noise distribution and / or background complexity of the image are different between one field and another.

[0015] Step S12: aligning the labeled source domain image to the unlabeled target domain image in the frequency domain to obtain a labeled source domain target image that is aligned with the feature distribution of the unlabeled target domain image.

[0016] In this embodiment, the source domain image carrying labels can be aligned to the unlabeled target domain image in the frequency domain to adjust the source domain image carrying labels to obtain the source domain target image carrying labels that is aligned with the feature distribution of the unlabeled target domain image. That is to say, the source domain target image in this embodiment is the source domain image after the feature distribution alignment adjustment of the source domain image, the source domain target image and the unlabeled target domain image have achieved feature distribution alignment, the source domain target image carries a label, and the label carried by the source domain target image is the label carried by the source domain image before adjustment corresponding to the source domain target image.

[0017] Step S13: inputting the source domain target image into a source domain target detection model to obtain a target detection result of the source domain target image, wherein the target detection result of the source domain target image represents a category and a position box of a target in the source domain target image.

[0018] In this embodiment, after obtaining the source domain target image, the source domain target image can be input into the source domain target detection model to obtain the target detection result of the source domain target image, and the target detection result of the source domain target image represents the category and position box of the target in the source domain target image. The source domain target detection model is a target detection model that has been trained in the source domain.

[0019] Step S14: using the target detection result of the source domain target image as a pseudo label of the unlabeled target domain image, labeling the unlabeled target domain image, and obtaining a target domain image carrying the pseudo label.

[0020] In this embodiment, the target detection result of the source domain target image is used as a pseudo label of the unlabeled target domain image, and the unlabeled target domain image is labeled to obtain a target domain image carrying the pseudo label.

[0021] In this embodiment, the target detection result corresponding to the source domain target image after feature distribution alignment is used as the pseudo-label of the unlabeled target domain image, and labeled data (target domain image carrying pseudo-label) that is highly close to the target domain distribution is generated. Compared with related pseudo-label generation methods, the introduction of potential errors in the pseudo-label generation process is fundamentally avoided.

[0022] Step S15: using the target domain image carrying the pseudo-label, updating the parameters of the source domain object detection model to obtain the target domain object detection model.

[0023] In this embodiment, after obtaining the target domain image with the pseudo-label, the model parameters of the source domain target detection model can be updated by at least using the target domain image with the pseudo-label to obtain the target domain target detection model. In one example, the result obtained after the source domain target detection model processes the target domain image can be used to calculate the loss with the pseudo-label corresponding to the target domain image, and the model parameters of the source domain target detection model can be updated based on the loss until the loss converges to fix the model parameters, thereby obtaining the target domain target detection model, which is the target detection model trained in the target domain.

[0024] Step S16: performing target detection on the target domain image to be detected by using the target detection model of the target domain to obtain a target detection result of the target domain image to be detected, wherein the target detection result of the target domain image to be detected represents the category and position box of the target in the target domain image to be detected.

[0025] In this embodiment, after obtaining the target detection model of the target domain, the target detection model of the target domain is used to perform target detection of images in the target domain, and the target domain image to be detected can be input into the target detection model of the target domain, and the target detection is performed on the target domain image to be detected through the target detection model of the target domain, and the target detection result of the target domain image to be detected output by the target detection model of the target domain is obtained. Among them, the target domain image to be detected is an image in the target domain that needs to be recognized as a target, and the target detection result of the target domain image to be detected represents the category and position box of the target in the target domain image to be detected.

[0026] In this embodiment, a labeled source domain image is aligned to an unlabeled target domain image in the frequency domain to obtain a labeled source domain target image that is aligned with the feature distribution of the unlabeled target domain image. In this way, this embodiment significantly alleviates the problem of feature distribution differences between domains by aligning the feature distribution of source domain and target domain features. Next, the source domain target image is input into a source domain target detection model to obtain a target detection result of the source domain target image as a pseudo-label for the unlabeled target domain image. In this way, this embodiment performs style transfer on the source domain data in the frequency domain, maps the features of the source domain data to a feature space that is close to the target domain distribution, thereby generating high-quality labeled data (i.e., pseudo-labels) that matches the target domain, and further guides model optimization. Thereafter, the target domain image carrying the pseudo-label is used to update the parameters of the source domain target detection model to obtain a target detection model for the target domain to achieve domain transfer of the target detection model. In this embodiment, target domain distribution data generation is achieved through feature distribution alignment, and unsupervised optimization is performed on the target domain data (that is, the target domain images are not labeled, but pseudo labels are generated through feature distribution alignment to achieve unsupervised optimization). This not only effectively avoids the performance loss caused by pseudo label generation errors, but also significantly improves the adaptability and robustness of the target detection model in multi-domain scenarios, and improves the target recognition performance and generalization ability in cross-domain complex tasks, thereby achieving efficient and robust target recognition in cross-domain scenarios.

[0027] In combination with the above embodiments, in one implementation, the present invention further provides a cross-domain target recognition method, in which the above step S12 may specifically include steps S21 to S24: Step S21: screening out, from the labeled source domain images, a first source domain image that is labeled and aligned with the category of the unlabeled target domain image.

[0028] In this embodiment, there are multiple source domain images carrying labels, and the source domain images carrying labels that are aligned with the category of the unlabeled target domain image can be screened out from the source domain images carrying labels to serve as the first source domain image carrying labels. Among them, the category alignment of this embodiment is characterized by: the category of the target in the first source domain image is the same as or similar to the category of the target in the unlabeled target domain image. After the category alignment, this embodiment selects the part closest to the current target domain category (i.e., the first source domain image), and then performs subsequent feature distribution alignment based on the part closest to the current target domain category to better learn the knowledge of this category.

[0029] Step S22: performing Fourier transform on the first source domain image and the unlabeled target domain image respectively to obtain frequency domain features of the first source domain image and the unlabeled target domain image respectively, wherein the frequency domain features include a phase spectrum and an amplitude spectrum.

[0030] In this embodiment, after obtaining the first source domain image with a label, Fourier transform can be performed on the first source domain image and the unlabeled target domain image respectively, and the first source domain image and the unlabeled target domain image can be mapped from the spatial domain to the frequency domain, and the frequency domain features of the first source domain image and the unlabeled target domain image can be extracted, wherein the frequency domain features include: phase spectrum and amplitude spectrum. In other words, this step can obtain the phase spectrum and amplitude spectrum of the first source domain image, and the phase spectrum and amplitude spectrum of the unlabeled target domain image.

[0031] Step S23: While keeping the phase spectrum of the first source domain image unchanged, keep other areas of the amplitude spectrum of the first source domain image unchanged, replace the central area of ​​the amplitude spectrum of the first source domain image with the corresponding central area of ​​the amplitude spectrum of the unlabeled target domain image, and obtain the adjusted frequency domain features of the first source domain image.

[0032] In this embodiment, the frequency domain features of the first source domain image are adjusted based on the amplitude spectrum of the unlabeled target domain image: while keeping the phase spectrum of the first source domain image unchanged, the central area of ​​the amplitude spectrum of the first source domain image is replaced with the corresponding central area of ​​the amplitude spectrum of the unlabeled target domain image, and the other areas of the amplitude spectrum of the first source domain image are kept unchanged, so as to obtain the adjusted frequency domain features of the first source domain image, so as to achieve distribution alignment in the frequency domain space. Among them, the other areas are the areas other than the central area in the amplitude spectrum. In an optional example, the size of the central area can be set as a learnable parameter b, and the central area can be flexibly adjusted to different geometric shapes according to the specific requirements of the target recognition task.

[0033] Step S24: performing inverse Fourier transform on the adjusted frequency domain features of the first source domain image to obtain the source domain target image carrying the label.

[0034] In this embodiment, after obtaining the adjusted frequency domain features of the first source domain image, an inverse Fourier transform is performed on the adjusted frequency domain features of the first source domain image, and the adjusted frequency domain features of the first source domain image are restored from the frequency domain to the spatial domain to obtain a source domain target image carrying a label. The source domain target image has a distribution closer to the unlabeled target domain image, and the source domain target image is a labeled source domain image that is sufficiently close in style to the unlabeled target domain image, that is, a labeled source domain image that is aligned with the unlabeled target domain image category, and can be used for subsequent target detection model optimization to adapt to the target domain data.

[0035] In this embodiment, the phase spectrum is kept unchanged to ensure that the image content in the source domain target image is still the image content of the original source domain image. By locally replacing the frequency domain features (replacement of the central area of ​​the amplitude spectrum), the spectral characteristics of the source domain target image carrying the label are closer to the distribution characteristics of the unlabeled target domain image. In this process, the alignment of the domain distribution and the dynamic adjustment of key features in the target recognition task are realized, thereby achieving accurate modeling of the target domain characteristics and improving the recognition performance. This embodiment generates labeled data that is highly close to the target domain distribution by aligning the distribution of images of the same category in the source domain and the target domain (i.e., the first source domain image and the target domain image) in the frequency domain, fundamentally avoiding the introduction of potential errors in the pseudo-label generation process. In addition, the frequency domain distribution alignment method of this embodiment does not need to rely on the training of a large amount of data or complex models, and has higher efficiency and flexibility.

[0036] In combination with the above embodiments, in an optional embodiment, under extreme conditions, a single unlabeled target domain image can achieve distribution alignment, significantly enhancing the adaptability and generalization ability of the target detection model. It is worth noting that in this embodiment, each new unlabeled target domain image can even be theoretically regarded as an image in an independent new domain to perform the above-mentioned domain migration processing, thereby providing great flexibility and robustness for multi-domain application scenarios.

[0037] In combination with the above embodiments, the present invention further provides a cross-domain target recognition method, in which the above step S15 may specifically include step S31: Step S31: using the source domain target image carrying the label and the target domain image carrying the pseudo label, updating the parameters of the source domain target detection model to obtain the target domain target detection model.

[0038] In this embodiment, the target domain image carrying the pseudo-label is combined with the source domain target image carrying the label, and the source domain target detection model is re-optimized: the source domain target image carrying the label and the target domain image carrying the pseudo-label are used to update the parameters of the source domain target detection model to obtain the target domain target detection model. In one example, the source domain target image carrying the label and the target domain image carrying the pseudo-label can be input into the source domain target detection model to obtain the result obtained after the source domain target detection model processes the target domain image and the result obtained after the source domain target detection model processes the source domain target image; then, based on the result obtained after the source domain target detection model processes the target domain image and the pseudo-label corresponding to the target domain image, and the result obtained after the source domain target detection model processes the source domain target image and the label corresponding to the source domain target image, the loss is calculated, and the model parameters of the source domain target detection model are updated based on the loss, until the loss converges to fix the model parameters, and the target domain target detection model is obtained.

[0039] This embodiment is similar to the fine-tuning operation. It does not rely on the real label of the target domain image for loss calculation. Instead, it uses the source domain target image after distribution alignment as the optimization basis. After several rapid optimization iterations, the target detection model can efficiently adjust parameters and better adapt to the feature distribution of the target domain, thereby improving the target recognition performance and adaptability in unknown domains. Compared with the related domain migration method, this embodiment significantly optimizes the performance of the target detection model by dynamically calling the existing source domain training data during the target detection model reasoning process, and combining the frequency domain transformation and fine-tuning strategy, thereby effectively enhancing the generalization ability and adaptability of the target detection model in unknown domains.

[0040] In combination with the above embodiments, in one implementation, the present invention further provides a cross-domain target recognition method. In this method, in addition to the above steps, step S41 may also be included, and the above step S21 may specifically include steps S42 to S44: Step S41: extracting features and clustering the source domain images carrying labels stored in the memory bank to obtain k categories and the cluster centers of the k categories.

[0041] In this embodiment, a plurality of source domain images carrying labels are stored in the memory bank. In order to efficiently utilize the source domain images in the memory bank, the source domain images carrying labels stored in the memory bank may be preprocessed: feature extraction and clustering (such as K-means clustering) are performed on all source domain images carrying labels stored in the memory bank, and cluster centers (category centers) are generated to reduce the search range, and k categories and their respective cluster centers are obtained to facilitate the subsequent extraction of source domain images having similar categories to the unlabeled target domain images, where k is an integer greater than 1.

[0042] Step S42: Reading the cluster centers of the k categories from the memory bank.

[0043] In this embodiment, when determining the first source domain image, the cluster centers of the k categories can be read from the memory bank first.

[0044] Step S43: extracting features of the unlabeled target domain image, and determining a matching degree between the features of the unlabeled target domain image and the cluster center of each of the k categories read.

[0045] In this embodiment, the features of the unlabeled target domain image can be extracted, and the matching degree between the features of the unlabeled target domain image and the cluster center of each of the k categories read is determined. For example, based on the features of the unlabeled target domain image and the cluster center of each of the k categories read, the similarity between the features of the unlabeled target domain image and the cluster center of each category is calculated, and the similarity is used as the matching degree.

[0046] In an optional implementation, features of unlabeled target domain images and labeled source domain images may be extracted based on a pre-trained feature extraction network.

[0047] Step S44: determining n labeled first source domain images belonging to the category with the highest matching degree, where n is an integer greater than 1.

[0048] In this embodiment, based on the calculated multiple matching degrees, the category with the highest matching degree can be determined, and n labeled source domain images can be screened from the multiple labeled source domain images belonging to the category with the highest matching degree to serve as the first source domain image.

[0049] In an optional example, the size of n can be dynamically adjusted to meet the requirements of different target recognition tasks and provide high-quality support for subsequent model optimization.

[0050] In an optional example, when the number m of source domain images with labels belonging to the category with the highest matching degree is greater than n, the similarity between each source domain image with labels belonging to the category with the highest matching degree and the unlabeled target domain image is calculated, and n first source domain images with labels are selected in descending order of similarity; when the number m of source domain images with labels belonging to the category with the highest matching degree is less than n, the m source domain images with labels belonging to the category with the highest matching degree are first determined as the m first source domain images with labels, and then nm first source domain images with labels are obtained from the source domain images with labels belonging to the category with the second highest matching degree, thereby obtaining n first source domain images with labels. When the total number of images from the category with the highest matching degree and the category with the second highest matching degree is not enough to be selected, the category with the third highest matching degree is selected according to the above method, or so on, until n first source domain images with labels are obtained.

[0051] In combination with the above embodiments, in one implementation, the present invention further provides a cross-domain target recognition method. In this method, after the above step S16 of "using the target detection model of the target domain to perform target detection on the target domain image to be detected to obtain the target detection result of the target domain image to be detected", steps S51 to S54 may also be included: Step S51: using the target detection result of the target domain image to be detected as a label carried by the target domain image to be detected.

[0052] In this embodiment, after obtaining the target detection result of the target domain image to be detected, the target detection result of the target domain image to be detected may be used as a label carried by the target domain image to be detected.

[0053] Step S52: Determine whether the category of the target in the target domain image to be detected is the same as any category in the k categories.

[0054] In this embodiment, it can be determined whether the category of the target in the target domain image to be detected is the same as any category among the k categories after clustering in the memory library.

[0055] Step S53: When the category of the target in the target domain image to be detected is different from the k categories, all the target domain images to be detected carrying labels are added to the memory library to obtain a memory library with expanded categories.

[0056] In this embodiment, when the category of the target in the target domain image to be detected is different from all k categories, it is determined to be a new category. At this time, all the target domain images to be detected carrying labels of the new category are added to the memory bank to obtain a memory bank with expanded categories. That is to say, in this embodiment, when a new category is added, the data corresponding to the new category will be preferentially stored in the memory unit as new information.

[0057] Step S54: using the memory library after the category expansion, updating the parameters of the target detection model of the target domain to obtain a target detection model of another target domain.

[0058] In this embodiment, the memory library after the category expansion can be used as the memory library for the next domain migration. The memory library after the category expansion can be used to update the parameters of the target detection model of the target domain obtained in this domain migration to obtain the target detection model of another target domain, that is, the target detection model of the target domain for the next domain migration.

[0059] In this embodiment, the data categories stored in the memory bank will continue to expand as the scenarios expand, thereby enhancing the target detection model's ability to recognize new categories and complex scenarios, providing more comprehensive and accurate support for target recognition.

[0060] In combination with the above embodiments, in one implementation, the present invention further provides a cross-domain target recognition method. In this method, after the above step S16 of "using the target detection model of the target domain to perform target detection on the target domain image to be detected to obtain the target detection result of the target domain image to be detected", steps S61 to S64 may also be included: Step S61: using the target detection result of the target domain image to be detected as a label carried by the target domain image to be detected.

[0061] Step S62: Determine whether the category of the target in the target domain image to be detected is the same as any category in the k categories.

[0062] Step S61 and step S62 of this embodiment are the same as or similar to the above-mentioned step S51 and step S52, and are not described in detail.

[0063] Step S63: When the category of the target in the target domain image to be detected is one of the k categories, for this category, a portion of the target domain images to be detected carrying labels are selected in order of confidence from high to low and added to the memory bank to obtain the memory bank after domain expansion.

[0064] In this embodiment, when the category of the target in the target domain image to be detected is one of the k categories, a portion of the target domain images to be detected carrying labels can be selected in order of confidence from high to low for the category and added to the memory bank to obtain the memory bank after domain expansion. That is to say, in this embodiment, when there is no new category to be added, a clustering algorithm can be used to select a certain number of target domain images to be detected carrying labels from the category in the k categories to form new memory information and store it in the memory bank to achieve the expansion of the target domain that has been successfully migrated. Among them, the confidence of this embodiment is the confidence corresponding to the category of the target in the target domain image to be detected output by the target detection model of the target domain.

[0065] Step S64: using the domain-expanded memory library, update the parameters of the target detection model of the target domain to obtain a target detection model of another target domain.

[0066] In this embodiment, the memory bank after the domain expansion can be used as the memory bank for the next domain migration. The memory bank after the domain expansion can be used to update the parameters of the target detection model of the target domain obtained in this domain migration to obtain the target detection model of another target domain, that is, the target detection model of the target domain for the next domain migration.

[0067] In this embodiment, the initial data stored in the memory bank is composed of the labeled data of the source domain, which serves as the basic knowledge base for training the target detection model. With the application of the target detection model in different scenarios, the memory bank will dynamically absorb the data of the target domain, and through continuous updating and iteration, gradually enrich its content as the prior knowledge for the domain migration of the target detection model. Each update of the memory bank not only integrates the feature information of the target domain, but also improves the adaptability of the target detection model to the distribution of features in multiple fields. The dynamic update mechanism of the memory bank in this embodiment not only effectively avoids the performance loss caused by the error of pseudo-label generation, but also significantly improves the adaptability and robustness of the target detection model in multi-domain scenarios, and its performance in cross-domain complex tasks is significantly better than related technologies.

[0068] In combination with the above embodiments, in one implementation, the present invention further provides a cross-domain target recognition method. In this method, after obtaining target detection results of each of the multiple target domain images to be detected, steps S71 to S73 may also be included: Step S71: using the target detection results of each of the multiple target domain images to be detected as labels carried by each of the multiple target domain images to be detected.

[0069] In this embodiment, after obtaining the target detection results of the multiple target domain images to be detected, the target detection results of the multiple target domain images to be detected may be used as labels carried by each of the multiple target domain images to be detected.

[0070] Step S72: adding a plurality of target domain images to be detected that carry labels to the memory bank as source domain images that carry labels of different source domains to obtain a memory bank after domain expansion.

[0071] In this embodiment, a plurality of target domain images to be detected carrying labels may be added to the memory bank as source domain images carrying labels of different source domains to obtain a memory bank after domain expansion.

[0072] Step S73: using the domain-expanded memory library, update the parameters of the target detection model of the target domain to obtain a target detection model of another target domain.

[0073] In this embodiment, the memory bank after the domain expansion can be used as the memory bank for the next domain migration. The memory bank after the domain expansion can be used to update the parameters of the target detection model of the target domain obtained in this domain migration to obtain the target detection model of another target domain, that is, the target detection model of the target domain for the next domain migration.

[0074] In addition, in combination with the above embodiments, in another embodiment, the labeled source domain images and the unlabeled target domain images can be standardized and preprocessed to normalize the pixel values ​​of the labeled source domain images and the unlabeled target domain images to a unified numerical range (such as [0,1] or [0,255]) to reduce the feature inconsistency caused by the difference in numerical ranges of different data sources. And, in one embodiment, the source domain images stored in the memory library must ensure that the images are stored and processed in a consistent data format to avoid interference with target recognition performance due to data inconsistency. This standardization operation of the present embodiment not only improves the stability and convergence speed of target detection model training, but also provides a more reliable data basis for subsequent feature extraction and recognition tasks.

[0075] In one embodiment, if Figure 2 As shown, Figure 2 FIG. 1 is a schematic diagram showing an overall target recognition method based on source domain memory and category-level alignment according to an embodiment of the present invention. Figure 2In this paper, the method can receive image inputs from multiple fields, generate target domain distribution data through category-level alignment, and perform unsupervised optimization on the target domain data, thereby achieving efficient and robust target recognition in cross-domain scenarios. Specifically, the image data is first preprocessed to normalize the pixel values ​​of all images to a unified numerical range to reduce the feature inconsistency caused by differences in numerical ranges of different data sources. Then, the source domain image and the unlabeled target domain image are classified and calculated to determine the category with the highest matching degree with the unlabeled target domain image; the first source domain image with a label that is aligned with the category of the unlabeled target domain image is selected from the labeled source domain images belonging to the category with the highest matching degree; then, the first source domain image and the unlabeled target domain image are respectively subjected to Fourier transform to obtain the frequency domain features of the first source domain image and the unlabeled target domain image, and the central area of ​​the amplitude spectrum of the first source domain image is replaced with the corresponding central area of ​​the amplitude spectrum of the unlabeled target domain image while keeping the phase spectrum of the first source domain image unchanged and other areas of the amplitude spectrum of the first source domain image unchanged, so as to realize feature conversion and obtain the adjusted frequency domain features of the first source domain image, and then obtain the labeled source domain target image through data generation. Finally, based on the source domain target image and the source domain target detection model, the pseudo label of the target domain image is generated, and the source domain target detection model is updated with the target domain image with the pseudo label to obtain the target detection model of the target domain, so as to realize the target domain migration of the target detection model. After the target domain migration is successful, the target domain data (the target detection results output by the target domain detection model are used as labels to obtain the target domain image to be detected with labels) is dynamically stored in the memory unit (i.e., the memory bank) for subsequent domain migration.

[0076] In this embodiment, category-level alignment significantly alleviates the problem of feature distribution differences between domains by aligning the category-level distribution of source domain and target domain features. By performing style transfer on the source domain data in the frequency domain, the features of the source domain data are mapped to a feature space that is close to the target domain distribution, thereby generating high-quality labeled data that matches the target domain, further guiding model optimization. In addition, when new categories appear, the data information stored in the memory module can be updated using a dynamic update mechanism. The recognition method based on category-level alignment and dynamic memory update not only effectively avoids the performance loss caused by pseudo-label generation errors, but also significantly improves the adaptability and robustness of the model in multi-domain scenarios, greatly improving the performance in cross-domain complex tasks. The method provided in this embodiment can significantly improve the accuracy and speed of target detection in wide-field and high-resolution scenes, and provide an image content analysis device for a newer generation of imaging systems.

[0077] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0078] Based on the same inventive concept, an embodiment of the present invention provides a cross-domain target recognition device. Figure 3 , Figure 3 FIG. 1 is a block diagram of a cross-domain target recognition device provided by an embodiment of the present invention. Figure 3 As shown, the cross-domain target recognition device of this embodiment may include: An image acquisition module is used to acquire a source domain image carrying a label and an unlabeled target domain image, wherein the label carried by the source domain image represents the category and position box of the target in the source domain image; A distribution alignment module, used to align the labeled source domain image to the unlabeled target domain image in the frequency domain, so as to obtain a labeled source domain target image that is aligned with the feature distribution of the unlabeled target domain image; An image processing module, used for inputting the source domain target image into a source domain target detection model to obtain a target detection result of the source domain target image, wherein the target detection result of the source domain target image represents a category and a position box of a target in the source domain target image; a pseudo-label marking module, configured to use the target detection result of the source domain target image as the pseudo-label of the unlabeled target domain image, mark the unlabeled target domain image, and obtain a target domain image carrying the pseudo-label; A model updating module, used to update the parameters of the source domain target detection model using the target domain image carrying the pseudo-label, so as to obtain the target domain target detection model; The target detection module is used to perform target detection on the target domain image to be detected through the target detection model of the target domain, and obtain the target detection result of the target domain image to be detected, wherein the target detection result of the target domain image to be detected represents the category and position box of the target in the target domain image to be detected.

[0079] Optionally, a distribution alignment module includes: A category alignment module, configured to screen out a first source domain image carrying a label that is category-aligned with the unlabeled target domain image from the source domain images carrying the label; A first transformation module is used to perform Fourier transform on the first source domain image and the unlabeled target domain image respectively to obtain frequency domain features of the first source domain image and the unlabeled target domain image, wherein the frequency domain features include a phase spectrum and an amplitude spectrum; A local replacement module, used to keep the phase spectrum of the first source domain image unchanged and keep other regions of the amplitude spectrum of the first source domain image unchanged, and replace the central region of the amplitude spectrum of the first source domain image with the corresponding central region of the amplitude spectrum of the unlabeled target domain image to obtain the adjusted frequency domain features of the first source domain image; The second transformation module is used to perform inverse Fourier transform on the adjusted frequency domain features of the first source domain image to obtain the source domain target image carrying the label.

[0080] Optionally, the model updating module includes: The model training module is used to update the parameters of the source domain target detection model by using the source domain target image carrying the label and the target domain image carrying the pseudo label to obtain the target domain target detection model.

[0081] Optionally, the device further comprises: A clustering module, used for extracting features and clustering the source domain images carrying labels stored in the memory bank to obtain k categories and cluster centers of the k categories; Category alignment module, including: A reading module, used for reading the cluster centers of the k categories from the memory bank; A matching module, used to extract features of the unlabeled target domain image and determine a matching degree between the features of the unlabeled target domain image and a cluster center of each of the k categories read; The screening module is used to determine n labeled first source domain images belonging to the category with the highest matching degree, where n is an integer greater than 1.

[0082] Optionally, the device further comprises: A first label determination module is used to perform target detection on a target domain image to be detected by using a target detection model of the target domain, and after obtaining a target detection result of the target domain image to be detected, use the target detection result of the target domain image to be detected as a label carried by the target domain image to be detected; A first category determination module is used to determine whether the category of the target in the target domain image to be detected is the same as any category of the k categories; A first adding module is used to add all the target domain images to be detected that carry labels to the memory bank to obtain a memory bank with expanded categories when the category of the target in the target domain image to be detected is different from the k categories; The first updating module is used to update the parameters of the target detection model of the target domain by using the memory library after the category expansion to obtain the target detection model of another target domain.

[0083] Optionally, the device further comprises: A second label determination module is used to perform target detection on the target domain image to be detected by using the target detection model of the target domain, and after obtaining the target detection result of the target domain image to be detected, use the target detection result of the target domain image to be detected as the label carried by the target domain image to be detected; A second category determination module is used to determine whether the category of the target in the target domain image to be detected is the same as any category in the k categories; A second adding module is used for selecting a portion of the target domain images to be detected carrying labels in descending order of confidence to add to the memory bank when the category of the target in the target domain image to be detected is one of the k categories, for the category, to obtain the memory bank after domain expansion; The second updating module is used to update the parameters of the target detection model of the target domain by using the memory bank after the domain expansion to obtain the target detection model of another target domain.

[0084] Optionally, the device further comprises: A third label determination module is used to, after obtaining the target detection results of each of the multiple target domain images to be detected, use the target detection results of each of the multiple target domain images to be detected as the labels carried by each of the multiple target domain images to be detected; A third adding module is used to add a plurality of target domain images to be detected carrying labels as source domain images carrying labels of different source domains to the memory bank to obtain a memory bank after domain expansion; The third updating module is used to update the parameters of the target detection model of the target domain by using the memory bank after the domain expansion to obtain the target detection model of another target domain.

[0085] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the cross-domain target recognition method as described in any of the above embodiments of the present invention are implemented.

[0086] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, such as Figure 4 As shown, Figure 4 The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the cross-domain target recognition method described in any of the above embodiments of the present invention when executing the computer program.

[0087] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0088] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0089] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, embodiments of the present invention may take the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware. Furthermore, embodiments of the present invention may take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0090] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0093] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0094] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0095] The above is a detailed introduction to a cross-domain target recognition method, device, equipment and medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A cross-domain target recognition method, characterized in that: The method comprises: Acquire a source domain image carrying a label and an unlabeled target domain image, wherein the label carried by the source domain image represents the category and position box of the target in the source domain image; Aligning the labeled source domain image to the unlabeled target domain image in the frequency domain to obtain a labeled source domain target image aligned with the feature distribution of the unlabeled target domain image; Inputting the source domain target image into a source domain target detection model to obtain a target detection result of the source domain target image, wherein the target detection result of the source domain target image represents a category and a position box of a target in the source domain target image; Using the target detection result of the source domain target image as a pseudo label of the unlabeled target domain image, labeling the unlabeled target domain image, and obtaining a target domain image carrying the pseudo label; Using the target domain image carrying the pseudo-label, updating the parameters of the source domain object detection model to obtain the target domain object detection model; By using the target detection model of the target domain, target detection is performed on the target domain image to be detected to obtain a target detection result of the target domain image to be detected, wherein the target detection result of the target domain image to be detected represents a category and a position frame of the target in the target domain image to be detected.

2. The cross-domain target recognition method according to claim 1, characterized in that: Aligning the labeled source domain image to the unlabeled target domain image in the frequency domain to obtain the labeled source domain target image aligned with the feature distribution of the unlabeled target domain image includes: From the labeled source domain images, select a first labeled source domain image that is aligned with the category of the unlabeled target domain image; Performing Fourier transform on the first source domain image and the unlabeled target domain image respectively to obtain frequency domain features of the first source domain image and the unlabeled target domain image, wherein the frequency domain features include a phase spectrum and an amplitude spectrum; While keeping the phase spectrum of the first source domain image unchanged, keeping other regions of the amplitude spectrum of the first source domain image unchanged, replacing the central region of the amplitude spectrum of the first source domain image with the corresponding central region of the amplitude spectrum of the unlabeled target domain image, to obtain the adjusted frequency domain features of the first source domain image; Perform an inverse Fourier transform on the adjusted frequency domain features of the first source domain image to obtain the source domain target image carrying the label.

3. The cross-domain target recognition method according to claim 1 or 2, characterized in that: Using the target domain image carrying the pseudo label, updating the parameters of the source domain object detection model to obtain the target domain object detection model, including: The source domain target image carrying the label and the target domain image carrying the pseudo label are used to update the parameters of the source domain target detection model to obtain the target domain target detection model.

4. The cross-domain target recognition method according to claim 2, characterized in that: The method further comprises: Extracting features and clustering the labeled source domain images stored in the memory bank to obtain k categories and cluster centers of the k categories; The method further comprises: selecting a first source domain image carrying a label and aligning the category of the unlabeled target domain image from the source domain image carrying the label, comprising: Reading the cluster centers of the k categories from the memory bank; Extracting features of the unlabeled target domain image, and determining a matching degree between the features of the unlabeled target domain image and a cluster center of each of the k categories read; Determine n labeled first source domain images belonging to the category with the highest matching degree, where n is an integer greater than 1.

5. The cross-domain target recognition method according to claim 4, characterized in that: After performing target detection on the target domain image to be detected by using the target detection model of the target domain and obtaining the target detection result of the target domain image to be detected, the method further includes: Using the target detection result of the target domain image to be detected as a label carried by the target domain image to be detected; Determine whether the category of the target in the target domain image to be detected is the same as any category in the k categories; In the case where the category of the target in the target domain image to be detected is different from the k categories, all the target domain images to be detected carrying labels are added to the memory bank to obtain a memory bank with expanded categories; The memory library after the category expansion is used to update the parameters of the target detection model of the target domain to obtain the target detection model of another target domain.

6. The cross-domain target recognition method according to claim 4, characterized in that: After performing target detection on the target domain image to be detected by using the target detection model of the target domain and obtaining the target detection result of the target domain image to be detected, the method further includes: Using the target detection result of the target domain image to be detected as a label carried by the target domain image to be detected; Determine whether the category of the target in the target domain image to be detected is the same as any category in the k categories; When the category of the target in the target domain image to be detected is one of the k categories, for the category, a portion of the target domain images to be detected carrying labels are selected in descending order of confidence and added to the memory bank to obtain a memory bank after domain expansion; The memory bank after the domain expansion is used to update the parameters of the target detection model of the target domain to obtain the target detection model of another target domain.

7. The cross-domain target recognition method according to claim 4, characterized in that: After obtaining the target detection results of each of the plurality of target domain images to be detected, the method further includes: Using the target detection results of each of the multiple target domain images to be detected as the labels carried by each of the multiple target domain images to be detected; Adding a plurality of target domain images to be detected carrying labels as source domain images carrying labels of different source domains to the memory bank to obtain a memory bank after domain expansion; The memory bank after the domain expansion is used to update the parameters of the target detection model of the target domain to obtain the target detection model of another target domain.

8. A cross-domain target recognition device, characterized in that: The device comprises: An image acquisition module is used to acquire a source domain image carrying a label and an unlabeled target domain image, wherein the label carried by the source domain image represents the category and position box of the target in the source domain image; A distribution alignment module, used to align the labeled source domain image to the unlabeled target domain image in the frequency domain, so as to obtain a labeled source domain target image that is aligned with the feature distribution of the unlabeled target domain image; An image processing module, used for inputting the source domain target image into a source domain target detection model to obtain a target detection result of the source domain target image, wherein the target detection result of the source domain target image represents a category and a position box of a target in the source domain target image; a pseudo-label marking module, configured to use the target detection result of the source domain target image as the pseudo-label of the unlabeled target domain image, mark the unlabeled target domain image, and obtain a target domain image carrying the pseudo-label; A model updating module, used to update the parameters of the source domain target detection model using the target domain image carrying the pseudo-label, so as to obtain the target domain target detection model; The target detection module is used to perform target detection on the target domain image to be detected through the target detection model of the target domain, and obtain the target detection result of the target domain image to be detected, wherein the target detection result of the target domain image to be detected represents the category and position box of the target in the target domain image to be detected.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by the processor, the cross-domain target recognition method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the cross-domain target recognition method according to any one of claims 1 to 7 is implemented.

Images