A cross-domain target detection method, device, network and equipment

Abstract

This invention provides a cross-domain target detection method, apparatus, network, and device. The cross-domain target detection method includes: constructing a cross-domain target detector, which includes a basic target detector, a domain alignment module, and a variational instance decoupling module embedded in the basic target detector. The variational instance decoupling module assists the cross-domain target detector in separating feature elements irrelevant to the target task and feature elements relevant to the target task; constructing labeled source domain samples and unlabeled target domain samples, each composed of multiple images; training the cross-domain target detector based on the source domain samples and target domain samples; and performing cross-domain target detection processing on the target domain samples based on the trained cross-domain target detector. The cross-domain target detection method of this invention can separate information relevant to the target task, effectively adapts to the source and target domains, and performs further target detection based on this information, significantly improving detection performance.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular to a cross-domain target detection method, apparatus, network and device. Background Technology

[0002] With breakthroughs in deep learning technology, object detection has achieved remarkable success. However, if the distribution of the test dataset is inconsistent with that of the training dataset, the performance of object detectors will degrade significantly, greatly limiting the application of object detection, such as autonomous driving and video analysis. Meanwhile, instance-level annotation is expensive and time-consuming, hindering the achievement of satisfactory model generalization ability in new domains through supervised learning. Therefore, cross-domain object detection has attracted considerable interest in recent years.

[0003] Cross-domain object detection refers to enabling a detector trained in a labeled source domain to adapt to an unlabeled target domain. To achieve cross-domain object detection, many studies have adopted a general paradigm to alleviate the problems of domain transfer and data labeling: unsupervised domain adaptation (UDA) to align the distributions of different domains. Many existing methods learn transferable features of the detector through adversarial training, self-training, and distillation methods. While these methods have achieved satisfactory performance, they focus on aligning distributions in the overall representation of the image, ignoring the coupling of domain-specific factors, which hinders the full exploitation of potential performance. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of the present invention provide a cross-domain target detection method, comprising:

[0005] A cross-domain target detector is constructed, which includes a basic target detector, a domain alignment module, and a variational instance disentanglement (VID) module embedded in the cross-domain target detector. The variational instance disentanglement module is used to assist the cross-domain target detector in separating feature elements that are irrelevant to the target task and feature elements that are relevant to the target task.

[0006] Labeled source domain samples and unlabeled target domain samples are constructed, each of which consists of multiple images;

[0007] The cross-domain target detector is trained based on the source domain samples and the target domain samples;

[0008] Cross-domain target detection processing is performed on the target domain samples based on the trained cross-domain target detector.

[0009] As an optional embodiment, the cross-domain target detector includes an instance feature extraction module and a task head, and the variational instance decoupling module is embedded between the instance feature extraction module and the task head.

[0010] As an optional embodiment, it also includes:

[0011] A loss function is constructed to implement variational information bottleneck constraint. The variational information bottleneck constraint is used to filter out feature elements that are irrelevant to the target task in the hidden layer features of the cross-domain target detector, so as to retain feature elements that are relevant to the target task.

[0012] The variational instance decoupling module and the loss function are used to decouple irrelevant information of the target task.

[0013] As an optional embodiment, the construction loss function includes:

[0014] Determine the input data, hidden layer variables, and target output data of the cross-domain target detector;

[0015] The input data, hidden layer variables, and target output data are substituted into the objective function, which is L. ib =I(Y,Z)-βI(X,Z), where I is mutual information and β is a hyperparameter for adjusting weights. The first term in the formula is used to encourage the hidden layer variables to learn enough information, and the second term in the formula is used to ensure that feature elements that are not relevant to the target task are filtered.

[0016] The loss function is constructed based on the objective function.

[0017] As an optional embodiment, it also includes:

[0018] Based on the loss function, the variational instance decoupling module reconstructs the hidden layer variables to form new hidden layer variables that retain feature elements related to the target task.

[0019] As an optional embodiment, the loss function is:

[0020]

[0021] Where N represents the total number of samples, M represents the number of instance features in the feature map of a single image, and p(z|x) is a Gaussian distribution N(μ,σ). 2 ), z is a new hidden layer variable that retains feature elements related to the target task, z = μ + σ⊙∈, where μ and σ are parameters of the distribution that the hidden layer variable follows before reconstruction, ∈ is a standard Gaussian random variable, ⊙ represents element-wise multiplication, and r(z) is a standard Gaussian distribution.

[0022] Another embodiment of the present invention also provides a cross-domain target detection device, comprising:

[0023] The first construction module is used to construct a cross-domain target detector. The cross-domain target detector includes a basic target detector, a domain alignment module, and a variational instance decoupling module embedded in the cross-domain target detector. The variational instance decoupling module is used to assist the cross-domain target detector in separating feature elements that are irrelevant to the target task and feature elements that are relevant to the target task.

[0024] The second construction module is used to construct labeled source domain samples and unlabeled target domain samples, both of which are composed of multiple images.

[0025] The training module is used to train the cross-domain target detector based on the source domain samples and the target domain samples;

[0026] The detection module is used to perform cross-domain target detection processing on the target domain samples based on the trained cross-domain target detector.

[0027] Another embodiment of the present invention provides a cross-domain target detection network, including a cross-domain target detection network and a cross-domain target detection device integrated in the cross-domain target detection network.

[0028] As an optional embodiment, the cross-domain object detection network includes at least one of the Variational Instance Disentanglement Adversarial Network (VIDAN) based on adversarial learning and the Variational Instance Disentanglement Self-training Network (VIDSN) based on self-training.

[0029] Another embodiment of the present invention also provides a cross-domain target detection device, comprising:

[0030] At least one processor; and,

[0031] A memory communicatively connected to the at least one processor; wherein,

[0032] The memory stores instructions that can be executed by the at least one processor to implement the cross-domain target detection method as described above.

[0033] Based on the disclosure of the above embodiments, it can be understood that the beneficial effects of the embodiments of the present invention include: by designing a variational instance decoupling module and introducing it into a cross-domain object detection target detector, task-related and task-independent information in the input data can be decoupled based on the target task, enabling the cross-domain object detector to better learn transferable hidden layer representations and improve object detection performance. Furthermore, the variational instance decoupling module in this application can be embedded as a plug-and-play module into cross-domain object detection networks, such as adversarial networks and self-trained networks, to form corresponding object detection models. These models not only eliminate the influence of task-independent information on object detection but also retain all the advantages of the original detection model.

[0034] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0035] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0036] The accompanying drawings are provided to further illustrate the present application and form part of the specification. They are used together with the embodiments of the present application to explain the application and do not constitute a limitation thereof. In the drawings:

[0037] Figure 1 This is a flowchart of a cross-domain target detection method in an embodiment of the present invention.

[0038] Figure 2 This is a diagram illustrating the task-related feature decoupling process in an embodiment of the present invention.

[0039] Figure 3 This is an architecture diagram of the cross-domain target detection network in an embodiment of the present invention.

[0040] Figure 4 This is a structural block diagram of the cross-domain target detection device in an embodiment of the present invention. Detailed Implementation

[0041] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings, but these are not intended to limit the scope of the invention.

[0042] It should be understood that various modifications can be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this disclosure will be apparent to those skilled in the art.

[0043] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present disclosure and, together with the general description of the disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the disclosure.

[0044] These and other features of the invention will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0045] It should also be understood that although the invention has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of the invention, which have the features described in the claims and are therefore all within the scope of protection defined herein.

[0046] The above and other aspects, features and advantages of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0047] Specific embodiments of the present disclosure are described thereafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure, which may be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the present disclosure. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but merely to serve as the basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in a variety of substantially any suitable detailed structures.

[0048] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in still another embodiment,” all of which may refer to one or more of the same or different embodiments according to this disclosure.

[0049] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0050] like Figure 1 As shown in the figure, this application provides a cross-domain target detection method, including:

[0051] S101: Construct a cross-domain target detector, which includes a basic target detector, a domain alignment module, and a variational instance decoupling module embedded in the cross-domain target detector. The variational instance decoupling module is used to assist the cross-domain target detector in separating feature elements that are irrelevant to the target task and feature elements that are relevant to the target task.

[0052] S102: Construct labeled source domain samples and unlabeled target domain samples, each of which consists of multiple images;

[0053] S103: Training a cross-domain target detector based on source domain samples and target domain samples;

[0054] S104: Perform cross-domain target detection processing on target domain samples based on the trained cross-domain target detector.

[0055] Cross-domain object detection utilizes a labeled source domain to learn an object detector that performs well in a new unlabeled target domain. Most existing works primarily rely on knowledge of the entire image to achieve distribution alignment, neglecting the obstacle of task-irrelevant information in reducing domain differences. To address this issue, this application proposes an information-theoretic-based Variational Instance Disentanglement (VID) module. This module aims to decouple task-related information while filtering out task-irrelevant factors, thereby better enabling transferable hidden layer representation learning for cross-domain object detectors and ultimately improving cross-domain object detection performance.

[0056] Furthermore, in object detection scenarios, the focus is primarily on information such as image spatial distribution, semantic relationships, object structure, and texture; this information can be termed task-related information. Forcing domain adaptation by aligning features containing task-irrelevant information such as background and color will predictably affect the detector's discriminability. This application aims to eliminate the interference of task-irrelevant information to achieve better domain adaptation and maximize the satisfaction of downstream object localization and classification tasks. Specific results can be found in [reference needed]. Figure 2 As shown.

[0057] To address the aforementioned issues, this application incorporates the Information Bottleneck (IB) theory, introducing the Variational Information Bottleneck (VIB) technique—which makes information bottlenecks easier to apply to deep neural networks—into the field of cross-domain object detection. Specifically, based on information theory, this embodiment proposes a Variational Instance Decoupling (VID) module, located between the instance feature extraction module and the task head of the cross-domain object detector in this embodiment. The VID module maximizes the mutual information between hidden layer variables and task output, and minimizes the mutual information between hidden layer variables and input, allowing task-related information to pass through while filtering out task-irrelevant factors in the input data, thereby better achieving transferable hidden layer representation learning.

[0058] In form, this embodiment has N s A collection of labeled source domain samples and by N t A set of unlabeled target domain samples in This represents the i-th source domain image instance sampled. The bounding box label, Indicates the corresponding category label, Let represent the i-th sampled target domain image. The ultimate goal of cross-domain object detection is to utilize D... s and D t To design a target detector with domain-invariant properties.

[0059] Furthermore, the method in this embodiment also includes:

[0060] S105: Construct a loss function to implement variational information bottleneck constraint based on the loss function. The variational information bottleneck constraint is used to filter out feature elements that are irrelevant to the target task in the hidden layer features of the cross-domain target detector, so as to retain feature elements that are relevant to the target task.

[0061] S106: Decoupling of task-independent information is achieved based on the variational instance decoupling module and the loss function.

[0062] like Figure 2 As shown, this embodiment uses Faster R-CNN as the basis for the object detector. Under the supervised paradigm, it uses a loss function L det The training process uses a loss function that includes the RPN classification loss. RPN regression loss RoI classification loss RoI regression loss The proposed VID module is embedded before the RoI classification and regression heads to decouple features. The loss function can be expressed as follows:

[0063]

[0064] The construction of the loss function includes:

[0065] S107: Determine the input data, hidden layer variables, and target output data of the target detector;

[0066] S108: Substitute the input data, hidden layer variables, and target output data into the objective function, which is L. ib =I(Y,Z)-βI(X,Z), where I is mutual information and β is a hyperparameter for adjusting the weights. The first term in the formula is used to encourage the hidden layer variables to learn enough information, and the second term is used to ensure that feature elements that are irrelevant to the target task are filtered.

[0067] S109: Construct a loss function based on the objective function.

[0068] In this embodiment, based on the loss function, the variational instance decoupling module reconstructs the hidden layer variables to form new hidden layer variables that retain feature elements related to the target task.

[0069] The constructed loss function is:

[0070]

[0071] Where N represents the total number of samples, M represents the number of instance features in the feature map of a single image, and p(z|x) is a Gaussian distribution N(μ,σ). 2 ), z is a new hidden layer variable that retains features related to the target task, z=μ+σ⊙∈, where μ and σ are parameters of the distribution that the hidden layer variable follows before reconstruction, ∈ is a standard Gaussian random variable, ⊙ represents element-wise multiplication, and r(z) is a standard Gaussian distribution.

[0072] The construction process of the above loss function specifically includes:

[0073] The variational information bottleneck constraint is used to extract transferable hidden layer representations from cross-domain detectors. This application selects the variational information bottleneck to filter task-insensitive information in the hidden layer features while retaining task-related information. Following the formalization of information bottleneck theory, let X, Z, and Y represent the input, hidden layer variables, and target output, respectively. The objective function to be maximized is defined as follows:

[0074] L ib =I(Y, Z) - βI(X, Z)

[0075] Where I is mutual information and β is a hyperparameter for adjusting the weights. The first term is used to encourage the hidden variables to learn sufficient information, and the second term is used to ensure that redundant information is filtered out. Specifically, it can be expanded according to the definition of mutual information as follows:

[0076]

[0077]

[0078] Since p(y|z) and p(z) are difficult to compute here, q(y|z) and r(z) are used as variational approximations. q(y|z) is the decoder of the model, and r(z) can be regarded as a standard Gaussian distribution N(0,I). Given that the Kullback-Leibler divergence is non-negative, this application has the following equation:

[0079] ∫dyp(y|z)logp(y|z)≥∫dyp(y|z)logq(y|z)

[0080] ∫dzp(z)logp(z)≥∫dzp(z)logr(z)

[0081] Therefore, L ib The following variational lower bounds can be generated:

[0082]

[0083]

[0084] Based on the Markov chain assumption of X, Y, Z: Y and Z are independent of a given X. In this embodiment, we have:

[0085] p(y,z)=∫dxp(x)p(y|x)p(z|x)

[0086] The previous expression can then be rewritten as:

[0087]

[0088] Therefore, we can obtain the loss function that we are trying to maximize under the constraint of variational information bottleneck, as represented by the lower bound in the above equation.

[0089] exist Figure 3 In Faster R-CNN, firstly, a Region Proposal Network (RPN) is used to generate candidate boxes to determine the Regions of Interest (RoIs), and then the extracted instance features are used for target prediction. To make the VIB constraint more focused on accurate final prediction, this embodiment uses instance features as hidden layer variables to establish a Variational Instance Decoupling (VID) module.

[0090] For example, recalling the loss function (the expression for the lower bound) in the above equation, where E z～p(z|x) [-logq(y|z)] can be viewed as the classification cross-entropy of the detection loss in the supervised paradigm, where H(Y) is the label entropy independent of optimization in the supervised paradigm. Since this embodiment does not need to consider the first two terms in the loss function, the goal of this embodiment is to minimize E. x～p(x) KL[p(z|x), r(z)]. Assuming a single image generates multiple instance features corresponding to different outputs, the variational instance decoupling loss is:

[0091]

[0092] Where N represents the number of samples in the dataset, and M represents the number of instance features in the feature map of a single image. p(z|x) is defined as a Gaussian distribution N(μ,σ). 2 Since random distribution sampling is non-differentiable, this embodiment employs a reparameterization technique to generate an approximate value for the hidden variable z. Let z = μ + σ⊙∈, where μ and σ are represented by the previous hidden variable z′, i.e., the hidden variable before information reconstruction, and z represents the reconstructed hidden variable. For details, please refer to... Figure 3As shown, ∈ is a standard Gaussian random variable, and ⊙ represents element-wise multiplication. Therefore, this embodiment reconstructs a new hidden layer variable z in the above manner, obtaining: p(z|x)dz=p(∈)d∈. The above method in this embodiment does not introduce any additional network parameters, thus avoiding an increase in computation, and preserves the original appearance of the detector as much as possible, reducing the difficulty of implementation.

[0093] like Figure 4 As shown, another embodiment of the present invention provides a cross-domain target detection device, comprising:

[0094] The first construction module is used to construct a cross-domain target detector. The cross-domain target detector includes a basic target detector and a variational instance decoupling module embedded in the basic target detector. The variational instance decoupling module is used to assist the cross-domain target detector in separating feature elements that are not related to the target task and feature elements that are related to the target task.

[0095] The second construction module is used to construct labeled source domain samples and unlabeled target domain samples, both of which are composed of multiple images.

[0096] The training module is used to train the target detector based on the source domain samples and the target domain samples;

[0097] The detection module is used to perform cross-domain target detection processing on the target domain samples based on the trained cross-domain target detector.

[0098] As an optional embodiment, the cross-domain target detector includes an instance feature extraction module and a task head, and the variational instance decoupling module is embedded between the instance feature extraction module and the task head.

[0099] As an optional embodiment, it also includes:

[0100] A loss function is constructed to implement variational information bottleneck constraint. The variational information bottleneck constraint is used to filter out feature elements that are irrelevant to the target task in the hidden layer features of the cross-domain target detector, so as to retain feature elements that are relevant to the target task.

[0101] The variational instance decoupling module is constructed based on the loss function and the hidden layer variables of the cross-domain target detector.

[0102] As an optional embodiment, the construction loss function includes:

[0103] Determine the input data, hidden layer variables, and target output data of the target detector;

[0104] The input data, hidden layer variables, and target output data are substituted into the objective function, which is L.ib =I(Y,Z)-βI(X,Z), where I is mutual information and β is a hyperparameter for adjusting weights. The first term in the formula is used to encourage the hidden layer variables to learn enough information, and the second term in the formula is used to ensure that feature elements that are not relevant to the target task are filtered.

[0105] The loss function is constructed based on the objective function.

[0106] As an optional embodiment, the variational instance decoupling module is further configured to reconstruct the hidden layer variables according to the loss function, forming new hidden layer variables that retain feature elements related to the target task.

[0107] As an optional embodiment, the loss function is:

[0108]

[0109] Where N represents the total number of samples, M represents the number of instance features in the feature map of a single image, and p(z|x) is a Gaussian distribution N(μ,σ). 2 ), z is a new hidden layer variable that retains feature elements related to the target task, z = μ + σ⊙∈, where μ and σ are parameters of the distribution that the hidden layer variable follows before reconstruction, ∈ is a standard Gaussian random variable, ⊙ represents element-wise multiplication, and r(z) is a standard Gaussian distribution.

[0110] Furthermore, another embodiment of the present invention also provides a cross-domain target detection network, characterized in that it includes a cross-domain target detection network and a cross-domain target detection device as described above integrated in the cross-domain target detection network.

[0111] The cross-domain object detection network includes at least one of the Variational Instance Disentanglement Adversarial Network (VIDAN) based on adversarial learning and the Variational Instance Disentanglement Self-training Network (VIDSN) based on self-training. That is, the VID module proposed in the foregoing embodiments of this application can be used as a general module and integrated with various existing cross-domain object detection frameworks to further improve the performance of the detection network.

[0112] For example, continue to refer to Figure 3As shown in the upper and lower left sections, the cross-domain object detection network is a variational instance decoupled adversarial network, containing a cross-domain object detection device. In practical applications, the cross-domain object detection network can include a basic object detector. In this case, the VID module is directly integrated into the detector, and domain alignment is performed through an adversarial learning network, such as a strong-weak distribution alignment network. During training, a local domain classifier D is used. l and least squares loss L ls Strong alignment is performed on the low-level features F′(·), and a global domain classifier D is used. g and focus loss L fl Weak alignment is performed on the high-level features F(·). The adversarial alignment loss is the sum of the two losses mentioned above: L adv =L ls +L fl .

[0113] Combining adversarial training and decoupling, the overall objective function can be expressed as:

[0114]

[0115] The first two terms represent the detection loss and decoupling loss in the source domain, respectively, while β and λ are hyperparameters controlling the loss weights. Furthermore, when training the variational instance decoupling adversarial network, both source and target domain data are used for adversarial training, while only source domain data is used for detector training.

[0116] For example, cross-domain object detection networks are variational instance decoupled self-trained networks. Figure 3 The upper and lower right portions (shown in the diagram) include a cross-domain target detection device. In practical applications, the cross-domain target detection network can include a basic detector. In this case, the VID module is directly integrated into the detector, and domain alignment is performed through a self-trained network. For example, the self-trained network can be a separation adaptation network. Specifically, when training the network, this embodiment first uses... A Faster R-CNN detector with embedded VID modules is pre-trained on the source domain. This trained detector can generate a source object domain. and a target object domain Each object is obtained by cropping the entire image using the target bounding box. obj Composition. Due to Source domain D can be used s To the category and bounding box For labeling, this embodiment utilizes an adversarial domain adaptation model, such as a Conditional Domain Adversarial Network (CDAN), to obtain relatively accurate category labels for unlabeled target object domains. Similarly, adversarial learning can also be used for bounding box adaptation. Figure 3 The adversarial regressor AdvReg shown is designed to maximize its difference from the main regressor Reg in the target object domain while minimizing its difference from the main regressor Reg in the source object domain. This difference is measured using the Generalized Intersection over Union (GIoU). This yields the pseudo-labels for the target object domain. and bounding box pseudo-labels This is used for supervised training of the detector in the target domain. Therefore, supervised variational instance decoupling can be performed in both the source and target domains. In summary, the final objective function of the variational instance decoupling self-trained network is:

[0117]

[0118] The first two terms represent the detection and decoupling losses in the source domain, while the last two terms represent the losses in the target domain. Note that... The loss does not include RPN. β1 and β2 can balance the various loss terms.

[0119] Another embodiment of this application also provides a cross-domain target detection device, including:

[0120] At least one processor; and,

[0121] A memory communicatively connected to the at least one processor; wherein,

[0122] The memory stores instructions that can be executed by the at least one processor to implement the cross-domain target detection method as described in any of the embodiments above.

[0123] Furthermore, one embodiment of this application also provides a storage medium storing a computer program, which, when executed by a processor, implements the cross-domain target detection method described above. It should be understood that the various solutions in this embodiment have the corresponding technical effects in the above-described method embodiments, and will not be repeated here.

[0124] Furthermore, embodiments of this application also provide a computer program product, which is tangibly stored on a computer-readable medium and includes computer-readable instructions that, when executed, cause at least one processor to perform a target comprehensive recognition method based on high-dimensional feature maps, such as the embodiments described above.

[0125] It should be noted that the computer storage medium in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access storage media (RAM), read-only storage media (ROM), erasable programmable read-only storage media (EPROM or flash memory), optical fibers, portable compact disk read-only storage media (CD-ROM), optical storage media, magnetic storage media, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program configured for use by or in connection with an instruction execution system, system, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, antenna, optical fiber, RF, etc., or any suitable combination thereof.

[0126] Furthermore, those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0127] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0128] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0130] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

[0131] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the present invention. The scope of protection of the present invention is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to the present invention within its spirit and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of the present invention.

Claims

1. A method for cross-domain object detection, characterized in that, include: A cross-domain target detector is constructed, which includes a basic target detector, a domain alignment module, and a variational instance decoupling module embedded in the basic target detector. The variational instance decoupling module is used to assist the cross-domain target detector in separating feature elements that are irrelevant to the target task and feature elements that are relevant to the target task. Labeled source domain samples and unlabeled target domain samples are constructed, each of which consists of multiple images; The cross-domain target detector is trained based on the source domain samples and the target domain samples; Cross-domain target detection processing is performed on the target domain samples based on the trained cross-domain target detector; The cross-domain target detector includes an instance feature extraction module and a task head, and the variational instance decoupling module is embedded between the instance feature extraction module and the task head; Also includes: A loss function is constructed to implement variational information bottleneck constraints. The variational information bottleneck constraints are used to filter out feature elements that are irrelevant to the target task in the hidden layer features of the cross-domain target detector, so as to retain feature elements that are relevant to the target task. The variational instance decoupling module and the loss function are used to decouple irrelevant information of the target task. The variational instance decoupling module reconstructs the hidden layer variables to form new hidden layer variables that retain feature elements related to the target task.

2. The cross-domain object detection method of claim 1, wherein, The construction loss function includes: Determine the input data, hidden layer variables, and target output data of the target detector; The input data, the hidden layer variables, and the target output data are brought into an objective function, the objective function being where I is mutual information, β is a hyperparameter that regulates the weight, where the first term is used to encourage the hidden layer variables to learn enough information, and where the second term is used to ensure that irrelevant features are filtered out from the target task; The loss function is constructed based on the objective function.

3. The cross-domain target detection method according to claim 1, characterized in that, The loss function is: Where N represents the total number of samples, and M represents the number of instance features in the feature map of a single image. p ( | x ( ) is a Gaussian distribution N ( µ , σ 2 ), To retain new hidden layer variables containing feature elements relevant to the target task, = µ + σ ⊙ ,in µ and σ Hidden layer variables The parameters of the distribution it followed before reconstruction. Let be a standard Gaussian random variable, and ⊙ denote element-wise multiplication. r ( ) is a standard Gaussian distribution.

4. A cross-domain target detection device, characterized in that, include: The first construction module is used to construct a cross-domain target detector. The cross-domain target detector includes a basic target detector, a domain alignment module, and a variational instance decoupling module embedded in the basic target detector. The variational instance decoupling module is used to assist the cross-domain target detector in separating feature elements that are irrelevant to the target task and feature elements that are relevant to the target task. The second construction module is used to construct labeled source domain samples and unlabeled target domain samples, both of which are composed of multiple images. The training module is used to train the cross-domain target detector based on the source domain samples and the target domain samples; The detection module is used to perform cross-domain target detection processing on the target domain samples based on the trained cross-domain target detector; The cross-domain target detector includes an instance feature extraction module and a task head, and the variational instance decoupling module is embedded between the instance feature extraction module and the task head; Also includes: A construction module is used to construct a loss function to implement variational information bottleneck constraints. The variational information bottleneck constraints are used to filter out feature elements that are irrelevant to the target task in the hidden layer features of the cross-domain target detector, so as to retain feature elements that are relevant to the target task. The variational instance decoupling module and the loss function are used to decouple irrelevant information of the target task. The variational instance decoupling module reconstructs the hidden layer variables to form new hidden layer variables that retain feature elements related to the target task.

5. A cross-domain target detection network, characterized in that, It includes a cross-domain target detection network and the cross-domain target detection device as described in claim 4, which is integrated into the cross-domain target detection network.

6. The cross-domain target detection network according to claim 5, characterized in that, The cross-domain object detection network includes at least one of the following: a variational instance decoupling cross-domain object detection network based on adversarial learning and a variational instance decoupling cross-domain object detection network based on self-training.

7. A cross-domain target detection device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to implement the cross-domain target detection method as described in any one of claims 1-3.

Images