Passive domain adaptive semantic segmentation method and device for diffusion-guided pseudo-label enhancement

Abstract

The invention provides a passive domain adaptive semantic segmentation method and device for diffusion-guided pseudo-label enhancement, and relates to the technical field of semantic segmentation. The method comprises the following steps: acquiring and enhancing an original target domain image, and predicting the original image and the enhanced image through a source domain model to obtain an initial pseudo tag; performing confidence ranking and filtering on the initial pseudo labels to obtain a pseudo label set, constructing seed pseudo labels, and adding Gaussian noise to obtain pseudo labels with different noise levels; extracting context features from the original target domain image by using an encoder, training a diffusion model, and generating a pseudo tag; and pairing the pseudo tag and the original image, performing supervised training on the source domain model according to cross entropy loss to obtain a trained model, and completing semantic segmentation of target domain data. According to the method, a diffusion model is introduced, starting from a small number of high-quality pseudo labels easy to obtain, the pseudo labels are propagated to a complete pseudo label set on the premise that the quality is guaranteed, and therefore pseudo label optimization is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of semantic segmentation, in particular to a diffusion guided pseudo label enhanced unsupervised domain adaptive semantic segmentation method and device. BACKGROUND

[0002] Unsupervised domain adaptation is a method that does not require source domain data, but only uses target domain unlabeled data and a pre-trained source domain model to adapt the model to a new environment. This method can ensure the privacy and security of the source domain data, and solve the problem of sharing original data due to data protection or security requirements in practical work. It is particularly suitable for scenarios that require data security, such as remote sensing images, medical images, etc., so that the model can still achieve good performance in a new scenario without accessing sensitive data.

[0003] Self-training is a commonly used model optimization method, which refers to using the model's prediction results on unlabeled data as pseudo labels, and then using these pseudo labels to iteratively train the model itself. By continuously updating the pseudo labels and model parameters, the model can gradually improve its performance in new data or new environments. The self-training method can effectively utilize a large number of unlabeled samples, thereby improving the model's generalization ability and application effect, and is particularly suitable for scenarios where labeled data is scarce. The core lies in how to improve the accuracy and reliability of the pseudo labels to ensure the final adaptation effect of the model.

[0004] For the source-free domain adaptation method of semantic segmentation, the existing technology is usually based on self-training, which continuously generates and optimizes pseudo-labels to improve the performance of the model in the target domain. Early methods mainly use confidence filtering, which only uses the high-confidence results predicted by the model on the unlabeled data in the target domain as pseudo-labels for training to reduce the impact of pseudo-label noise. With the in-depth study, the source-free domain adaptation method has proposed various innovative strategies to improve the quality of pseudo-labels. For example, the SFDASEG (Source-Free Domain Adaptive Segmentation) method generates multiple source domain data through data augmentation and combines conditional prior constraints to improve the quality of pseudo-labels generated by the model; the DT-ST (Dynamic Teacher-Update and Self-Training) method introduces a dynamic teacher model update mechanism and combines consistency-based resampling to effectively enhance the stability of the self-training process and the reliability of the pseudo-labels; the CROTS (Cross-Domain Teacher-Student Learning) method uses spatial mixing and rare class mining to alleviate the problems of class imbalance and pseudo-label noise; the SND (Stable Neighbor Denoising) method filters pseudo-labels based on prediction stability and achieves denoising through stable neighbor retrieval and class compensation, further improving the quality of pseudo-labels. The above methods have achieved certain application results in the source-free domain adaptation semantic segmentation task, providing an effective technical path for improving the model's generalization ability and actual deployment performance under the condition of lacking labeled and inaccessible source domain data. However, due to the poor performance of the source domain model in the target domain, the initial pseudo-labels generated by the model are generally of low quality. In order to obtain as much supervision signal as possible, existing methods can only filter part of the pseudo-labels with low quality and high noise, resulting in a large amount of noise in the overall pseudo-labels, which limits the effect of pseudo-label optimization and self-training.

[0005] Therefore, existing methods extract part of the reliable supervision information from the initial pseudo-labels through confidence filtering, but due to the insufficient discrimination ability of the model in detail or ambiguous areas, the model may predict high confidence but be wrong in these areas. This part of the noise reduces the overall quality of the pseudo-labels and cannot be effectively filtered out.

[0006] At the same time, the initial pseudo-labels generated by the source domain model in the target domain usually contain a large amount of noise, and existing methods cannot filter these low-quality pseudo-labels on a large scale to ensure the richness of the supervision information. This part of the pseudo-labels with high confidence but insufficient actual accuracy is mixed with a large amount of noise, which is difficult to filter or correct, severely limiting the effect of subsequent pseudo-label optimization. SUMMARY

[0007] To solve the technical problem of how to effectively improve the accuracy and robustness of target domain semantic segmentation without source domain data, an embodiment of the present application provides a diffusion guide pseudo label enhanced source-free domain adaptive semantic segmentation method and device. The technical solution is as follows:

[0008] On the one hand, a diffusion guide pseudo label enhanced source-free domain adaptive semantic segmentation method is provided, which is realized by a source-free domain adaptive semantic segmentation device. The method comprises the following steps:

[0009] S1, an original target domain image is obtained, and the original target domain image is enhanced through a super-resolution model to obtain an enhanced target domain image.

[0010] S2, the original target domain image and the enhanced target domain image are respectively predicted through a source domain model to obtain an initial pseudo label of the original target domain image and an initial pseudo label of the enhanced target domain image.

[0011] S3, the initial pseudo label of the original target domain image and the initial pseudo label of the enhanced target domain image are respectively subjected to confidence sorting and confidence filtering to obtain a pseudo label set of the original target domain image and a pseudo label set of the enhanced target domain image; and a seed pseudo label is constructed according to the pseudo label set of the original target domain image and the pseudo label set of the enhanced target domain image.

[0012] S4, Gaussian noise is added to the seed pseudo label to obtain pseudo labels with different noise levels; and a context feature is extracted from the original target domain image by using an encoder.

[0013] S5, a diffusion model is trained according to the pseudo labels with different noise levels, the context feature and the seed pseudo label to obtain a trained diffusion model.

[0014] S6, random Gaussian noise and the context feature are input into the trained diffusion model to generate a pseudo label.

[0015] S7, the pseudo label is paired with the original target domain image, a source domain model is supervisedly trained according to a pairing result and a cross-entropy loss to obtain a trained model, and semantic segmentation of target domain data is completed according to the trained model.

[0016] Optionally, the confidence sorting and the confidence filtering of the initial pseudo label of the original target domain image and the initial pseudo label of the enhanced target domain image in S3 to obtain the pseudo label set of the original target domain image and the pseudo label set of the enhanced target domain image comprises the following steps:

[0017] The confidence of each initial pseudo label in the initial pseudo label of the original target domain image and the initial pseudo label of the enhanced target domain image is sorted in a percentage distribution sorting manner, and a preset percentage of pixels are selected as candidate pseudo labels for confidence filtering according to the sorting result to construct the pseudo label set of the original target domain image and the pseudo label set of the enhanced target domain image.

[0018] Optionally, the constructing of the seed pseudo label according to the pseudo label set of the original target domain image and the pseudo label set of the enhanced target domain image in S3 comprises:

[0019] The pseudo label set of the original target domain image and the pseudo label set of the enhanced target domain image are subjected to pixel-by-pixel consistency verification to obtain the seed pseudo label.

[0020] Optionally, the adding of the Gaussian noise to the seed pseudo label to obtain the pseudo label with different noise levels in S4 comprises:

[0021] A time step is randomly sampled, and Gaussian noise is added to the seed pseudo label according to the noise intensity corresponding to the time step to obtain the pseudo label with different noise levels.

[0022] On the other hand, a diffusion-guided pseudo label augmented unsupervised domain adaptation semantic segmentation device is provided, which is applied to a diffusion-guided pseudo label augmented unsupervised domain adaptation semantic segmentation method, and the device comprises:

[0023] An image enhancement module is configured to obtain an original target domain image, enhance the original target domain image through a super-resolution model, and obtain an enhanced target domain image.

[0024] An initial pseudo label construction module is configured to predict the original target domain image and the enhanced target domain image through a source domain model to obtain an initial pseudo label of the original target domain image and an initial pseudo label of the enhanced target domain image.

[0025] A seed pseudo label construction module is configured to perform confidence sorting and confidence filtering on the initial pseudo label of the original target domain image and the initial pseudo label of the enhanced target domain image to obtain a pseudo label set of the original target domain image and a pseudo label set of the enhanced target domain image, and construct a seed pseudo label according to the pseudo label set of the original target domain image and the pseudo label set of the enhanced target domain image.

[0026] A feature extraction module is configured to add Gaussian noise to the seed pseudo label to obtain a pseudo label with different noise levels, and extract context features from the original target domain image by using an encoder.

[0027] The training module is configured to train the diffusion model according to the pseudo labels with different noise levels, the context features and the seed pseudo labels, and obtain a trained diffusion model.

[0028] The generation module is configured to input the random Gaussian noise and the context features into the trained diffusion model, and generate the pseudo labels.

[0029] The output module is configured to pair the pseudo labels with the original target domain images, perform supervised training on the source domain model according to a pairing result and a cross-entropy loss, obtain a trained model, and complete semantic segmentation of the target domain data according to the trained model.

[0030] Optionally, the seed pseudo label construction module is further configured to:

[0031] The confidence of each initial pseudo label in the initial pseudo labels of the original target domain image and the initial pseudo labels of the enhanced target domain image is sorted in a percentage distribution sorting manner, and a preset percentage of pixels is selected as candidate pseudo labels according to a sorting result to realize confidence filtering, and to construct a pseudo label set of the original target domain image and a pseudo label set of the enhanced target domain image.

[0032] Optionally, the seed pseudo label construction module is further configured to:

[0033] The pseudo label set of the original target domain image and the pseudo label set of the enhanced target domain image are subjected to pixel-by-pixel consistency verification to obtain the seed pseudo label.

[0034] Optionally, the feature extraction module is further configured to:

[0035] A time step is randomly sampled, and Gaussian noise is added to the seed pseudo label according to a noise intensity corresponding to the time step to obtain the pseudo label with different noise levels.

[0036] In another aspect, a kind of passive field adaptive semantic segmentation equipment is provided, and the passive field adaptive semantic segmentation equipment includes: processor;Memory, computer readable instructions are stored on the memory, the computer readable instructions are executed by the processor, and any one method in the diffusion guided pseudo label enhanced passive domain adaptive semantic segmentation method described above is realized.

[0037] In another aspect, a kind of computer readable storage medium is provided, and at least one instruction is stored in the storage medium, the at least one instruction is loaded and executed by processor to realize any one method in the diffusion guided pseudo label enhanced passive domain adaptive semantic segmentation method described above.

[0038] The technical scheme provided by the embodiment of the application has at least the following beneficial effects:

[0039] In the present application, the accuracy and robustness of target domain semantic segmentation can be effectively improved without source domain data. By fully mining and utilizing the structure and context information in the target domain, combined with the diffusion model, high-quality pseudo-label automatic generation and efficient propagation are realized, providing efficient and reliable technical support for the field adaptive segmentation task under the limited conditions such as unavailability of source data. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0041] Figure 1 is a diffusion guided pseudo-label enhanced source-free domain adaptive semantic segmentation method flowchart provided by the embodiment of the present application;

[0042] Figure 2 is a structural schematic diagram of a diffusion guided pseudo-label enhanced source-free domain adaptive semantic segmentation method provided by the embodiment of the present application;

[0043] Figure 3 is a diffusion guided pseudo-label enhanced source-free domain adaptive semantic segmentation device block diagram provided by the embodiment of the present application;

[0044] Figure 4 is a structural schematic diagram of a source-free domain adaptive semantic segmentation device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the present application will be described below in combination with the drawings.

[0046] In the embodiments of the present application, the words such as "example", "for example" are used to represent as an example, illustration or description. Any embodiment or design scheme described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be either one of the two.

[0047] In the embodiments of the present application, "image" and "picture" can be used interchangeably, and it should be pointed out that the meanings expressed are consistent when the distinction is not emphasized. "Of", "corresponding" and "relevant" can be used interchangeably, and it should be pointed out that the meanings expressed are consistent when the distinction is not emphasized.

[0048] In the embodiments of the present application, sometimes the subscript such as W1 can be written in the form of non-subscript such as W1, and the meanings expressed are consistent when the distinction is not emphasized.

[0049] In order to make the technical problems, technical solutions and advantages of the present application clearer, specific embodiments will be described in detail below with reference to the drawings.

[0050] The embodiments of the present application provide a diffusion guided pseudo label enhanced passive domain adaptive semantic segmentation method, which can be implemented by a passive domain adaptive semantic segmentation device, which can be a terminal or a server. As shown in the flow chart of the diffusion guided pseudo label enhanced passive domain adaptive semantic segmentation method, the processing flow of the method can include the following steps: Figure 1 、 Figure 2 As shown in the flow chart of the diffusion guided pseudo label enhanced passive domain adaptive semantic segmentation method, the processing flow of the method can include the following steps:

[0051] S1, obtaining an original target domain image, enhancing the original target domain image through a super-resolution model to obtain an enhanced target domain image.

[0052] In a feasible implementation, an existing super-resolution model is used to improve the definition and detail performance of the target domain image to obtain the enhanced target domain image.

[0053] S2, predicting the original target domain image and the enhanced target domain image through a source domain model respectively to obtain initial pseudo labels of the original target domain image and initial pseudo labels of the enhanced target domain image.

[0054] In a feasible implementation, a segmentation model trained on the source domain (source domain model) is used to predict the original target domain image and the enhanced image respectively to obtain two groups of initial pseudo labels (including class and confidence).

[0055] S3, performing confidence sorting and confidence filtering on the initial pseudo labels of the original target domain image and the initial pseudo labels of the enhanced target domain image respectively to obtain a pseudo label set of the original target domain image and a pseudo label set of the enhanced target domain image; and constructing seed pseudo labels according to the pseudo label set of the original target domain image and the pseudo label set of the enhanced target domain image.

[0056] Optionally, step S3 above may include:

[0057] S31. For each type of initial pseudo-label in the initial pseudo-label of the original target domain image and the initial pseudo-label of the enhanced target domain image, sort the confidence by percentage distribution. Select a preset percentage of pixels as candidate pseudo-labels according to the sorting results to achieve confidence filtering, and construct the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image.

[0058] S32. Perform pixel-by-pixel consistency verification on the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image to obtain seed pseudo-labels.

[0059] In one feasible implementation, confidence filtering is performed for each category using a percentage distribution sorting method. For each category, the confidence scores of all pixels belonging to that category are sorted, and the top percentage of pixels are selected as candidate pseudo-labels. This process is performed separately on the original image pseudo-labels and the enhanced image pseudo-labels, resulting in two sets of high-confidence pseudo-labels.

[0060] Furthermore, a pixel-by-pixel consistency check is performed on the two filtered pseudo-label sets. Only those pixels that exist in both sets of pseudo-labels and whose category predictions are consistent are retained as the final seed pseudo-labels.

[0061] S4. Add Gaussian noise to the seed pseudo-labels to obtain pseudo-labels with different noise levels; use the encoder to extract contextual features from the original target domain image.

[0062] Optionally, Gaussian noise is added to the seed pseudo-tags in S4 to obtain pseudo-tags with different noise levels, including:

[0063] A time step is randomly sampled, and Gaussian noise is added to the seed pseudo-label based on the noise intensity corresponding to the time step, resulting in pseudo-labels with different noise levels.

[0064] In one feasible implementation, for each training sample in the target domain, a time step t is first randomly sampled within a diffusion modeling framework. This time step corresponds to a specific noise intensity. Based on this noise level, Gaussian noise is added to the original seed pseudo-label to obtain a noisy pseudo-label.

[0065] Meanwhile, the encoder extracts contextual features from the original target domain image. These features are provided as conditional inputs to the diffusion model, guiding the model to learn how to recover relevant pseudo-labels based on the image context.

[0066] S5. Train the diffusion model based on pseudo-labels with different noise levels, context features, and seed pseudo-labels to obtain the trained diffusion model.

[0067] In one feasible implementation, during training, the diffusion model uses pseudo-labels with different noise levels and contextual features of the target domain image as joint inputs. Under the supervision of seed pseudo-labels, it learns how to recover accurate segmentation predictions from different noise levels.

[0068] S6. Input random Gaussian noise and contextual features into the trained diffusion model to generate pseudo-labels.

[0069] In one feasible implementation, after training, the diffusion model enters the inference phase. At this point, it no longer relies on prior label input, but uses random Gaussian noise as the initial state of the pseudo-labels. Through the interaction with the contextual features of the target domain image, and after multiple iterative denoising steps, it extracts and gradually forms segmentation regions that conform to the true semantics from the pure noise, ultimately generating a complete pseudo-label segmentation map.

[0070] S7. Pair the pseudo-labels with the original target domain image, and perform supervised training on the source domain model based on the pairing results and cross-entropy loss to obtain the trained model. Then, use the trained model to complete the semantic segmentation of the target domain data.

[0071] In one feasible implementation, model optimization involves obtaining high-quality pseudo-labels generated by the diffusion model. These labels are then considered pseudo-labels within the target domain and used in subsequent model optimization stages. In this stage, the pseudo-labels are paired with unlabeled images in the target domain, and the segmentation model undergoes end-to-end supervised training using cross-entropy loss. In each iteration, the model predicts segmentation of the target image based on the current parameters and compares the prediction results with the pseudo-labels at the pixel level, calculating the cross-entropy loss. This guides the model to continuously adjust its parameters, improving its adaptability to the distribution characteristics of the target domain.

[0072] For example, remote sensing image data, medical image data, etc., to be semantically segmented can be acquired. The acquired data can be augmented based on the trained model, initial pseudo-labels can be generated and seed pseudo-labels can be constructed. Then, a diffusion model can be used to generate complete high-quality pseudo-labels to supervise the training of the model in the target domain, thereby improving the semantic segmentation effect of the model in the target domain.

[0073] The experimental results show that, compared with the previous best results, the present invention improves the semantic segmentation index miou by 2.5% on the transfer task GTA5→Cityscapes, by 2.3% on the transfer task SYNTHIA→Cityscapes, by 2.45% on the transfer task Vaihingen→Potsdam, and by 2.18% on the transfer task Rural→Urban.

[0074] In this embodiment of the invention, the accuracy and robustness of target domain semantic segmentation can be effectively improved without the need for source domain data. By fully mining and utilizing the structural and contextual information in the target domain, combined with a diffusion model, high-quality pseudo-labels are automatically generated and efficiently propagated, providing efficient and reliable technical support for domain adaptive segmentation tasks under constraints such as unavailable source data.

[0075] Figure 3 This is a block diagram of a diffusion-guided pseudo-label-enhanced passive domain adaptive semantic segmentation apparatus according to an exemplary embodiment. The apparatus is used in a diffusion-guided pseudo-label-enhanced passive domain adaptive semantic segmentation method. (Refer to...) Figure 3 The device includes an image enhancement module 310, an initial pseudo-label construction module 320, a seed pseudo-label construction module 330, a feature extraction module 340, a training module 350, a generation module 360, and an output module 370. Wherein:

[0076] Image enhancement module 310 is used to acquire the original target domain image, enhance the original target domain image through a super-resolution model, and obtain the enhanced target domain image.

[0077] The initial pseudo-label construction module 320 is used to predict the original target domain image and the enhanced target domain image respectively through the source domain model to obtain the initial pseudo-label of the original target domain image and the initial pseudo-label of the enhanced target domain image.

[0078] The seed pseudo-label construction module 330 is used to sort and filter the initial pseudo-labels of the original target domain image and the initial pseudo-labels of the enhanced target domain image with confidence scores, respectively, to obtain the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image; and to construct seed pseudo-labels based on the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image.

[0079] The feature extraction module 340 is used to add Gaussian noise to the seed pseudo-labels to obtain pseudo-labels with different noise levels; and to extract contextual features from the original target domain image using the encoder.

[0080] Training module 350 is used to train the diffusion model based on pseudo-labels with different noise levels, contextual features, and seed pseudo-labels to obtain the trained diffusion model.

[0081] The generation module 360 ​​is used to input random Gaussian noise and contextual features into the trained diffusion model to generate pseudo-labels.

[0082] Output module 370 is used to pair pseudo-labels with the original target domain image, perform supervised training on the source domain model based on the pairing results and cross-entropy loss, obtain a trained model, and complete semantic segmentation of the target domain data based on the trained model.

[0083] In this embodiment of the invention, the accuracy and robustness of target domain semantic segmentation can be effectively improved without the need for source domain data. By fully mining and utilizing the structural and contextual information in the target domain, combined with a diffusion model, high-quality pseudo-labels are automatically generated and efficiently propagated, providing efficient and reliable technical support for domain adaptive segmentation tasks under constraints such as unavailable source data.

[0084] Figure 4 This is a schematic diagram of the structure of a passive domain adaptive semantic segmentation device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, a passive domain adaptive semantic segmentation device may include the above-mentioned Figure 3 The illustrated passive domain adaptive semantic segmentation apparatus features diffusion-guided pseudo-label enhancement. Optionally, the passive domain adaptive semantic segmentation apparatus 410 may include a first processor 2001.

[0085] Optionally, the passive domain-adaptive semantic segmentation device 410 may also include a memory 2002 and a transceiver 2003.

[0086] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0087] The following is combined Figure 4 The components of the passive domain adaptive semantic segmentation device 410 are described in detail below:

[0088] The first processor 2001 is the control center of the passive domain adaptive semantic segmentation device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0089] Optionally, the first processor 2001 can perform various functions of the passive domain adaptive semantic segmentation device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0090] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0091] In a specific implementation, as one example, the passive domain-adaptive semantic segmentation device 410 may also include multiple processors, for example... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0092] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0093] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be accessed through the interface circuit of the passive domain adaptive semantic segmentation device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0094] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0095] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0096] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected to the interface circuit of the passive domain adaptive semantic segmentation device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0097] It should be noted that, ​ The structure of the passive domain adaptive semantic segmentation device 410 shown in the figure does not constitute a limitation on the router. Actual knowledge structure recognition devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0098] Furthermore, the technical effect of the passive domain adaptive semantic segmentation device 410 can be referred to the technical effect of the diffusion-guided pseudo-label-enhanced passive domain adaptive semantic segmentation method described in the above method embodiments, and will not be repeated here.

[0099] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0100] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0101] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0102] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0103] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0104] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0105] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0106] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0107] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0108] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0109] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0110] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A diffusion-guided pseudo-label-enhanced passive domain adaptive semantic segmentation method, characterized in that, The method includes: S1. Obtain the original target domain image, and enhance the original target domain image through a super-resolution model to obtain the enhanced target domain image; S2. The source domain model is used to predict the original target domain image and the enhanced target domain image respectively, so as to obtain the initial pseudo-labels of the original target domain image and the initial pseudo-labels of the enhanced target domain image. S3. Rank and filter the initial pseudo-labels of the original target domain image and the enhanced target domain image according to their confidence scores to obtain the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image; construct seed pseudo-labels based on the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image. S4. Add Gaussian noise to the seed pseudo-labels to obtain pseudo-labels with different noise levels; use the encoder to extract contextual features from the original target domain image. S5. Train the diffusion model based on pseudo-labels with different noise levels, context features, and seed pseudo-labels to obtain the trained diffusion model. S6. Input random Gaussian noise and contextual features into the trained diffusion model to generate pseudo-labels; S7. Pair the pseudo-labels with the original target domain image, and perform supervised training on the source domain model based on the pairing results and cross-entropy loss to obtain the trained model. Then, use the trained model to complete the semantic segmentation of the target domain data.

2. The passive domain adaptive semantic segmentation method with diffusion-guided pseudo-label enhancement according to claim 1, characterized in that, In step S3, confidence ranking and confidence filtering are performed on the initial pseudo-labels of the original target domain image and the initial pseudo-labels of the enhanced target domain image, respectively, to obtain the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image, including: For each type of initial pseudo-label in the original target domain image and the enhanced target domain image, the confidence level is sorted by percentage distribution. Based on the sorting results, a preset percentage of pixels are selected as candidate pseudo-labels to achieve confidence filtering, thus constructing the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image.

3. The passive domain adaptive semantic segmentation method with diffusion-guided pseudo-label enhancement according to claim 1, characterized in that, The step S3, which involves constructing seed pseudo-labels based on the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image, includes: Pixel-by-pixel consistency verification is performed on the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image to obtain seed pseudo-labels.

4. The passive domain adaptive semantic segmentation method with diffusion-guided pseudo-label enhancement according to claim 1, characterized in that, In step S4, Gaussian noise is added to the seed pseudo-tags to obtain pseudo-tags with different noise levels, including: A time step is randomly sampled, and Gaussian noise is added to the seed pseudo-label based on the noise intensity corresponding to the time step, resulting in pseudo-labels with different noise levels.

5. A diffusion-guided pseudo-label-enhanced passive domain adaptive semantic segmentation apparatus, wherein the diffusion-guided pseudo-label-enhanced passive domain adaptive semantic segmentation apparatus is used to implement the diffusion-guided pseudo-label-enhanced passive domain adaptive semantic segmentation method as described in any one of claims 1-4, characterized in that, The device includes: The image enhancement module is used to acquire the original target domain image, and enhance the original target domain image through a super-resolution model to obtain the enhanced target domain image. The initial pseudo-label construction module is used to predict the original target domain image and the enhanced target domain image respectively through the source domain model, so as to obtain the initial pseudo-label of the original target domain image and the initial pseudo-label of the enhanced target domain image. The seed pseudo-label construction module is used to sort and filter the initial pseudo-labels of the original target domain image and the initial pseudo-labels of the enhanced target domain image with confidence scores, respectively, to obtain the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image; and to construct seed pseudo-labels based on the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image. The feature extraction module adds Gaussian noise to the seed pseudo-labels to obtain pseudo-labels with different noise levels; the encoder extracts contextual features from the original target domain image. The training module is used to train the diffusion model based on pseudo-labels with different noise levels, contextual features, and seed pseudo-labels to obtain a trained diffusion model. The generation module is used to input random Gaussian noise and contextual features into the trained diffusion model to generate pseudo-labels; The output module is used to pair pseudo-labels with the original target domain image, perform supervised training on the source domain model based on the pairing results and cross-entropy loss, obtain the trained model, and complete the semantic segmentation of the target domain data based on the trained model.

6. The passive domain adaptive semantic segmentation device with diffusion-guided pseudo-label enhancement according to claim 5, characterized in that, The step of performing confidence ranking and confidence filtering on the initial pseudo-labels of the original target domain image and the initial pseudo-labels of the enhanced target domain image, respectively, to obtain the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image, includes: For each type of initial pseudo-label in the original target domain image and the enhanced target domain image, the confidence level is sorted by percentage distribution. Based on the sorting results, a preset percentage of pixels are selected as candidate pseudo-labels to achieve confidence filtering, thus constructing the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image.

7. The passive domain adaptive semantic segmentation device with diffusion-guided pseudo-label enhancement according to claim 5, characterized in that, The step of constructing seed pseudo-labels based on the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image includes: Pixel-by-pixel consistency verification is performed on the pseudo-label set of the original target domain image and the pseudo-label set of the enhanced target domain image to obtain seed pseudo-labels.

8. The passive domain adaptive semantic segmentation device with diffusion-guided pseudo-label enhancement according to claim 5, characterized in that, The process of adding Gaussian noise to seed pseudo-tags to obtain pseudo-tags with different noise levels includes: A time step is randomly sampled, and Gaussian noise is added to the seed pseudo-label based on the noise intensity corresponding to the time step, resulting in pseudo-labels with different noise levels.

9. A passive domain-adaptive semantic segmentation device, characterized in that, The passive domain adaptive semantic segmentation device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 4.

Images