A robot cross-domain visual scene recognition method and system

Abstract

The application discloses a kind of robot cross-domain visual scene identification method and system, method includes: obtaining current domain sample and the feature thereof is frequency domain decomposition, separate low-frequency component and high-frequency component;Two kinds of components are modeled and fused respectively using mapping function, obtain frequency domain regulation feature, mapping is pseudo sample data, with current domain sample common input scene identification model training, constrain the discrimination ability of model to historical domain and current domain scene in training;The model parameters before and after training are gradually fused, and a cross-domain visual scene recognition model is obtained for the identification of the current observation scene. By frequency domain decomposition and regulation, the structural consistency and texture expression of cross-domain style transfer are enhanced, combined with pseudo sample generation and distribution replay training strategy, without relying on historical real data, effectively alleviating catastrophic forgetting, significantly improving the long-term cross-domain scene recognition robustness of robot under the drastic appearance change of season, illumination and the like.

Description

Technical Field

[0001] This invention relates to the field of robot visual navigation technology, and in particular to a method and system for robot cross-domain visual scene recognition. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is a core technology for autonomous robot navigation, enabling robots to simultaneously estimate their pose and build an environmental map in unknown environments. Visual SLAM, using cameras as its primary sensor, offers advantages such as low cost, small size, and ease of deployment, and has become an important technological approach in robot navigation. In a visual SLAM system, the loop closure detection module is responsible for identifying whether the robot has revisited previously visited scenes, and introducing global constraints to eliminate accumulated pose drift when a loop is detected. As the core support for loop closure detection, scene recognition technology needs to accurately determine whether the currently observed scene and the historical scene form a loop relationship during long-term robot operation.

[0003] Unlike static visual tasks such as image classification, robot scene recognition processes data not from isolated images randomly collected from the network, but from image sequences captured by onboard cameras as the robot moves continuously in a real environment. These image sequences exhibit significant temporal consistency and spatial continuity, with adjacent frames highly correlated in geometric structure and scene layout. When a robot performs long-term, cross-time-period autonomous operations in the same area, almost all appearance changes stem from environmental changes experienced by the robot itself, such as vegetation changes due to seasonal transitions, lighting differences due to day-night cycles, and decreased visibility caused by fluctuating weather conditions. These environmental factors triggered by the robot's cross-domain movement cause a significant shift in the scene appearance distribution during system operation, resulting in inconsistent data distribution between the training and deployment phases of the model.

[0004] In the development of scene recognition methods, early work mainly relied on manually designed local feature descriptors, such as SIFT, SURF, and ORB, combined with a bag-of-words visual model to quantize local features into a global image representation. These methods are simple in structure and easy to implement, but due to the limited representational power of hand-crafted features, they are sensitive to dynamic lighting and seasonal changes common in real-world robot operations, making it difficult to maintain stable performance. Subsequently, deep learning-based methods significantly improved discriminative capabilities under complex conditions by automatically learning high-level semantic features from data. However, most of these methods are based on the assumption of offline training and stable data distribution, failing to fully consider the reality of continuous, streaming, online arrival of data during robot cross-domain operations. If newly acquired data from the current domain is directly used to fine-tune the deployed model, the model parameters will rapidly shift towards the new domain distribution, disrupting the original feature space structure and leading to a significant decline in the ability to recognize scenes from the old domain—a catastrophic forgetting.

[0005] To address this issue, continuous learning methods have been introduced to enable models to retain old knowledge while continuously learning multiple tasks. Based on different optimization strategies, existing continuous learning methods can generally be divided into three categories: model architecture-based methods achieve explicit isolation between different tasks by introducing task-specific parameters, but the model size expands continuously with the number of tasks, making them unsuitable for robot embedded platforms with limited storage and computing power; regularization-based methods limit the over-updating of key weights by imposing parameter importance constraints on the loss function, but struggle to achieve a balance between plasticity and stability in the face of drastic seasonal and weather changes encountered by robots; and replay-based methods maintain feature space stability by reviewing data from past tasks. However, real sample caching requires storing a large number of historical images, placing a significant burden on the robot's local storage, while generative replay relies on the quality of the generative model's mapping from the new domain to the old domain; if the generated samples are distorted, additional noise interference will be introduced.

[0006] In recent years, a few studies have applied continuous learning to scene recognition tasks, but existing methods mostly rely on simple regularization strategies to alleviate the forgetting problem. In the face of significant appearance changes encountered by robots during long-term cross-domain operation, the cross-domain generalization ability is still insufficient, making it difficult to maintain stable and reliable recognition performance in actual deployment. Summary of the Invention

[0007] Therefore, the technical problem to be solved by the present invention is to overcome the problem that existing robot vision scene recognition methods suffer from catastrophic forgetting and insufficient cross-domain generalization ability due to significant shifts in environmental appearance distribution in cross-domain operation scenarios.

[0008] To address the aforementioned technical problems, this invention provides a robot cross-domain visual scene recognition method, comprising: S1: Obtain the current domain sample, perform frequency domain decomposition on the features of the current domain sample, and separate the low-frequency component and the high-frequency component. S2: The low-frequency component and the high-frequency component are modeled using mapping functions respectively, and the modeled low-frequency features are fused with the high-frequency features to obtain frequency domain modulation features; S3: Map the frequency domain modulation features to pseudo-sample data; input the pseudo-sample data and the current domain samples into the scene recognition model for training, and constrain the scene recognition model's ability to distinguish between historical domain scenes and current domain scenes during the training process; S4: Progressively fuse the parameters of the trained scene recognition model with the parameters of the pre-trained scene recognition model to obtain a cross-domain visual scene recognition model; use the cross-domain visual scene recognition model to perform cross-domain visual scene recognition on the robot's current observation scene.

[0009] In one embodiment of the present invention, the method for performing frequency domain decomposition on the features of the current domain sample in step S1 to separate low-frequency and high-frequency components is as follows: using a parameterized Gaussian low-pass operator on the input features Perform channel-wise depthwise convolution to extract the low-frequency components. The formula is: , in, For standard deviation Parameterized two-dimensional Gaussian kernel, This represents channel-wise convolution; Calculate the input features With the low-frequency component The difference is used to obtain the high-frequency residual, and the high-frequency residual is amplitude modulated to generate the high-frequency component. The formula is: , in, This is the high-frequency enhancement factor.

[0010] In one embodiment of the present invention, the method for fusing the modeled low-frequency features with high-frequency features to obtain frequency domain modulation features in step S2 is as follows: The modeled low-frequency features... and high-frequency features after modeling The data is concatenated along the channel dimension, and the fused features are obtained through convolution operations. ; The fusion features Features of the current domain samples are incorporated in the form of residuals. Through learnable gating parameters By controlling the degree of its influence, the frequency domain modulation characteristics are obtained. The formula is: .

[0011] In one embodiment of the present invention, in step S3, the method of mapping the frequency domain modulation feature to pseudo sample data is as follows: the frequency domain modulation feature is decoded, and the intermediate features of the decoding process are enhanced at high frequencies, and the frequency domain modulation feature is mapped to the pseudo sample data through the decoding process.

[0012] In one embodiment of the present invention, the method for performing high-frequency enhancement on intermediate decoding features during the decoding process is as follows: [The method involves] enhancing the intermediate decoding features... High-frequency components are extracted and high-frequency enhancement mapping is applied to obtain enhanced high-frequency features. ; For the decoding intermediate features Adaptive weighted modulation is performed to output channel attention weights. ; The enhanced high-frequency features With the channel attention weights Perform channel-by-channel multiplication and then combine with the decoded intermediate features. By summing the residuals, we obtain the enhanced output features. , is represented as: , Here, ⊙ represents channel-by-channel multiplication.

[0013] In one embodiment of the present invention, step S3, the method for constraining the scene recognition model's ability to distinguish between historical and current domain scenes includes using triplet loss for constraint, specifically: A fixed-capacity data buffer is constructed, which stores cached historical images and corresponding loop closure matrix. The loop closure matrix is ​​used to record whether a loop closure relationship exists between samples. Using the currently queried image as the anchor sample, images that have a loop relationship with the anchor sample are selected from the data buffer as positive samples, and images that do not have a loop relationship are selected as negative samples to construct triples; The anchor sample, the positive sample, and the negative sample are respectively mapped into global descriptors through a feature extractor and an aggregation module; The cosine similarity function is used to measure the similarity between the global descriptors. A triplet loss function is constructed, which constrains the similarity between positive samples and anchor samples to be higher than that between negative samples and anchor samples.

[0014] In one embodiment of the present invention, step S3, the method for constraining the scene recognition model's ability to distinguish between historical and current domain scenes, further includes using similarity knowledge distillation loss for constraint, specifically as follows: During the training process of the scene recognition model, the current batch data is obtained from the pseudo sample data and the current domain sample; Using the old scene recognition model before the update and the new scene recognition model to be updated, the global descriptor of the current batch of data is extracted respectively; The similarity between samples is calculated using cosine similarity, and a first similarity relation matrix corresponding to the old scene recognition model and a second similarity relation matrix corresponding to the new scene recognition model are constructed respectively. After normalizing the first similarity relation matrix and the second similarity relation matrix, the similarity knowledge distillation loss is constructed using KL divergence; Using the similarity knowledge distillation loss and the triplet loss function, a total loss function for the new scene recognition model is constructed, and the parameters of the new scene recognition model are optimized and updated based on the total loss function.

[0015] In one embodiment of the present invention, the total loss function Represented as: , in, The loss corresponding to the current domain sample, The loss corresponding to the pseudo-sample data, These are the preset weight parameters.

[0016] In one embodiment of the present invention, in step S4, the method for progressively fusing the trained scene recognition model parameters with the pre-trained scene recognition model parameters to obtain a cross-domain visual scene recognition model is as follows: weighted fusion is performed using an exponential moving average strategy to obtain fusion coefficients. By controlling the weight ratio of the scene recognition model parameters before training and the scene recognition model parameters after training in the parameter space using the fusion coefficient, a cross-domain visual scene recognition model is obtained.

[0017] Based on a unified inventive concept, this invention also provides a robot cross-domain visual scene recognition system, comprising: The frequency domain decomposition module is used to acquire the current domain sample, perform frequency domain decomposition on the features of the current domain sample, and separate the low-frequency components and high-frequency components. The frequency domain fusion module is used to model the low-frequency component and the high-frequency component using mapping functions respectively, and to fuse the modeled low-frequency features with the high-frequency features to obtain frequency domain modulation features; The generation and training module is used to map the frequency domain modulation features into pseudo sample data, and input the pseudo sample data and the current domain samples into the scene recognition model for training. During the training process, the scene recognition model is constrained to distinguish between historical domain scenes and current domain scenes. The fusion recognition module is used to progressively fuse the parameters of the trained scene recognition model with the parameters of the pre-trained scene recognition model to obtain a cross-domain visual scene recognition model; and to use the cross-domain visual scene recognition model to perform cross-domain visual scene recognition on the robot's current observation scene.

[0018] The technical solution of the present invention has the following advantages over the prior art: This invention constructs a frequency-domain-aware style transfer network to generate pseudo-samples with appearance features from the historical domain using only current domain samples, without caching real historical domain data, thus achieving effective replay of historical appearance distributions. The method introduces an adaptive Gaussian frequency-domain separation module in the generator decoding stage, decomposing features into low-frequency appearance and high-frequency texture components and modeling them separately. Combined with an attention-guided high-frequency enhancement module and multi-band consistency loss, it significantly improves structural consistency and texture representation quality during cross-domain transfer. Based on this, a generative distribution replay-driven training strategy is adopted, combining triplet loss and similarity knowledge distillation loss, and balancing plasticity and stability through progressive parameter fusion. This method significantly outperforms existing methods in terms of anti-forgetting ability and overall recognition performance, effectively mitigating catastrophic forgetting and improving the robustness of robots in long-term cross-domain autonomous navigation. Attached Figure Description

[0019] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0020] Figure 1 This is a flowchart illustrating the robot cross-domain visual scene recognition method provided in this embodiment of the invention. Figure 2 This is a schematic diagram of the network structure of the adaptive Gaussian frequency domain separation module in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the multi-scale discriminator in an embodiment of the present invention; Figure 4 This is a schematic diagram of the network structure of the attention-guided high-frequency enhancement module in an embodiment of the present invention; Figure 5 This is a schematic diagram of the training process driven by generative distributed replay in an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the quantitative comparison results of the method of this invention and the comparative method on the Robotcar dataset for a cross-seasonal style transfer task. Figure 7 This is a schematic diagram illustrating the quantitative comparison results of the method of this invention and the comparative method on three cross-seasonal style transfer tasks on the Nordland dataset. Figure 8 This is a visual comparison diagram of the style transfer results of the method of this invention and the comparison method on two tasks with significant cross-domain differences in the experiment. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0022] Example 1: like Figure 1 As shown, the present invention provides a robot cross-domain visual scene recognition method, including: S1: Obtain the current domain sample, perform frequency domain decomposition on the features of the current domain sample, and separate the low-frequency component and the high-frequency component. S2: The low-frequency component and the high-frequency component are modeled using mapping functions respectively, and the modeled low-frequency features are fused with the high-frequency features to obtain frequency domain modulation features; S3: Map the frequency domain modulation features to pseudo-sample data; input the pseudo-sample data and the current domain samples into the scene recognition model for training, and constrain the scene recognition model's ability to distinguish between historical domain scenes and current domain scenes during the training process; S4: Progressively fuse the parameters of the trained scene recognition model with the parameters of the pre-trained scene recognition model to obtain a cross-domain visual scene recognition model; use the cross-domain visual scene recognition model to perform cross-domain visual scene recognition on the robot's current observation scene.

[0023] This invention provides a method for cross-domain visual scene recognition in robots. It acquires samples from the current domain and performs frequency domain decomposition on their features, separating low-frequency and high-frequency components. Then, it models and fuses these two types of components using mapping functions to obtain frequency-domain modulated features. These features are then mapped to pseudo-sample data, which, along with samples from the current domain, is input into a scene recognition model for training. During training, the model's ability to distinguish between historical and current domain scenes is constrained. Finally, the model parameters before and after training are progressively fused to obtain a cross-domain visual scene recognition model for the robot to recognize the currently observed scene. This method enhances the structural consistency and texture representation of cross-domain style transfer through frequency domain decomposition and modulation. Combined with pseudo-sample generation and distributed replay training strategies, it effectively mitigates catastrophic forgetting without relying on historical real data, significantly improving the robot's robustness and generalization ability in long-term cross-domain scene recognition under drastic changes in appearance such as seasons and lighting.

[0024] As a preferred embodiment of the present invention, the robot cross-domain visual scene recognition method will be described in detail below.

[0025] A Frequency-Aware Style Transfer Network (FAST-Net) is constructed to learn appearance distribution mappings between different environment domains without relying on paired data. FAST-Net employs a bidirectional generative adversarial framework, comprising two generators (one that maps an image from domain A to a style image from domain B). And a generator that maps domain B images to domain A style images. The generator consists of an encoder and two discriminators. Each generator contains an encoder and a decoder. The encoder consists of nine residual blocks, and the decoder integrates an Adaptive Gaussian Frequency Separation Module (AGFS) and an Attention-Guided High-Frequency Enhancement Module (AG-HFE).

[0026] In step S1, the scene image currently captured by the robot is acquired as the current domain sample. Frequency domain decomposition is performed on the features of the current domain sample to separate low-frequency and high-frequency components. The AGFS module can explicitly decompose features into low-frequency appearance style components and high-frequency texture detail components, and model and fuse them separately, thereby reducing the interference of multiple targets and drastic background changes on structural consistency in complex scenes. Figure 2 As shown, the AGFS module contains a low-frequency branch, a high-frequency branch, and a feature fusion unit. Its specific working process is as follows.

[0027] A parameterized Gaussian low-pass operator is used to extract low-frequency features from the input. The depth-separable Gaussian smoothing operator is defined as follows: , in, For standard deviation Parameterized two-dimensional Gaussian kernel, This represents channel-wise depthwise convolution. Using channel-wise convolution can avoid the disruption of frequency statistics caused by cross-channel aliasing. This is to ensure Gaussian standard deviation. Differentiable and satisfying Learnable parameters It is processed by the softplus activation function and a tiny constant is added. ,Right now The bandwidth of the Gaussian kernel can be adaptively adjusted during network training.

[0028] Furthermore, regarding input features Adopted by The low-frequency components are obtained by performing channel-wise depthwise convolution with a parameterized 2D Gaussian kernel. .

[0029] Further, the input features are calculated. With the low-frequency component The difference is used to obtain the high-frequency residual, and the high-frequency residual is amplitude modulated to generate the high-frequency component: ,in, This is the high-frequency enhancement coefficient, used to control the amplitude of high-frequency injection. It is adjusted... The value can flexibly balance the preservation of high-frequency details and the effect of noise suppression.

[0030] After obtaining the frequency domain components, different mapping functions are used to model the low-frequency and high-frequency components respectively, which can be expressed as: , Among them, the low-frequency branch Primarily used for modeling the overall structure and semantic layout of a scene, it is insensitive to noise but requires high stability. This branch employs a more conservative network structure, consisting of... A shallow transform implementation consisting of convolutional layers, normalization layers, and ReLU activation functions. High-frequency branch. This requires a stronger non-linear feature to depict the more sensitive texture, edge and detail information in cross-domain changes.

[0031] To avoid amplifying noise and inducing artifacts by forcibly enhancing details, this branch introduces a flexible nonlinearity and channel recalibration mechanism. Specifically, Employing an asymmetric structure incorporating LeakyReLU, and using Convolution adaptively reorganizes the channel dimensions. Compared to low-frequency branches, this structure, while maintaining expressive power, can improve the suppression of high-frequency noise and anomalous responses, thereby achieving controllable texture transfer.

[0032] Furthermore, in step S2, the modeled low-frequency features are... and high frequency characteristics Input feature fusion unit to obtain output features .

[0033] Specifically, and The splicing is performed along the channel dimension, through a... Convolution yields fused features The fusion feature is incorporated into the original input feature as a residual. In, and through a learnable gating parameter Controlling the extent of its impact, that is: , in, It is a learnable scalar (or channel-wise scaling vector) initialized to 0, ensuring that the perturbation of the backbone features by the module is almost zero in the early stages of training, thus guaranteeing training stability; as training progresses, the model can be gradually increased. The absolute value of the residual plus gating design is used to enhance the frequency domain modulation capability. This design achieves a good balance between structural stability and texture representation capability, while effectively reducing the risk of artifacts caused by overmodulation; resulting in frequency domain modulation features that fuse low-frequency structural information and high-frequency detail information. This feature preserves the semantic layout of the original scene while enhancing discriminative details that are sensitive to cross-domain changes.

[0034] After completing the feature decomposition and fusion of the AGFS module, an attention-guided high-frequency enhancement module (AG-HFE) was designed at the decoding end of the FAST-Net generator to prevent the attenuation of high-frequency textures and edge details during the decoding process.

[0035] The discriminator in FAST-Net is used to distinguish between real and generated images to assist in the adversarial training of the generator. For example... Figure 3As shown, to improve discriminative ability and training stability, the discriminator employs a multi-scale network structure and image pooling mechanism. The single-scale discriminator judges the authenticity of generated samples at the local patch level. To further enhance the discriminator's constraint capability at different scales, a multi-scale PatchGAN network structure is used. The input image is fed into a discriminator with the same structure but independent parameters at both the original resolution and two downsampling scales, resulting in global and local output matrices. This simultaneously covers fine-grained texture consistency and a wider range of appearance distribution consistency. Simultaneously, the discriminator uses an image pooling mechanism during training to constrain the generated results, aiming to improve the stability of adversarial training while maintaining discriminative ability.

[0036] Specifically, when updating, the discriminator does not directly use the latest sample output by the generator. Instead, it randomly samples generated samples from a fixed-capacity image pool for training. This image pool buffers the samples output by the generator over a period of time, allowing the discriminator to be exposed to more diverse and smoothly changing generated samples during training. This avoids the discriminator from over-adapting to the instantaneous output of the current generator and helps to alleviate instability in adversarial training.

[0037] Furthermore, the generator of FAST-Net is trained and optimized. The total loss of the generator consists of four parts: adversarial loss, cycle consistency loss, identity preservation loss, and multi-band consistency loss. At the same time, an exponential moving average (EMA) mechanism is introduced during training to suppress parameter mutations and improve training stability.

[0038] Adversarial loss, acting as an adversarial constraint between the generator and discriminator, can alleviate the gradient vanishing problem of the generator when the discriminator is too strong, making the generated result approximate the target domain samples at the distribution level. This is exemplified by a generator that maps an image from domain A to a style image from domain B. For example, its adversarial loss is defined as: , in, This represents an image from domain A. The discriminator is used to distinguish between real B-domain samples and generated B-domain pseudo-samples. Similarly, the reverse generator... The losses from the confrontation were The total loss from the confrontation is the sum of the two: .

[0039] Cyclic consistency loss is used to constrain structural and semantic content preservation, suppressing structural drift and geometric distortion. This loss ensures that bidirectional mappings can be reconstructed back to the original domain, avoiding content drift caused by cross-domain mappings only satisfying the target domain distribution. Its definition is: , in This represents an image from domain B.

[0040] Identity loss is used to reduce unnecessary appearance modifications imposed by the generator on inputs already in the target domain distribution, which helps stabilize low-frequency appearance statistics such as color and improves the fidelity of mapping and training stability. .

[0041] Multi-band consistency loss constructs a Laplacian pyramid decomposer to perform frequency band decomposition on the image representation, constraining the consistency of frequency components between the generated result and the target domain image at multiple frequency band levels. This allows the loss function to provide more refined supervision of the cross-domain generation process at the frequency domain level. For example, let the generator output be... The target domain real image is A recursive approach using Gaussian smoothing and residual downsampling is employed to construct the Laplace pyramid. Let the number of decomposition levels be . Then the image It can be decomposed into: , in, Indicates the first Laplacian bandpass components (texture and edge residual information) in each frequency band. The lowest frequency residual component (structure and brightness distribution) is the topmost layer.

[0042] Based on this, the low-frequency consistency loss is defined as: , This constraint is used to assess the stability of the generated results in terms of global structure and low-frequency appearance statistics. The high-frequency consistency loss imposes constraints on all Laplacian bandpass layers and is weighted using geometric decay weights. , in The attenuation coefficient makes the constraints corresponding to the high-frequency bandwidth relatively stronger, while the constraint strength of the low-frequency bandpass component is gradually weakened to avoid excessive restriction on the generator's expressive ability.

[0043] The multi-band consistency loss is: ; Similarly, we can obtain The two together constitute .

[0044] In summary, the total generator loss function of the style transfer network FAST-Net is: , in , and These are the preset hyperparameters.

[0045] Furthermore, to avoid potential transient mode collapse, an exponential moving average (EMA) mechanism is introduced during training to smoothly model the generator parameters. Let the... The generator's parameters at the next iteration are: Then its corresponding EMA parameter Update as follows: , Where the attenuation coefficient This mechanism controls the relative contribution of historical and current parameters to the weighted average. It makes the evolution of generator parameters more continuous and robust, effectively reducing parameter noise introduced by transient anomalous gradients or adversarial training instability. After training FAST-Net is complete, its generator is frozen.

[0046] Furthermore, in step S3, the frequency domain modulation features obtained in the above steps are mapped to pseudo-sample data using the trained generator.

[0047] Specifically, the generator's decoding process progressively upsamples and convolves the frequency domain modulation features to recover the samples in the image space. However, multiple upsampling and normalization operations can easily lead to the attenuation of high-frequency textures and edge details. Therefore, this invention integrates an AG-HFE module at the end of the decoder. Figure 4 As shown, the AG-HFE module handles any intermediate feature map during the decoding process. The high-frequency components are extracted using the same Gaussian filter as in the adaptive Gaussian frequency domain separation module. Specifically, after obtaining the low-frequency component through Gaussian low-pass filtering, the high-frequency residual is obtained by subtracting the low-frequency component from the original feature. ;right Apply a two-layer High-frequency enhancement mapping composed of convolution and ReLU activation functions To restore and enhance the high-frequency response, thus obtaining the enhanced high-frequency characteristics. .

[0048] Meanwhile, the channel attention branch in the module Perform adaptive weighted modulation: Sequentially through global average pooling, two Convolutional layers (with ReLU interleaved) and a Sigmoid activation function, output channel attention weights. .

[0049] The enhanced high-frequency features are multiplied channel by channel attention weights, and then multiplied by the original features. Adding the residuals yields the enhanced output features: , in, This indicates channel-by-channel multiplication.

[0050] Through this controlled high-frequency compensation, discriminative texture and edge information is moderately enhanced during decoding, while high-frequency noise in smooth regions is suppressed by attention weights. This significantly improves the detail quality of the generated pseudo-samples while maintaining overall structural stability. After complete decoding, the frequency domain modulation features are finally mapped to a pseudo-sample image of the same size as the input image. Using current domain samples (e.g., a sunny summer scene) as input, corresponding pseudo-sample data with style transferred to the historical domain (e.g., a winter snow scene) can be generated.

[0051] After obtaining the pseudo-sample data, it is input together with the real samples from the current domain into the scene recognition model for training. The scene recognition model uses VGG-19 as the feature extractor and introduces generalized mean pooling (GeM) as an aggregation module to map each image into a compact global descriptor. Figure 5 As shown, to enable the model to maintain its discriminative ability in historical domain scenes while learning new features in the current domain, a generative distribution replay-driven cross-domain place recognition training (GDR-CPRT) strategy is adopted. This strategy consists of three core parts: triplet loss based on data buffer, similarity knowledge distillation loss based on feature relationships, and progressive parameter fusion.

[0052] Specifically, to accommodate the streaming input characteristics of continuous learning, a fixed-capacity data buffer is constructed to store a limited number of historical images and their corresponding loop-closure relation matrices. Let the current time be denoted as . The set of historical images in the buffer is The corresponding cyclic relation matrix is This matrix records whether there are loop relationships between the samples. When querying images in a new domain... Upon arrival, using this image as the anchor sample, images with a loop relationship to it are selected from the buffer as positive samples. At the same time, images that do not form a loop relationship are selected as negative samples. Thus, triples are constructed. .

[0053] This triple is input into the feature extractor and aggregation module of the scene recognition model, respectively, and mapped to a global descriptor. The cosine similarity function is used to measure the similarity between descriptors, thus obtaining the similarity between anchor samples and positive samples. And the similarity between anchor samples and negative samples .

[0054] Based on this, the feature embedding space is constrained by the triplet loss, ensuring that the similarity between positive samples and anchor samples is higher than that between negative samples and anchor samples. The triplet loss function is defined as follows: Among them, the margin parameter This is used to ensure that positive and negative samples maintain at least a minimum similarity difference in the embedding space, thereby maintaining a stable discriminative structure.

[0055] In addition to triplet loss, similarity knowledge distillation loss based on feature relationships is also introduced to promote knowledge transfer between the old and new models, so that the model can maintain the original similarity structure while learning new domain data.

[0056] Specifically, the scene recognition model that has just been trained in the old domain is frozen and referred to as the old model. The model currently awaiting update is denoted as the new model. When retrieving a batch of data from a new domain (batch size is...) When ), extract the global descriptor for this batch of data using both the old and new models, denoted as . and .

[0057] Cosine similarity was used to calculate the similarity between each pair of samples within a batch, and corresponding similarity relationship matrices were constructed accordingly. and , . Softmax normalization is applied to each of the two matrices along their rows to obtain the probability distribution form. Then, KL divergence is used to construct the similarity knowledge distillation loss: .

[0058] This loss constraint ensures that the new model maintains similarity relationships between samples in the embedding space as the old model, thus effectively suppressing catastrophic forgetting. Adding the triplet loss and the knowledge distillation loss yields the optimization objective function for the current domain data. .

[0059] When new domain data Upon arrival, in addition to using real samples from the current domain, a frequency-domain-aware style transfer network that has been trained and frozen is also used to generate pseudo-sample data with the style of the old domain. To fully utilize these pseudo-samples, the same triplet loss and similarity knowledge distillation loss are applied to them to obtain the loss corresponding to the old domain style data. Ultimately, the complete loss function of the scene recognition model driven by generative distributed replay is: ,in This is a preset weight parameter used to balance the relationship between learning new domain knowledge and forgetting old domain knowledge (typically taken as 0.5 to 1.0). By minimizing... The parameters of the new model are then optimized and updated. The above training process is repeated each time new domain data arrives, thereby effectively reproducing the historical appearance distribution using generated pseudo-samples without accessing any historical real images, significantly mitigating catastrophic forgetting.

[0060] Furthermore, in step S4, the parameters of the trained scene recognition model are progressively fused with those of the untrained scene recognition model to further balance the stability and plasticity of the model during continuous learning. Since the old and new models have the same network structure and parameter space, they can be fused in the parameter space, thereby achieving smooth knowledge inheritance and avoiding drastic changes in model parameters between different visual domains.

[0061] Specifically, an exponential moving average strategy is used for weighted fusion. Let the model parameters frozen immediately after training in the old domain be denoted as . The new model parameters obtained by optimizing and updating the current domain data through step S3 are: After completing the new domain learning, a smooth update is constructed for both models: , in This is the fusion coefficient, used to control the weighting of historical and new knowledge in the parameter space. It is typically... Set to a near The value (e.g.) This allows the fused parameters to be based primarily on historical knowledge, with only a small amount of newly learned changes adopted. This ensures that the model maintains its ability to stably distinguish between old and new domain scenarios while gradually absorbing features from the new domain.

[0062] After the above progressive parameter fusion, the cross-domain visual scene recognition model is finally obtained. Using this model, the robot performs forward propagation on the scene image it is currently observing, extracts global descriptors, and performs similarity matching with the historical descriptor library. This allows the model to determine whether the current observation forms a loop with the historical scene, thereby achieving robust visual scene recognition during long-term cross-domain operation of the robot. This further suppresses catastrophic forgetting and ensures that the model can maintain stable discrimination ability for historical scenes even after experiencing multiple appearance domains (such as changes in season, weather, and lighting).

[0063] Experiment 1: Performance Evaluation of Cross-Domain Style Transfer Using FAST-Net To comprehensively evaluate the performance of the frequency-domain aware style transfer network proposed in this invention in terms of generation quality and structure preservation, four commonly used evaluation metrics were selected from three levels: distribution consistency, perceptual similarity, and pixel-level reconstruction quality: Fréchet Inception Distance (FID), Kernel Inception Distance (KID), Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index (SSIM). FID and KID measure the difference in feature distribution between the generated image and the real image in the target domain; both are reference-free metrics, and smaller values ​​indicate that the generated result is closer to the target domain in terms of overall distribution. PSNR and SSIM measure the consistency between the generated result and the reference image from the perspectives of pixel error and structural similarity, respectively. These require comparing the generated image with the corresponding reference image; larger PSNR and SSIM values ​​generally indicate better reconstruction quality and structure preservation capabilities.

[0064] The experiment used two visual scene recognition datasets, Nordland and Robotcar, which exhibit significant appearance domain distribution shifts. The style transfer order between domains in the style transfer experiment was the reverse of the training order of the subsequent visual scene recognition model. The style transfer order for the Nordland dataset was set to winter to autumn, autumn to summer, and summer to spring; the transfer order for the Robotcar dataset was foggy day to night and night to sunny day, consistent with the playback process. Since both datasets are large-scale visual scene recognition datasets, and the style transfer model only requires a limited number of samples to learn the appearance distribution features between different domains, this experiment randomly selected 2000 images from each subdomain, with 1500 used for model training and the remaining 500 used for quantitative evaluation.

[0065] Classic style transfer models CycleGAN, CUT, and U-GAT-IT, as well as FDIT, which also utilizes frequency domain information, were chosen as comparative methods. Since the Robotcar dataset is unpaired, the FID and KID metrics were evaluated only on this dataset. Figure 6 As shown, CycleGAN and CUT have limited ability to model transfer tasks involving drastic weather changes. U-GAT-IT and FDIT, which introduce attention mechanisms and frequency domain modeling, have achieved better performance in both metrics. FAST-Net, proposed in this invention, has achieved the best performance in both transfer tasks, indicating that FAST-Net can more effectively constrain the distribution shift of the generated results under complex lighting conditions.

[0066] Quantitative results from the Nordland dataset, such as Figure 7As shown, this invention achieves optimal or near-optimal performance on most metrics, particularly demonstrating a stable advantage in distribution alignment and structure preservation. In the most challenging winter-to-autumn migration task, this invention achieves optimal results on all four metrics: FID, KID, PSNR, and SSIM, indicating that frequency-domain perceptual modeling significantly improves migration tasks with large differences in appearance distribution. This invention also exhibits stable advantages in autumn-to-summer and summer-to-spring tasks. Taking autumn-to-summer as an example, compared to U-GAT-IT and FDIT, this invention reduces FID by 11.45 and 5.52 respectively, KID by 0.0029 and 0.0007 respectively, and improves SSIM by 0.074; in the summer-to-spring task, this invention achieves the highest SSIM value of 0.629, an improvement of 0.02 over FDIT, while simultaneously reducing KID by 0.0051. Although FID is slightly higher than FDIT in individual tasks, the overall improvement of KID and SSIM shows that this invention introduces richer local details by explicitly modeling and enhancing high-frequency information, which leads to slight fluctuations in the global statistical distribution, but the overall generation quality is still better.

[0067] In addition, two tasks with significant cross-domain differences—winter to autumn (Nordland) and night to clear weather (Robotcar)—were selected for visualization comparison. For example... Figure 8 As shown, CycleGAN and CUT are prone to color shifts and detail blurring under drastic appearance changes, leaving obvious dark areas during the transition from night to day and producing unnatural vegetation colors during seasonal changes. U-GAT-IT improves the overall style to some extent through an attention mechanism, but texture instability still exists in local areas. FDIT, using frequency domain masking modeling, can better preserve scene structure, but the detail rendering is relatively smooth. In contrast, this invention can better preserve key structures such as roads and tracks while completing the overall style conversion, and restore clearer texture details, making the generated results closer to real images in terms of brightness distribution, color consistency, and structure preservation, consistent with the advantages of the aforementioned quantitative indicators.

[0068] Experiment 2: Performance Evaluation of Cross-Domain Visual Scene Recognition To further verify the overall performance of the proposed method in cross-domain continuous visual scene recognition tasks, representative continuous learning methods were selected for comparative experiments, including EWC and SI based on parameter regularization, as well as the latest visual scene recognition methods AirLoop and VIPeR. The experiments were evaluated on the Nordland and Robotcar datasets, both exhibiting significant domain shift characteristics. The maximum recall (MR) metric at 100% accuracy was used to measure the model's visual scene recognition accuracy, and the average performance (AP) of MR was used as the overall accuracy. Backward transfer (BWT) and forward transfer (FWT) were used to measure the forgetting resistance and generalization performance of continuous learning. AP, BWT, and FWT were compared comprehensively in terms of overall performance, forgetting resistance, and generalization ability, respectively.

[0069] The experimental results are summarized in Table 1, with bold text indicating the best results and underlined text indicating the second-best performance. As can be seen from the table, EWC and SI show similar AP performance on both datasets, with negative BWT values ​​(-0.009 and -0.011 on Nordland, and -0.041 and -0.045 on Robotcar), indicating that strategies relying solely on parameter regularization are insufficient to effectively suppress catastrophic forgetting when faced with significant changes in appearance. AirLoop improved its BWT to 0.008 and 0.003 on both datasets, indicating that it alleviated the forgetting problem to some extent, but the overall improvement was limited. In contrast, VIPeR achieved a BWT of 0.020 on the Nordland dataset, demonstrating good stability, but its BWT remained at -0.030 on the Robotcar dataset, showing insufficient robustness under more complex weather conditions.

[0070] Table 1:

[0071] The proposed method achieves BWTs of 0.027 and 0.018 on two datasets, significantly outperforming other comparative methods, demonstrating that it can more effectively maintain historical loop closure discrimination capabilities during cross-domain scene recognition training. Simultaneously, on the Robotcar dataset, the proposed method achieves an AP of 0.528, improving upon AirLoop (0.465) and VIPeR (0.490) by 0.063 and 0.038 respectively, validating its performance advantage in open cross-domain scenes. On the Nordland dataset, the proposed method achieves an AP of 0.636, second only to VIPeR's 0.639, but with a significantly higher BWT. Overall, the proposed method achieves a more balanced performance between forgetting resistance (BWT) and overall performance (AP), fully demonstrating the effectiveness of the generative distributed replay-driven training strategy combined with a frequency-domain aware style transfer network in mitigating catastrophic forgetting and improving the long-term robustness of robots in cross-domain visual scene recognition.

[0072] Example 2: Based on the same inventive concept as Embodiment 1, the present invention also provides a robot cross-domain visual scene recognition system for implementing the steps of the robot cross-domain visual scene recognition method described in Embodiment 1, including the following modules: The frequency domain decomposition module is used to acquire the current domain sample, perform frequency domain decomposition on the features of the current domain sample, and separate the low-frequency components and high-frequency components. The frequency domain fusion module is used to model the low-frequency component and the high-frequency component using mapping functions respectively, and to fuse the modeled low-frequency features with the high-frequency features to obtain frequency domain modulation features; The generation and training module is used to map the frequency domain modulation features into pseudo sample data, and input the pseudo sample data and the current domain samples into the scene recognition model for training. During the training process, the scene recognition model is constrained to distinguish between historical domain scenes and current domain scenes. The fusion recognition module is used to progressively fuse the parameters of the trained scene recognition model with the parameters of the pre-trained scene recognition model to obtain a cross-domain visual scene recognition model; and to use the cross-domain visual scene recognition model to perform cross-domain visual scene recognition on the robot's current observation scene.

[0073] The frequency domain decomposition module, frequency domain fusion module, generation and training module, and fusion recognition module of the robot cross-domain visual scene recognition system proposed in this embodiment are used to implement steps S1, S2, S3, and S4 in the robot cross-domain visual scene recognition method in Embodiment 1, respectively. To avoid redundancy, they will not be described in detail here.

[0074] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0076] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0077] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0078] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for cross-domain visual scene recognition in robots, characterized in that, include: S1: Obtain the current domain sample, perform frequency domain decomposition on the features of the current domain sample, and separate the low-frequency component and the high-frequency component. S2: The low-frequency component and the high-frequency component are modeled using mapping functions respectively, and the modeled low-frequency features are fused with the high-frequency features to obtain frequency domain modulation features; S3: Map the frequency domain modulation features to pseudo-sample data; input the pseudo-sample data and the current domain samples into the scene recognition model for training, and constrain the scene recognition model's ability to distinguish between historical domain scenes and current domain scenes during the training process; S4: Progressively fuse the parameters of the trained scene recognition model with the parameters of the pre-trained scene recognition model to obtain a cross-domain visual scene recognition model; use the cross-domain visual scene recognition model to perform cross-domain visual scene recognition on the robot's current observation scene.

2. The robot cross-domain visual scene recognition method according to claim 1, characterized in that: In step S1, the method for performing frequency domain decomposition on the features of the current domain sample to separate low-frequency and high-frequency components is as follows: a parameterized Gaussian low-pass operator is used to decompose the input features. Perform channel-wise depthwise convolution to extract the low-frequency components. The formula is: ， in, For standard deviation Parameterized two-dimensional Gaussian kernel, This represents channel-wise convolution; Calculate the input features With the low-frequency component The difference is used to obtain the high-frequency residual, and the high-frequency residual is amplitude modulated to generate the high-frequency component. The formula is: ， in, This is the high-frequency enhancement factor.

3. The robot cross-domain visual scene recognition method according to claim 1, characterized in that: In step S2, the method for fusing the modeled low-frequency features with the high-frequency features to obtain the frequency domain modulation features is as follows: The modeled low-frequency features... and high-frequency features after modeling The data is concatenated along the channel dimension, and the fused features are obtained through convolution operations. ; The fusion features Features of the current domain samples are incorporated in the form of residuals. Through learnable gating parameters By controlling the degree of its influence, the frequency domain modulation characteristics are obtained. The formula is: 。 4. The robot cross-domain visual scene recognition method according to claim 1, characterized in that: In step S3, the method for mapping the frequency domain modulation feature to pseudo sample data is as follows: the frequency domain modulation feature is decoded, and the intermediate decoding feature is enhanced at high frequencies during the decoding process. The frequency domain modulation feature is then mapped to the pseudo sample data through the decoding process.

5. The robot cross-domain visual scene recognition method according to claim 4, characterized in that: The method for high-frequency enhancement of intermediate decoding features during the decoding process is as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] High-frequency components are extracted and high-frequency enhancement mapping is applied to obtain enhanced high-frequency features. ; For the decoding intermediate features Adaptive weighted modulation is performed to output channel attention weights. ; The enhanced high-frequency features With the channel attention weights Perform channel-by-channel multiplication and then combine with the decoded intermediate features. By summing the residuals, we obtain the enhanced output features. , represented as: ， Here, ⊙ represents channel-by-channel multiplication.

6. The robot cross-domain visual scene recognition method according to claim 1, characterized in that: In step S3, the method for constraining the scene recognition model's ability to distinguish between historical and current domain scenes includes using triplet loss for constraint, specifically: A fixed-capacity data buffer is constructed, which stores cached historical images and corresponding loop closure matrix. The loop closure matrix is ​​used to record whether a loop closure relationship exists between samples. Using the currently queried image as the anchor sample, images that have a loop relationship with the anchor sample are selected from the data buffer as positive samples, and images that do not have a loop relationship are selected as negative samples to construct triples; The anchor sample, the positive sample, and the negative sample are respectively mapped into global descriptors through a feature extractor and an aggregation module; The cosine similarity function is used to measure the similarity between the global descriptors. A triplet loss function is constructed, which constrains the similarity between positive samples and anchor samples to be higher than that between negative samples and anchor samples.

7. The robot cross-domain visual scene recognition method according to claim 6, characterized in that: In step S3, the method for constraining the scene recognition model's ability to distinguish between historical and current domain scenes further includes using similarity knowledge distillation loss for constraint, specifically as follows: During the training process of the scene recognition model, the current batch data is obtained from the pseudo sample data and the current domain sample; Using the old scene recognition model before the update and the new scene recognition model to be updated, the global descriptor of the current batch of data is extracted respectively; The similarity between samples is calculated using cosine similarity, and a first similarity relation matrix corresponding to the old scene recognition model and a second similarity relation matrix corresponding to the new scene recognition model are constructed respectively. After normalizing the first similarity relation matrix and the second similarity relation matrix, the similarity knowledge distillation loss is constructed using KL divergence; Using the similarity knowledge distillation loss and the triplet loss function, a total loss function for the new scene recognition model is constructed, and the parameters of the new scene recognition model are optimized and updated based on the total loss function.

8. The robot cross-domain visual scene recognition method according to claim 7, characterized in that: The total loss function Represented as: ， in, The loss corresponding to the current domain sample, The loss corresponding to the pseudo-sample data, These are the preset weight parameters.

9. The robot cross-domain visual scene recognition method according to claim 1, characterized in that: In step S4, the method for progressively fusing the parameters of the trained scene recognition model with the parameters of the pre-trained scene recognition model to obtain the cross-domain visual scene recognition model is as follows: weighted fusion is performed using an exponential moving average strategy to obtain the fusion coefficients. By controlling the weight ratio of the scene recognition model parameters before training and the scene recognition model parameters after training in the parameter space using the fusion coefficient, a cross-domain visual scene recognition model is obtained.

10. A robot cross-domain visual scene recognition system, characterized in that, include: The frequency domain decomposition module is used to acquire the current domain sample, perform frequency domain decomposition on the features of the current domain sample, and separate the low-frequency components and high-frequency components. The frequency domain fusion module is used to model the low-frequency component and the high-frequency component using mapping functions respectively, and to fuse the modeled low-frequency features with the high-frequency features to obtain frequency domain modulation features; The generation and training module is used to map the frequency domain modulation features into pseudo sample data, and input the pseudo sample data and the current domain samples into the scene recognition model for training. During the training process, the scene recognition model is constrained to distinguish between historical domain scenes and current domain scenes. The fusion recognition module is used to progressively fuse the parameters of the trained scene recognition model with the parameters of the pre-trained scene recognition model to obtain a cross-domain visual scene recognition model; and to use the cross-domain visual scene recognition model to perform cross-domain visual scene recognition on the robot's current observation scene.

Images