Chiral molecule cross-domain detection method based on frequency domain prototype domain alignment of STM image

Abstract

STM image chiral molecule cross-domain detection method based on frequency domain prototype domain alignment. It belongs to the cross field of STM image intelligent analysis and deep learning domain adaptive technology. It includes: data preparation and preprocessing; construction of multi-scale feature extraction network; frequency domain perception of multi-scale feature alignment; prototype alignment based on optimal transport; region proposal and target detection; three-stage curriculum training strategy; reasoning and detection output. Through frequency domain decomposition, the style and structure information are separated and aligned, and the optimal transport theory is used to establish the explicit matching of cross-domain class-level features, which eliminates the differences in imaging conditions while maintaining the chiral discrimination ability, and realizes the cross-condition robust detection of chiral molecules in STM images.

Description

TECHNICAL FIELD

[0001] The application belongs to the cross field of intelligent analysis of scanning probe microscope images and deep learning domain adaptive technology, and particularly relates to an STM image chiral molecule cross-domain detection method based on frequency domain prototype domain alignment. BACKGROUND

[0002] The domain shift of STM images (Scanning Tunneling Microscope, STM) has distinct physical characteristics and is essentially different from the domain shift commonly seen in natural images (such as photos and videos). Understanding the physical nature of the domain shift of STM images is the basis for designing effective domain adaptive methods. The main sources of domain shift of STM images can be analyzed from the imaging physical mechanism. The first source is the change of bias voltage, the second source is the change of tunneling current setting value, the third source is the change of probe tip state, and the fourth source is the change of scanning parameters and environmental conditions.

[0003] From the perspective of signal processing, the domain shift caused by the above imaging condition changes can be divided into two types of feature changes. The first type is global characteristic change, including overall brightness shift, contrast scaling, background tilt, etc. large-scale smooth changes, which mainly affect the low-frequency components of the image. The second type is local characteristic change, including fine structure definition, texture feature, edge sharpness, etc. These changes are mainly encoded in the high-frequency components of the image. This frequency domain decomposition perspective provides a theoretical basis for designing targeted domain adaptive strategies.

[0004] Traditional image preprocessing methods (such as histogram equalization, background subtraction, and filter denoising) can partially correct global differences, but it is difficult to handle inter-domain changes of local textures. More importantly, excessive preprocessing may damage chiral-related fine features, thereby reducing detection accuracy. Dimensionality reduction visualization methods such as t-SNE can visually demonstrate the existence of domain shift: projecting the features of STM images under different imaging conditions to a two-dimensional space, it can be observed that there is a clear separation of clustering structure, and images under different conditions form independent clusters rather than mixed distribution. This separation of feature space is the fundamental reason for the decline in cross-condition detection performance.

[0005] Domain adaptation (DA) aims to transfer the knowledge learned from a source domain (labeled training data) to a target domain (unlabeled testing data) and is an effective way to solve the domain shift problem. However, existing DA methods are mainly designed for natural images and have the following limitations when applied to STM images: first, existing methods fail to fully consider the physical characteristics of STM image domain shift, second, existing methods lack a class-aware alignment mechanism, third, existing methods lack stability and convergence in training, and fourth, existing methods lack consideration of the multi-scale characteristics of microscopic images.

[0006] In summary, existing DA methods have obvious limitations in the task of chiral molecule detection in STM images, and there is an urgent need to develop specialized DA techniques for STM image characteristics.

[0007] Frequency domain analysis provides a new perspective for understanding image domain shift. Studies have shown that the domain difference of images mainly manifests in the amplitude spectrum of Fourier transform, while the phase information is relatively stable between different domains. The amplitude spectrum encodes the "style" information of the image (such as texture statistics, color distribution), and the phase spectrum encodes the "content" information of the image (such as edge position, structure layout). However, existing frequency domain methods mainly perform frequency decomposition and style transfer at the image pixel level, and have not been systematically explored at the deep feature level. In addition, existing methods mainly target semantic segmentation and image classification tasks, and have less application in object detection tasks. Optimal transport (OT) theory provides a principled framework for measuring and minimizing distribution differences. Compared with adversarial learning methods, OT has the following advantages. First, OT explicitly models the distance measure between distributions, with a clear mathematical definition and geometric meaning, while the alignment degree of adversarial learning is difficult to quantify. Second, the optimization objective function of OT is convex (under Kantorovich relaxation), and the training is more stable, without problems such as mode collapse. Third, OT can naturally incorporate class information by performing transport in the class prototype space rather than the overall feature space, achieving class-aware domain alignment. However, existing OT-based DA methods are mainly designed for classification tasks, and extending them to chiral molecule object detection in STM images requires handling a variable number of object instances in each image and maintaining spatial localization accuracy. SUMMARY

[0008] In order to solve the technical problems of domain offset and poor robustness existing in the related technologies of existing STM image chiral molecule detection, the present application provides an STM image chiral molecule cross-domain detection method based on frequency domain prototype domain alignment. The style and structure information are separated and aligned through frequency domain decomposition, and the explicit matching of cross-domain class-level features is established by using the optimal transmission theory, so as to eliminate the differences of imaging conditions while maintaining the chiral discrimination ability, and realize the cross-condition robust detection of chiral molecules in STM images.

[0009] The method comprises:

[0010] S1, data preparation and preprocessing: collecting STM source domain images and target domain images of chiral molecules and performing preprocessing;

[0011] S2, constructing a multi-scale feature extraction network: in the network, the source domain images and the target domain images are subjected to hierarchical feature extraction through a backbone network with shared weights, and multi-scale features are constructed through a feature pyramid network;

[0012] S3, multi-scale feature alignment based on frequency domain perception: the multi-scale features are decomposed into low-frequency components and high-frequency components through Fourier transform, the low-frequency components and the high-frequency components are subjected to adversarial alignment through a frequency domain discriminator, and a frequency domain adversarial loss is obtained ;

[0013] S4, prototype alignment based on optimal transmission: a prototype optimal transmission alignment module is constructed, in which the distribution alignment of class-level features of L-type and R-type chiral molecules between the source domain images and the target domain images is performed, and a prototype alignment loss is obtained ;

[0014] S5, region proposal and target detection: a region proposal network RPN is constructed, the multi-scale features are input into the region proposal network RPN, and candidate regions are obtained; in the selected regions, positioning and classification are performed through a detection head, and a bounding box and its corresponding probability of belonging to background / L-type / R-type three categories are output, and a detection loss is obtained ;

[0015] S6, three-stage curriculum training strategy: based on the frequency domain adversarial loss, the prototype alignment loss and the detection loss, the multi-scale feature extraction network is subjected to three-stage curriculum training, and a trained multi-scale feature extraction network is obtained;

[0016] S7, reasoning and detection output: the target domain STM image after preprocessing is input, the trained multi-scale feature extraction network is used to extract features of the STM image, and multi-scale features are obtained, and step S5 is repeated to detect the chiral molecules in the STM image.

[0017] Furthermore, the source domain data consists of high-quality STM images acquired under standard imaging conditions, with complete bounding boxes and chiral class annotations; the target domain data consists of images acquired under different imaging conditions, which remain unlabeled during the training phase and are only annotated during the evaluation phase.

[0018] Furthermore, the preprocessing specifically includes: adjusting all input images to a uniform pixel size; normalizing the images; and applying data augmentation strategies to increase the diversity of the training data.

[0019] Furthermore, the backbone network is improved based on the ConvNeXt model, specifically as follows:

[0020] First, a 7×7 large-kernel depth-separable convolution is used to replace the traditional 3×3 convolution;

[0021] Second, layer normalization is adopted instead of batch normalization;

[0022] Third, the GELU activation function is used instead of the ReLU activation function;

[0023] Fourth, the input image is downsampled using 4×4 convolutions in the stem layer of the ConvNeXt network, instead of traditional pooling layers.

[0024] Furthermore, the multi-scale features are decomposed into low-frequency and high-frequency components using Fourier transform. Specifically, for the feature pyramid network layers... Feature map on First, apply a two-dimensional discrete Fourier transform independently to each channel to obtain the frequency domain representation: Subsequently, the frequency domain representation is decomposed into low-frequency and high-frequency components through frequency-selective filtering, and a low-pass filter mask is defined. ,in The cutoff frequency threshold is used for low-frequency components. and high frequency components The extraction formula is:

[0025] ; ;in This represents element-wise multiplication;

[0026] Frequency domain adversarial loss through Obtain, among which, Weighing the contributions of each level of the pyramid, Balancing low-frequency and high-frequency adaptation, Indicates the total number of levels in the pyramid network. For low-frequency alignment loss, This represents the high-frequency alignment loss.

[0027] Furthermore, the prototype alignment loss is obtained through... obtained, wherein, , is a learnable parameter vector, the Softmax function maps to positive weights that sum to 1, , denotes the quality proportion of transferring from the source domain's class prototype to the target domain's class prototype at the hierarchy level , wherein, denotes the square of the Euclidean distance, denotes the prototype vector of the source domain's class prototype at the hierarchy level , denotes the pseudo prototype vector of the target domain's class pseudo prototype at the hierarchy level .

[0028] Further, the region proposal network RPN independently runs at each pyramid level and is responsible for generating candidate regions that may contain chiral molecules. The RPN performs classification and regression prediction at each spatial position through two parallel 1x1 convolution branches, respectively. The classification branch outputs a binary classification score indicating whether each anchor box belongs to the foreground or background. The regression branch outputs the positional offset of the anchor box relative to the true target box.

[0029] The detection head performs fine classification and positioning on the screened region proposals. First, the RoI Align operation is used to extract fixed-size region features from the corresponding pyramid level. After flattening, the extracted region features are input into two fully connected networks for encoding. The first fully connected layer uses a ReLU activation function, and the second fully connected layer also uses a ReLU activation function. These two fully connected networks encode the region features for subsequent classification and regression.

[0030] Finally, the classification head and the regression head perform prediction on the encoded features. The classification head outputs the probability distribution of background / L-type / R-type, i.e., the posterior probability of each proposal belonging to each class. The regression head outputs the fine adjustment offset of the bounding box, which is used to adjust the region proposal to a more accurate target position.

[0031] Further, the total detection loss wherein, and are weight coefficients, is the RPN loss, is the detection head loss,

[0032] ;

[0033] wherein is the The predicted foreground probability of each anchor box. To give it its true label, For binary cross-entropy loss; For the predicted position offset, This is the actual position offset. Smooth L1 loss; and As the normalization factor, This is the balance coefficient;

[0034] ,in, For three-class cross-entropy loss, The loss is for Smooth L1 regression.

[0035] Furthermore, the three-stage course training strategy is as follows:

[0036] The first stage is the source domain pre-training stage, corresponding to 20% of the training iterations. In this stage, standard supervised object detection training is performed only on the source domain data to optimize the detection loss. ;

[0037] The second stage is the frequency domain alignment introduction stage, corresponding to 20%-60% of the training iterations. In this stage, frequency domain adversarial alignment is gradually introduced, and the training objective becomes: Among them, the adversarial loss weight It increases linearly from 0 to the final value of 1.0;

[0038] The third stage is the prototype alignment activation stage, corresponding to 60%-100% of the training iterations. In this stage, the prototype alignment loss based on optimal transmission is activated. The complete training objective is:

[0039] ;

[0040] Among them, the prototype alignment loss weight It gradually increases from 0 to the final value of 1.0.

[0041] Furthermore, during inference and detection output, category-level NMS is applied to all non-background category detection results to remove highly overlapping redundant detections. The IoU threshold of NMS is set to 0.5. Finally, detection results with a confidence score higher than 0.5 are retained as output. The output format includes: bounding box coordinates, category label, and confidence score for each detection box. Attached Figure Description

[0042] Figure 1 This is a detailed diagram of the overall architecture of the FPDA-FRCNN framework in an embodiment of the present invention;

[0043] Figure 2A schematic diagram of the principle of the frequency domain decoupling alignment strategy in the embodiment of the present application is shown in the figure.

[0044] Figure 3 A schematic diagram of the alignment mechanism in the prototype optimal transmission alignment module in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0045] The technical solutions of the present application will be described clearly and completely in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0046] Embodiment 1,

[0047] The present embodiment proposes a STM image chiral molecule cross-domain detection method based on frequency domain prototype domain alignment, named FPDA-FRCNN (Frequency-domain Prototype Domain Alignment Faster R-CNN). The method realizes the separation and alignment of style and structure information through frequency domain decomposition, and uses optimal transport theory to establish explicit matching of cross-domain class-level features, eliminating the differences in imaging conditions while maintaining the chiral discrimination ability, and realizing the cross-condition robust detection of chiral molecules in STM images.

[0048] The overall design concept of the present embodiment can be summarized as three core principles. The first principle is multi-scale feature representation to adapt to different resolution conditions. The multi-scale characteristics of STM images are a practical problem that must be addressed. The number of pixels occupied by a single molecule under different magnifications can differ by more than ten times. A feature pyramid network (FPN) is used to construct a feature representation covering multiple scales, providing a foundation for detecting molecular targets under different resolution conditions.

[0049] The second principle is frequency domain decoupling alignment to separate style information and structure information. Based on a deep understanding of the physical mechanism of STM imaging, the present application recognizes that domain differences caused by changes in imaging conditions are mainly reflected in changes in low-frequency components (reflecting overall illumination, contrast, and other "style" information), while chiral structure details are mainly encoded in high-frequency components (reflecting molecular fine structure and other "content" information). By performing Fourier transform to decompose the features into low-frequency and high-frequency components, respectively, and performing targeted adversarial alignment, the domain bias can be eliminated while protecting the chiral discrimination information.

[0050] The third principle is the class-aware transfer of prototype guidance to maintain the chiral discriminability. To avoid the blurring of class boundaries caused by global alignment, the invention introduces an optimal transport-based prototype alignment mechanism. By maintaining a representative prototype vector for each chiral class, cross-domain alignment is performed in the prototype space rather than the overall feature space, ensuring that cross-domain samples of the same class are brought closer together while samples of different classes remain separated, thereby ensuring class-balanced domain adaptation from a mechanism perspective.

[0051] These three principles work together and progress in layers to form a complete technical solution to address the STM image domain shift problem. The overall technical process of the embodiment includes the following seven core steps: data preparation and preprocessing, multi-scale feature extraction, multi-scale feature alignment in frequency domain, optimal transport-based prototype alignment, region proposal and target detection, three-stage curriculum training, and inference and detection output. Each step will be described in detail below through specific embodiments.

[0052] The overall workflow of the method includes five core steps. Step 1 is the input of the source and target domains, which inputs labeled source domain STM images and unlabeled target domain STM images, both of which have domain shifts (such as blurring, noise, and contrast changes) caused by imaging condition differences. Step 2 is the backbone network and multi-scale feature extraction, in which the source and target domain images are extracted through a shared ConvNeXt backbone network to form hierarchical features, and a feature pyramid network (FPN) is used to construct multi-scale feature representations {P2, P3, P4, P5}. Step 3 is the core innovation of the frequency domain decoupling alignment strategy, which decomposes the features into low-frequency (LF) and high-frequency (HF) components through fast Fourier transform (FFT), and the low-frequency components are strongly aligned through a domain discriminator and a gradient reversal layer (GRL) to eliminate the style shift caused by imaging conditions, while the high-frequency components are weakly aligned to preserve the discriminative details of the chiral structure. Step 4 is the prototype optimal transport alignment module, which implements class-level cross-domain feature matching based on the Sinkhorn algorithm. Before alignment, the source and target domain features are mixedly distributed in the feature space, and after alignment, the same class prototypes (L and R) are pulled closer together across domains while the inter-class boundaries remain clearly separated. Step 5 is the final implementation of robust cross-condition chiral molecule detection on the target domain, with a detection accuracy mAP@50 of 92.9%.

[0053] Embodiment 2,

[0054] This embodiment is a further limitation of Embodiment 1, which further describes Step 1, data preparation and preprocessing.

[0055] The source domain data is high-quality STM image acquired under standard imaging conditions, with complete bounding box and chirality class annotation; the target domain data is image acquired under different imaging conditions, which remains unlabeled during the training stage and only uses annotation during the evaluation stage. This setting truly simulates the actual application scenario: researchers have accumulated labeled data under a certain standard condition and hope to apply the trained model to new imaging conditions without re-labeling.

[0056] The source domain image is collected under standard imaging parameters, typical settings include: bias voltage -0.5 V to -1.0 V, tunneling current about 100 pA, each molecule occupies about 50x50 pixel area. The source domain image should have clear molecular outline and recognizable chirality features, and the image quality should meet the requirements of manual annotation. The annotation work is completed by experienced STM researchers, and the double independent annotation and consensus mechanism is adopted to ensure the quality consistency. The annotation content includes the bounding box coordinates (left upper corner x, y coordinates, width and height) of each molecule and the chirality class (L type or R type). The annotation result is stored in COCO format, which is convenient for compatibility with mainstream target detection framework.

[0057] The target domain image covers different imaging conditions, including but not limited to: different bias voltage range (-1.5 V to +1.5 V), different tunneling current setting value (10 pA to 500 pA), different probe state, different scanning rate, etc. These changes result in target domain images with different contrast, resolution and texture features from the source domain. The target domain image strictly remains unlabeled during the training process to truly reflect the cross-condition transfer ability of the domain adaptive method. The annotation of the target domain is only used for model evaluation and is invisible in training.

[0058] The image preprocessing process includes the following steps. First, all input images are uniformly adjusted to 256x256 pixel size, and the bilinear interpolation method is used to maintain image quality. Choosing 256x256 as the uniform size is a balance between computational efficiency and detection accuracy: larger size can retain more details but increase computational overhead, smaller size is computationally efficient but may lose fine features. Experiments show that 256x256 size is a suitable choice for this task.

[0059] Second, the image is normalized. The pixel value is normalized from the range [0, 255] to the range [0, 1], and then the mean ([0.485, 0.456, 0.406]) of the ImageNet dataset is subtracted and the standard deviation ([0.229, 0.224, 0.225]) is divided. This normalization method is consistent with the ImageNet pre-training setting of the backbone network, which helps the transfer learning effect. For single-channel STM grayscale images, they are copied into three channels before inputting into the network to meet the network input requirements.

[0060] Third, data augmentation strategies are applied to increase the diversity of training data. Data augmentation methods include: random horizontal flip (probability 0.5), random vertical flip (probability 0.5), color jitter (brightness, contrast, saturation are randomly disturbed within ±10% of the original value). These enhancement methods simulate the changes that may occur in STM images, which helps to improve the generalization ability of the model. It should be noted that for the chiral molecule detection task, horizontal flip and vertical flip will change the chiral label of the molecule (change L type to R type or vice versa), so the label needs to be updated synchronously when applying flip enhancement.

[0061] Example 3,

[0062] This embodiment is a further limitation of Example 1, which further describes the multi-scale feature extraction of Step 2.

[0063] The source domain and target domain images are extracted by the backbone network with shared weights. Weight sharing is a key design of domain adaptation, which ensures that the source domain and target domain data are processed in the same feature space, providing a foundation for subsequent domain alignment.

[0064] The backbone network adopts the ConvNeXt-Tiny architecture, which is an important design choice of this embodiment for the chiral molecule detection task of STM images. ConvNeXt is a pure convolutional network architecture proposed in 2022, which draws on multiple design elements of Vision Transformer (ViT) and achieves comparable modeling capabilities while maintaining the simplicity of convolution operations. Compared with traditional backbone networks such as ResNet and VGG, ConvNeXt has shown stronger performance in multiple visual tasks.

[0065] The core design features of ConvNeXt-Tiny include the following aspects: based on the existing ConvNeXt model, the following improvements are made:

[0066] First, 7x7 depthwise separable convolution is used to replace the traditional 3x3 convolution. The large receptive field effectively expands the field of view of a single convolution layer, which helps to capture the six-fold symmetry structure of the six-arylbenzene molecules in the STM image. Depthwise separable convolution separates spatial convolution and channel mixing, which reduces the number of parameters and calculations while maintaining the expression ability.

[0067] Second, layer normalization is used to replace batch normalization. Layer normalization normalizes within each sample and does not depend on batch statistics, making it more stable in small batch training. For the STM image data with limited data volume and limited batch size, layer normalization is a more suitable choice.

[0068] Third, GELU (Gaussian Error Linear Unit) activation function is used to replace ReLU. GELU is a smooth nonlinear activation function widely used in the Transformer architecture, which helps to improve the expression ability and training stability of the model.

[0069] Fourth, the stem design of "Patchify" is used, that is, a larger step convolution (4x4 convolution, step 4) is used in the stem layer at the beginning of the network to downsample the input image, instead of the traditional pooling layer. This design is consistent with the patch embedding idea of ViT, which can more effectively extract the initial feature representation of the image.

[0070] ConvNeXt-Tiny contains four hierarchical stages (Stages), and the structure configuration of each stage is as follows. The first stage contains 3 ConvNeXt Blocks, with an output feature channel dimension of 96 and a spatial resolution of 1 / 4 of the input image (i.e. 64x64). The second stage contains 3 ConvNeXt Blocks, with an output feature channel dimension of 192 and a spatial resolution of 1 / 8 of the input image (i.e. 32x32). The third stage contains 9 ConvNeXt Blocks, with an output feature channel dimension of 384 and a spatial resolution of 1 / 16 of the input image (i.e. 16x16). The fourth stage contains 3 ConvNeXt Blocks, with an output feature channel dimension of 768 and a spatial resolution of 1 / 32 of the input image (i.e. 8x8). Between each stage, a convolution layer with a step of 2 is used for downsampling, while expanding the channel dimension.

[0071] The backbone network is initialized with ImageNet-1K pre-trained weights. ImageNet is a large-scale image classification dataset containing over 1 million natural images and 1000 classes. The weights pre-trained on ImageNet encode rich visual feature knowledge, including edge, texture, shape, and other general visual patterns. Although there are significant differences in visual characteristics between STM images and natural images, experiments show that ImageNet pre-training still provides effective feature initialization, significantly better than random initialization. This transfer learning strategy effectively alleviates the problem of sparse SPM domain labeled data, providing a favorable foundation for learning domain-invariant and discriminative feature representations.

[0072] The Feature Pyramid Network (FPN) constructs multi-scale feature representations from the outputs of the four stages of ConvNeXt-Tiny. FPN is an important technical innovation in the field of object detection, providing semantic-rich and appropriately-resolved feature maps at different scales through bottom-up and top-down feature fusion paths, effectively solving the problem of multi-scale object detection.

[0073] The construction process of FPN is as follows. First, denote the outputs of the four stages of ConvNeXt as {C2, C3, C4, C5}, with spatial resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image, and channel dimensions of 96, 192, 384, and 768, respectively. Then, apply a 1x1 convolution to each stage output to unify the channel dimension to 256, obtaining {M2, M3, M4, M5}. The unification of channel dimensions is a key design of FPN, allowing subsequent feature fusion and detection heads to be processed on a unified feature dimension.

[0074] Next, a top-down path is used to pass deep semantic information to the shallow layer. Starting from the deepest layer M5, a 2x upsampling (bilinear interpolation) is performed to expand its spatial resolution, and it is element-wise added to the previous layer M4 to obtain the fused feature. This process is recursively performed until the shallowest layer. The upsampling operation passes the semantic information of the deep layer to the shallow layer, and the element-wise addition realizes the fusion of multi-scale features.

[0075] Finally, a 3x3 convolution is applied to the fused feature map for smoothing, eliminating the aliasing effect introduced by upsampling, and obtaining the final feature pyramid {P2, P3, P4, P5}. These four pyramid levels cover multiple scales from fine texture (P2, resolution 64x64) to global semantics (P5, resolution 8x8), providing a solid foundation for detecting molecular targets in STM images of different resolutions.

[0076] Example 4,

[0077] This embodiment is a further limitation of embodiment 1, which further illustrates the step three frequency domain perception multi-scale feature alignment.

[0078] The frequency domain perception feature alignment strategy is the first core technical innovation of this method. Based on the in-depth understanding of the imaging physical mechanism of STM, the features are decomposed into low-frequency and high-frequency components through Fourier transform, and a differentiated alignment strategy is adopted to eliminate the style differences caused by imaging conditions while protecting the discriminative details of chiral structures.

[0079] From the perspective of signal processing, STM images can be decomposed into the superposition of different frequency components. Low-frequency components correspond to overall brightness, contrast gradient, and background tilt, etc. These changes mainly reflect the influence of imaging conditions (bias, current, probe state), which are "domain noise" that need to be eliminated. High-frequency components correspond to sharp local changes such as molecular edge profile, internal fine structure, etc. These changes carry the structural information of the molecule, including the relative spatial distribution of the six methyl protrusions that determine the chiral configuration, and are the discriminative features that need to be preserved.

[0080] This frequency domain decomposition is highly consistent with the "style-content separation" paradigm in computer vision. In the study of image style transfer, researchers found that the "style" of an image (such as texture statistics, color distribution) is mainly encoded in the low-frequency part of the Fourier amplitude spectrum, while the "content" (such as edge position, structure layout) is mainly encoded in the phase spectrum and high-frequency part. Applying this idea to domain adaptation, we can regard low-frequency as "style" (imaging conditions) and high-frequency as "content" (molecular structure), and achieve precise domain adaptation by processing them separately.

[0081] The specific implementation of frequency domain decomposition is as follows. For feature maps on the pyramid level ,where C is the number of channels, H and W are spatial dimensions, first apply two-dimensional discrete Fourier transform (2D DFT) to each channel independently to obtain the frequency domain representation:

[0082] ;

[0083] where FFT2D represents two-dimensional fast Fourier transform. Fourier transform converts spatial domain feature maps into frequency domain representation, with the frequency domain center corresponding to the direct current component (zero frequency), and the closer to the edge, the higher the frequency. The result of Fourier transform is a complex matrix, containing amplitude and phase components.

[0084] Subsequently, the frequency domain representation is decomposed into low-frequency and high-frequency components through frequency selection filtering. Define the low-pass filter mask , where is the cutoff frequency threshold. The low-pass filter in the frequency domain center The value is 1 within the radius range, and 0 at other positions. A Gaussian or cosine type of smooth transition can also be used to reduce the ringing effect. The extraction formula of low and high frequency components is:

[0085] ;

[0086] ;

[0087] where represents element-wise multiplication (Hadamard product). The low frequency component contains frequency components within the cutoff frequency, and the high frequency component contains frequency components outside the cutoff frequency.

[0088] The selection of the cutoff frequency threshold is a key parameter of the frequency domain decomposition. Considering that features of different pyramid levels have different frequency characteristics - shallow features (P2, P3) have high spatial resolution and contain more high frequency information; deep features (P4, P5) have low spatial resolution and are mainly expressed in semantics - this embodiment adopts a scale-dependent threshold selection strategy:

[0089] ;

[0090] where corresponds to four pyramid levels P2 to P5. This adaptive design makes the shallow layer use a lower cutoff frequency (retaining more high frequencies), and the deep layer use a higher cutoff frequency (more low frequencies are included in the low frequency component), which conforms to the feature distribution characteristics of each level.

[0091] The frequency domain discriminator is used to realize the adversarial alignment of low and high frequency components. This embodiment designs independent domain discriminators and for low and high frequency components, respectively, which receive features restored to the spatial domain by inverse Fourier transform as input, and output domain prediction maps between [0, 1], where close to 0 indicates that the discriminator considers the input to come from the source domain, and close to 1 indicates from the target domain.

[0092] The frequency domain discriminator employs a lightweight fully convolutional architecture, consisting of three cascaded convolutional layers. The first convolutional layer has 256 input channels (consistent with the FPN output), 128 output channels, a 3×3 kernel, a stride of 2, and padding of 1. The second convolutional layer has 128 input channels and 64 output channels, a 3×3 kernel, a stride of 2, and padding of 1. The third convolutional layer has 64 input channels and 1 output channel, a 3×3 kernel, a stride of 2, and padding of 1. Each convolutional layer is followed by batch normalization and a LeakyReLU activation function (with a negative slope of 0.2). Finally, the output is mapped to the [0,1] range using a Sigmoid activation function.

[0093] The fully convolutional architecture allows the discriminator to output domain predictions for each spatial location in the feature map, forming spatially resolved domain prediction maps, rather than a single global prediction. This design helps capture the spatial distribution information of domain differences, which is beneficial for molecular detection tasks that require maintaining spatial localization accuracy.

[0094] Adversarial training employs a Gradient Reversal Layer (GRL) to implement a minimax game between the feature extractor and the domain discriminator. GRL, proposed by Ganin et al., is a core technique in domain adversarial training. During forward propagation, GRL acts as an identity mapping, not altering the input; during backward propagation, it inverts the gradients and multiplies them by a scaling factor. :

[0095] Forward propagation: ;

[0096] Backpropagation: ;

[0097] in It is the identity matrix. is the gradient scaling factor. This design transforms the adversarial game between the discriminator and the feature extractor into a single end-to-end optimization problem: the discriminator learns to distinguish features between the source and target domains by minimizing the domain classification loss, while the feature extractor learns domain-invariant features that "trick" the discriminator by inverting the gradient.

[0098] To achieve the design principle of "strong alignment for low frequencies and weak alignment for high frequencies," this embodiment sets different GRL scaling factors for low-frequency and high-frequency components. A larger scaling factor is set for the low-frequency components. This is to completely eliminate style differences caused by imaging conditions. For high-frequency components, a smaller scaling factor is set. This reduces domain offset while preserving chiral discrimination information and avoids over-alignment that erases fine chiral features.

[0099] The GRL scaling factor is linearly increased from 0 to the final value during training. This gradual strategy avoids instability at the beginning of adversarial training: at the early stage of training, the feature extractor has not learned a meaningful feature representation, and forcing domain alignment at this stage can lead to training collapse; as training proceeds, the feature quality gradually improves, and the gradually increased adversarial strength allows smooth domain alignment.

[0100] The frequency-domain adversarial loss is defined as the sum of binary cross-entropy losses of source and target domain samples. Let the input batch contain source domain samples and target domain samples, the low-frequency alignment loss is:

[0101]

[0102] where IFFT denotes the inverse Fourier transform, which restores the frequency-domain features to the spatial domain. The high-frequency alignment loss has a similar form:

[0103]

[0104] The total loss of multi-scale frequency-domain alignment is defined as the weighted sum of alignment losses of each level and each frequency component:

[0105] ;

[0106] where balances the contribution of each pyramid level, balances the low-frequency and high-frequency adaptation. A larger is set for deeper layers because semantic expression is more biased towards deep features, and structure preservation is more important there.

[0107] As shown in Figure 2 , the core mechanism of frequency-domain decoupled alignment is demonstrated. Figure 2 The left side of the figure shows the input feature map, the middle shows the frequency-domain representation after FFT transformation and the low-pass filter mask, and the right side shows the low-frequency component and the high-frequency component obtained by decomposition. The lower part shows the architecture of the low-frequency discriminator and the high-frequency discriminator, as well as the gradient reversal mechanism of GRL. The figure also shows the differentiated strategy of low-frequency strong alignment ( ) and high-frequency weak alignment ( ).

[0108] Embodiment 5,

[0109] This embodiment is a further limitation of embodiment 1, and further explains the prototype alignment based on optimal transport in step four.

[0110] The prototype alignment mechanism based on optimal transport is the second core technical innovation of the method. The mechanism explicitly models the alignment of the class-level feature distribution between the source domain and the target domain through the Sinkhorn algorithm, while preserving the discriminative structure between the left-handed and right-handed molecules, solving the problem of blurred class boundary caused by global alignment.

[0111] The motivation of prototype learning comes from the analysis of the limitations of global alignment. Frequency domain alignment eliminates the distribution difference between the source domain and the target domain at the feature level, but this global alignment strategy has potential risks: if the class information is not considered, the alignment process may confuse samples of different classes together, resulting in blurred class decision boundary. Assuming that in the feature space, the L-type molecules of the source domain are concentrated in region A, and the R-type molecules are concentrated in region B, the two regions are obviously separated; due to domain shift, the L-type and R-type molecules of the target domain are concentrated in regions A' and B', which may be different from the region positions of the source domain. If the global alignment simply pulls the overall distribution of the target domain to the overall distribution of the source domain without considering the class, the situation of A' mixed with B or B' mixed with A may occur, resulting in the interlacing of originally separated classes in the feature space. For the chiral molecule detection task, the separability of L-type and R-type molecules in the feature space is a prerequisite for accurate classification, and any alignment operation that damages this separability will directly reduce the detection accuracy.

[0112] Prototype is the representative representation of a class in the feature space, which is usually defined as the centroid or mean vector of all sample features of the class. By introducing the constraint of class prototype in the domain alignment process, it can be ensured that the alignment operation maintains the intra-class compactness while maintaining the inter-class separability. The goal of prototype alignment is to pull the L-type prototype of the source domain closer to the L-type prototype of the target domain, and to pull the R-type prototype of the source domain closer to the R-type prototype of the target domain, while maintaining the separation distance between the L-type and R-type prototypes. This class-aware alignment strategy ensures that domain migration is performed within the class and does not destroy the discriminative structure between classes.

[0113] In this embodiment, the prototype alignment mechanism based on optimal transport is implemented by constructing a prototype optimal transport alignment module, and the specific workflow in the module is as follows:

[0114] A dynamically updated class prototype vector is maintained for each chiral class (L and R) and each pyramid level. The initialization of the prototype is based on the feature statistics of the source domain data: after the pre-training phase is completed (i.e., after the detection head has certain classification ability), the feature vectors of the correctly classified L-type and R-type molecule regions are collected respectively, and the feature mean of each class is calculated as the initial prototype. This initialization strategy ensures that the prototype is located at a reasonable feature space position from the beginning, providing a stable starting point for subsequent dynamic updates.

[0115] The prototype is dynamically updated using an exponential moving average (EMA) mechanism. EMA is a commonly used online statistical estimation method, which takes the weighted average of the current estimate and the new observation as the updated estimate, and the weight is controlled by the momentum coefficient. Let represent the region proposal feature set of the level , the update formula of the source domain prototype is:

[0116]

[0117] wherein represents the source domain category at the th iteration, the prototype vector of the level , represents the region proposal set belonging to the level category , and is the momentum coefficient. is the feature vector of a specific region proposal (RegionProposal) belonging to the level

[0118] In this embodiment, a larger momentum coefficient makes the prototype change smooth, and the influence of new samples is significantly attenuated, which helps to stabilize the prototype position in the training process. This EMA mechanism provides stable prototype estimation by accumulating information in the training iteration, avoiding the excessive influence of single batch sample fluctuation on the prototype position.

[0119] The class labels of the source domain samples are directly available, so the prototypes of the corresponding categories can be reliably updated: when a batch of source domain samples is processed by the detection head, the correctly matched detection region features are grouped by category, and the L-type and R-type prototypes are updated respectively.

[0120] The target domain samples have no real labels, and this embodiment uses the pseudo-labels predicted by the current model to guide the prototype association. Specifically, for the detection results in the target domain image, if the classification confidence of a certain detection box exceeds a certain threshold , the predicted category is taken as the pseudo-label, and the corresponding region features are included in the target domain pseudo-prototype update of the category. This self-training strategy enables the target domain prototype to gradually capture the class feature distribution of the target domain, providing more and more accurate reference for cross-domain alignment.

[0121] The pseudo-label confidence threshold ​The setting of the threshold is crucial for controlling the quality of pseudo labels. A too high threshold will result in too few available pseudo labeled samples, failing to fully exploit the target domain data; a too low threshold will introduce too many incorrect labels, and the accumulated noise can mislead the model learning. This embodiment adopts a progressive relaxation strategy: the initial confidence threshold is set to 0.9 to filter most of the noisy labels, and then linearly decreases according to a schedule:

[0122] ;

[0123] where is the current iteration number, is the total number of training iterations. The design philosophy of this progressive relaxation strategy is: in the early stage of training, the model is not well adapted to the target domain, and the quality of pseudo labels is low, so a strict threshold is needed for filtering; as the training proceeds, the domain adaptation ability of the model is enhanced, and the quality of pseudo labels is improved, so the threshold can be appropriately relaxed to utilize more target domain samples. The final threshold decreases to 0.5, allowing most predictions above the confidence of random guessing to be used as pseudo labels.

[0124] Optimal transport theory provides a principled framework for measuring and minimizing distributional discrepancies. Compared with adversarial learning methods, optimal transport has the following unique advantages. First, optimal transport explicitly models the cost of transferring one distribution to another, with a clear geometric interpretation. The minimum of the transfer cost (Wasserstein distance) quantifies the discrepancy between two distributions, and the optimal transport scheme gives the specific mapping relationship of how to achieve distribution alignment with the minimum cost. Second, the optimization problem of optimal transport is convex (under Kantorovich relaxation), and the training is more stable, without the problems such as mode collapse and oscillation commonly seen in adversarial learning. Third, optimal transport can naturally incorporate class information, by performing transport in the class prototype space rather than the overall feature space, achieving class-aware domain alignment.

[0125] Applying optimal transport to prototype alignment, this embodiment regards the set of class prototypes of the source domain and the set of class pseudo prototypes of the target domain as the support points of two discrete distributions, respectively. Let the source domain have class prototypes (in the chiral molecule detection task , corresponding to L-type and R-type, respectively), and the target domain has corresponding pseudo prototypes. We want to find a transport scheme to "transfer" the source domain prototypes to the positions of the target domain prototypes, so that the total transport cost is minimized.

[0126] Given a hierarchy of source domain prototype sets and target domain pseudo prototype sets , where denotes the set of source domain prototypes at the given hierarchy The total number of classes included, which serves as an upper bound on the count, defines the size of the prototype set.

[0127] The cost matrix is first computed whose elements are defined as the squared Euclidean distance between the corresponding prototypes:

[0128] ;

[0129] where denotes the prototype vector of the th class prototype in the source domain at level , and denotes the pseudo prototype vector of the th class pseudo prototype in the target domain at level .

[0130] The cost matrix is a symmetric matrix whose th element represents the cost of transferring the th class prototype in the source domain to the th class prototype position in the target domain. The cost is defined as the squared Euclidean distance, which reflects the geometric structure of the feature space.

[0131] The optimal transport problem seeks a transport matrix (also known as a coupling matrix or transport plan) that minimizes the total transport cost. The th element of the transport matrix represents the proportion of mass transferred from the th class prototype in the source domain to the th class prototype in the target domain. The transport matrix must satisfy marginal constraints to ensure the integrity of the transfer:

[0132] ;

[0133] ;

[0134] where and are the pre-defined marginal distributions representing the weights of each class in the source and target domains. Under the assumption of class balance, the marginal distributions are set to uniform distribution, i.e. .

[0135] The classical optimal transport problem can be formulated as a linear programming:

[0136] ;

[0137] where denotes the set of valid transport plans that satisfy the marginal constraints, denotes the Frobenius inner product. This problem can be solved by the simplex method or interior point method, but the computational complexity is high and it is difficult to extend to the iterative optimization of deep learning.

[0138] The Sinkhorn algorithm proposed by Cuturi transforms the optimal transport problem into an efficiently iteratively solvable form by introducing an entropy regularization term:

[0139] ;

[0140] where is the entropy of the transport plan, is the entropy regularization strength parameter. The entropy regularization term encourages the transport plan to tend to be uniform (maximum entropy principle), which has the following effects: first, makes the objective function strictly convex, ensuring a unique optimal solution; second, makes the optimal solution dense rather than sparse, avoiding the instability of the corner solution that may occur in the original problem; third, transforms the optimization problem into a form that can be efficiently solved by matrix scaling. The embodiment sets , balancing between computational efficiency and alignment accuracy.

[0141] The Sinkhorn algorithm solves the entropy-regularized optimal transport problem by iteratively scaling matrices. First, calculate the kernel matrix: ;

[0142] Then update the scaling vectors and alternately:

[0143] ;

[0144] ;

[0145] where denotes element-wise division, and are the marginal distribution vectors, initialized (the all-1 vector), is used to mark the state of vector updates with the number of steps in the Sinkhorn algorithm iteration process, and the iteration continues until convergence (the embodiment sets a fixed iteration of 100 times), and the optimal transport plan is:

[0146] ;

[0147] All operations of the Sinkhorn algorithm (matrix exponentiation, vector-matrix multiplication, element-wise division) are differentiable, and the gradient can be passed back to the prototype representation and the underlying feature extractor through the automatic differentiation mechanism, realizing end-to-end joint optimization. The embodiment uses an efficient GPU implementation of the Sinkhorn algorithm, which can be calculated in real time on modern GPUs.

[0148] obtaining optimal transport matrix After that, the prototype alignment loss is defined as the transport cost:

[0149]

[0150] This loss measures the optimal transport distance between the source and target domain prototypes, and minimizing it is equivalent to driving the class-level distribution alignment of the two domains.

[0151] To balance the contribution of different pyramid levels to the final detection performance, this embodiment introduces a learnable weight to weight the cross-scale optimal transport loss:

[0152] where , is a learnable parameter vector. The Softmax function maps to positive weights that sum to 1, and this mechanism allows the model to automatically learn which scales need stronger class-level alignment constraints. During training, is optimized together with other network parameters by gradient descent.

[0153] As shown in Figure 3 , the prototype alignment mechanism based on the Sinkhorn algorithm is shown. The upper part of the figure is a feature space visualization, showing the state of the mixed and intertwined L-type and R-type molecular feature distributions of the source and target domains before alignment. The middle of the figure is the prototype construction and optimal transport solving process, including the calculation of the cost matrix , Sinkhorn iterative update and vector, and the obtaining of the optimal transport plan . The lower part of the figure is a visualization of the feature space after alignment, showing that the prototypes of the same class are pulled closer across domains (the source domain L-type and target domain L-type prototypes are close, and the source domain R-type and target domain R-type prototypes are close), while the inter-class boundary remains clearly separated (the L-type and R-type prototypes remain distant).

[0154] Embodiment 6,

[0155] This embodiment is a further limitation of embodiment 1, and further explains the region proposal of step five and target detection.

[0156] The region proposal network (RPN) runs independently at each pyramid level and is responsible for generating candidate regions that may contain chiral molecules. The RPN is a core component of Faster R-CNN, which learns to generate high-quality target proposals in an end-to-end manner, replacing the time-consuming selective search in traditional methods.

[0157] ​​The input of RPN is the feature pyramid {P2, P3, P4, P5} output by FPN, and the size of feature map at each level is 64x64, 32x32, 16x16, 8x8 respectively (for input image of 256x256).

[0158] The design of anchor box is a key factor for the performance of RPN. The commonly used anchor box in traditional object detection has aspect ratios of {0.5, 1.0, 2.0}, which is suitable for various shapes of objects (such as pedestrians, vehicles, buildings, etc.) in natural images. However, the hexaarylbenzene molecules in STM images present a nearly circular six-fold symmetric appearance, and the traditional anchor box design is not suitable. The aspect ratios of the anchor box are optimized to {0.8, 1.0, 1.25} according to the shape characteristics of the molecules, which is closer to the actual shape of the molecules. This targeted anchor box design can improve the ability of RPN to generate high-quality proposals.

[0159] RPN performs classification and regression prediction at each spatial location through two parallel 1x1 convolution branches. The classification branch outputs a binary classification score for each anchor box as foreground (containing objects) or background; the regression branch outputs the position offset (center point offset and scale) of the anchor box relative to the real target box.

[0160] The training of RPN uses a multi-task loss function:

[0161] ;

[0162] where is the predicted foreground probability of the th anchor box, is its true label (1 for foreground and 0 for background), is the binary cross-entropy loss of RPN; is the predicted position offset, is the true position offset, is the Smooth L1 loss; and are normalization factors, is a balance coefficient.

[0163] During inference, RPN first sorts all anchor boxes by foreground score, and keeps the top candidate (the present invention sets ). Then Non-Maximum Suppression (NMS) is applied to these proposals to remove highly overlapped redundant proposals. The IoU threshold of NMS is set to 0.7, and the top 300 proposals with the highest scores are finally reserved for subsequent processing.

[0164] The RoI Head performs fine classification and localization on the filtered region proposals. First, the fixed-size region features are extracted from the corresponding pyramid level by the RoIAlign operation.

[0165] RoI Align is an improvement of the traditional RoI Pooling. RoI Pooling maps the proposal region of arbitrary size to a fixed-size feature map by quantization, but the quantization operation introduces position error, affecting the detection accuracy. RoI Align uses bilinear interpolation instead of quantization to accurately calculate the feature value at any position, avoiding the quantization error. For the detection of chiral molecules with fine structure targets, the accurate positioning ability of RoI Align is particularly important.

[0166] The output size of RoI Align is set to 7x7 in this embodiment, and the number of channels is consistent with the FPN output (256), so each region proposal is extracted as a 7x7x256 = 12544-dimensional feature vector.

[0167] The extracted region features are flattened and input into a two-layer fully connected network for encoding. The first fully connected layer: input dimension 12544, output dimension 1024, using ReLU activation function; the second fully connected layer: input dimension 1024, output dimension 1024, using ReLU activation function. The two-layer fully connected network encodes the region features into a compact 1024-dimensional representation for subsequent classification and regression.

[0168] The final classification head and regression head respectively predict from the 1024-dimensional encoded features. The classification head outputs the probability distribution of background / L-type / R-type three classes (normalized by Softmax), i.e., the posterior probability of each proposal belonging to each class. The regression head outputs the fine adjustment offset of the bounding box (center point offset and scale), which is used to adjust the region proposal to a more accurate target position.

[0169] The training of the detection head uses a multi-task loss function: ;

[0170] where is the three-class cross-entropy loss of the detection head, is the Smooth L1 regression loss. The total detection loss includes the weighted sum of RPN loss and detection head loss.

[0171] Example 7,

[0172] This embodiment is a further limitation of Example 1, and further explains the step six three-stage course training.

[0173] The present application adopts a three-stage curriculum learning strategy for training, aiming to stabilize the training process and prevent the negative impact of early unstable pseudo-labels on model convergence. The core idea of curriculum learning is to simulate the human learning process, gradually transitioning from simple tasks to complex tasks, so that the model can obtain effective learning signals at each stage.

[0174] The first stage is the source domain pre-training stage, corresponding to 20% of the training iterations (0-800 iterations out of a total of 4000 iterations). In this stage, only standard supervised object detection training is performed on the source domain data, and the detection loss is optimized. The domain adaptation components (frequency domain discriminator and prototype optimal transport alignment module) do not participate in training. The goal of this stage is to establish a baseline detection capability, enabling the model to learn basic molecular localization and chirality classification capabilities. At the same time, the class prototypes are initialized at the end of this stage: the feature vectors of correctly classified L-type and R-type molecular regions are collected, and the mean of the features of each class is calculated as the initial prototype.

[0175] The second stage is the frequency domain alignment introduction stage, corresponding to 20%-60% of the training iterations (800-2400 iterations). In this stage, the frequency domain adversarial alignment is gradually introduced, and the training target becomes: ;

[0176] Where the adversarial loss weight increases linearly from 0 to the final value 1.0. Specifically:

[0177] ;

[0178] This gradual introduction strategy ensures the stability of adversarial training: in the early stages of training, the adversarial signal is weak and does not interfere with the consolidation of basic detection capabilities; as training progresses, the adversarial strength gradually increases, driving the feature extractor to learn domain-invariant representations. The frequency domain decomposition strategy plays a role in this stage, with low-frequency strong alignment eliminating imaging condition differences and high-frequency weak alignment protecting chirality discrimination information.

[0179] The third stage is the prototype alignment activation stage, corresponding to 60%-100% of the training iterations (2400-4000 iterations). In this stage, the prototype alignment loss based on optimal transport is activated, and the complete training target is:

[0180] ;

[0181] Where the prototype alignment loss weight increased from 0 to the final value 1.0:

[0182] ;

[0183] The reason why the prototype alignment is activated only in the third stage is that the prototype alignment relies on high-quality class prototypes, while the updating of the target domain prototypes requires reliable pseudo labels. In the first two stages, the model gradually acquires the domain adaptation ability, and the quality of the pseudo labels gradually improves; at this time, activating the prototype alignment can perform class-level alignment on the basis of reliable prototype estimation, avoiding the misleading of training by noisy prototypes in the early stage.

[0184] The complete training objective function integrates three components: detection loss, frequency domain adversarial loss, and prototype alignment loss:

[0185] ;

[0186] The weight hyperparameters of the three loss terms The optimal configuration is determined by Bayesian optimization. Bayesian optimization is an efficient black-box optimization method that builds a Gaussian process proxy model of the objective function and uses the Expected Improvement (EI) acquisition function to guide the search, efficiently exploring the high-dimensional hyperparameter space within a limited evaluation budget. The final optimal hyperparameter configuration is: , indicating that the three loss components contribute equally to the final performance.

[0187] The model is trained using the Stochastic Gradient Descent (SGD) optimizer. The configuration of SGD is as follows: the momentum is set to 0.9 to accelerate convergence and reduce oscillation; the weight decay is set to as L2 regularization to prevent overfitting; the initial learning rate is set to , which remains constant throughout the training process. The total number of training iterations is 2000, and for an input image of 256x256 pixels and a batch size of 256, a single iteration takes about 0.8 seconds, and the complete training takes about 27 minutes.

[0188] Example 8,

[0189] This example is a further limitation of Example 1, and further explains the reasoning and detection output of Step Seven.

[0190] The inference stage has a different flow from the training stage. The key difference is that the domain adaptation components (frequency discriminator and prototype alignment module) are not involved in the inference computation, and the model is simplified to the standard Faster R-CNN detection flow. This is because the role of the domain adaptation components is to guide the feature extractor to learn domain-invariant representations in the training stage. Once the training is completed, the feature extractor already has the ability to generalize across domains, and these auxiliary components are no longer needed in inference. This design makes the computational efficiency of the inference stage comparable to that of the standard Faster R-CNN, meeting the real-time analysis requirements.

[0191] The inference flow is as follows. First, input the target domain STM image to be detected. Image preprocessing includes: adjusting to 256x256 pixel size, normalization processing (subtracting mean value and dividing by standard deviation), and copying the grayscale image to three channels. The preprocessed image is input into the ConvNeXt-Tiny backbone network for feature extraction, obtaining four-stage hierarchical features.

[0192] Then, the FPN constructs a feature pyramid {P2, P3, P4, P5} from the outputs of the four stages of the backbone network. The RPN runs independently at each pyramid level, generating candidate region proposals. After NMS filtering, the top 300 region proposals with the highest scores are retained.

[0193] Next, the detection head classifies and regresses the bounding box for each region proposal. The classification head outputs the probability that the proposal belongs to the background, L-type molecule, or R-type molecule; the regression head outputs the refinement offset of the bounding box. Apply the offset to the original proposal box to obtain the refined bounding box coordinates.

[0194] Finally, apply class-level NMS to the detection results of all non-background classes to remove redundant detections with high overlap. The IoU threshold of NMS is set to 0.5, and the final detection results with a confidence score higher than 0.5 are retained as output. The output format includes: the bounding box coordinates (left upper corner x, y coordinates, width, and height) of each detection box, the class label (L-type or R-type), and the confidence score.

[0195] The inference time of a single 256x256 pixel STM image is about 0.05 seconds (on an NVIDIA RTX 5090 GPU), corresponding to a processing speed of about 20 frames / second, fully meeting the real-time analysis requirements. Compared to manual annotation (which usually takes 20-30 minutes to annotate an image containing 200 molecules), the present invention achieves an efficiency improvement of more than 6000 times.

[0196] Embodiment 9,

[0197] This embodiment is a further limitation of embodiments 1-8, and further describes the methods, principles, and beneficial effects involved in this embodiment.

[0198] Summary of the invention points and innovations:

[0199] The first core innovation of the method is the multi-scale feature alignment strategy based on frequency domain perception. Based on the deep understanding of the physical mechanism of STM imaging, the domain shift is decomposed into low-frequency style difference and high-frequency structure difference, and a differentiated strategy of "low-frequency strong alignment and high-frequency weak alignment" is adopted for processing. Compared with the existing global alignment method, this strategy has the following advantages: from the physical principle, it ensures the effectiveness of domain adaptation, and the low-frequency alignment eliminates the overall contrast and brightness difference caused by the imaging conditions (bias voltage, current, probe state); high-frequency weak alignment avoids the damage to the chiral discrimination information that may be caused by excessive alignment, and protects the fine structure features (relative spatial distribution of six methyl protrusions) that determine the L / R type classification.

[0200] The second core innovation of the method is the prototype alignment mechanism based on optimal transport. This mechanism explicitly models the alignment of class-level feature distribution between source and target domains through Sinkhorn algorithm, solving the class boundary ambiguity problem that may be caused by global alignment. Compared with traditional adversarial learning methods, this mechanism has the following advantages: optimal transport provides a geometrically meaningful distribution measurement method, making the training process more stable and avoiding the pattern collapse and oscillation problems commonly seen in adversarial training; by performing transport in the class prototype space rather than the overall feature space, class-aware domain alignment is achieved, and prototypes are maintained and aligned independently for each chiral class, ensuring class-balanced domain adaptation from a mechanism perspective; all operations of Sinkhorn algorithm are differentiable, and the gradient can be back-propagated end-to-end, realizing seamless integration with deep learning.

[0201] The third innovation of the method is the three-stage curriculum training strategy. This strategy gradually introduces domain adaptation components to stabilize the training process and prevent the negative impact of early unstable pseudo-labels on model convergence. The three stages have different focuses: the first stage establishes the baseline detection capability, the second stage gradually introduces adversarial alignment, and the third stage activates prototype alignment. This curriculum learning design ensures that each component is involved in the optimization process at the appropriate time, avoiding the convergence difficulty caused by the mutual interference of multiple task objectives in the early training.

[0202] The fourth innovation of the method is the network architecture design optimized for chiral molecule detection tasks of STM images. Specifically, it includes: using ConvNeXt-Tiny backbone network, which effectively captures the six-fold symmetry structure of the molecule with 7x7 large kernel convolution, and layer normalization design is more stable when training in small batches; The aspect ratio of the anchor box is optimized to {0.8, 1.0, 1.25}, which is closer to the actual shape of the approximately circular molecule, improving the ability of RPN to generate high-quality proposals; Use ImageNet pre-trained weight to initialize the backbone network, which effectively alleviates the problem of scarce labeled data in the SPM field.

[0203] Advantages:

[0204] The advantages of the method are reflected in detection accuracy, positioning accuracy, class balance, computational efficiency, and practicality.

[0205] In terms of detection accuracy, the method achieves 92.9% mAP@50 detection accuracy on the F-HPB dense packing system, which is the evaluation result in the target domain (different imaging conditions). Compared with the baseline model trained only using source domain data (Source Only, 83.9%), the method improves by 9.0 percentage points, clearly quantifying the negative impact of domain shift on detection performance and the effectiveness of the domain adaptation strategy of the method. Compared with the current best domain adaptation method CMT (Contrastive Mean Teacher, 89.1%), the method still maintains a 3.8 percentage point advantage, verifying the targeted advantage of the frequency domain perception alignment strategy over the general domain adaptation method in STM image processing.

[0206] Testing on different molecular systems further verifies the generalization ability of the method. On the HPB dense packing system, the method achieves 90.1% mAP@50, which is 7.2 percentage points higher than the baseline. HPB molecules lack fluorine atom substituents, have smaller chiral contrast differences, and have more blurred molecular boundaries, making detection more difficult, but the method still exhibits robust detection capabilities. On the sparse packing system, the method achieves 88.6% mAP@50, which is 6.5 percentage points higher than the baseline. In the sparse system, the molecules are distributed in isolation, lacking context information, and the proportion of background areas increases. The method effectively addresses these challenges through multi-scale feature fusion and prototype alignment, as shown in Table 1.

[0207] Table 1: FPDA-FRCNN method comparison experiment table

[0208]

[0209] In terms of positioning accuracy, the mAP@75 of the method reaches 74.6%, and the COCO mAP@[0.5:0.95] reaches 61.2%, leading the suboptimal methods CMT by 4.2 and 5.1 percentage points, respectively. mAP@75 uses a more stringent IoU threshold (0.75) to examine the model's accurate regression ability of the molecular bounding box; COCO mAP is the average value at multiple IoU thresholds (0.5 to 0.95), comprehensively measuring the model's robustness under different positioning accuracy requirements. The excellent performance of the invention on these two strict indicators shows that the model not only accurately detects the presence of molecules, but also accurately regresses the position of the molecular bounding box, which is of great significance for subsequent quantitative analysis (such as molecular coverage statistics, chiral domain size measurement, intermolecular distance analysis, etc.).

[0210] In terms of class balance, the detection accuracy difference of L-type and R-type molecules on the three datasets is less than 1.0 percentage point. Specifically: in the F-HPB system, AP_L is 91.8%, AP_R is 92.7%, the difference is only 0.9 percentage points; in the HPB system, AP_L is 89.2%, AP_R is 90.2%, the difference is 1.0 percentage point; in the sparse system, AP_L is 88.0%, AP_R is 88.5%, the difference is only 0.5 percentage point. This consistent class balance shows that the prototype alignment mechanism of the method successfully maintains the discriminant boundary between L-type and R-type molecules in the cross-domain migration process, avoiding the class confusion problem that may be caused by global feature alignment. As a comparison, the CMT method shows serious class imbalance on the sparse system (AP_L is 83.6%, AP_R is 88.1%, the gap is 4.5 percentage points), which means that the model has a systematic bias in recognizing the two chiral configurations, which may lead to incorrect statistics of the molecular chiral organization pattern in practical applications.

[0211] In terms of computational efficiency, the method has good training and inference efficiency. In the training stage, the backbone network is initialized with ImageNet pre-trained weights, and the domain adaptive joint training takes about 0.8 seconds per iteration under the configuration of input image size 256x256 pixels and batch size 256. The total training of 4000 iterations takes about 53 minutes, and the peak GPU memory usage is about 10GB. This training efficiency can be smoothly realized on mainstream research-level GPUs such as NVIDIA RTX 5090, A100, etc. In the inference stage, since the domain adaptive component does not participate in the calculation, the model is simplified to the standard Faster R-CNN detection process. The inference time of a single 256x256 pixel STM image is only about 0.05 seconds, corresponding to a processing speed of about 20 frames / second, fully meeting the real-time analysis requirements. Taking an STM image containing 200 molecules as an example, the complete inference process takes about 0.2 seconds, which is about 6000 times faster than manual annotation (usually takes 20-30 minutes).

[0212] In terms of practicality, the method has the following advantages. First, cross-condition robustness: the method has good robustness to imaging condition changes, and researchers do not need to retrain the model for each new experimental condition, greatly reducing the use threshold of the method. As long as the labeled data is accumulated and the model is trained under a set of standard conditions, it can be applied to image analysis under different bias, current, probe state, etc. Second, no target domain labeling: the method uses an unsupervised domain adaptation strategy, and the target domain image does not need to be labeled during training. It only needs to provide unlabeled images for domain adaptation training. This greatly reduces the workload of data preparation, making the method more easily applied. Third, plug and play: the trained model can be directly applied to new target domain images without additional fine-tuning or adaptation steps. Fourth, scalability: the technical framework of the method has good scalability and can be extended to other molecular systems and other microscopic imaging modalities (such as AFM, SEM) image analysis tasks.

[0213] As shown in Figure 1 , the complete network architecture of FPDA-FRCNN is shown. The figure includes: source domain and target domain image input, shared ConvNeXt-FPN backbone network, frequency domain decoupling alignment (including FFT decomposition, low frequency discriminator, high frequency discriminator, GRL), prototype OT alignment module (including source domain prototype, target domain pseudo prototype, cost matrix, Sinkhorn algorithm), region proposal network (RPN), detection head (RoI Head). The data flow and connection relationship between each component are clearly marked. The figure also shows three loss terms in the training stage (detection loss , frequency domain adversarial loss , prototype alignment loss the calculated position and backpropagated path of the source domain data.

[0214] Embodiment 10,

[0215] This embodiment is a further limitation of embodiments 1-8, applying the method described in embodiments 1-8 in a specific practical use.

[0216] 1. Cross-condition chiral molecule detection on F-HPB dense packing system.

[0217] This embodiment describes in detail the complete process and experimental results of applying the present application for cross-condition chiral molecule detection on a fluorinated hexakis-2,6-dimethylphenylbenzene (F-HPB) dense packing system.

[0218] Experimental data was collected on an ultrahigh vacuum low-temperature scanning tunneling microscopy system. The key parameters of the experimental environment are as follows: the base vacuum is better than 1.0 x 10-10 Pa, ensuring the cleanliness of the sample surface and the stability of the molecular structure; the sample temperature is maintained at 4.4 K (liquid helium temperature), suppressing the thermal motion and diffusion of the molecules, and obtaining stable high-resolution images; a commercial Omicron low-temperature STM system is used for data collection.

[0219] The sample preparation process is as follows: first, the Ag(111) single crystal substrate is subjected to standard cleaning treatment, including multiple argon ion sputtering (energy 1 keV, time 30 minutes) and annealing cycles (temperature 800 K, time 10 minutes) to obtain an atomically flat clean surface. Subsequently, F-HPB molecules are deposited on the room temperature substrate surface by a Knudsen evaporation source, with the evaporation temperature set to 180°C and the deposition rate controlled at about 0.1 monolayer / min, and the deposition time is about 5 minutes, finally obtaining a sub-monolayer coverage of molecular thin film. After deposition, the sample is cooled to 4.4 K for STM imaging.

[0220] The source domain data is collected under standard imaging conditions, with the main parameters being: bias voltage -0.8 V, tunneling current 100 pA, scanning range 50 nm x 50 nm, image resolution 512 x 512 pixels. Under this condition, each F-HPB molecule occupies an area of about 50 x 50 pixels, the six-fold symmetric structure of the molecule is clearly distinguishable, and the chiral difference (clockwise / counterclockwise arrangement of the six protrusions) of L-type and R-type can be accurately identified. The source domain data set contains a total of 150 STM images, each containing about 50-100 molecules, a total of about 12000 molecular instances.

[0221] ​The annotation work was completed by two experienced STM researchers independently, and a consensus mechanism was used to ensure the quality of annotation. For each molecule, the annotation content includes the boundary box coordinates (left upper corner x, y coordinates, width and height) and the chirality category (L type or R type). The annotation consistency rate of the two annotators is about 97%, and for inconsistent annotations, a consensus is reached through discussion. The annotation results are stored in COCO format, which is compatible with the target detection framework. The source domain data is divided into training set and validation set in the ratio of 8:2.

[0222] The target domain data covers different imaging conditions, including: bias voltage range -1.5V to +0.5V (different bias voltages detect different molecular orbits), tunneling current setting value 20 pA to 300 pA (different currents correspond to different resolution and contrast), and different probe states (obtained by natural drift during the scanning process). These changes result in target domain images that present different overall brightness, contrast and texture features from the source domain. The molecular resolution of the target domain image is relatively low, with each molecule occupying an area of about 30x30 pixels. The target domain data set contains a total of 100 STM images, about 8000 molecular instances. The target domain data is strictly kept unlabeled during the training process, and annotation is only used for final performance evaluation.

[0223] The network parameter settings are as follows. The backbone network uses the ConvNeXt-Tiny architecture, initialized with ImageNet-1K pre-trained weights. The feature pyramid network (FPN) is constructed from the outputs of the four stages of ConvNeXt to build a multi-scale feature representation, with the output channel dimension unified to 256. The cutoff frequency threshold for frequency domain decomposition is , where corresponds to the four pyramid levels. The frequency domain discriminator uses a three-layer fully convolutional architecture (128-64-1 channels), with the low-frequency GRL scaling coefficient and the high-frequency GRL scaling coefficient .

[0224] Prototype update momentum , Sinkhorn algorithm entropy regularization parameter , 100 iterations. The pseudo-label confidence threshold initial value is , linearly decreased to 0.5 according to . The RPN anchor box aspect ratio is {0.8, 1.0, 1.25}, and the IoU threshold for NMS is 0.5. The detection head outputs 3 classes (background, L type, R type), and the RoI Align output size is 7x7.

[0225] The training configuration is as follows. The optimizer uses SGD, the learning rate is , the momentum is 0.9, and the weight decay is . Total training iterations 4000, batch size 256 (source domain 128 + target domain 128). Three-stage curriculum training: first stage (0-800 times) only source domain supervised training; second stage (800-2400 times) from 0 to 1.0; third stage (2400-4000 times) from 0 to 1.0. Loss weights .

[0226] The experiment was carried out on a workstation equipped with an NVIDIA RTX 5090 GPU (24GB video memory). A single complete training took about 53 minutes, and the peak GPU memory occupancy was about 10GB.

[0227] The experimental results are as follows. On the target domain test set, the present application achieved a detection accuracy of 92.9% mAP@50. Compared with the comparative method: the Source Only baseline (trained only using source domain data without domain adaptation) has an mAP@50 of 83.9%, and the present application has a relative improvement of 9.0 percentage points; the DiffAlign method has an mAP@50 of 88.6%, and the present application has a relative improvement of 4.3 percentage points; the CAT method has an mAP@50 of 87.5%, and the present application has a relative improvement of 5.4 percentage points; the CMT method has an mAP@50 of 89.1%, and the present application has a relative improvement of 3.8 percentage points.

[0228] Under more stringent evaluation indicators, the present application also performs excellently. The mAP@75 (strict positioning accuracy) reaches 74.6%, leading the suboptimal method CMT (70.4%) by 4.2 percentage points. The COCO mAP@[0.5:0.95] (multi-threshold comprehensive indicator) reaches 61.2%, leading CMT (56.1%) by 5.1 percentage points. These results show that the present application not only accurately detects the presence of molecules, but also accurately regresses the bounding box position of molecules.

[0229] In terms of class balance, the present application exhibits excellent performance. AP_L (L-type molecule detection accuracy) is 91.8%, and AP_R (R-type molecule detection accuracy) is 92.7%, with a difference of only 0.9 percentage points. As a comparison, the CMT method has an AP_L of 88.3% and an AP_R of 89.9%, with a difference of 1.6 percentage points; the Source Only baseline has an AP_L of 82.5% and an AP_R of 85.3%, with a difference of 2.8 percentage points. The class balance of the present application is significantly better than that of the comparative methods, indicating that the prototype alignment mechanism effectively maintains the discriminative boundary between L-type and R-type molecules.

[0230] In terms of inference efficiency, the inference of a single 256x256 pixel image takes about 0.05 seconds, corresponding to a processing speed of about 20 frames / second. For a typical STM image containing about 80 molecules, the complete detection procedure takes about 0.1 seconds, which is about 12000 times more efficient than manual annotation (about 20 minutes).

Claims

1. A method for cross-domain detection of chiral molecules in STM images based on frequency domain prototype domain alignment, characterized in that, include: S1. Data preparation and preprocessing: Acquire STM source domain images and target domain images of chiral molecules and perform preprocessing. S2. Construct a multi-scale feature extraction network: In the network, source domain images and target domain images are subjected to hierarchical feature extraction through a backbone network with shared weights, and multi-scale features are constructed through a feature pyramid network. S3. Frequency Domain Aware Multi-Scale Feature Alignment: Multi-scale features are decomposed into low-frequency and high-frequency components using Fourier transform. A frequency domain discriminator is then used to achieve adversarial alignment between the low-frequency and high-frequency components, yielding a frequency domain adversarial loss. ; S4. Prototype Alignment Based on Optimal Transmission: Construct a prototype optimal transmission alignment module. In this module, perform distribution alignment of chiral molecule L-type and R-type class-level features between the source and target domain images to obtain the prototype alignment loss. ; S5. Region Proposal and Object Detection: Construct a Region Proposal Network (RPN), inputting multi-scale features into the RPN to obtain candidate regions. Within the selected regions, a detection head is used for localization and classification, outputting bounding boxes and their corresponding probabilities of belonging to background, L-shaped, or R-shaped classes, and obtaining the detection loss. ; S6. Three-stage course training strategy: The multi-scale feature extraction network is trained in three stages based on frequency domain adversarial loss, prototype alignment loss and detection loss to obtain a qualified multi-scale feature extraction network. S7. Inference and Detection Output: Input the preprocessed target domain STM image, use a trained multi-scale feature extraction network to extract features from the STM image, obtain multi-scale features, repeat step S5, and perform STM image chiral molecule detection.

2. The method for cross-domain detection of chiral molecules in STM images based on frequency domain prototype domain alignment according to claim 1, characterized in that, The source domain data consists of high-quality STM images acquired under standard imaging conditions, with complete bounding boxes and chiral class annotations; the target domain data consists of images acquired under different imaging conditions, which remain unlabeled during the training phase and are only annotated during the evaluation phase.

3. The method for cross-domain detection of chiral molecules in STM images based on frequency domain prototype domain alignment according to claim 2, characterized in that, The preprocessing specifically involves: adjusting all input images to a uniform pixel size; normalizing the images; and applying data augmentation strategies to increase the diversity of the training data.

4. The method for cross-domain detection of chiral molecules in STM images based on frequency domain prototype domain alignment according to claim 3, characterized in that, The backbone network is an improvement on the ConvNeXt model, specifically: First, a 7×7 large-kernel depth-separable convolution is used to replace the traditional 3×3 convolution; Second, layer normalization is adopted instead of batch normalization; Third, the GELU activation function is used instead of the ReLU activation function; Fourth, the input image is downsampled using 4×4 convolutions in the stem layer of the ConvNeXt network, instead of traditional pooling layers.

5. The method for cross-domain detection of chiral molecules in STM images based on frequency domain prototype domain alignment according to claim 4, characterized in that, The multi-scale features are decomposed into low-frequency and high-frequency components using Fourier transform. Specifically, for the feature pyramid network layers... Feature map on First, apply a two-dimensional discrete Fourier transform independently to each channel to obtain the frequency domain representation: Subsequently, the frequency domain representation is decomposed into low-frequency and high-frequency components through frequency-selective filtering, and a low-pass filter mask is defined. ,in The cutoff frequency threshold is used for low-frequency components. and high frequency components The extraction formula is: ; ;in This represents element-wise multiplication; Frequency domain adversarial loss through Obtain, among which, Weighing the contributions of each level of the pyramid, Balancing low-frequency and high-frequency adaptation, Indicates the total number of levels in the pyramid network. For low-frequency alignment loss, This represents the high-frequency alignment loss.

6. The method for cross-domain detection of chiral molecules in STM images based on frequency domain prototype domain alignment according to claim 5, characterized in that, Prototype alignment loss through Obtain, among which, , The Softmax function will be a learnable parameter vector. Mapped to positive weights that sum to 1. , Indicates at the level From the source domain Class prototype transferred to the target domain The quality ratio of class prototypes, ,in, This represents the square of the Euclidean distance. Represents the source domain. Class prototypes in hierarchy prototype vector, Indicates the target domain. class pseudo-prototype at the hierarchy The pseudo-prototype vector.

7. The method for cross-domain detection of chiral molecules in STM images based on frequency domain prototype domain alignment according to claim 6, characterized in that, The Region Proposal Network (RPN) operates independently at each pyramid level and is responsible for generating candidate regions that may contain chiral molecules. At each spatial location, the RPN performs classification and regression prediction through two parallel 1×1 convolutional branches. The classification branch outputs a binary score for each anchor box, indicating whether it belongs to the foreground or the background. The regression branch outputs the position offset of the anchor box relative to the true target box; The detection head performs fine classification and localization on the filtered region proposals. First, it extracts fixed-size region features from the corresponding pyramid level through the RoI Align operation. The extracted region features are flattened and then input into two fully connected networks for encoding. The first fully connected layer uses the ReLU activation function; the second fully connected layer uses the ReLU activation function. These two fully connected networks encode the region features for subsequent classification and regression. The final classification head and regression head make predictions based on the encoded features. The classification head outputs the probability distribution of the three classes: background, L-shape, and R-shape, which is the posterior probability of each proposal belonging to each class. The regression head outputs the refined offset of the bounding box, which is used to adjust the region proposal to a more accurate target location.

8. The method for cross-domain detection of chiral molecules in STM images based on frequency domain prototype domain alignment according to claim 7, characterized in that, Total detection loss ,in, and These are the weighting coefficients. For RPN loss, To detect head damage, ； in For the first The predicted foreground probability of each anchor box. To give it its true label, For binary cross-entropy loss; For the predicted position offset, This is the actual position offset. Smooth L1 loss; and As the normalization factor, This is the balance coefficient; ,in, To detect the cross-entropy loss of the first three classifications, The loss is for Smooth L1 regression.

9. The method for cross-domain detection of chiral molecules in STM images based on frequency domain prototype domain alignment according to claim 8, characterized in that, The three-stage course training strategy is as follows: The first stage is the source domain pre-training stage, corresponding to 20% of the training iterations. In this stage, standard supervised object detection training is performed only on the source domain data to optimize the detection loss. ; The second stage is the frequency domain alignment introduction stage, corresponding to 20%-60% of the training iterations. In this stage, frequency domain adversarial alignment is gradually introduced, and the training objective becomes: Among them, the adversarial loss weight It increases linearly from 0 to the final value of 1.0; The third stage is the prototype alignment activation stage, corresponding to 60%-100% of the training iterations. In this stage, the prototype alignment loss based on optimal transmission is activated. The complete training objective is: ； Among them, the prototype alignment loss weight It gradually increases from 0 to the final value of 1.

0.

10. The method for cross-domain detection of chiral molecules in STM images based on frequency domain prototype domain alignment according to claim 9, characterized in that, During inference and detection output, category-level NMS is applied to all non-background category detection results to remove highly overlapping redundant detections. The IoU threshold of NMS is set to 0.

5. Finally, detection results with a confidence score higher than 0.5 are retained as output. The output format includes: bounding box coordinates, category label and confidence score for each detection box.

Images