A chemical structure recognition method based on visual transformer
By combining visual transformers and adapter networks, the problems of diversity in chemical structure recognition and preservation of molecular conformation in traditional methods are solved, achieving efficient and accurate chemical structure recognition, which is applicable to drug discovery and materials science.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANGXIN BIOTECHNOLOGY CO LTD
- Filing Date
- 2026-03-20
- Publication Date
- 2026-07-31
AI Technical Summary
Traditional chemical structure identification methods cannot effectively handle the diversity of molecular descriptions in scientific literature, and deep learning models cannot accurately preserve molecular conformation and orientation details, leading to delays and errors in the drug discovery process.
A chemical structure recognition method based on visual transformers is adopted. Basic features are extracted through deep convolutional neural networks, and features are enhanced using adapters. Combined with FPN-PAN fusion network and uncertainty minimization selection strategy, high-quality fusion images are screened, and the bounding boxes and label types of molecular structures are iteratively optimized through decoder to finally obtain self-assembled chemical molecular maps.
It improves the accuracy and efficiency of chemical structure identification, reduces bond angle prediction errors, ensures the rationality and integrity of molecular structures, and enhances the ability to identify complex molecules.
Smart Images
Figure CN121884069B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of automatic identification technology, specifically to a chemical structure identification method based on a visual transformer. Background Technology
[0002] Accurate identification and reconstruction of chemical structures from molecular images is crucial in cheminformatics, with applications including drug discovery, materials science, and patent analysis. However, traditional rule-based and machine learning-based methods often encounter overlapping atoms, blurred bonds, and low-quality image data.
[0003] Accurately identifying molecular structures from graphical descriptions remains one of the most challenging aspects of computational chemistry and drug discovery. Despite significant advancements in digital technologies, the vast majority of chemical and biological analytical data remains presented as static images or tables, making computational analysis impossible. This lack of machine-readable data creates a major bottleneck in the modern drug discovery pipeline, as researchers must manually manage molecular structures from publications—a time-consuming process that can lead to significant delays and profound impacts on patient outcomes.
[0004] Traditional chemical structure identification methods are rule-based systems that have long been the cornerstone of molecular data extraction. However, these methods are inherently limited because they rely on predefined heuristics and cannot handle the vast diversity of molecular descriptions found in the scientific literature.
[0005] Some deep learning models, such as decamer, img2mol, and ChemVLM, treat chemical structure identification as an image-to-capture or image-to-sequence task. These models can only predict the SMILES sequence of a chemical structure, completely disregarding coordinate-related information, thus losing crucial molecular conformational and orientational details from the original image. However, this orientation-related structural information is vital for medicinal chemists in the early stages of drug discovery, particularly in designing homologous sequences and building quantitative structure-activity relationship models.
[0006] Chinese invention patent CN118781326A discloses a target detection method based on a depth-aware RGB multi-scale fusion network. The method includes the following steps: Step 1: In the RGB branch and the depth branch, feature downsampling is performed on the RGB image and the depth image to obtain RGB features and depth features respectively; After downsampling in the RGB branch, a depth-aware RGB feature optimization module is inserted to enhance the RGB features; Step 2: The RGB features and depth features are fused based on a multi-scale attention enhancement fusion module; Step 3: A dual-attention guidance module uses deeper features to guide the fused features for further filtering; Step 4: The sigmoid function is used to optimize the detection of salient regions to obtain the final detected target.
[0007] In the above scheme, although basic features are collected first and then feature fusion is performed, the network does not limit the collected basic features, resulting in insufficient accuracy. Furthermore, the fused multi-scale feature images are not further filtered to select high-quality images that meet the requirements, making the decoding process more complex and the results less accurate. The lack of a validity assessment of the output multi-scale images may lead to errors in their validity. Summary of the Invention
[0008] To address the problems existing in the prior art, this invention proposes a chemical structure recognition method based on a visual transformer. Basic features are extracted using a deep convolutional neural network, then refined through three convolutional layers. An adapter further enhances the backbone, customizing pre-trained features for the chemical structure recognition task. After feature extraction, multi-scale feature maps are passed to the encoder and FPN-PAN fusion network to convert them into a series of image representations. A minimum uncertainty selection strategy is used to filter high-quality fusion images that meet preset conditions. Finally, the transformer decoder uses an auxiliary prediction head to iteratively optimize these object queries, predicting and generating bounding boxes and label types for molecular structures. A matching post-processing model and algorithm are then used to self-assemble the bounding boxes and label types of the molecular structures into a chemical molecular map, maximizing the preservation of chemical structure information from the original image.
[0009] The technical solution of the present invention is as follows: On the one hand, this invention proposes a chemical structure recognition method based on a visual transformer, the specific steps of which include: Obtain the chemical structure diagram to be identified, and normalize and preprocess the chemical structure diagram to be identified; A pre-trained visual transformer is used as the backbone feature extraction network to extract the basic features of chemical structure images and generate hierarchical feature maps based on the basic features. An adapter hybrid encoder performs domain adaptation and cross-scale fusion on hierarchical feature maps to generate a fused image; the adapter hybrid encoder includes an adapter module and a feature fusion module. Based on the fusion features of the fused image, a minimum uncertainty selection strategy is adopted to filter out the fusion features that meet the preset conditions and their corresponding fused images. The decoder with an auxiliary prediction head is used to decode the filtered fused image and predict the bounding box coordinates and label types of each molecular structure in the generated fused image. A post-processing model for verifying the rationality of molecular structures is constructed to verify the rationality of the label type and label type name of each molecular structure; the label type of each verified molecular structure is combined with its bounding box and mapped to a chemical structure diagram.
[0010] In a preferred embodiment, the steps of normalizing and preprocessing the chemical structure image specifically include: Enhance the dataset by using predefined r groups to introduce or replace existing substructures; The size and resolution of each chemical structure map to be identified are normalized by rotating, scaling, adding Gaussian noise, and introducing obstacle objects.
[0011] In a preferred embodiment, the adapter module employs an Adapter network to learn and understand knowledge of chemical structure expressions, and merges adapter modules in the same domain based on the domain to which the input image belongs.
[0012] As a preferred embodiment, the uncertainty-minimizing selection strategy specifically includes: Using a preset confidence score, select the top-ranking fusion features from the adapter hybrid encoder as the initial query objects; Based on the initial query object, a minimum selection strategy is adopted to calculate the fusion feature with the lowest prediction uncertainty as the selected fusion feature. The specific calculation formula is as follows:
[0013] In the formula, The probability of selecting the query object. To initialize the query object, To predict the confidence level variance; q i Let be the i-th candidate fusion feature vector generated by the encoder.
[0014] In a preferred embodiment, the decoder with an auxiliary prediction head is divided into several layers, and each layer uses a multi-head self-attention mechanism and cross-scale feature fusion to iteratively refine the target query.
[0015] In a preferred embodiment, the auxiliary prediction head is a lightweight neural network layer attached to each layer of the decoder.
[0016] In a preferred embodiment, the molecular structure includes atoms, charges, and chemical bonds.
[0017] As a preferred implementation, a post-processing model for verifying the rationality of molecular structures is constructed. The specific steps for verifying the rationality of the label type names of each molecular structure are as follows: For unidentifiable atomic boxes, dual-resolution OCR recognition is performed, and they are converted into chemical semantics through regularization and replacement tables before being injected into the molecular structure diagram. For atomic frames that cannot be recognized by OCR, placeholders are first used to construct a molecular skeleton, and then the complete structure is replaced in batches while preserving stereochemical information.
[0018] As a preferred embodiment, the post-processing model for verifying the rationality of molecular structure rationality, specifically the step of verifying the rationality of the label type for each molecular structure, includes: Assigning appropriate connecting atoms to chemical bonds through double-anchor-KDTree bond-atomic matching; Self-healing topological repair includes single-end bond repair, duplicate bond deduplication, and isolated atom attachment. Charge-atom mapping distributes dense charges in steps based on whether charges and atoms overlap and the distance between corner points.
[0019] In a preferred embodiment, the step of assigning suitable connecting atoms to the chemical bond through double-anchor-KDTree bond-atom matching specifically includes: Take the four vertices of each chemical bond frame and divide them into two groups of anchor points along the diagonal, i.e., four anchor points; use the KDTree algorithm to obtain the atom center closest to the four anchor points, and obtain the candidate atom closest to the four anchor points; If there are multiple groups of candidate atoms, they are grouped diagonally, and the sum of the distances from each candidate atom pair to its respective anchor point is calculated. The group with the smaller sum of distances is selected as the final confirmed atom pair.
[0020] The present invention has the following beneficial effects: 1. This invention employs an adapter hybrid encoder to achieve multi-scale feature complementarity: low-level features capture atomic-level edge textures, high-level features extract semantic information of the molecular skeleton, and a cross-attention mechanism dynamically associates cross-scale features; thereby improving atomic positioning accuracy, reducing bond angle prediction errors, and enhancing the ability to recognize complex chemical molecular structures.
[0021] 2. This invention proposes a selection strategy with minimal uncertainty, which filters high-quality query objects by using confidence variance; it achieves dynamic filtering of low-confidence features and adaptive adjustment of the number of queries.
[0022] 3. The adapter in this invention adopts an Adapter network structure, which can learn according to the target structural features of a specific field, thereby limiting the target features to be collected in the backbone network. In this application, the target field is the chemical field, and the target feature to be collected is the molecular structure.
[0023] 4. This invention determines the rationality of chemical molecular structures from four verification dimensions: Atomic valence constraint check: verifying the legality of atomic valence through RDKit; Bond angle rationality check: correcting abnormal bond angles based on geometric topological constraints; Stereochemical matching: checking the consistency between chiral center markers and spatial configuration; Topological integrity verification: ensuring that the molecular diagram has no dangling bonds / isolated nodes.
[0024] 5. This invention uses a decoder with an auxiliary prediction head, combined with multi-head self-attention and cross-scale feature fusion iteration, to optimize and iterate the prediction of the fused image and output the best prediction result.
[0025] 6. This invention introduces 8-anchor-diagonal constraints into chemical bond recognition for the first time through double-anchor-KDTree bond-atom pairing, significantly reducing false connections.
[0026] 7. This invention utilizes self-healing topological repair: fully automatic discovery and repair of isolated atoms, single-ended bonds, and repeating bonds, ensuring the connectivity of the molecular graph.
[0027] 8. This invention uses OCR recognition to directly integrate semantics before structural reconstruction, thereby increasing the ability to recognize complex symbols / abbreviations.
[0028] 9. This invention utilizes placeholder delayed expansion + stereo synchronization: the skeleton is reconstructed first and then replaced in batches, while the placeholders simultaneously retain key orientation and stereo information.
[0029] 10. This invention distributes dense charges in steps based on whether the charge and the atom overlap and the corner distance, thereby improving the accuracy of dense charge positioning. Attached Figure Description
[0030] Figure 1 This is a schematic diagram of the steps of the present invention; Figure 2 This is a schematic diagram of charge-atom mapping. Detailed Implementation
[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0032] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.
[0033] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention.
[0034] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.
[0035] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.
[0036] Example 1: See Figure 1 A chemical structure recognition method based on visual transformers, comprising the following steps: Obtain the chemical structure diagram to be identified, and normalize and preprocess the chemical structure diagram to be identified; In this embodiment, the training data comes from the CHEMBL dataset. RDKit was used to remove invalid molecules and exclude molecules containing rare elements. After this cleaning process, the remaining dataset was augmented by introducing or replacing existing substructures with predefined r groups to generate new data points. Constraints were also applied, limiting the maximum molecular weight to 800 and the number of heavy atoms to 60.
[0037] To create the synthetic dataset, the size and resolution of each image were normalized, and various augmentation techniques were applied, including rotation, scaling, adding Gaussian noise, and introducing obstacle objects such as spheres, triangles, and strings containing numbers. These augmentations were designed to enhance the model's robustness and generalization ability. During dataset preparation, bounding boxes and label annotations, including atoms, bonds, and charges, were dynamically generated. A total of 221,314 images were created for the training and validation sets. The dataset has been released on Google Drive for public access.
[0038] The synthetic dataset was divided into training, validation, and test sets in an 8:1:1 ratio. I2M was trained using the training set, with the encoder initialized from pre-trained weights from RT-DETR. These weights were further refined using an ensemble adapter technique. The primary objective of the training process was to minimize the discrepancy between the predicted chemical maps and their corresponding ground-to-real structures.
[0039] Six publicly available datasets were used: CLEF, JPO, UOB, USPTO, STAKER, and ACS. These datasets cover a wide range of chemical structures, providing a comprehensive evaluation of model capabilities. Data points that could not be read into smiles and converted to 2D representations using RDKit were removed. Despite using the same smile data, the images differ significantly, making the datasets particularly suitable for testing different models.
[0040] A pre-trained visual transformer is used as the backbone feature extraction network to extract the basic features of chemical structure images and generate hierarchical feature maps based on the basic features. In this embodiment, the visual transformer is a deep learning model based on the Transformer architecture; the pre-trained visual transformer is specifically ResNet-50, with the backbone extracting basic features, which are then refined through three convolutional layers (C1, C2, and C3). These layers generate hierarchical feature maps—f1, f2, and f3—that capture multi-scale information, progressively encoding low-level edges, textures, and high-level semantic features, such as atomic and bond patterns.
[0041] An adapter hybrid encoder performs domain adaptation and cross-scale fusion on hierarchical feature maps to generate a fused image; the adapter hybrid encoder includes an adapter module and a feature fusion module. In this embodiment, these adapters serve as effective fine-tuning layers, enabling the model to adapt to pre-trained features specifically designed for chemical structure recognition. The self-attention mechanism in the encoder primarily focuses on high-level features, such as those from C3, capturing rich semantic concepts and relationships between entities. This allows subsequent modules to effectively locate and recognize objects. Low-level features, such as f1 and f2, are excluded from scale-based interactions to avoid redundancy and interference with high-level feature processing.
[0042] The feature fusion module employs the FPN-PAN fusion network, which effectively merges multi-scale information by combining features from different scales (F1, F2, F3) through cross-scale fusion. By integrating these features, the model captures fine-grained details while improving training and inference efficiency. This approach ensures accurate target detection, regardless of their size or location within the image.
[0043] Based on the fusion features of the fused image, a minimum uncertainty selection strategy is adopted to filter out the fusion features that meet the preset conditions and their corresponding fused images. The filtered fused image is decoded using a decoder with an auxiliary prediction head, and the bounding box coordinates and label types of each molecular structure in the generated fused image are predicted. The bounding box of the molecular structure is the spatial coordinate of the molecular structure, and the label type of the molecular structure includes atoms, bonds, and charges. In this embodiment, the transformer decoder employs an iterative query optimization mechanism based on a transformer architecture. This method integrates a query selection strategy that minimizes uncertainty, ensuring higher quality initial object queries and thus producing more accurate predictions. Each layer of the decoder iteratively refines the target query using multi-head self-attention and cross-scale feature fusion. Furthermore, the decoder has auxiliary prediction heads at each layer to generate intermediate predictions, guiding the continuous iterative refinement process. These auxiliary heads facilitate the model's progressive refinement of predictions for both object categories (atoms, bonds, and charges) and bounding boxes. Through intermediate feedback and iterative refinement, the decoder effectively guarantees the accurate and precise reconstruction of molecular structures, capturing the semantic and spatial relationships necessary for structure recognition.
[0044] A post-processing model for verifying the rationality of molecular structures is constructed to verify the rationality of the label type and label type name of each molecular structure; the label type of each verified molecular structure is combined with its bounding box and mapped to a chemical structure diagram.
[0045] In this embodiment, the detected atoms, charges, and bonds are mapped onto a graph structure, where atoms and charges are represented as nodes, and bonds are represented as edges. Charges are assigned to the nearest atom based on the predicted center of the bounding box, while the relationships between atoms and bonds are determined by the relative positions of their respective box centers. To ensure chemical validity, we validated the valence of each atom and constructed a chemically sound graph.
[0046] In a preferred embodiment of this invention, the steps of normalizing and preprocessing the chemical structure image specifically include: The RDKit toolkit is used to remove invalid molecules and exclude molecules containing rare elements, generating a valid dataset. Use predefined r groups to introduce or replace existing substructures to enhance the effectiveness of the dataset; The size and resolution of each chemical structure map to be identified are normalized by rotating, scaling, adding Gaussian noise, and introducing obstacle objects.
[0047] In a preferred embodiment of this invention, the adapter module employs an Adapter network to learn and understand knowledge of chemical structure expressions, and merges adapter modules of the same domain based on the domain to which the input image belongs.
[0048] In this embodiment, the Adapter network, as the core module of domain adaptation, achieves efficient learning and cross-task transfer of chemical features through a lightweight pluggable design.
[0049] As a preferred embodiment of this example, the uncertainty minimization selection strategy is specifically as follows: Using a preset confidence score, select the top-ranking fusion features from the adapter hybrid encoder as the initial query objects; Based on the initial query object, a minimum selection strategy is adopted to calculate the fusion feature with the lowest prediction uncertainty as the selected fusion feature. The specific calculation formula is as follows:
[0050] In the formula, The probability of selecting the query object. To initialize the query object, To predict the confidence level variance; q i Let be the i-th candidate fusion feature vector generated by the encoder.
[0051] In this embodiment, the uncertainty-minimum selection strategy uses confidence scores to select the top K features from the encoder to initialize the object query. By combining this robust approach, the decoder receives high-quality queries, significantly improving its performance in classification and localization tasks.
[0052] In a preferred embodiment of this invention, the decoder with an auxiliary prediction head is divided into several layers. Each layer uses a multi-head self-attention mechanism and cross-scale feature fusion to iteratively refine the target query. Moreover, each layer contains an auxiliary prediction head to generate intermediate predictions to guide the continuous iterative refinement process.
[0053] In this embodiment, the enhanced image and the original image are input into the encoder, and the decoder outputs bounding boxes (Box(x, y, w, h)) and label types (atom, bond, charge). If misidentification occurs, such as labeling an "O" atom as "S", manual correction is used to adjust the box coordinates and label type. These corrected samples and their corresponding images are then augmented using data augmentation techniques to create more pairs of images and samples, which are then added back to the training dataset. Through continuous iteration, the model's ability to resolve molecular structures is improved, ultimately enabling this method to reliably interpret new molecular images with minimal user intervention.
[0054] In a preferred embodiment of this invention, the auxiliary prediction head is a lightweight neural network layer attached to each layer of the decoder.
[0055] In a preferred embodiment of this invention, the molecular structure includes atoms, charges, and chemical bonds.
[0056] In this embodiment, the label type of the molecular structure includes atomic type (C, H, O, N, S, P, F, Br, Cl, I, etc.), chemical bond type (single bond, double bond, triple bond, aromatic bond, wedge bond, etc.) and charge type (integer from -3 to 3).
[0057] In a preferred embodiment of this invention, the step of constructing a post-processing model for verifying the rationality of molecular structure rationality, and verifying the rationality of the label type names of each molecular structure, specifically includes: Dual-resolution OCR recognition is performed on sub-images defined by unidentifiable atomic boxes. The images are then converted into chemical semantics through regularization and replacement tables, and then injected into the molecular structure diagram. For atomic frames that cannot be recognized by OCR, placeholders are first used to construct a molecular skeleton, and then the complete structure is replaced in batches while preserving stereochemical information.
[0058] In this embodiment, the specific steps of dual-resolution OCR recognition are as follows: Perform dual-resolution OCR on atomic boxes categorized as 'other', and the one with the higher score will be used. Use regular expressions and substitution tables to normalize text (NO2, SO2, etc.); The results are stored in placeholder_atoms and then expanded using molExpanding after the RDKit graph is built.
[0059] In this embodiment, the specific operations corresponding to the step of first constructing a molecular skeleton using placeholders for atomic frames that cannot be recognized by OCR, and then replacing them in batches with complete structures while simultaneously preserving stereochemical information are as follows: Atom addition: Unknown symbols are first placed as * and written to a custom atomLabel; Key addition: Batch writing based on the corrected bonds; FormalCharge: Applied according to c2a; 3D Conformer: Press (x, y) / 100 generates 2D coordinates; Stereo alignment: After writing the wedge and dashed lines using bondWithdirct, call AssignStereochemistryFrom3D.
[0060] As a preferred embodiment of this example, the step of constructing a post-processing model for verifying the rationality of molecular structure rationality, and verifying the rationality of the label type for each molecular structure, specifically includes: Verification was conducted using double-anchor-KDTree bond-atomic matching to ensure that large bond frames were not mistakenly connected by adjacent small bond frames. Self-healing topological repair includes single-end bond repair, duplicate bond deduplication, and isolated atom attachment. Charge-atom mapping: Step-by-step distribution of dense charges based on whether charges and atoms overlap and corner distance; In this embodiment, the self-healing topology repair step specifically includes: Single-ended bond repair: Automatically detects unconnected atoms and completes the structure by searching with KDTree or dynamically inserting new atoms (default C); Key deduplication: Based on confidence level, retain the most reliable key and remove redundancies; Isolated atom connection: Automatically connects isolated atoms based on geometric thresholds (bond width statistics) to eliminate dangling nodes; a two-level strategy is used to connect isolated atoms: A. Border overlap → directly connect single bonds; B. Minimum corner distance < minimum bond width → connect single bonds.
[0061] The charge-atom mapping steps are as follows: First, determine whether the charge and the atom overlap: if they directly overlap, they are bound together; if there is no overlap, find the nearest center using a KDtree query; if the centers conflict, verify by the corner distance.
[0062] In a preferred embodiment of this invention, the step of assigning suitable connecting atoms to chemical bonds through double-anchor-KDTree bond-atomic matching specifically includes: Take the four vertices of each chemical bond frame and divide them into two groups of anchor points along the diagonal, i.e., four anchor points; use the KDTree algorithm to obtain the atom center closest to the four anchor points, and obtain the candidate atom closest to the four anchor points; If there are multiple groups of candidate atoms, they are grouped diagonally, and the sum of the distances from each candidate atom pair to its respective anchor point is calculated. The group with the smaller sum of distances is selected as the final confirmed atom pair.
[0063] In this embodiment, the charge-atom mapping step adopts the nearest neighbor algorithm to calculate the coordinates of the charge center and the atom center, and assign the charge to the nearest atom.
[0064] like Figure 2 As shown, the KDTree algorithm is used to obtain the nearest atom centers to the four anchor points (the four points a, b, c, d of the bond frame), and the four candidate atoms closest to the four anchor points (such as 2, 3, 4, 6) are obtained, while the atom centers that are farther away are excluded (such as 1, 5, 7). Calculate the distance from the candidate atom to the nearest anchor point, group them by the two diagonals of the bond frame and sum them separately, compare the sum of the distances between the atom centers and anchor points of the two groups, and select the group of candidate atoms with the smaller sum of distances as the connecting atoms at both ends of the bond.
[0065] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.
Claims
1. A chemical structure recognition method based on visual transformers, characterized in that, The specific steps include: Obtain the chemical structure diagram to be identified, and normalize and preprocess the chemical structure diagram to be identified; A pre-trained visual transformer is used as the backbone feature extraction network to extract the basic features of chemical structure images and generate hierarchical feature maps based on the basic features. An adapter hybrid encoder performs domain adaptation and cross-scale fusion on hierarchical feature maps to generate a fused image; the adapter hybrid encoder includes an adapter module and a feature fusion module. Based on the fusion features of the fused image, a minimum uncertainty selection strategy is adopted to filter out the fusion features that meet the preset conditions and their corresponding fused images. The filtered fused image is decoded using a decoder with an auxiliary prediction head, and the bounding box coordinates and label types of each molecular structure in the generated fused image are predicted; the molecular structure includes atoms, charges, and chemical bonds. A post-processing model for verifying the rationality of molecular structures is constructed to verify the rationality of the label type and label type name of each molecular structure; the label type of each verified molecular structure is combined with its bounding box and mapped to a chemical structure diagram. The post-processing model for verifying the rationality of molecular structures, specifically the step of verifying the rationality of the label types and label type names for each molecular structure, includes the following: Assigning appropriate connecting atoms to chemical bonds through double-anchor-KDTree bond-atomic matching; Self-healing topological repair includes single-end bond repair, duplicate bond deduplication, and isolated atom attachment. Charge-atom mapping distributes dense charges in steps based on whether charges and atoms overlap and the corner distance; The specific steps for assigning suitable connecting atoms to chemical bonds through double-anchor-KDTree bond-atomic matching are as follows: Take the four vertices of each chemical bond frame and divide them into two groups of anchor points along the diagonal, i.e., four anchor points; use the KDTree algorithm to obtain the atom center closest to the four anchor points, and obtain the candidate atom closest to the four anchor points; If there are multiple groups of candidate atoms, they are grouped diagonally, and the sum of the distances from each candidate atom pair to its respective anchor point is calculated. The group with the smaller sum of distances is selected as the final confirmed atom pair.
2. The chemical structure recognition method based on a visual transformer according to claim 1, characterized in that, The normalization and preprocessing steps for the chemical structure diagram to be identified specifically include: Enhance the dataset by using predefined r groups to introduce or replace existing substructures; The size and resolution of each chemical structure map to be identified are normalized by rotating, scaling, adding Gaussian noise, and introducing obstacle objects.
3. The chemical structure recognition method based on a visual transformer according to claim 1, characterized in that, The adapter module uses an Adapter network to learn and understand the knowledge of chemical structure expressions, and merges adapter modules in the same field based on the field to which the input image belongs.
4. The chemical structure recognition method based on a visual transformer according to claim 1, characterized in that, The uncertainty-minimizing selection strategy is specifically as follows: Using a preset confidence score, select the top-ranking fusion features from the adapter hybrid encoder as the initial query objects; Based on the initial query object, a minimum selection strategy is adopted to calculate the fusion feature with the lowest prediction uncertainty as the selected fusion feature. The specific calculation formula is as follows: In the formula, The probability of selecting the query object. To initialize the query object, To predict the confidence level variance; q i Let be the i-th candidate fusion feature vector generated by the encoder.
5. The chemical structure recognition method based on a visual transformer according to claim 1, characterized in that, The decoder with an auxiliary prediction head is divided into several layers. Each layer uses a multi-head self-attention mechanism and cross-scale feature fusion to iteratively refine the target query.
6. The chemical structure recognition method based on a visual transformer according to claim 1, characterized in that, The auxiliary prediction head is a lightweight neural network layer that is appended to each layer of the decoder.
7. The chemical structure recognition method based on a visual transformer according to claim 1, characterized in that, The post-processing model for verifying the rationality of molecular structures, specifically the step of verifying the rationality of the label type names of each molecular structure, includes the following: For unidentifiable atomic boxes, dual-resolution OCR recognition is performed, and they are converted into chemical semantics through regularization and replacement tables before being injected into the molecular structure diagram. For atomic frames that cannot be recognized by OCR, placeholders are first used to construct a molecular skeleton, and then the complete structure is replaced in batches while preserving stereochemical information.