Molecular structure identification method and device, electronic equipment and storage medium

Abstract

The application provides a molecular structure recognition method and device, electronic equipment and storage medium, wherein the method comprises: acquiring a molecular image; initializing an empty angle set; in the case of decoding a branch angle based on image features of the molecular image and decoding to the branch angle for the first time, storing the branch angle in the angle set; taking out a branch angle from the angle set, taking the branch angle as a guide, decoding a molecular structure under the branch angle based on image features of the molecular image, and updating the angle set based on a new branch angle obtained by decoding for decoding of a molecular structure under a next branch angle until the angle set is empty; and determining a molecular structure corresponding to the molecular image based on decoding results under each branch angle. The method, device, electronic equipment and storage medium provided by the application improve the reliability and accuracy of molecular structure decoding.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a molecular structure recognition method, apparatus, electronic device, and storage medium. Background Technology

[0002] Chemical molecular structure recognition can be widely applied in fields including pharmaceutical research and development, human-computer interaction, biochemistry, education, and organic synthesis.

[0003] Currently, research on handwritten chemical molecular structure recognition relies excessively on rule-based post-processing methods, which simplifies the complexity of molecular structure recognition itself, making it difficult to analyze and recognize complex handwritten molecular layouts under existing schemes. Summary of the Invention

[0004] This invention provides a molecular structure recognition method, apparatus, electronic device, and storage medium to address the shortcomings of existing technologies in recognizing complex handwritten molecular structures.

[0005] This invention provides a molecular structure identification method, comprising:

[0006] Acquire molecular images;

[0007] Initialize an empty angle set. When decoding the molecular structure based on the image features of the molecular image and first decoding a branch angle, store the branch angle in the angle set. Take a branch angle from the angle set, and using the branch angle as a guide, decode the molecular structure under the branch angle based on the image features of the molecular image. Update the angle set based on the new branch angle obtained from the decoding for decoding the molecular structure under the next branch angle, until the angle set is empty.

[0008] Based on the decoding results from each branch angle, the molecular structure corresponding to the molecular image is determined.

[0009] According to a molecular structure identification method provided by the present invention, updating the angle set based on new branch angles obtained through decoding includes:

[0010] The new branch angle is compared with each existing branch angle in the angle set to detect chemical bonds.

[0011] The angle set is updated based on the detection results of the chemical bond detection.

[0012] According to a molecular structure identification method provided by the present invention, updating the angle set based on the detection results of the chemical bond detection includes:

[0013] If the detection result indicates that no chemical bonds exist, the new branch angle is stored in the angle set;

[0014] If the detection result indicates the presence of a chemical bond, the branch angle that forms a chemical bond with the new branch angle will be removed from the angle set.

[0015] According to a molecular structure recognition method provided by the present invention, the step of decoding the molecular structure at the branch angle based on the image features of the molecular image, guided by the branch angle, includes:

[0016] Based on the decoding features of the branch angle and the decoding state of the previous decoding moment under the branch angle, the visual context features of the current decoding moment under the branch angle are determined.

[0017] Based on the visual context features, the molecular structure of the image features is decoded at the current decoding moment to obtain the decoding state at the current decoding moment, and the current decoding moment is returned as the previous decoding moment for decoding until the decoding at the branch angle ends.

[0018] According to a molecular structure recognition method provided by the present invention, the step of extracting a branch angle from an angle set, using the branch angle as a guide, decoding the molecular structure at the branch angle based on the image features of the molecular image, and updating the angle set based on the decoded new branch angle for decoding the molecular structure at the next branch angle, until the angle set is empty, includes:

[0019] Based on the recognition model, a branch angle is taken from the angle set. Guided by the branch angle, the molecular structure under the branch angle is decoded based on the image features of the molecular image. The angle set is updated based on the new branch angle obtained by decoding so that the molecular structure under the next branch angle can be decoded, until the angle set is empty.

[0020] The recognition model is trained based on sample images and the molecular structure labels corresponding to the sample images;

[0021] The molecular structure tag is obtained by connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image to form a molecular structure diagram and then performing graph traversal.

[0022] According to a molecular structure identification method provided by the present invention, the step of determining the molecular structure tag includes:

[0023] Connect the atomic groups and chemical bonds in the molecular formula corresponding to the sample image to form a molecular structure diagram;

[0024] The molecular structure diagram is traversed, and molecular structure tags are generated based on the atomic groups, chemical bonds, angles, nested symbols, and reconnection markers obtained from the traversal.

[0025] According to a molecular structure recognition method provided by the present invention, the training steps of the recognition model include:

[0026] Based on the initial model, molecular structure recognition is performed on the sample image to obtain the chemical bond detection results between the sample branch angle decoded during the molecular structure recognition process and the existing branch angles in the sample angle set, as well as the structure recognition results of the sample image.

[0027] Based on the structure recognition results and the molecular structure tag, as well as the chemical bond detection and the reconnection markers in the molecular structure tag, the initial model is iterated to obtain the recognition model.

[0028] The present invention also provides a molecular structure recognition device, comprising:

[0029] Acquisition unit, used to acquire molecular images;

[0030] The identification unit is used to initialize an empty angle set. When decoding the molecular structure based on the image features of the molecular image and decoding a branch angle for the first time, the branch angle is stored in the angle set. A branch angle is taken out from the angle set, and the molecular structure under the branch angle is decoded based on the image features of the molecular image, guided by the branch angle. The angle set is updated based on the new branch angle obtained by decoding for decoding the molecular structure under the next branch angle, until the angle set is empty.

[0031] The output unit is used to determine the molecular structure corresponding to the molecular image based on the decoding results at each branch angle.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the molecular structure recognition method as described above.

[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the molecular structure identification method as described above.

[0034] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the molecular structure recognition method as described above.

[0035] The molecular structure recognition method, device, electronic device, and storage medium provided by this invention add a mechanism for decoding various branch angles in chemical molecular structures and maintaining branch angles to be explored, and use the branch angles in chemical molecular structures as guiding conditions during decoding, so as to enrich the information of molecular structure decoding, improve the reliability of molecular structure decoding, and improve the decoding accuracy for complex chemical molecular structures. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is one of the flowcharts illustrating the molecular structure recognition method provided by the present invention;

[0038] Figure 2 This is a schematic diagram of molecular structure decoding provided by the present invention;

[0039] Figure 3 The present invention provides a handwritten molecular image;

[0040] Figure 4 This is a molecular structure diagram provided by the present invention;

[0041] Figure 5 This is a molecular structure tag provided by the present invention;

[0042] Figure 6 This is a schematic diagram of the molecular structure recognition device provided by the present invention;

[0043] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0045] In recent years, advancements in computer vision technology, particularly in object detection and semantic segmentation, have yielded satisfactory recognition performance for simple and controllable printed molecular images. Simultaneously, significant progress has been made in the recognition of handwritten chemical molecular structures. However, these methods may overlook the importance of recognizing handwritten molecular images in human-computer interaction, or oversimplify the complexity of molecular structure recognition, relying excessively on rule-based post-processing methods. Consequently, recognizing large and complex molecular structures remains a major challenge.

[0046] Figure 1 This is one of the flowcharts illustrating the molecular structure recognition method provided by the present invention, such as... Figure 1 As shown, the method includes:

[0047] Step 110: Obtain molecular images.

[0048] The molecular image here refers to an image depicting a chemical molecular structure. Furthermore, the chemical molecular structure in the molecular image can be printed or hand-drawn; this embodiment of the invention does not impose specific limitations on this.

[0049] Step 120: Initialize an empty angle set. When decoding the molecular structure based on the image features of the molecular image and decoding a branch angle for the first time, store the branch angle in the angle set. Take a branch angle from the angle set, and using the branch angle as a guide, decode the molecular structure under the branch angle based on the image features of the molecular image. Update the angle set based on the new branch angle obtained from the decoding for decoding the molecular structure under the next branch angle, until the angle set is empty.

[0050] Considering the complexity of chemical molecular structures, when performing molecular structure recognition on molecular images, an angle decoding and maintenance mechanism for each branch in the chemical molecular structure can be added to the conventional encoder + decoder architecture. The branch angles in the chemical molecular structure can be used as guiding conditions during decoding to improve the reliability and accuracy of decoding for complex chemical molecular structures.

[0051] In practical implementation, branch-by-branch decoding can be performed based on the image features of the molecular image. This branch-by-branch decoding is understood to mean a single graph traversal of the molecular image.

[0052] Figure 2 This is a schematic diagram of molecular structure decoding provided by the present invention, as shown below. Figure 2 As shown, in Figure 2In (a), at the beginning of the graph traversal, an empty angle set Memory can be initialized. During the traversal, a branch point is encountered. At this branch point, two branch angles can be decoded, namely branch angle ① and branch angle ②. Branch angle ① points upward and branch angle ② points downward to the right. Branch angle ① and branch angle ② are the first branch angles decoded and are stored in the angle set Memory.

[0053] Subsequently, Figure 2 In (b), branch angle ① is retrieved from the angle set Memory, and the process is traversed upwards along branch angle ① until a new branch point is encountered. Figure 2 In step (c), decoding yields two branch angles: branch angle ③ and branch angle ④. Branch angle ③ points to the right, and branch angle ④ points downwards. Both branch angles ③ and ④ are stored in the angle set Memory. Next, in... Figure 2 In (d), branch angles ①, ③, ⑤, and ⑦ are successively retrieved from the angle set Memory and traversed in the graph until a branch traversal is completed.

[0054] exist Figure 2 In (e), after completing the traversal of one branch, branch angle ⑥ is retrieved from the angle set Memory, and the traversal continues along branch angle ⑥. At this point, it is found that the traversed atom is connected to the previously visited atom, that is, in Figure 2 In (e), atoms are marked as connected by two solid circles. For this situation, in Figure 2 In (f), it can be predicted whether branch angle ⑧, which is already stored in the angle set Memory, is a loop key. If it is, then branch angle ⑧ can be considered to have been traversed and removed from the angle set Memory. The loop key here is understood as whether two branch angles form a reconnection relationship, that is, whether a cycle of branch angles is formed.

[0055] Based on the above traversal method, after completing the branch decoding traversal of all branch angles in the angle set Memory, the graph traversal for the molecular image can be considered complete.

[0056] Understandably, the angle set is a collection used to store the angles of branches to be traversed. At the beginning of graph traversal, an angle set can be initialized, initially empty; that is, at the beginning of the graph traversal, the angle set does not contain any branch angles to be traversed. During the graph traversal, when a branch point is first encountered, i.e., the first branch angle is decoded, this first decoded branch angle can be stored in the angle set. At this point, the angle set is no longer empty; it stores the branch angles to be traversed. Furthermore, in subsequent traversals, branch angles can be retrieved from the angle set for exploration. Therefore, the angle set is used to store and maintain the branch angles to be traversed, and during the decoding process of a branch angle, the angle set can be updated and adjusted accordingly based on the newly decoded branch angles to facilitate the decoding of the next branch angle.

[0057] Furthermore, during the decoding process for any branch angle, it is necessary to first retrieve the branch angle from the angle set, that is, to confirm that the branch angle is a branch angle that is being explored or has been explored, so that the angle set can always maintain branch angles that have not yet been explored.

[0058] After obtaining the branch angle, the molecular structure at that branch angle can be decoded. During decoding, the molecular structure can be decoded along the branch angle, that is, the branch angle can be used as conditional information to guide the decoding of the molecular structure, thereby enriching the information of the molecular structure decoding and improving the reliability of the molecular structure decoding.

[0059] During the decoding of the molecular structure at a given branch angle, if a new branch angle is decoded, the angle set can be updated based on this new angle. Specifically, the new branch angle can be added to the angle set as an unexplored branch angle for subsequent decoding. Furthermore, before adding the new branch angle as an unexplored branch angle to the angle set, the relationship between this branch angle and the existing branch angles in the angle set can be determined. That is, it can be determined whether the new branch angle and the existing branch angles that form a reconnection relationship in the molecular structure are reconnected. If a reconnection relationship is formed, it can be considered that both the new branch angle and the existing branch angles that form a reconnection relationship with it have been explored. The new branch angle does not need to be stored in the angle set, and the existing branch angles that form a reconnection relationship with it can also be deleted from the angle set. This ensures that the angle set can always maintain unexplored branch angles.

[0060] Step 130: Based on the decoding results at each branch angle, determine the molecular structure corresponding to the molecular image.

[0061] Specifically, after decoding the molecular structure at each branch angle in the molecular image, the decoding results at each branch angle can be obtained. These decoding results can include chemical bonds, angles, and atomic groups at that branch angle. Based on this, by combining the decoding results at each branch angle, the molecular structure corresponding to the molecular image can be obtained. This molecular structure can be a string recorded using a markup language based on organic chemical structural formulas, a representation diagram of the molecular structure, or a vector representation of the molecular structure in a specific encoding language. This embodiment of the invention does not specifically limit this.

[0062] The method provided in this invention adds a mechanism for decoding each branch angle in the chemical molecular structure and maintaining the branch angle to be explored, and uses the branch angle in the chemical molecular structure as a guiding condition during decoding, so as to enrich the information of molecular structure decoding, improve the reliability of molecular structure decoding, and improve the decoding accuracy for complex chemical molecular structures.

[0063] Based on the above embodiments, step 120, which involves updating the angle set based on the new branch angle obtained through decoding, includes:

[0064] The new branch angle is compared with each existing branch angle in the angle set to detect chemical bonds.

[0065] The angle set is updated based on the detection results of the chemical bond detection.

[0066] Specifically, considering the possibility of reconnection in molecular structures, such as the 6-chain single bonds in a benzene ring being connected end to end, if we do not consider whether the new branch angle obtained by decoding forms a chemical bond with the branch angles that have not yet been explored in the angle set, it is very likely to lead to repeated traversal, that is, decoding the molecular structure under the two branch angles separately, causing the molecular structure that should originally be under one branch to be incorrectly decoded as two molecular structures, affecting the accuracy of molecular structure recognition.

[0067] Therefore, after decoding the new branch angle, chemical bond detection can be performed between the new branch angle and each existing branch angle in the angle set. Here, for the new branch angle and any branch angle in the angle set, it is possible to detect whether a chemical bond is formed between these two branch angles. The detection result can include whether a chemical bond exists, and if it is determined that a chemical bond is included, it can further include the type of chemical bond, such as single bond, double bond, etc.

[0068] After obtaining the detection results of chemical bonds between the new branch angle and each existing branch angle in the angle set, the angle set can be updated accordingly. It is understood that if there are no chemical bonds between the new branch angle and any existing branch angle, it means that a reconnection relationship has not yet been decoded, and the new branch angle can be added to the angle set as an unexplored branch angle. If there is a chemical bond between the new branch angle and one of the existing branch angles, it means that a reconnection relationship has been decoded, and the existing branch angle with a chemical bond to the new branch angle is deleted from the angle set, and the new branch angle is not added to the angle set.

[0069] Based on any of the above embodiments, in step 120, updating the angle set based on the detection results of the chemical bond detection includes:

[0070] If the detection result indicates that no chemical bonds exist, the new branch angle is stored in the angle set;

[0071] If the detection result indicates the presence of a chemical bond, the branch angle that forms a chemical bond with the new branch angle will be removed from the angle set.

[0072] Specifically, for a new branch angle, if all the detection results of the chemical bond detection between the new branch angle and each existing branch angle indicate that there is no chemical bond, that is, there is no chemical bond between the new branch angle and each existing branch angle, it means that the reconnection relationship has not been decoded yet, and the new branch angle also needs to be explored. At this time, the new branch angle can be stored in the angle set so that the angle set can continue to maintain all the branch angles that have not yet been explored.

[0073] If a detection result indicates the presence of a chemical bond, meaning that there is a chemical bond between the new branch angle and an existing branch angle in the angle set, then there is a reconnection relationship between the new branch angle and the existing branch angle. It can be considered that the new branch angle and the existing branch angle that forms a reconnection relationship with it have been explored. The new branch angle does not need to be stored in the angle set, and the existing branch angle that forms a reconnection relationship with it can also be deleted from the angle set. This allows the angle set to always maintain the branch angles that have not yet been explored.

[0074] Based on any of the above embodiments, step 120, which involves decoding the molecular structure at the branch angle based on the image features of the molecular image, includes:

[0075] Based on the decoding features of the branch angle and the decoding state of the previous decoding moment under the branch angle, the visual context features of the current decoding moment under the branch angle are determined.

[0076] Based on the visual context features, the molecular structure of the image features is decoded at the current decoding moment to obtain the decoding state at the current decoding moment, and the current decoding moment is returned as the previous decoding moment for decoding until the decoding at the branch angle ends.

[0077] Specifically, in the process of decoding a molecular structure at a branch angle, the branch angle can be used as a guiding condition for decoding the molecular structure. When a branch is encountered, decoding can continue along the direction of the branch angle taken from the angle set, and the branch angle applied at this time can guide the generation of visual context features when decoding the molecular structure at that branch angle.

[0078] That is, when generating visual context features at a decoding moment, compared to existing technologies that only apply the decoding state of the previous decoding moment to determine the visual context features at the current decoding moment, the decoding features of the branch angle are also considered. It can be understood that here, the decoding features of the branch angle are stored when that branch angle is decoded during the decoding process of the molecular structure, and the decoding features of the branch angle can specifically include the attention weight information and state features of that branch angle.

[0079] For example, the calculation of visual context features at the current decoding moment can be expressed as the following formula:

[0080]

[0081]

[0082]

[0083] In the formula, e t,i Let w and W be the energy at position i in the feature map at time t during decoding. x W y W S W a W sp W ap These are all projection parameters of the attention module. i The image features of the molecular image at position i. Let s be the decoding result at the (t-1)th decoding time. t-1 Let a be the decoding state at the (t-1)th decoding time. t-1 a t The attention weights at decoding time t-1 and t-th are respectively, a b s b These represent the attention weights and state features during branch-angle decoding, respectively. It should be noted that when decoding is not performed at a branch angle, a...b and s b Both are zero vectors, at which point e t,i The calculation formula for e differs from the visual context feature acquisition method in conventional string decoders. t,i The calculation formula is consistent.

[0084] a t,i Let h be the attention weight at position i at time t during decoding, and w be the height and width of the molecular image, respectively; c t Let be the visual context features at the t-th decoding time.

[0085] After obtaining the visual context features at the current decoding moment, the molecular structure at the current decoding moment can be decoded based on these features, thus obtaining the decoding state at the current decoding moment, which can be expressed as the following formula:

[0086]

[0087] p t =softmax(W c s t );

[0088] In the formula, s te That is, the decoding state at the current decoding moment; GRU represents the computational unit of a recurrent neural network. te W represents the probability distribution of the decoding result output at the current decoding moment. c These are the parameters for the classification layer.

[0089] It is understandable that after completing the molecular structure decoding at the current decoding moment, the decoding state at the current decoding moment can be used as the decoding state at the previous decoding moment, and the molecular structure decoding at the next decoding moment can be executed. Furthermore, during this process, at different moments when decoding the molecular structure from a branch angle, the decoding features of the same branch angle are applied.

[0090] In this embodiment of the invention, the neural network structure used to decode molecular structures is referred to as a stochastic condition-guided decoder. The stochastic condition-guided decoder significantly outperforms traditional string decoders in generalizing complex structures without adding significant extra parameters or computational burden. The use of conditional guidance mechanisms and path selection effectively improves the accuracy of multi-path decoding.

[0091] Based on any of the above embodiments, step 120 includes:

[0092] Based on the recognition model, a branch angle is taken from the angle set. Guided by the branch angle, the molecular structure under the branch angle is decoded based on the image features of the molecular image. The angle set is updated based on the new branch angle obtained by decoding so that the molecular structure under the next branch angle can be decoded, until the angle set is empty.

[0093] The recognition model is trained based on sample images and the molecular structure labels corresponding to the sample images;

[0094] The molecular structure tag is obtained by connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image to form a molecular structure diagram and then performing graph traversal.

[0095] Specifically, step 120 can be implemented using a pre-trained recognition model. Here, the recognition model can be in the form of an encoder-decoder. Training the recognition model can be achieved through supervised learning. It can be understood that the samples for supervised learning are images of samples depicting chemical molecular structures, and the labels for supervised learning are the molecular structure labels corresponding to the sample images. Here, the molecular structure labels reflect the molecular structure in a linguistic form.

[0096] Conventional notation languages ​​for organic chemical structures include Chemfig and SMILES. Compared to the most commonly used notation language SMILES, Chemfig requires almost no abstract chemical knowledge; its design aims to improve relevance and accuracy by utilizing the chemical structure diagram itself. Chemfig's syntax also provides descriptions for "angles," which can be used to specify the orientation of bonds in a molecule. Figure 3 In the handwritten molecular image shown, the atom "N" is connected to two double bonds, one pointing to the upper right and the other to the lower right. Subscripts in the Chemfig syntax, such as "[1]" and "[-1]", can be used to represent the approximate orientation of these two double bonds, thus making the visual morphology of the molecule more accurate and comprehensive.

[0097] However, if the original Chemfig markup language is used directly as the label for training the recognition model, the following problems still exist:

[0098] First, Chemfig suffers from syntactic ambiguity: due to different starting points and traversal orders, multiple correct Chemfig sequences can represent the same molecular image. This ambiguity is unavoidable and increases the difficulty of model learning.

[0099] Secondly, the Chemfig syntax is complex: prior rules and domain knowledge need to be incorporated into the syntax to maintain its conciseness. For example, rules such as using brackets to group atoms, and the need to remove redundant chemical symbols and valence dashes to reduce syntactic complexity. Furthermore, domain knowledge such as IUPAC (International Union of Pure and Applied Chemistry) nomenclature rules is necessary to ensure the correct labeling of compound structures. The use of these rules and domain knowledge increases the complexity of the Chemfig syntax. For example, a benzene ring is simply represented as "*6(––)," but the orientation of the six single bonds needs to be automatically inferred from the rules governing benzene rings. These prior rules are complex, increasing the difficulty of model learning.

[0100] To overcome the above problems, this embodiment of the invention proposes to analyze and connect the atomic groups and chemical bonds in the molecular formula corresponding to the sample image to form a molecular structure diagram, and then perform graph traversal to obtain molecular structure labels.

[0101] Here, considering that the molecular formula corresponding to the sample image can be represented by different strings, in order to resolve this grammatical ambiguity, in this embodiment of the invention, the atomic groups and chemical bonds are manually parsed and recovered from the string of the molecular formula, and then all the obtained atomic groups and chemical bonds are connected to generate a diagram, which is referred to here as the molecular structure diagram.

[0102] For example, “HO-**6(---(-COOH)---)”, “COOH-[4]**6(---(-OH)---)”, and “**6([:30]-(-OH)---(-COOH)--)” are different string representations of the same molecular formula. By parsing and restoring the atomic groups and chemical bonds, it is possible to construct such... Figure 4 The molecular structure diagram shown achieves a unified representation in the form of a structure diagram.

[0103] Figure 4 In this code, strings are represented as atom groups, such as "HO" and "COOH". The cyclic benzene molecule is considered a special atom group. Lines between atom groups represent chemical bonds, which can be single bonds "-", double bonds "=", or triple bonds. One atom group can be connected to another atom group by one or more chemical bonds. By using atom groups as vertices and chemical bonds as edges, a molecular structure diagram can be obtained. Therefore, even if different Chemfig tag strings are used, the graphical representation of the same molecule remains the same, eliminating potential ambiguity in the tags.

[0104] The resulting molecular structure diagram is a complex data structure and needs to be transformed into a label format suitable for model training through graph traversal. Specifically, this involves converting the molecular structure diagram into an equivalent one-dimensional text representation through graph traversal. The transformation rule can be a fixed rule, which can be to start traversing the molecular structure diagram from a specific point and append the traversed nodes (atomic groups) and edges (chemical bonds) to the end of the string, thereby obtaining concise molecular structure labels without syntactic ambiguity.

[0105] The method provided in this invention connects the atomic groups and chemical bonds in the molecular formula to form a molecular structure diagram and then performs graph traversal to obtain concise molecular structure labels without grammatical ambiguity. This reduces the difficulty of supervised learning of the recognition model and improves the reliability of molecular structure recognition based on the recognition model.

[0106] Based on any of the above embodiments, the step of determining the molecular structure tag includes:

[0107] Connect the atomic groups and chemical bonds in the molecular formula corresponding to the sample image to form a molecular structure diagram;

[0108] The molecular structure diagram is traversed, and molecular structure tags are generated based on the atomic groups, chemical bonds, angles, nested symbols, and reconnection markers obtained from the traversal.

[0109] Specifically, the molecular structure diagram obtained by structural analysis and connection of molecular formulas is a complex data structure that must be converted into a form suitable for model training. For encoder-decoder recognition models, the suitable training form is usually a one-dimensional string or standardized labels. Standardized labels involve converting the molecular structure diagram into an equivalent one-dimensional text representation.

[0110] To ensure consistency and unambiguity in the rules for obtaining molecular structure labels, the specific rules can be pre-defined fixed rules. For traversing the molecular structure diagram, one can start from a specific point and append nodes (atomic groups) and edges (chemical bonds) to the end of the string. After traversal, standardized labels are obtained. For example, for... Figure 4 Traversing the shown molecular structure diagram yields the following standardized labels: HO-[:0]? [a](-[:60]-[:0]-[:300](-[:0]COOH)-[:240]-[:180]?[a,{-}](--[:60]\circle))

[0111] In this diagram, each unit is separated by a space. Graph traversal can be performed using Depth-First Search (DFS). The standardized labels above can contain four types of elements: "atomic group", "chemical bond", "angle", "nested symbol", and "reconnection symbol". Specifically, in the standardized labels above, "HO" represents an atomic group, "?[a]" and "?[a,{-}]" are reconnection symbols, "-" represents a chemical bond, "[:0]" represents an angle, and "(" represents a nested symbol.

[0112] Based on this, standardized tags can be integrated to obtain structure-specific molecular structure tags. If an atom group does not contain any characters, it can be omitted. Entity characters can be represented using LaTeX notation. Since "angles" and "chemical bonds" are closely related in visual information, each chemical bond can be appended with a corresponding angle as "[:]" in the standardized tag to represent a "bond angle" modeling unit. The angle value can be calculated according to the format during the graph construction and output process. Furthermore, nested branches can be represented using the nested symbol "()". Additionally, the original Chemfig syntax can be retained for reconnection markers, represented by "?[a]", which represents a unit surrounded by a loop unit represented by "?[a,-]". When a reconnection occurs, the reconstructed angle value is not represented. Finally, a specific atom in benzene can be represented by "○", and a virtual bond angle unit "–[:]" can be constructed to represent the connection between "○" and any atom on the ring.

[0113] For example, the standardized labels shown above can be integrated into... Figure 5 The molecular structure label shown Figure 5 The molecular structure labels shown indicate nesting relationships and reconnection relationships at different angles. Specifically, “\eob” indicates decoding completion at a branch angle, an arrow connecting “\angle[:60]” and “-[:60]” indicates a nesting relationship, and a double-arrow connecting “\angle[:300]” and “\angle[:120]” and “reconnection” indicate a reconnection marker.

[0114] The method provided in this invention connects atomic groups and chemical bonds in a molecular formula to form a molecular structure diagram, and forms molecular structure tags by traversing the molecular structure diagram. Compared to the mainstream language SMILES, the resulting molecular structure tags have the advantage of higher consistency with the image, which is more conducive to model learning. Furthermore, the syntax of the molecular structure tags is not limited by chemical knowledge, allowing it to represent incorrect or non-existent molecular structures and making it more universal. The above method provides an innovative approach to molecular structure characterization and has potential applications in various fields of chemical structure identification and analysis.

[0115] Based on any of the above embodiments, the training steps of the recognition model include:

[0116] Based on the initial model, molecular structure recognition is performed on the sample image to obtain the chemical bond detection results between the sample branch angle decoded during the molecular structure recognition process and the existing branch angles in the sample angle set, as well as the structure recognition results of the sample image.

[0117] Based on the structure recognition results and the molecular structure tag, as well as the chemical bond detection and the reconnection markers in the molecular structure tag, the initial model is iterated to obtain the recognition model.

[0118] Specifically, for the recognition model, an initial model with an encoder + decoder structure can be obtained, where the model parameters can be initialized. During training, molecular structure recognition can be performed on sample images based on the initial model. Specifically, image features can be extracted from the sample images based on the initial model, and then a sample branch angle can be taken from the sample angle set. Guided by this sample branch angle, the molecular structure under that sample branch angle is decoded based on the image features of the sample image. The sample angle set is then updated based on the decoded new sample branch angle for decoding the molecular structure under the next sample branch angle, until the sample angle set is empty.

[0119] It is understandable that when the initial model decodes to obtain new sample branch angles, the sample angle set needs to be updated based on the new sample branch angles. When updating the sample angle set, the new sample branch angles need to be chemically bonded with each existing branch angle in the sample angle set. This allows us to obtain the chemical bond detection results between the sample branch angles and the existing branch angles in the sample angle set.

[0120] Here, the presence or absence of chemical bonds in the chemical bond detection results reflects whether there is a reconnection relationship between the two sample branch angles. The chemical bond detection results can be compared with the reconnection markers in the molecular structure labels, thereby measuring the training loss of the initial model from the perspective of reconnection relationship detection.

[0121] In addition, the structure recognition results can be compared with the molecular structure labels to measure the training loss of the initial model from the perspective of molecular structure recognition.

[0122] Furthermore, the loss L for molecular structure recognition can be expressed based on the following formula. ce :

[0123]

[0124] In the formula, Lce For cross-entropy loss, V represents the number of units in the model, and y ti To determine whether the molecular structure tag is the i-th character at the t-th decoding time, p ti This represents the probability that the structure recognition result is the i-th character at the t-th decoding time.

[0125] The detection loss L for reconnection relationships can be expressed by the following formula. bc That is, the loss caused by the cycle:

[0126]

[0127] q tb =softmax(W m s b +W o s t );

[0128] In the formula, s t W represents the decoding state of the branch angle obtained at the current decoding time t. m and W o These are all parameters used for chemical bond detection. The state features s of the existing branch angle b stored in the angle set. b The probability distribution of chemical bond detection with respect to the branching angle t is expressed as follows: Where N is the total number of chemical bond types, and N+1 includes N types of chemical bonds as well as the case where no chemical bonds exist.

[0129] N is the total number of branch angles in the angle set, z tbi To determine whether the chemical bond type between the molecular structure tag and the branch angle b is i and q at the t-th decoding time. tb This represents the probability that a chemical bond of type i is detected at decoding time t, which is related to the branch angle b.

[0130] Combining the two losses mentioned above, we can obtain the overall loss used for training the recognition model. This overall loss L can be expressed as Li ce +L bc It can also be represented as L ce and L bc The weighted summation result is not specifically limited in this embodiment of the invention. After obtaining the overall loss, the parameters of the initial model can be iterated based on the overall loss to obtain a fully trained recognition model.

[0131] The method provided in this invention achieves molecular structure recognition based on molecular images through end-to-end modeling and training.

[0132] Based on any of the above embodiments, the initial model is an encoder + decoder structure. The encoder is a DenseNet, comprising three dense blocks used to convert the input RGB three-channel image into high-dimensional features. The growth rate and depth of each dense block are set to 24 and 32, respectively. Furthermore, both the encoder and decoder use GRUs with a hidden state dimension of 256 as the recurrent units of the RNN, and the dimension of the attention projection parameters is set to 128. Additionally, the dimension of the embedding parameters is set to 256, and a 15% drop-out rate is applied. For the decoder, the dimension of the projection parameters for chemical bond detection classification is set to 256.

[0133] The optimizer used for parameter iteration of the initial model can be Adam, with an initial learning rate of 2e-4. The learning rate decay strategy is multi-step decay, using PyTorch's MultiStepLR to adjust the learning rate, and the decay factor gamma is set to 0.5.

[0134] Based on any of the above embodiments, this invention provides a molecular structure recognition method, which is implemented based on a recognition model. The recognition model here includes a random conditional decoder, which differs from a string decoder. The random conditional decoder combines three mechanisms to address problems existing in the prior art: conditional guided attention, looping, and path selection.

[0135] Conditional guided attention mechanisms utilize the natural graph structure of molecular structures, treating the recognition process as a graph traversal problem. When the recognition model traverses the graph, it encounters multiple branch angles, which follow a fixed counter-clockwise order within the proposed modeling units. However, if decoding is performed solely based on this fixed angle order, the model may forget which angle units have not yet been decoded due to the extended decoding time later on. To address this issue, the conditional guided attention mechanism uses branch angles as conditional information to guide the decoding process. When the recognition model encounters a branch, it continues decoding along the specified branch angle direction. The updated decoder no longer uses parentheses () and commas to indicate the start and end of branches. Instead, it first predicts the branch angle of each branch, i.e., "\angle[:<angle value> The decoding features of each branch angle are stored separately in the angle set Memory, which contains the context and attention weight information of each branch angle.

[0136] Subsequently, during the decoding process of the molecular structure at a branch angle, when “\eob” is decoded, it indicates that there are no additional branch angles to predict, and the decoding feature of a branch angle can be selected from the angle set Memory as a condition to continue the decoding process.

[0137] Furthermore, the main difference between graph structures and tree structures lies in the cyclic nature of graph structures. In the molecular structure recognition method provided in this embodiment of the invention, the branch angles corresponding to cycles have been stored in the angle set Memory and have not yet been decoded. Therefore, a simple multi-label classification module can be constructed for chemical bond detection to determine the direction corresponding to the loop angle, while classifying the type of loop bond (such as single bond, double bond, etc.). The result of chemical bond detection may be empty or the type of bond; empty indicates that there is no cycle with a corresponding branch angle that has not yet been decoded. If a cyclic chemical bond exists in the detection result, the corresponding branch angle is deleted from the angle set Memory, and the branch angle decoded at the current moment is not stored. Conversely, if there is no cycle, i.e., the detection result is empty, the branch angle at the current moment is saved in the angle set Memory for future decoding selection.

[0138] Finally, the recognition process of the stochastic conditional decoder can be viewed as a graph traversal problem, where different traversals of the graph can generate multiple target sequences. Therefore, a single graph can have multiple training labels. Although the conditionally guided attention mechanism still decodes according to a fixed counter-clockwise order, it can lead to overfitting and recognition errors in complex or uncommon structures. Therefore, embodiments of the invention can incorporate a path selection mechanism for decoding, which randomly selects different paths during training to improve the alignment between visual information and the decoded characters using the conditionally guided attention mechanism. During inference, the recognition model can attempt to decode all branch angles stored in the angle set and participate in calculating the beam search path score PK, thereby selecting the path with the highest score to continue decoding.

[0139] Through the above methods, the recognition model can have better versatility and recognition efficiency.

[0140] Furthermore, for training the recognition model, this embodiment of the invention uses the publicly accessible CASIA-CSDB dataset as a benchmark to establish a handwritten dataset as sample images. Specifically, real-world handwritten molecular structures can be obtained from an educational environment. This dataset includes many instances of structural data with writing errors and those that do not exist.

[0141] Furthermore, the handwritten dataset consists of 52,987 handwritten molecular structure images collected in educational settings. These images were obtained using various devices such as cameras, scanners, and screens and were labeled with native Chemfig strings. The dataset also contains a mixture of molecular structures and regular molecular formulas. Besides isolated molecular structures, many instances in the dataset also include combinations of formulas and molecular structures. In addition, the dataset integrates different structural writing styles, such as the use / absence of abbreviations, the type of Keckler ring notation for benzene, and the inclusion / exclusion of hydrogen atoms. The dataset also contains a large number of instances of human error that violates chemical principles or even non-existent structures; models trained on this basis can potentially be used to correct and modify handwritten answers. Furthermore, to test the model's generalization performance, the molecular structures in the dataset need to maintain a certain level of complexity. Specifically, the complexity level of molecular structures can be evaluated using the sum of the number of atoms and bonds; approximately 10% of the dataset has a complexity level exceeding that of the most complex samples in the regular training set.

[0142] Based on any of the above embodiments, after the recognition model training is completed, experimental testing can be conducted on the recognition model.

[0143] Specifically, considering that the molecular structure labels themselves are concise enough, we can directly compare whether the predicted recognition result string and the label string are the same. We can convert both the label string and the recognition result string into molecular structure diagrams, and then compare whether these two molecular structure diagrams match. Assuming T is the number of samples and R represents the number of recognition results that match the label, the Exact Match (EM) score can be calculated using the following formula:

[0144]

[0145] Furthermore, considering the mixture of formulas and molecular structures in handwritten datasets, a single image may contain multiple molecular structures. Therefore, this embodiment of the invention defines two auxiliary indicators as follows:

[0146] Structure-Exact Matching (Structure EM): For samples containing mixed molecular structures and regular formulas, the structure in the sample is considered correctly identified when all molecular structure identification results match the labeled graphic. Let T be the number of samples, and R... struct The number of correctly identified structural samples is represented by the structural EM score, which is as follows:

[0147]

[0148] Structure EM fraction EM structThe single EM score EM can measure the recognition performance of the recognition model on molecular structures in handwritten datasets in mixed modes.

[0149] Experimental results show that the molecular structure recognition method provided in this invention significantly improves recognition performance compared to the SMILES-based method. This is mainly due to reduced ambiguity of molecular structure labels and stronger consistency between images and labels. Furthermore, the random conditional decoder proposed in this invention has significant advantages over the string decoder. It should be emphasized that the computational and parameter costs of the random conditional decoder are almost identical to those of the string decoder.

[0150] Based on any of the above embodiments Figure 6 This is a schematic diagram of the molecular structure recognition device provided by the present invention, as shown below. Figure 6 As shown, the device includes:

[0151] Acquisition unit 610 is used to acquire molecular images;

[0152] The identification unit 620 is used to initialize an empty angle set. When decoding the molecular structure based on the image features of the molecular image and decoding a branch angle for the first time, the branch angle is stored in the angle set. A branch angle is taken out from the angle set, and the molecular structure under the branch angle is decoded based on the image features of the molecular image, guided by the branch angle. The angle set is updated based on the new branch angle obtained by decoding for decoding the molecular structure under the next branch angle, until the angle set is empty.

[0153] The output unit 630 is used to determine the molecular structure corresponding to the molecular image based on the decoding results at each branch angle.

[0154] The apparatus provided in this invention adds a mechanism for decoding each branch angle in the chemical molecular structure and maintaining the branch angle to be explored, and uses the branch angle in the chemical molecular structure as a guiding condition during decoding, so as to enrich the information of molecular structure decoding, improve the reliability of molecular structure decoding, and improve the decoding accuracy for complex chemical molecular structures.

[0155] Based on any of the above embodiments, the identification unit 620 includes a set update unit, used for:

[0156] The new branch angle is compared with each existing branch angle in the angle set to detect chemical bonds.

[0157] The angle set is updated based on the detection results of the chemical bond detection.

[0158] Based on any of the above embodiments, the set update unit is specifically used for:

[0159] If the detection result indicates that no chemical bonds exist, the new branch angle is stored in the angle set;

[0160] If the detection result indicates the presence of a chemical bond, the branch angle that forms a chemical bond with the new branch angle will be removed from the angle set.

[0161] Based on any of the above embodiments, the identification unit 620 includes a branch decoding unit, used for:

[0162] Based on the decoding features of the branch angle and the decoding state of the previous decoding moment under the branch angle, the visual context features of the current decoding moment under the branch angle are determined.

[0163] Based on the visual context features, the molecular structure of the image features is decoded at the current decoding moment to obtain the decoding state at the current decoding moment, and the current decoding moment is returned as the previous decoding moment for decoding until the decoding at the branch angle ends.

[0164] Based on any of the above embodiments, the identification unit 620 is specifically used for:

[0165] Based on the recognition model, a branch angle is taken from the angle set. Guided by the branch angle, the molecular structure under the branch angle is decoded based on the image features of the molecular image. The angle set is updated based on the new branch angle obtained by decoding so that the molecular structure under the next branch angle can be decoded, until the angle set is empty.

[0166] The recognition model is trained based on sample images and the molecular structure labels corresponding to the sample images;

[0167] The molecular structure tag is obtained by connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image to form a molecular structure diagram and then performing graph traversal.

[0168] Based on any of the above embodiments, the device further includes a tag conversion unit, used for:

[0169] Connect the atomic groups and chemical bonds in the molecular formula corresponding to the sample image to form a molecular structure diagram;

[0170] The molecular structure diagram is traversed, and molecular structure tags are generated based on the atomic groups, chemical bonds, angles, nested symbols, and reconnection markers obtained from the traversal.

[0171] Based on any of the above embodiments, the device further includes a model training unit, used for:

[0172] Based on the initial model, molecular structure recognition is performed on the sample image to obtain the chemical bond detection results between the sample branch angle decoded during the molecular structure recognition process and the existing branch angles in the sample angle set, as well as the structure recognition results of the sample image.

[0173] Based on the structure recognition results and the molecular structure tag, as well as the chemical bond detection and the reconnection markers in the molecular structure tag, the initial model is iterated to obtain the recognition model.

[0174] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a molecular structure recognition method, which includes:

[0175] Acquire molecular images;

[0176] Initialize an empty angle set. When decoding the molecular structure based on the image features of the molecular image and first decoding a branch angle, store the branch angle in the angle set. Take a branch angle from the angle set, and using the branch angle as a guide, decode the molecular structure under the branch angle based on the image features of the molecular image. Update the angle set based on the new branch angle obtained from the decoding for decoding the molecular structure under the next branch angle, until the angle set is empty.

[0177] Based on the decoding results from each branch angle, the molecular structure corresponding to the molecular image is determined.

[0178] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0179] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the molecular structure recognition method provided by the above methods, the method comprising:

[0180] Acquire molecular images;

[0181] Initialize an empty angle set. When decoding the molecular structure based on the image features of the molecular image and first decoding a branch angle, store the branch angle in the angle set. Take a branch angle from the angle set, and using the branch angle as a guide, decode the molecular structure under the branch angle based on the image features of the molecular image. Update the angle set based on the new branch angle obtained from the decoding for decoding the molecular structure under the next branch angle, until the angle set is empty.

[0182] Based on the decoding results from each branch angle, the molecular structure corresponding to the molecular image is determined.

[0183] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the molecular structure recognition method provided by the methods described above, the method comprising:

[0184] Acquire molecular images;

[0185] Initialize an empty angle set. When decoding the molecular structure based on the image features of the molecular image and first decoding a branch angle, store the branch angle in the angle set. Take a branch angle from the angle set, and using the branch angle as a guide, decode the molecular structure under the branch angle based on the image features of the molecular image. Update the angle set based on the new branch angle obtained from the decoding for decoding the molecular structure under the next branch angle, until the angle set is empty.

[0186] Based on the decoding results from each branch angle, the molecular structure corresponding to the molecular image is determined.

[0187] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0188] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0189] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for molecular structure recognition, characterized in that, include: Acquire molecular images; Initialize an empty angle set. When decoding the molecular structure based on the image features of the molecular image and first decoding a branch angle, store the branch angle in the angle set. Take a branch angle from the angle set, and using the branch angle as a guide, decode the molecular structure under the branch angle based on the image features of the molecular image. Update the angle set based on the new branch angle obtained from the decoding for decoding the molecular structure under the next branch angle, until the angle set is empty. Based on the decoding results from each branch angle, the molecular structure corresponding to the molecular image is determined; The step of decoding the molecular structure at the branch angle based on the image features of the molecular image, guided by the branch angle, includes: Based on the decoding features of the branch angle and the decoding state of the previous decoding moment under the branch angle, the visual context features of the current decoding moment under the branch angle are determined. Based on the visual context features, the molecular structure of the image features is decoded at the current decoding time to obtain the decoding state at the current decoding time, and the current decoding time is returned as the previous decoding time for decoding until the decoding at the branch angle ends. The step of updating the angle set based on the new branch angle obtained from decoding includes: The new branch angle is compared with each existing branch angle in the angle set to detect chemical bonds. If the detection result of the chemical bond detection indicates that no chemical bond exists, the new branch angle is stored in the angle set; If the detection result indicates the presence of a chemical bond, the branch angle that forms a chemical bond with the new branch angle will be removed from the angle set.

2. The molecular structure recognition method according to claim 1, characterized in that, The step of selecting a branch angle from the angle set, using the branch angle as a guide, decoding the molecular structure at the branch angle based on the image features of the molecular image, and updating the angle set based on the decoded new branch angle for decoding the molecular structure at the next branch angle, until the angle set is empty, includes: Based on the recognition model, a branch angle is taken from the angle set. Guided by the branch angle, the molecular structure under the branch angle is decoded based on the image features of the molecular image. The angle set is updated based on the new branch angle obtained from the decoding for the decoding of the molecular structure under the next branch angle, until the angle set is empty. The recognition model is trained based on sample images and the molecular structure labels corresponding to the sample images; The molecular structure tag is obtained by connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image to form a molecular structure diagram and then performing graph traversal.

3. The molecular structure recognition method according to claim 2, characterized in that, The steps for determining the molecular structure tag include: Connect the atomic groups and chemical bonds in the molecular formula corresponding to the sample image to form a molecular structure diagram; The molecular structure diagram is traversed, and molecular structure tags are generated based on the atomic groups, chemical bonds, angles, nested symbols, and reconnection markers obtained from the traversal.

4. The molecular structure recognition method according to claim 2, characterized in that, The training steps of the recognition model include: Based on the initial model, molecular structure recognition is performed on the sample image to obtain the chemical bond detection results between the sample branch angle decoded during the molecular structure recognition process and the existing branch angles in the sample angle set, as well as the structure recognition results of the sample image. Based on the structure recognition results and the molecular structure tag, as well as the chemical bond detection and the reconnection markers in the molecular structure tag, the initial model is iterated to obtain the recognition model.

5. A molecular structure recognition device, characterized in that, include: Acquisition unit, used to acquire molecular images; The identification unit is used to initialize an empty angle set. When decoding the molecular structure based on the image features of the molecular image and decoding a branch angle for the first time, the branch angle is stored in the angle set. A branch angle is taken out from the angle set, and the molecular structure under the branch angle is decoded based on the image features of the molecular image, guided by the branch angle. The angle set is updated based on the new branch angle obtained by decoding for decoding the molecular structure under the next branch angle, until the angle set is empty. The output unit is used to determine the molecular structure corresponding to the molecular image based on the decoding results at each branch angle. The step of decoding the molecular structure at the branch angle based on the image features of the molecular image, guided by the branch angle, includes: Based on the decoding features of the branch angle and the decoding state of the previous decoding moment under the branch angle, the visual context features of the current decoding moment under the branch angle are determined. Based on the visual context features, the molecular structure of the image features is decoded at the current decoding time to obtain the decoding state at the current decoding time, and the current decoding time is returned as the previous decoding time for decoding until the decoding at the branch angle ends. The step of updating the angle set based on the new branch angle obtained from decoding includes: The new branch angle is compared with each existing branch angle in the angle set to detect chemical bonds. If the detection result of the chemical bond detection indicates that no chemical bond exists, the new branch angle is stored in the angle set; If the detection result indicates the presence of a chemical bond, the branch angle that forms a chemical bond with the new branch angle will be removed from the angle set.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the molecular structure recognition method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the molecular structure recognition method as described in any one of claims 1 to 4.

Images