Apparatus and method for measuring the reliability of molecular structure prediction models

Abstract

One embodiment of the present disclosure provides a system, computer program, apparatus, and method for measuring the reliability of a molecular structure prediction model. The method may include the steps of: acquiring an image of a first molecular structure; acquiring a graph of the first molecular structure using the molecular structure prediction model; performing image rendering on the image of the first molecular structure based on the graph of the first molecular structure; and determining the reliability of the graph of the first molecular structure based on the image rendering result and the graph of the first molecular structure.

Description

Technical Field

[0001] The present disclosure relates to an apparatus and a method for measuring the reliability of a molecular structure prediction model. More specifically, the present disclosure is intended to provide convenience by additionally providing reliability for a result when a molecular structure prediction model provides a predicted molecular structure. Background Art

[0002] A structural formula is a graphic representation of a chemical structure or a molecular structure, and can indicate how atoms are arranged in a three-dimensional space. A structural formula can also explicitly or implicitly indicate chemical bonds in a molecule. In particular, unlike a molecular formula which can only provide limited description with a limited number of symbols, a structural formula can provide geometric information of a molecular structure. For example, isomers having the same molecular formula but different atomic structures and arrangements can be represented.

[0003] In various documents, papers, patents, etc., structural formulas are sometimes provided in the form of images. However, unlike text, images are difficult to search, and there is a problem that it is difficult to find a document including the corresponding structural formula. Accordingly, various methods for searching images such as structural formulas have been developed. Models that analyze images and extract structural formulas are mainly used to create academic databases, but if wrong data is entered into such academic databases due to wrong prediction, it becomes a fatal flaw for research. Therefore, there is a current need for a method that can additionally provide reliability information for a predicted structural formula and distinguish which of the predicted structural formulas is determined as reliable information to be stored in a database. Summary of the Invention Problem to be Solved by the Invention

[0004] This disclosure aims to provide a method and apparatus for a model that predicts molecular structure using images, which also provides a confidence score when predicting molecular structure. [Means for solving the problem]

[0005] One embodiment of this disclosure can provide an apparatus and method for measuring the reliability of a molecular structure prediction model.

[0006] One embodiment of the present disclosure aims to provide a system for measuring the reliability of a molecular structure prediction model. The system includes a memory for storing one or more instructions, and at least one processor for executing the one or more instructions stored in the memory, wherein the at least one processor can, by executing the one or more instructions, acquire an image of a first molecular structure, acquire a graph of the first molecular structure determined using the molecular structure prediction model, perform image rendering on the image of the first molecular structure based on the graph of the first molecular structure, and determine the reliability of the graph of the first molecular structure based on the image rendering result and the graph of the first molecular structure.

[0007] In one embodiment, the at least one processor identifies at least one of the first and second components based on a graph of the first molecular structure, identifies a first portion corresponding to the first component in the image of the first molecular structure, identifies a second portion corresponding to the second component in the image of the first molecular structure, and performs image rendering by separating the first portion and the second portion with different labels, wherein each of the first and second components may include one of a first atom, a second atom, a first bond, and a second bond.

[0008] In one embodiment, the molecular structure prediction model may include a first learning model, which may be trained to extract a chemical table file graph from molecular structural image input.

[0009] In one embodiment, the at least one processor can take the image rendering result and the graph of the first molecular structure as input and output a confidence score using a second learning model.

[0010] In one embodiment, the second learning model may include an image backbone model for extracting features of the image rendering result; a graph backbone model for extracting features of the graph of the first molecular structure; a feature concatenation unit for linking the features of the image rendering result and the features of the graph of the first molecular structure; and a linear layer model that determines confidence using the output of the feature concatenation unit as input.

[0011] In one embodiment, the second learning model may be trained to output a first value when the image rendering result and the graph of the first molecular structure match, and to output a second value when the image rendering result and the graph of the first molecular structure do not match.

[0012] In one embodiment, graphs of molecular structures whose reliability is above a predetermined level can be stored in a database.

[0013] One embodiment of the present disclosure aims to provide a method for measuring the reliability of a molecular structure prediction model, which is performed by at least one processor. The method may include the steps of: acquiring an image of a first molecular structure; acquiring a graph of the first molecular structure using the molecular structure prediction model; performing image rendering on the image of the first molecular structure based on the graph of the first molecular structure; and determining the reliability of the graph of the first molecular structure based on the image rendering result and the graph of the first molecular structure.

[0014] In one embodiment, the step of performing image rendering on an image of the first molecular structure includes: identifying at least one of a first component and a second component based on a graph of the first molecular structure; identifying a first portion in the image of the first molecular structure that corresponds to the first component; identifying a second portion in the image of the first molecular structure that corresponds to the second component; and performing image rendering by separating the first portion and the second portion with different labels, wherein each of the first component and the second component may include one of a first atom, a second atom, a first bond, and a second bond.

[0015] In one embodiment, the molecular structure prediction model may include a first learning model, which may be trained to extract a chemical table file graph from molecular structural image input.

[0016] In one embodiment, the step of outputting the confidence level of the graph of the first molecular structure may include the step of outputting the confidence level of the graph of the first molecular structure using a second learning model, with the image rendering result and the graph of the first molecular structure as input.

[0017] In one embodiment, the second learning model may include an image backbone model for extracting features of the image rendering result; a graph backbone model for extracting features of the graph of the first molecular structure; a feature concatenation unit for linking the features of the image rendering result and the features of the graph of the first molecular structure; and a linear layer model that determines confidence using the output of the feature concatenation unit as input.

[0018] In one embodiment, the second learning model may be trained to output a first value when the image rendering result and the graph of the first molecular structure match, and to output a second value when the image rendering result and the graph of the first molecular structure do not match.

[0019] In one embodiment, graphs of molecular structures having the reliability equal to or greater than a predetermined value may be stored in a database.

[0020] One embodiment of the present disclosure includes a program stored in a recording medium to cause a computer to execute the method according to one embodiment of the present disclosure.

[0021] One embodiment of the present disclosure includes a computer-readable recording medium having recorded thereon a program for causing a computer to execute the method according to one embodiment of the present disclosure.

[0022] One embodiment of the present disclosure includes a computer-readable recording medium having recorded thereon a database used in one embodiment of the present disclosure. Effects of the Invention

[0023] According to one embodiment of the present disclosure, by additionally providing the reliability of a result of a prediction model, accuracy information for the result of the prediction model can be provided, which can further help a user determine what kind of information is to be stored in the database. Brief Description of the Drawings

[0024] [Figure 1] It is a diagram illustrating a method of extracting a molecular structural formula from an image using a first learning model according to an embodiment of the present disclosure. [Figure 2] It is a flowchart illustrating a method of providing reliability of a result value of a first learning model using a second learning model according to an embodiment of the present disclosure. [Figure 3] It is a flowchart illustrating a method of measuring reliability of a molecular structure prediction model according to an embodiment of the present disclosure. [Figure 4a] It is a diagram illustrating a method of measuring reliability when a molecular structure prediction model according to an embodiment of the present disclosure makes an incorrect prediction. [Figure 4b] It is a diagram illustrating a method of measuring reliability when a molecular structure prediction model according to an embodiment of the present disclosure makes a correct prediction. [Figure 5] It is an illustrative diagram of a second learning model according to an embodiment of the present disclosure. [Figure 6] It is a block diagram of a reliability measurement apparatus for a molecular structure prediction model according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0025] To clarify the technical idea of the present disclosure, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In describing the present disclosure, if it is determined that a specific description of related well-known functions and components will unnecessarily obscure the gist of the present disclosure, detailed description thereof will be omitted. In the drawings, components having substantially the same function or configuration are denoted by the same reference numerals and symbols as much as possible even when they are shown in other drawings. Also, for convenience of explanation, both the apparatus and the method may be described together as necessary. Note that the operations described in the present disclosure do not necessarily need to be performed in the described order, and may be performed in parallel, selectively, or individually.

[0026] The terms used in the embodiments of the present disclosure are selected from general terms that are currently widely used as much as possible in consideration of their functions in the present disclosure. However, these terms may be changed depending on the intention of engineers engaged in the relevant technical field, judicial precedents, or the emergence of new technologies. In addition, in certain cases, some terms are arbitrarily selected by the applicant, and in such cases, the meanings of these terms will be described in detail in the description of the relevant embodiments. Therefore, the terms used in this specification should be defined based not only on the names of the terms themselves, but also on the meanings of the terms and the overall content of the present disclosure.

[0027] Throughout this disclosure, singular expressions can be pluralized unless the context clearly indicates otherwise. Terms such as “contains” or “has” should be understood as intending to specify the presence of a feature, number, step, action, component, part, or combination thereof, and not as preemptively excluding the possibility of the presence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. In other words, throughout this disclosure, when a part “contains” a component, it means that, unless otherwise stated, it may contain other components rather than excluding them.

[0028] The phrase "at least one" modifies the entire list of components, not the individual components within that list. For example, the phrases "at least one of A, B, and C" and "at least one of A, B, or C" refer to "A only," "B only," "C only," "both A and B," "both B and C," "both A and C," "all of A, B, and C," or any combination thereof.

[0029] Furthermore, terms such as "...part" and "...module" as used in this disclosure mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software.

[0030] Throughout this disclosure, "connected" to another part includes not only "directly connected" parts but also "electrically connected" parts with other materials interposed between them. Furthermore, "contains" a component of a part, unless otherwise stated, means that it may contain other components rather than excluding them.

[0031] Throughout this disclosure, the expression “configured to” may be replaced, depending on the context, with expressions such as “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” does not necessarily mean “specifically designed to” in terms of hardware. Instead, in some contexts, the expression “a system configured to” may mean that the system is “capable” to do something in combination with other devices or components. For example, the expression “a processor configured to perform A, B, and C” may mean a dedicated processor for performing those operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) capable of performing those operations by running one or more software programs stored in memory.

[0032] The artificial intelligence-related functions relating to this disclosure operate via a processor and memory. The processor may consist of one or more processors. In this case, one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. One or more processors are controlled to process input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed as hardware structures specialized for processing a particular artificial intelligence model.

[0033] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, "created through learning" means that a basic artificial intelligence model is trained using a learning algorithm with a large amount of training data to create predefined rules of operation or artificial intelligence models configured to perform desired characteristics (or objectives). Such learning may be performed on the device on which the artificial intelligence relating to this disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0034] Throughout this disclosure, the term "device" may include, but is not limited to, servers, smartphones, tablet PCs, PCs, TVs, smart TVs, mobile phones, PDAs (personal digital assistants), speakers, laptops, media players, microservers, e-book object recognition devices, digital broadcasting object recognition devices, kiosks, MP3 players, digital cameras, robotic vacuum cleaners, home appliances, other mobile or non-mobile computing devices, watches, glasses, hairbands, and rings equipped with communication and data processing functions. In one embodiment, the device may run a web or module-based application related to the system. For example, a molecular structure prediction model confidence measurement device may refer to a server, on which a web-based application related to a system for measuring the confidence of a molecular structure prediction model may run. In other words, the server may provide a web service or software for measuring the confidence of a molecular structure prediction model.

[0035] One embodiment of this disclosure aims to predict a molecular structural formula from an image of a molecular structure and to provide a confidence level for the predicted molecular structural formula. However, this disclosure is not limited to embodiments of molecular structures and can be applied to a variety of technical fields that extract image information from non-textual images.

[0036] Figure 1 shows a method for extracting molecular structural formulas from an image using a first learning model according to one embodiment of this disclosure.

[0037] Referring to Figure 1, the device including a molecular structure prediction model can identify an image 120 of a molecular structure from document 110, analyze the image 120, and extract the molecular structure corresponding to the image. In one embodiment, one device may include a molecular structure prediction model and a confidence measurement model, and the molecular structure prediction model and the confidence measurement model may be included in different devices, each device being able to calculate output values ​​using its respective model.

[0038] In one embodiment, a device including a molecular structure prediction model can extract an image 120 of a molecular structure from a document 110. For example, the device including the molecular structure prediction model can extract the image 120 of a molecular structure from a document 110 using panoptic segmentation technology. Furthermore, the device including the molecular structure prediction model can identify atoms and interatomic bonds contained in the image 120 based on the image. In one embodiment, the device including the molecular structure prediction model can analyze the types and positions of atoms, the types and positions of bonds, etc., contained in the image 120. For example, vertices implicitly representing carbon, atoms, superatoms, etc., and their positions, and the types of bonds between them (e.g., single / double / triple bonds, up / down bonds, etc.) can be identified. Optical character recognition (OCR) technology may be used to analyze the types of atoms or superatoms. In one embodiment, the device including a molecular structure prediction model can determine a linear notation structural formula 130 corresponding to an image 120 of the molecular structure based on the types and positions of identified atoms, the types and positions of bonds, etc. Linear notation structural formulas include, but are not limited to, SMILES (Simplified Molecular Input Line Entry System) notation, InChI (International Chemical Identifier), WLN (Wiswesser line notation), ROSDAL (Representation of Organic Structure Descriptions Arranged Linearly), SLN (SYBYL Line Notation), etc. Furthermore, the device including the molecular structure prediction model can determine a graph corresponding to the linear notation structural formula 130. The graph corresponding to the linear notation structural formula 130 includes a CT file (Chemical table file) graph, and for example, a Molfile graph.

[0039] In providing the predicted linear notation structural formulas 130 or graphs in this manner, conventional methods have been used to determine the reliability of the predicted results by creating chemical structural formulas from the predicted results and measuring the reliability if the structural formula is actually capable of bonding. However, since such methods do not take actual images into consideration, even if the prediction is incorrect, if the structural formula is judged to be actually capable of bonding, it is highly likely that a high confidence score will be output and erroneous data will be accumulated. Therefore, a method is needed to provide a more accurate confidence score for the prediction model, which will be described in more detail later with reference to Figures 2 to 5.

[0040] Figure 2 is a flowchart illustrating a method for providing the confidence level of the result values ​​of the first learning model using a second learning model according to one embodiment of the present disclosure.

[0041] Referring to Figure 2, a graph 203 of the predicted first molecular structure corresponding to the image 201 of the first molecular structure can be obtained using the molecular structure prediction model 210. The method for obtaining the graph 203 of the predicted first molecular structure can be the method described above, referring to Figure 1.

[0042] In one embodiment, image rendering may be performed based on a graph 203 of a first molecular structure predicted by a molecular structure prediction model 210. Image rendering may be performed by identifying a plurality of components based on the graph of the first molecular structure, identifying the portion corresponding to each of the plurality of components, and displaying them in the original image of the first molecular structure, image 201, with each different component separated by different labels. Furthermore, the position of each of the plurality of components may also be identified in order to identify the portion corresponding to each of the plurality of components in the image 201 of the first molecular structure. The plurality of components may include a first atom, a second atom, a third atom, a first bond, a second bond, a third bond, and so on. For example, referring to the example in Figure 2, the reliability measurement device for the molecular structure prediction model identifies bromine (Br), four nitrogen atoms (N), eight vertices representing carbon (C), seven single bonds, five double bonds, and one triple bond based on the graph 203 of the first molecular structure. Elements are represented by circles, and bonds by line segments. Different elements or different bonds can be represented using different colors. This allows for the acquisition of the image rendering result 205 in Figure 2.

[0043] In one embodiment, the image rendering result 205 and the predicted graph of the first molecular structure 203 can be used as inputs to a confidence model 230, and a confidence score 207 can be obtained according to the similarity level of the two input pieces of information. The more similar the input pieces of information are, the higher the confidence score 207 will be, and the more dissimilar they are, the lower the confidence score 207 will be.

[0044] Figure 3 is a flowchart showing a method for measuring the reliability of a molecular structure prediction model according to one embodiment of this disclosure.

[0045] Referring to Figure 3, in operation 310, the confidence measurement device for the molecular structure prediction model can acquire an image of the first molecular structure. In one embodiment, the confidence measurement device for the molecular structure prediction model can acquire an image of the first molecular structure by directly extracting an image of the first molecular structure from a document, or by acquiring an image of the first molecular structure extracted from another device. In other words, the confidence measurement device for the molecular structure prediction model or an external device can extract an image of the first molecular structure using an AI (artificial intelligence) algorithm or a predefined operating rule.

[0046] In operation 330, the confidence measurement device for the molecular structure prediction model can acquire a graph of the first molecular structure determined using the molecular structure prediction model. That is, the graph of the first molecular structure may be a graph corresponding to an image of the first molecular structure predicted by the molecular structure prediction model. In one embodiment, the molecular structure prediction model may include a first trained model that takes an image of a molecular structure as input and outputs a chemical table file (CT file) graph. The CT file graph may include information on each atom of the molecule, the xyz coordinate information of the corresponding atom, bond information between atoms, etc. Similar to operation 310, the confidence measurement device for the molecular structure prediction model can either directly acquire a graph of the first molecular structure determined using the molecular structure prediction model, or it can receive and acquire a graph of the first molecular structure determined using the molecular structure prediction model from an external device.

[0047] In operation 350, the reliability measuring device for the molecular structure prediction model can perform image rendering on an image of the first molecular structure based on a graph of the first molecular structure. In one embodiment, the reliability measuring device for the molecular structure prediction model can perform image rendering by identifying multiple components based on a graph of the first molecular structure, identifying the parts corresponding to each of the multiple components in the image of the first molecular structure, and separating the components that are different from each other with different labels. Here, the multiple components may include atoms, bonds between atoms, etc. Separating them with different labels can include, but is not limited to, separating them with different colors or different shapes, and can include a variety of forms in which they are separated and displayed in order to be distinguished from each other.

[0048] For example, if the first component is carbon and the second component is nitrogen, the reliability measurement device for the molecular structure prediction model can identify the position coordinates of the carbon atoms, which are the first component, and the position coordinates of the nitrogen atoms, which are the second component, in the graph of the first molecular structure. Based on these position coordinates, it can identify the first part corresponding to the first component and the second part corresponding to the second component in the image of the first molecular structure, and perform image rendering by distinguishing them with different labels, such as displaying the first part in yellow and the second part in red.

[0049] To give another example, if the first component is carbon, the second component is nitrogen, the third component is a single bond, and the fourth component is a double bond, the reliability measurement device for the molecular structure prediction model can identify the positions of carbon atoms (the first component), nitrogen atoms (the second component), single bonds (the third component), and double bonds (the fourth component) in the graph of the first molecular structure. Furthermore, based on the position of each component, the reliability measurement device for the molecular structure prediction model can identify the first part corresponding to the first component, the second part corresponding to the second component, the third part corresponding to the third component, and the fourth part corresponding to the fourth component in the image of the first molecular structure. The first and second parts, which correspond to elements, are displayed in the form of circles, but in different colors to indicate different elements, and the third and fourth parts are displayed in the form of bars, but in different colors to indicate different bonds, thereby enabling image rendering.

[0050] In operation 370, the reliability measuring device for the molecular structure prediction model can determine the reliability of the graph of the first molecular structure based on the image rendering result and the graph of the first molecular structure. In one embodiment, the reliability of the graph of the first molecular structure can be determined using a second learning model. The second learning model may be a learning model that takes the image rendering result and the graph of the first molecular structure as input and outputs a reliability value (e.g., a reliability score). In one embodiment, the learning model may include an image backbone network model that extracts features from the image rendering result, a graph backbone network model that extracts features from the graph of the first molecular structure, a feature coupling unit that connects the extracted features from the image rendering result and the extracted features from the graph of the first molecular structure, and an extraction network model that extracts reliability using the output of the feature coupling unit as input. This will be described in more detail later with reference to Figure 5.

[0051] In one embodiment, the second learning model may be trained to output a first value if the image rendering result and the graph of the first molecular structure match, and to output a second value if the image rendering result and the graph of the first molecular structure do not match. For example, the second learning model may be trained in the learning step to output 1 if the image rendering result and the graph of the first molecular structure match, and 0 if they do not match. Even in actual use after training, if the prediction result of the molecular structure prediction model is correctly predicted, it can extract a value similar to 1, and if there is an error in the prediction, it can output a value similar to 0. The confidence measuring device for the molecular structure prediction model can determine the confidence level of the graph of the first molecular structure based on the value extracted by the second learning model. For example, the confidence level may be determined by a confidence value (or score).

[0052] According to one embodiment, the reliability measurement device for a molecular structure prediction model provides reliability along with the predicted graph of the first molecular structure, thereby providing the accuracy of the predicted graph of the first molecular structure and helping to determine whether it can be used. Therefore, the user or the user's device can decide to store the graph in the database only if the reliability is above a predetermined level (or value), thereby enabling accurate data to be used in research and other applications.

[0053] Figure 4a shows a method for measuring the confidence level when a molecular structure prediction model according to one embodiment of this disclosure makes an incorrect prediction.

[0054] Referring to Figure 4a, the first molecular structure can be predicted by a molecular structure prediction model based on the image 410 of the first molecular structure. For example, the first molecular structure may be a structure in which single bonds and double bonds coexist. However, unlike humans, it is not possible to expect 100% accuracy when a machine predicts the first molecular structure using the image 410 of the first molecular structure. Therefore, a process is needed to verify whether the first molecular structure predicted based on the image 410 of the first molecular structure was correctly predicted.

[0055] In one embodiment, the reliability measurement device for a molecular structure prediction model can acquire an image 410 of the first molecular structure and a graph of the first molecular structure predicted by the first learning model. The reliability measurement device for a molecular structure prediction model can also identify the types of elements, the positions of elements, the types of bonds, the positions of bonds, etc., based on the predicted graph of the first molecular structure, and render this on the image 410 of the first molecular structure. For example, if the first bond 420 is identified as a double bond from the graph of the first molecular structure, the reliability measurement device for a molecular structure prediction model can generate an image rendering result 430 by displaying the bonds identified as double bonds, including the first bond 440, and the bonds identified as single bonds in different colors on the image 410 of the first molecular structure. That is, although the first bond 420 is actually a single bond, the image rendering result may show that the first bond 420 has a double bond.

[0056] In one embodiment, a molecular structure prediction model reliability measuring device can determine the reliability of a graph of a first molecular structure based on image rendering results. For example, the molecular structure prediction model reliability measuring device can identify elemental and bond portions using segmentation of image 410 of the first molecular structure, and can determine whether they are the same element or different elements, or whether they are the same bond or different bonds. If the segmentation results of image 410 of the first molecular structure and the image rendering results 430 are similar, the molecular structure prediction model reliability measuring device can determine that the reliability is high; if they are not similar, it can determine that the reliability is low. In the example in Figure 4a, the image rendering result 430 determines that the first bond 440 is a double bond, but the image 410 of the first molecular structure determines that the first bond 420 is a single bond, so the reliability of the graph of the first molecular structure may be determined to be low. Alternatively, the reliability score of the graph of the first molecular structure may be determined to be low.

[0057] Alternatively, in one embodiment, the reliability measurement device for the molecular structure prediction model can determine the reliability of the graph of the first molecular structure using both the image rendering result and the graph of the predicted first molecular structure. According to one embodiment, the graph of the molecular structure is also used in determining the reliability, thereby increasing the accuracy of the determined reliability. This will be described in more detail later with reference to Figure 5.

[0058] Figure 4b shows a method for measuring the confidence level when a molecular structure prediction model according to one embodiment of this disclosure makes a correct prediction.

[0059] Referring to Figure 4b, the second molecular structure can be predicted by a molecular structure prediction model based on image 450 of the second molecular structure. However, as explained in Figure 4a above, unlike humans, a machine cannot be expected to predict the second molecular structure using image 450 with 100% accuracy. Therefore, a process is needed to verify whether the second molecular structure predicted based on image 450 was correct.

[0060] In one embodiment, the molecular structure prediction model confidence measurement device can acquire an image 450 of the second molecular structure and a graph of the second molecular structure predicted by the first learning model. The molecular structure prediction model confidence measurement device can also identify the types of elements, the positions of elements, the types of bonds, the positions of bonds, etc., based on the predicted graph of the second molecular structure, and render this on the image 450 of the second molecular structure. For example, if the graph of the second molecular structure shows that the bonds excluding the second bond 460 and the third bond 470 are single bonds, and the second bond 460 is a bond that extends forward in the plane, and the third bond 470 is a bond that recedes backward in the plane, the molecular structure prediction model confidence measurement device can generate an image rendering result 480 by displaying the second bond 490, the third bond 495, and the other bonds in different colors on the image 450 of the second molecular structure. Furthermore, the molecular structure prediction model confidence measurement device can also generate an image rendering result 480 that shows the bond positions of the second and third bonds.

[0061] In one embodiment, the reliability measuring device for a molecular structure prediction model can determine the reliability of the graph of the second molecular structure based on the image rendering result 480. For example, the reliability measuring device for a molecular structure prediction model can identify elemental and bonding parts using segmentation of the image 450 of the second molecular structure, and can determine whether they are the same element or different elements, or whether they are the same bond or different bond. If the segmentation result of the image 450 of the second molecular structure and the image rendering result 480 are similar, the reliability measuring device for a molecular structure prediction model can determine that the reliability is high; if they are not similar, it can determine that the reliability is low. In the example in Figure 4b, since the image rendering result 480 and the image 450 of the second molecular structure are similar, the reliability of the graph of the second molecular structure can be determined to be high. Alternatively, the reliability score of the graph of the second molecular structure can be determined to be high.

[0062] Alternatively, in one embodiment, the reliability measurement device for the molecular structure prediction model can determine the reliability of the graph of the first molecular structure based on the image rendering results and the graph of the predicted first molecular structure. This will be described in more detail later with reference to Figure 5.

[0063] Figure 5 is an illustrative diagram of a second learning model according to one embodiment of the present disclosure.

[0064] Referring to Figure 5, the second learning model 530 for measuring confidence may include an image backbone model 540 that extracts features from the image rendering result 510, a graph backbone model 550 that extracts features from the molecular structure graph 520, a feature concatenation unit 560 that connects the extracted features from the image rendering result 510 and the extracted features from the molecular structure graph 520, and a network model 570 that determines confidence using the output of the feature concatenation unit 560 as input. The network model 570 that determines confidence may also be a model that classifies the concatenated features through a linear layer.

[0065] In one embodiment, the second learning model 530 may be an artificial intelligence model that extracts a confidence value 580 using an image rendering result 510 and a molecular structure graph 520 as inputs. In this case, the image rendering result 510 becomes the input to the image backbone model 540, and the molecular structure graph 520 becomes the input to the graph backbone model 550, which can be inputs to different network models. The image backbone model 540 can extract features from the image rendering result 510, and the graph backbone model 550 can extract features from the molecular structure graph 520. The feature concatenation unit 560 concatenates the features extracted from each backbone model, and the network model 570 that determines the confidence level can determine a confidence value based on the extracted features. For example, it can output a higher value the higher the confidence level, and a lower value the lower the confidence level.

[0066] Figure 6 is a block diagram of a reliability measurement device for a molecular structure prediction model according to one embodiment of this disclosure.

[0067] Referring to Figure 6, the molecular structure prediction model reliability measurement device 600 may include a transmitting / receiving unit 610, a memory 620, a database 630, and a processor 640. However, not all of the components shown in Figure 6 are essential components of the molecular structure prediction model reliability measurement device 600. The molecular structure prediction model reliability measurement device 600 may be realized with more components than those shown in Figure 6, or with fewer components than those shown in Figure 6. Furthermore, the transmitting / receiving unit 610, the memory 620, and the processor 640 may be realized in the form of a single chip.

[0068] In one embodiment, the transmitting / receiving unit 610 can communicate with a terminal or other electronic device connected by wire or wirelessly to the reliability measuring device 600 of the molecular structure prediction model. For example, the transmitting / receiving unit 610 can acquire an image of the first molecular structure, a graph of the first molecular structure determined using the molecular structure prediction model, etc., from the other electronic device.

[0069] Memory 620 can install and store various types of data, such as applications, programs, and files. The processor 640 can access and use the data stored in memory 620, or store new data in memory 620. Memory 620 can also store one or more instructions. The processor 640 can execute one or more instructions stored in memory.

[0070] The processor 640 controls the overall operation of the molecular structure prediction model confidence measuring device 600 and may include at least one processor such as a CPU or GPU. The processor 640 can control other components included in the molecular structure prediction model confidence measuring device 600 to perform operations to operate the molecular structure prediction model confidence measuring device 600. For example, the processor 640 can acquire an image of a first molecular structure, acquire a graph of the first molecular structure determined using the molecular structure prediction model, perform image rendering on the image of the first molecular structure based on the graph of the first molecular structure, and determine the confidence of the graph of the first molecular structure based on the results of the image rendering and the graph of the first molecular structure.

[0071] The database 630 can store various training data for training the learning model. The database 630 may also store material information, phase information, simulation result information, and in various embodiments, calculated data output by the learning model. In Figure 6, the molecular structure prediction model reliability measurement device 600 is shown to include the database 630, but the database 630 may be located outside the device. In this case, the database 630 can be connected to the molecular structure prediction model reliability measurement device 600 via wired or wireless connection.

[0072] Furthermore, the learning model may be implemented outside the molecular structure prediction model confidence measurement device 600 (for example, on a cloud-based basis), or it may be included inside the molecular structure prediction model confidence measurement device 600.

[0073] One embodiment of the present disclosure may also be realized in the form of a recording medium containing computer-executable instructions, such as a program module executed by a computer. Computer-readable media are any available media accessible by a computer, and include all volatile and non-volatile media, removable and non-removable media. Computer-readable media may also include both computer storage media and communication media. Computer storage media include all volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media generally include computer-readable instructions, data structures, or program modules, and include any information transmission media.

[0074] The descriptions of this disclosure described herein are illustrative and will be readily understood by those with ordinary skill in the art to which this disclosure pertains, as they can be easily modified into other specific forms without altering the technical idea or essential features of the invention. Accordingly, the embodiments described above are illustrative in all respects and should not be construed as limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0075] The scope of this disclosure is defined by the claims, which are set forth below rather than by the detailed description above, and all modifications or alterations derived from the meaning and scope of the claims, as well as the concept of equivalents thereof, should be interpreted as being included within the scope of this disclosure.

Claims

1. A system for measuring the reliability of molecular structure prediction models, Memory for storing one or more instructions; and Includes at least one processor that executes one or more instructions stored in the memory, The at least one processor executes the one or more instructions, We obtained an image of the first molecular structure, A graph of the first molecular structure determined using the molecular structure prediction model is obtained. Based on the graph of the first molecular structure, image rendering is performed on the image of the first molecular structure. A system for determining the reliability of the graph of the first molecular structure based on the image rendering results and the graph of the first molecular structure.

2. The aforementioned at least one processor is Based on the graph of the first molecular structure, at least one of the first and second components is identified. In the image of the first molecular structure, the first portion corresponding to the first component is identified, In the image of the first molecular structure, the second portion corresponding to the second component is identified, The image rendering is performed by dividing the first part and the second part with different indicators. Each of the first and second components is, The system according to claim 1, comprising one of a first atom, a second atom, a first bond, and a second bond.

3. The aforementioned molecular structure prediction model is Including the first learning model, The first learning model described above is The system according to claim 1, which is trained to extract a chemical table file graph from a molecular structural formula image as input.

4. The aforementioned at least one processor is The system according to claim 1, wherein the image rendering result and the graph of the first molecular structure are inputs, and a confidence score is output using a second learning model.

5. The second learning model described above is An image backbone model for extracting features from the aforementioned image rendering results; A graph backbone model for extracting the characteristic features of the graph of the first molecular structure; A feature linking section that connects the features of the image rendering result and the features of the graph of the first molecular structure; and The system according to claim 4, comprising a linear layer model that determines the reliability using the output of the feature coupling section as input.

6. The second learning model described above is The system according to claim 4, which is trained to output a first value when the image rendering result and the graph of the first molecular structure match, and to output a second value when the image rendering result and the graph of the first molecular structure do not match.

7. The system according to claim 1, wherein graphs of molecular structures whose reliability is above a predetermined level are stored in a database.

8. A method for measuring the reliability of a molecular structure prediction model, which is performed by at least one processor, Steps to obtain an image of the first molecular structure; A step of obtaining a graph of the first molecular structure using the molecular structure prediction model; A step of rendering an image of the first molecular structure based on the graph of the first molecular structure; and A method comprising the step of determining the reliability of the graph of the first molecular structure based on the image rendering results and the graph of the first molecular structure.

9. The step of performing image rendering on the image of the first molecular structure is: A step of identifying at least one of the first and second components based on the graph of the first molecular structure; A step of identifying a first portion corresponding to the first component in the image of the first molecular structure; A step of identifying a second portion corresponding to the second component in the image of the first molecular structure; and The step of rendering the image by separating the first part and the second part with different indicators is included. Each of the first and second components is, The method according to claim 8, comprising one of a first atom, a second atom, a first bond, and a second bond.

10. The aforementioned molecular structure prediction model is Including the first learning model, The first learning model described above is The method according to claim 8, which is trained to extract a chemical table file graph from a molecular structural image as input.

11. The step of outputting the confidence level of the graph of the first molecular structure is: The method according to claim 8, further comprising the step of outputting the confidence level of the graph of the first molecular structure using a second learning model, with the image rendering result and the graph of the first molecular structure as input.

12. The second learning model described above is An image backbone model for extracting features from the aforementioned image rendering results; A graph backbone model for extracting the characteristic features of the graph of the first molecular structure; A feature linking section that connects the features of the image rendering result and the features of the graph of the first molecular structure; and The method according to claim 11, comprising a linear layer model that determines the reliability using the output of the feature coupling section as input.

13. The second learning model described above is The method according to claim 11, wherein the system is trained to output a first value when the image rendering result and the graph of the first molecular structure match, and to output a second value when the image rendering result and the graph of the first molecular structure do not match.

14. The method according to claim 8, wherein graphs of molecular structures whose confidence level is above a predetermined value are stored in a database.

15. A program stored on a computer-readable recording medium to cause a computer to perform the method according to any one of claims 8 to 14.

Images