Chemical synthesis literature information extraction method and system and storage medium

Abstract

The invention provides a chemical synthesis literature information extraction method. The chemical synthesis literature information extraction method comprises the steps of obtaining a chemical combination reaction schematic diagram and a chemical combination reaction text in chemical literatures; detecting a target entity of the combination reaction schematic diagram through a target detection model; identifying reaction equation information in the chemical combination reaction schematic diagram by utilizing a reaction equation detection model; fusing and filtering composition entities of the target entities and the reaction equation information to obtain reaction data and residual target entities after filtering; determining a compound name corresponding to the molecular structure based on the reaction data of the molecular structure type; performing character string matching on the compound name and the chemical combination reaction text, and fusing the successfully matched compound name and corresponding reaction information into a cue word of a large language model; and performing reaction parameter extraction on the combination reaction text by utilizing a large language model, and combining an extraction result with the reaction data and the filtered residual target entities to determine complete combination reaction parameters.

Description

Technical Field

[0001] This invention relates to the field of chemical synthesis information extraction, and more particularly to a method for extracting chemical synthesis literature information, a system for extracting chemical synthesis literature information, and a computer-readable storage medium. Background Technology

[0002] Information extraction technology is a key means of automating the parsing and conversion of unstructured data into structured data. Scientific literature (such as journal articles and patents) has always been an important source of information in the field of organic synthetic chemistry, providing methods for specific chemical reactions or synthetic steps of interest. To reduce the time and cost required for information retrieval and facilitate the acquisition of reaction data, current technologies have focused on building structured databases from unstructured literature, such as Reaxys and SciFinder. The content of these databases is usually manually extracted from the literature by human experts, which is expensive, time-consuming, and requires specialized knowledge, especially given the exponential growth of scientific chemistry publications in recent years.

[0003] Automatically extracting reaction data from unstructured literature data can address this challenge. However, existing technologies primarily target single sources of information, such as extracting structured reaction parameters (including substrate, product, catalyst, solvent, etc.) from a text describing a chemical reaction, or extracting reaction parameters or SMILES representations of molecular structures from schematic diagrams in literature. Furthermore, the vast amounts of reaction data described in literature are scattered across different contexts and have different modalities; even the same reaction data may exist in different forms. Therefore, chemical reasoning is often required to fully determine the information during extraction.

[0004] It is evident that existing technologies can extract structured information from a single modality, but they lack precision and overlook the fact that chemical reaction diagrams are summaries of textual descriptions. The structured information of chemical reactions in text and image modalities is fragmented, information mining is incomplete, and no correlation between modalities has been established.

[0005] In order to overcome the above-mentioned defects of the existing technology, there is an urgent need in this field for a chemical synthesis literature information extraction technology that can fully mine the information contained in the chemical reaction diagram and integrate the information contained in the chemical reaction diagram into the process of text modality structured information extraction, so as to integrate the schematic information of different modalities and improve the accuracy and completeness of information extraction. Summary of the Invention

[0006] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0007] To overcome the aforementioned deficiencies in the existing technology, this invention provides a method for extracting chemical synthesis literature information, a system for extracting chemical synthesis literature information, and a computer-readable storage medium. This method enables comprehensive mining of the information contained in chemical reaction diagrams and integrates the information contained in the chemical reaction diagrams into the process of extracting structured information from text modalities. This integrates schematic information from different modalities, thereby improving the accuracy and completeness of information extraction.

[0008] Specifically, the chemical synthesis literature information extraction method provided by the first aspect of the present invention includes the following steps: acquiring chemical reaction diagrams and chemical reaction texts from chemical literature; detecting target entities in the chemical reaction diagrams using a target detection model; identifying reaction equation information in the chemical reaction diagrams using a reaction equation detection model, wherein the constituent entities of the reaction equation information include reaction substrates, reaction conditions, and reaction products; fusing and filtering the target entities and the constituent entities of the reaction equation information to obtain reaction data and the remaining target entities after filtering; determining the compound name corresponding to the molecular structure based on the reaction data of the molecular structure type; performing string matching between the compound name and the chemical reaction text, and integrating the successfully matched compound name and corresponding reaction information into the prompt words of a large language model; and extracting reaction parameters from the chemical reaction text using the large language model, and combining the extraction results with the reaction data and the remaining target entities after filtering to determine the complete chemical reaction parameters.

[0009] Preferably, in one embodiment of the present invention, the pre-training process of the object detection model includes the steps of: obtaining a pre-training dataset; annotating the schematic diagrams in the pre-training dataset based on the defined type of the target entity; and training the object detection model using the annotated pre-training dataset, wherein the object detection model is built based on YOLO11.

[0010] Preferably, in one embodiment of the present invention, the target entity and the constituent entities of the reaction equation information are fused and filtered based on bounding boxes. The fusion and filtering steps include: in response to the constituent entity being of type molecular structure, traversing the bounding boxes of the target entity of type molecular structure based on the bounding box of the constituent entity; removing the target entity when the cross-union ratio between the bounding box of the constituent entity and the bounding box of the target entity is greater than a preset value; calculating the distance between the bounding box of the target entity of type identifier and the bounding box of the constituent entity; identifying and binding the constituent entity with the nearest target entity of type identifier; and repeating the above steps until all constituent entities of molecular structure type have been traversed.

[0011] Preferably, in one embodiment of the present invention, the step of calculating the distance between the bounding box of the target entity of type identifier and the bounding box of the constituent entity includes: expanding the bounding box range outward based on the bounding box of the constituent entity; and determining the distance between the target entity of type identifier and the constituent entity based on the bounding box center point in response to the target entity of type identifier being located within the new bounding box range.

[0012] Preferably, in one embodiment of the present invention, the step of fusing and filtering the target entity and the constituent entities of the reaction equation information based on bounding boxes includes: in response to the constituent entity being of type text, traversing the bounding boxes of the target entity of type text based on the bounding boxes of the constituent entities; removing the target entity when the intersection-union ratio between the bounding boxes of the constituent entities and the bounding boxes of the target entity is greater than a preset value; performing OCR text recognition and classification on the constituent entities using a large language model; and repeating the above steps until all constituent entities of text types have been traversed.

[0013] Preferably, in one embodiment of the present invention, the step of determining the compound name corresponding to the molecular structure based on the reaction data of the molecular structure type includes: converting the molecular structure entity in the reaction data of the molecular structure type into a SMILES representation; and determining the corresponding compound name in a compound name query database according to the SMILES representation, wherein the corresponding compound name includes synonymous compound names.

[0014] Preferably, in one embodiment of the present invention, the compound name lookup database is constructed based on the PubChem public dataset.

[0015] Preferably, in one embodiment of the present invention, the complete chemical reaction parameters include reaction substrate, reaction product, solvent, catalyst, general reagent, reaction temperature, reaction time and / or yield / proportion.

[0016] Furthermore, the chemical synthesis literature information extraction system provided according to the second aspect of the present invention includes a memory and a processor. The memory stores computer instructions. The processor is connected to the memory and configured to execute the computer instructions stored in the memory to implement the chemical synthesis literature information extraction method provided in any of the above embodiments.

[0017] Furthermore, the computer-readable storage medium provided according to the third aspect of the present invention stores computer instructions. When the computer instructions are executed by a processor, the chemical synthesis literature information extraction method provided in any of the above embodiments is implemented. Attached Figure Description

[0018] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0019] Figure 1 A schematic diagram of a chemical synthesis literature information extraction system provided according to some embodiments of the present invention is shown;

[0020] Figure 2 A flowchart of a method for extracting chemical synthesis literature information according to some embodiments of the present invention is shown;

[0021] Figure 3 A schematic diagram of a chemical literature is shown;

[0022] Figure 4 A flowchart illustrating information extraction from a chemical reaction diagram according to some embodiments of the present invention is shown.

[0023] Figure 5 A schematic diagram illustrating the pre-training process of a target detection model provided according to some embodiments of the present invention is shown;

[0024] Figure 6 The illustration shows an execution diagram of fusion and filtering using bounding boxes according to some embodiments of the present invention;

[0025] Figure 7 An overall flowchart of a chemical synthesis literature information extraction method according to some embodiments of the present invention is shown;

[0026] Figure 8The evaluation results of the trained object detection model on the validation set are shown; and

[0027] Figure 9 The evaluation results of the trained object detection model on the validation set are shown.

[0028] Figure label:

[0029] 100: Image processing device;

[0030] 110: Memory;

[0031] 111: Computer-readable storage medium;

[0032] 120: Processor;

[0033] S210~S270: Steps;

[0034] 301: Schematic diagram of a combination reaction; and

[0035] 302: Combination reaction text. Detailed Implementation

[0036] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the aspects described below with reference to the accompanying drawings and specific embodiments are merely exemplary and should not be construed as limiting the scope of protection of the present invention in any way.

[0037] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0038] Furthermore, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," and "vertical" used in the following description should be understood as the orientations shown in the relevant paragraphs and accompanying drawings. These relative terms are for illustrative purposes only and do not imply that the described apparatus must be manufactured or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0039] It is understood that although terms such as "first," "second," and "third" may be used herein to describe various components, regions, layers, and / or parts, these components, regions, layers, and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers, and / or parts. Therefore, the first components, regions, layers, and / or parts discussed below may be referred to as second components, regions, layers, and / or parts without departing from some embodiments of the present invention.

[0040] As mentioned above, existing technologies primarily target single sources of information, such as extracting structured reaction parameters (including substrate, product, catalyst, solvent, etc.) from a text describing a chemical reaction, or extracting reaction parameters or SMILES representations of molecular structures from schematic diagrams in literature. Furthermore, the vast amounts of reaction data described in literature are scattered across different contexts and have different modalities; even the same reaction data may exist in different forms of expression. Therefore, chemical reasoning is often required to fully determine the information extracted.

[0041] It is evident that existing technologies can extract structured information from a single modality, but they lack precision and overlook the fact that chemical reaction diagrams are summaries of textual descriptions. The structured information of chemical reactions in text and image modalities is fragmented, information mining is incomplete, and no correlation between modalities has been established.

[0042] To overcome the aforementioned deficiencies in the existing technology, this invention provides a method for extracting chemical synthesis literature information, a system for extracting chemical synthesis literature information, and a computer-readable storage medium. This method enables comprehensive mining of the information contained in chemical reaction diagrams and integrates the information contained in the chemical reaction diagrams into the process of extracting structured information from text modalities. This integrates schematic information from different modalities, thereby improving the accuracy and completeness of information extraction.

[0043] In some non-limiting embodiments, the chemical synthesis literature information extraction method provided in the first aspect of the present invention can be implemented via the chemical synthesis literature information extraction system provided in the second aspect of the present invention.

[0044] Please refer to Figure 1 , Figure 1 A schematic diagram of a chemical synthesis literature information extraction system provided according to some embodiments of the present invention is shown.

[0045] like Figure 1As shown, the chemical synthesis literature information extraction system 100 may be configured with a memory 110 and a processor 120. The memory 110 includes, but is not limited to, the computer-readable storage medium 111 provided in the third aspect of the present invention, which stores computer instructions thereon. The processor 120 is connected to the memory 110 and is configured to execute the computer instructions stored in the memory 110 to implement the chemical synthesis literature information extraction method provided in the first aspect of the present invention.

[0046] The working principle of the above-mentioned chemical synthesis literature information extraction system will first be described with reference to some embodiments of chemical synthesis literature information extraction methods. Those skilled in the art will understand that these embodiments of chemical synthesis literature information extraction methods are merely non-limiting implementations provided by the present invention, intended to clearly demonstrate the main concepts of the invention and provide specific solutions convenient for public implementation, rather than limiting all functions or all working methods of the chemical synthesis literature information extraction system. Similarly, the chemical synthesis literature information extraction system is also only a non-limiting implementation provided by the present invention, and does not constitute a limitation on the executing entity and execution order of each step in these chemical synthesis literature information extraction methods.

[0047] Please refer to Figure 2 , Figure 2 A flowchart of a method for extracting chemical synthesis literature information according to some embodiments of the present invention is shown.

[0048] like Figure 2 As shown, the chemical synthesis literature information extraction method 200 may include step S210: obtaining the chemical reaction diagram and chemical reaction text from the chemical literature.

[0049] A chemical synthesis literature information extraction system can first acquire chemical literature from which the information to be extracted can be obtained. In some embodiments, the acquired chemical literature from which the information to be extracted can be as follows: Figure 3 As shown, chemical literature may include a chemical reaction diagram 301 and a chemical reaction text 302.

[0050] A chemical reaction diagram may include at least one reaction equation for the preparation of a product from a substrate. Furthermore, the chemical reaction diagram may also contain other reaction information, including reaction conditions, R group information, etc. In some embodiments, the collected chemical reaction diagrams may be saved in JPEG format.

[0051] Subsequently, the chemical synthesis literature information extraction system can execute step S220: detect the target entity in the chemical reaction diagram using the target detection model.

[0052] Figure 4A flowchart illustrating information extraction from a chemical reaction diagram according to some embodiments of the present invention is shown. Figure 4 As shown, for the structured reaction information extraction of chemical reaction diagrams in image modalities, the chemical synthesis literature information extraction system can first use a pre-trained target detection model to detect the chemical reaction diagrams to determine the target entities in the chemical reaction diagrams.

[0053] Here, the type of the target entity can include molecular structure, text, identifier, and / or supplementary information.

[0054] Molecular structure describes the three-dimensional arrangement of atoms in a molecule; compounds are typically represented using molecular structures in chemical reaction diagrams. Text provides supplementary information to the chemical reaction diagram, such as reaction conditions. Identifiers are numbers or letters used to denote molecular structures in the diagram. Supplementary information may include information about R groups, yield / proportion information, or molecular structure identifiers.

[0055] In some embodiments, the chemical synthesis literature information extraction method includes a pre-training process for a target detection model.

[0056] Please refer to Figure 5 , Figure 5 A schematic diagram of the pre-training process of an object detection model provided according to some embodiments of the present invention is shown.

[0057] like Figure 5 As shown, the chemical synthesis literature information extraction system can first obtain a pre-training dataset. The source data for the chemical reaction diagrams included in the pre-training dataset can be collected from organic chemistry journals such as the Journal of Organic Chemistry.

[0058] Next, based on the defined target entity types, the schematic diagrams in the pre-training dataset are labeled. The defined target entities are molecular structures, text, identifiers, and supporting information. In some embodiments, the chemical synthesis literature information extraction system can use the target detection data labeling tool LabelMe for data labeling. More preferably, when the downstream referencing reduction task is implemented using bounding boxes, the data labeling results, in addition to including the type of the labeled target entity, can also include the bounding boxes of the target entities. The bounding box labeling results can consist of the boundary coordinates of the upper left and lower right corners of the bounding box.

[0059] Then, an object detection model is trained using the labeled pre-trained dataset. The object detection model can be built based on YOLO11. In some embodiments, the constructed object detection model is a YOLO11x object detection model.

[0060] The trained object detection model can serve as a preliminary entry point for image information mining, detecting all target entities in the chemical reaction diagram. The input to the trained object detection model is the chemical reaction diagram, and the output is all entities in the diagram along with their corresponding entity types.

[0061] Please continue to refer to this. Figure 2 The method for extracting chemical synthesis literature information may include step S230: using a reaction equation detection model to identify reaction equation information in a chemical reaction diagram, wherein the constituent entities of the reaction equation information include the reaction substrate, reaction conditions, and reaction products.

[0062] like Figure 4 As shown, the chemical synthesis literature information extraction system can identify reaction equation information in chemical reaction diagrams using a reaction equation detection model. In some embodiments, the reaction equation detection model can be the RxnScribe reaction equation detection model. The RxnScribe reaction equation detection model, through the Pix2Seq object detection language modeling framework, defines a sequence representation for describing the reaction structure contained in the reaction equation diagram. It constructs sequences according to the hierarchy of entities, reaction roles, and complete reactions, and trains a sequence generation model for reaction structures by combining the image features of the reaction equation diagram, thereby achieving the function of predicting reaction structures.

[0063] Reaction equation information centers on the reaction substrate, reaction conditions, and reaction products. Therefore, the constituent entities of the reaction equation information in the chemical reaction diagram identified by the reaction equation detection model can include the reaction substrate, reaction conditions, and reaction products. The type of each constituent entity can include molecular structure and / or text. Typically, the reaction substrate and reaction products are molecular structure-type constituent entities, while the reaction conditions are text-type constituent entities.

[0064] Because reaction equation detection models only focus on reaction structure information and ignore other important additional information in chemical reaction diagrams, such as identifiers and R group referencing relationships, relying solely on the reaction equation information identified by these models to extract chemical reaction diagrams from chemical synthesis literature is incomplete and insufficient.

[0065] Please refer to Figure 2 and Figure 4The chemical synthesis literature information extraction system can perform step S240: fuse and filter the constituent entities of the target entity and the reaction equation information to obtain the reaction data and the remaining target entities after filtering.

[0066] Specifically, the fusion of the target entity and the constituent entities of the reaction equation information can be achieved by performing referential reduction and / or identifier binding processing on the target entity determined by the target detection model and the constituent entities of the reaction equation information determined by the reaction equation detection model. This process involves replacing the referential information in the target entity and the constituent entities of the reaction equation information that may point to the same entity, and binding the identifier to the entity of the molecular structure type.

[0067] For example, in some embodiments, the molecular structure in the chemical reaction diagram may contain an R group, which may be shown in a table or other place in the chemical reaction diagram. The chemical synthesis literature information extraction system can first bind the identifier corresponding to the R group to clarify the content referred to by the R group, and then perform the referencing replacement to replace the referencing content of the R group on the molecular structure.

[0068] In some embodiments, the molecular structure in the chemical reaction diagram can be identified by an identifier A, and the identifier A can be used to refer to the molecular structure elsewhere. The chemical synthesis literature information extraction system can first bind the identifier, let the identifier A represent the molecule, and then perform the referencing replacement.

[0069] Furthermore, because the target entities identified by the target detection model are more comprehensive and complete, the chemical synthesis literature information extraction system can filter out entities that might point to the same component entities as the reaction equation information from the target entities; that is, it removes duplicate entities from the target entities that are components of the reaction equation information. For example, the chemical synthesis literature information extraction system can filter out entities from the target entities that may have the same molecular structure as components of the reaction equation information, or it can filter out entities from the target entities that may have the same text as components of the reaction equation information.

[0070] The processed reaction equation information comprises various entities, i.e., reaction data. This reaction data can include molecular structure reaction data bound to identifiers and textual reaction data. By fusing and binding the identifier-type target entity with the corresponding molecular structure-type component entities of the reaction equation information, molecular structure reaction data can be obtained. Preferably, textual reaction data is obtained by performing OCR (Optical Character Recognition) text recognition and classification on the component entities of the text-type reaction equation information.

[0071] The remaining target entities after filtering are other important additional information in the chemical reaction diagram that was ignored by the reaction equation detection model.

[0072] Thus, the obtained reaction data and the filtered remaining target entities constitute the complete image extraction information extracted from the chemical synthesis literature information extraction system from the chemical reaction diagram.

[0073] Those skilled in the art will understand that the methods of referential reduction and identifier binding used to fuse the constituent entities of the target entity and the reaction equation information can include a variety of methods, such as using text rules based on symbol normalization, using placeholder replacement based on molecular graph expansion, traditional computer vision methods, feature extraction methods, and so on.

[0074] Preferably, in Figure 4 In the embodiments shown, the chemical synthesis literature information extraction system provided by the present invention can use bounding boxes to achieve the fusion and filtering of target entities and constituent entities of reaction equation information.

[0075] like Figure 5 As shown, during the pre-training process of the object detection model, when the chemical synthesis literature information extraction system annotates the schematic diagrams in the pre-training dataset, the annotation results can also include entity bounding boxes. Here, the annotated entity bounding boxes can be composed of the boundary coordinates of the upper left and lower right corners of the bounding box.

[0076] After that, as Figure 4 As shown, the results of the target detection model detecting target entities in a chemical reaction diagram can also include the boundary coordinates of all target entities, thereby determining the bounding boxes of all target entities. Similarly, the results of the reaction equation detection model identifying reaction equation information in a chemical reaction diagram can include the boundary coordinates of all entities that constitute the reaction equation information, thereby determining the bounding boxes of all entities that constitute the reaction equation information.

[0077] In other words, the target detection model predicts the boundary coordinates of all target entities and their corresponding type (molecular structure, text, identifier, or supporting information). The reaction equation detection model predicts all reaction equation information in the chemical reaction diagram. Each reaction equation includes three parts: reaction substrate, reaction conditions, and reaction products. That is, the constituent entities of the reaction equation information include the reaction substrate, reaction conditions, and reaction products. The prediction results from the reaction equation detection model include the boundary coordinates of each constituent entity and its type (molecular structure or text).

[0078] Then, the bounding box is used to fuse and filter the target entity and the constituent entities of the reaction equation information.

[0079] Specifically, such as Figure 4 As shown, when the prediction result of the reaction equation detection model is not empty, that is, when the chemical reaction diagram includes reaction equation information, all reaction equations are traversed and each reaction equation is iterated.

[0080] The iterative process for each reaction equation involves traversing the constituent entities of each reaction equation, namely, traversing the reaction substrate, reaction conditions, and reaction products of each reaction equation, and iteratively processing the reaction substrate, reaction conditions, and reaction products of each reaction equation.

[0081] The iterative processing of the reaction substrate, reaction conditions, and reaction products for each reaction equation includes first obtaining the boundary coordinates and type of each component entity, and then classifying the component entities according to their type.

[0082] Please refer to Figure 6 , Figure 6 The diagram illustrates the implementation of fusion and filtering using bounding boxes according to some embodiments of the present invention.

[0083] In response to the constituent entity being of type molecular structure, the bounding boxes of the target entity of type molecular structure are traversed based on the bounding boxes of the constituent entities. For example... Figure 6 As shown in the bounding box filtering logic, when the intersection-over-union (IOU) ratio between the bounding boxes of the constituent entities and the bounding box of the target entity is greater than a preset value (such as...), the bounding box filtering logic will be applied. Figure 6 When the value is 0.8 (as shown), the target entity is removed.

[0084] When the Intersection over Union (IoU) between two different bounding boxes is greater than a certain threshold, the two bounding boxes can be considered to point to the same entity in the image. Therefore, when the IoU between the bounding box of a constituent entity and the bounding box of a target entity is greater than a preset value, the constituent entity and the target entity can be considered to point to the same molecular structure type in the chemical reaction diagram, and the target entity is filtered out and removed from all target entities.

[0085] In some embodiments, the origin of the image coordinate system in the image processing library is the top-left corner. The X-axis value increases to the right, and the Y-axis value increases downwards until the bottom-right corner of the image. The coordinates of the top-left and bottom-right corners of the bounding box A that makes up the entity can be respectively... and The coordinates of the top-left and bottom-right corners of the bounding box B of the target entity can be respectively and The coordinates of the top-left and bottom-right corners of the intersection region of the bounding box A of the entity and the bounding box B of the target entity are the overlapping parts of the coordinates of the two bounding boxes:

[0086] ,

[0087] ,

[0088] ,

[0089] ,

[0090] ,

[0091] That is, when the overlapping part is or When the boundary box A of the constituent entity and the boundary box B of the target entity do not overlap, the intersection-union ratio is 0.

[0092] Continuing the calculations, we can obtain the areas of bounding box A, bounding box B, intersection area, and union area. The area of ​​bounding box A can be:

[0093] ,

[0094] The area of ​​bounding box B can be:

[0095] ,

[0096] The intersection area can be:

[0097] ,

[0098] The area of ​​the union can be:

[0099] ,

[0100] Therefore, the intersection-union ratio of bounding boxes A and B can be obtained:

[0101] .

[0102] After that, as Figure 6 As shown in the identifier binding logic and bounding box distance calculation logic, the chemical synthesis literature information extraction system can use the bounding box distance calculation logic to calculate the distance between the bounding box of the target entity of type identifier and the bounding box of the constituent entities. Then, based on the identifier binding logic, the constituent entities are identified and bound to the nearest target entity of type identifier, forming molecular structure reaction data.

[0103] In this preferred embodiment, the core idea of ​​the identifier binding logic is to use the candidate identifier closest to the bounding box of the constituent entity of the molecular structure type as the identifier of that molecular structure. Here, the identifier binding logic includes two key points: First, it needs to store the relationship between already bound molecular structures and identifiers to address the issue of the same compound playing different roles in multiple reaction steps (e.g., a product in one step and a substrate in another). Second, it utilizes... Figure 6 The bounding box distance calculation logic in the code calculates the distance between the bounding box of the target entity (type identifier) ​​and the bounding boxes of the constituent entities.

[0104] Figure 6 The logic for calculating the distance between bounding boxes in the code first needs to expand the bounding boxes of the constituent entities of the molecular structure type, limiting the identifiers to the expanded bounding boxes of the constituent entities of the molecular structure type, preventing incorrect identifier binding and avoiding binding the identifiers of other molecular structures to this molecular structure.

[0105] Specifically, such as Figure 6 As shown in the bounding box distance calculation logic, after the bounding box of the constituent entity is expanded outward, the center point of the bounding box of the target entity of type identifier is calculated. When the center point of the bounding box is within the new bounding box range, the distance between the target entity of type identifier and the constituent entity is determined based on the center point of the bounding box.

[0106] Then, as Figure 6 As shown in the identifier binding logic, the constituent entities are identified and bound to the nearest target entity of type identifier. Preferably, after the identifier and molecular structure are successfully bound, the textual content of the identifier can be understood using a multimodal general large language model.

[0107] The chemical synthesis literature information extraction system can repeat the above steps to iteratively process the reaction substrate, reaction conditions, and reaction products of each molecular structure type in each reaction equation until all constituent entities of all molecular structure types have been traversed.

[0108] Please continue to refer to this. Figure 4 The types of entities that make up the system can also be text; a chemical synthesis literature information extraction system can use this. Figure 6 The bounding box filtering logic in the code implements the fusion and filtering of text-type constituent entities.

[0109] Specifically, when the type of the constituent entity is text, the bounding boxes of the target entity of type text are traversed based on the bounding boxes of the constituent entity. When the intersection-union ratio between the bounding boxes of the constituent entity and the target entity is greater than a preset value, it can be considered that the constituent entity and the target entity point to the same text type entity in the chemical reaction diagram, and the target entity is filtered out and removed from all target entities.

[0110] Subsequently, a large language model is used to perform OCR text recognition and classification on the constituent entities, understand the text content of the image, determine the reaction parameters (such as reaction temperature, reaction time, yield / proportion or solvent), and obtain the original text content corresponding to the text-type constituent entities and the corresponding reaction parameters.

[0111] The chemical synthesis literature information extraction system can repeat the above steps to iteratively process the reaction substrate, reaction conditions, and reaction products of each reaction equation in text type until all text type constituent entities have been traversed.

[0112] After iteration, the chemical synthesis literature information extraction system can store the results of the chemical reaction schematic diagram information mining as a JSON file. The results of the chemical reaction schematic diagram information mining can contain two parts: one part is the reaction data composed of entities from multiple reaction equations, and the other part is the target entities remaining after filtering from the results predicted by the target detection model. For example... Figure 4 As shown, the mining results can be saved by category.

[0113] After that, as Figure 2 As shown, the chemical synthesis literature information extraction system can perform steps S250 and S260: based on the reaction data of the molecular structure type, determine the compound name corresponding to the molecular structure; perform string matching between the compound name and the chemical reaction text, and integrate the successfully matched compound name and corresponding reaction information into the prompt words of the large language model.

[0114] Large language models possess powerful text understanding and generation capabilities, efficiently processing complex textual modal data from chemical literature, demonstrating enormous potential in chemical text mining. Building upon this foundation, combining large language models with data-driven and prompt-based engineering methods enables faster and more accurate information extraction.

[0115] For structured information extraction from chemical reaction texts in text modalities, a large language model based on cue word engineering can achieve or even surpass the performance of dedicated entity extraction models without relying on a large amount of training data. In some embodiments, the extraction targets of the large language model based on cue word engineering can be eight reaction parameters: substrate, product, solvent, catalyst, general reagent, reaction temperature, reaction time and / or yield / proportion.

[0116] In some embodiments, the prompt words constructed by the large language model can incorporate information extracted from the chemical reaction diagram.

[0117] Please refer to the reference. Figure 4 and Figure 7 , Figure 7 An overall flowchart of a chemical synthesis literature information extraction method provided according to some embodiments of the present invention is shown.

[0118] The reaction data extracted from the chemical reaction diagram can include reaction data of molecular structure type. Based on the reaction data of molecular structure type, the name of the compound corresponding to the molecular structure is determined.

[0119] Preferably, in such Figure 4 and Figure 7 In the preferred embodiment shown, the chemical synthesis literature information extraction system can convert molecular structure entities into SMILES (Simplified molecular input line entry system) representations, and then determine the corresponding compound names in the compound name query database based on the SMILES representations. The corresponding compound names include synonym compound names.

[0120] For molecular structure diagrams in image modalities from chemical literature, the chemical synthesis literature information extraction system can convert molecular structure entities into SMILES representations using the MolScribe model. The MolScribe model can construct atomic sequences using atomic coordinates and types in the molecular structure diagram, and construct chemical bond sequences using the types of chemical bonds between atoms. Combining these, a sequence representation of the molecular structure is obtained. By training a sequence generation model of the molecular structure using the image features of the molecular structure diagram, and then using the RDKit toolkit to construct the SMILES representation of the molecule, the system can identify molecular structures.

[0121] Subsequently, the chemical synthesis literature information extraction system can determine the compound name corresponding to the molecular structure from the compound name lookup database based on the SMILES representation. Here, the compound name lookup database can be constructed based on the PubChem public dataset.

[0122] like Figure 7As shown, the chemical synthesis literature information extraction system can construct a mapping database for compound name retrieval based on the PubChem public datasets CID-SMILES, CID-IUPAC, and CID-Synonym-unfiltered. This database maps SMILES to corresponding IUPAC (International Union of Pure and Applied Chemistry) compound names and synonyms. The PubChem public datasets cover over 170 million SMILES representations and their corresponding IUPAC compound names and synonyms.

[0123] In this way, the chemical synthesis literature information extraction system can use the SMILES representation obtained by converting the reaction data of molecular structure types in the schematic diagram mining results to construct an SQL query statement to obtain the compound name corresponding to the molecular structure. The determined compound name includes all synonym compound names.

[0124] Then, the chemical synthesis literature information extraction system can perform string matching between compound names and chemical reaction texts, and integrate the successfully matched compound names and corresponding reaction information into the prompt words of the large language model.

[0125] Please combine Figure 7 The chemical synthesis literature information extraction system can perform string matching between the obtained compound names and the text descriptions in the chemical reaction text, identify the reaction parameters corresponding to the compound names, and construct structured information from the compound names and corresponding reaction information. This structured information is then integrated into the prompts of a large language model.

[0126] The chemical synthesis literature information extraction system integrates molecular structure reaction data (usually reaction substrates and reaction products) obtained from the identified chemical reaction diagrams into prompt words, which can assist the large language model in parameter determination and improve the accuracy of the extracted reaction parameter determination.

[0127] The chemical synthesis literature information extraction system can perform step S270: extract reaction parameters from the chemical reaction text using a large language model, combine the extraction results with the reaction data and the remaining filtered target entities to determine the complete chemical reaction parameters.

[0128] like Figure 7 As shown, the chemical synthesis literature information extraction system can call a large language model incorporating prompt words to extract reaction parameters from the chemical reaction text to be processed. The extraction results are then combined with the reaction data obtained in step S240 and the filtered remaining target entities to determine the complete chemical reaction parameters.

[0129] Thus, the chemical synthesis literature information extraction system, through information mining of chemical reaction diagrams and modal fusion text information extraction, can ultimately obtain structured reaction data from the chemical literature data to be processed. Complete chemical reaction parameters can include reaction substrates, reaction products, solvents, catalysts, general reagents, reaction temperature, reaction time, and / or yield / proportion. Preferably, the content displayed through molecular structures of reaction substrates and reaction products can simultaneously include corresponding SMILES expressions.

[0130] The following is a specific, non-limiting preferred embodiment, which is used to elaborate on the chemical synthesis literature information extraction method proposed in this invention.

[0131] Before extracting information, the chemical synthesis literature information extraction system collects data as samples from organic chemistry journals such as the Journal of Organic Chemistry and similar open-source datasets for object detection tasks. The sample dataset is randomly divided into a training set containing 1240 data points and a validation set containing 138 data points. The training set is used to train the YOLO11x object detection model, while the validation set is not used for model training but is used to monitor overfitting during the training process.

[0132] Please refer to Figure 8 and Figure 9 , Figure 8 and Figure 9 The evaluation results of the trained object detection model on the validation set are shown.

[0133] Figure 8 and Figure 9 The evaluation results of the object detection model on the validation set are shown using a confusion matrix. Figure 8 The confusion matrix shown can represent the distribution of correct and incorrect predictions of the trained object detection model by absolute numbers, and is the basis for calculating various performance indicators. Figure 9 The normalized confusion matrix shown can eliminate the influence of data size through scaling, more intuitively revealing the advantages and disadvantages of the trained object detection model at the category level. From Figure 8 and Figure 9 The observations show that the trained target detection model has an accuracy rate of over 85% in detecting target entities and an accuracy rate of close to 1 in recognizing molecular structures, indicating that the trained target detection model has a good fitting effect.

[0134] Subsequently, the chemical synthesis literature information extraction system extracts target entities from the chemical reaction diagram based on the trained target detection model, and identifies the reaction equation information in the chemical reaction diagram using the reaction equation detection model. The prediction results of the target detection model and the identification results of the reaction equation detection model are then subjected to subtraction and identifier binding to determine the mining results of the chemical reaction diagram.

[0135] To demonstrate the advantages of the chemical synthesis literature information extraction method provided by this invention in integrating information from chemical reaction diagrams into the chemical text reaction parameter extraction process, chemical literature was collected and organized from the Reaxys database and organic synthesis chemistry journals (referred to as "General" in the data presentation below). Based on the reaction text data in the collected chemical literature, the extraction effect after integrating the reaction substrate and reaction product information from the chemical reaction diagrams was tested and evaluated. Among the collected chemical literature data, 154 data points came from the Reaxys database, and 49 data points came from organic synthesis chemistry journals. The text data in the collected chemical literature data was annotated using the annotation tool LabelStudio (e.g., annotating the substrate, product, solvent, catalyst, or general reagent in the text, etc.). The large language model used to test the extraction effect was the open-source deep thinking model DeepSeek-R1.

[0136] In this embodiment, to test the effectiveness of entity extraction from the response text, the evaluation method for the extraction task in this embodiment is set with reference to the evaluation method for entity extraction tasks. For each response parameter considered in the extraction task, precision, recall, and F1 score are calculated to reflect the hit effect of the response parameter extraction results. The formulas for calculating precision, recall, and F1 score are as follows:

[0137] ,

[0138] ,

[0139] ,

[0140] in, This indicates that the predicted entity matches the labeled entity; in the extraction task of this embodiment, the predicted entity exists in the labeled entity list. This indicates that the predicted entity does not match the labeled entity; in the extraction task of this embodiment, it means that the predicted entity does not exist in the labeled entity list. There are no general, unified predefined entity types in entity extraction tasks, and they are not defined in the extraction task of this embodiment. This indicates that an entity in the label does not exist in the predicted entity list. Furthermore, and The sum equals the predicted total number of entities. and The sum of these numbers equals the total number of entities in the tag.

[0141] Furthermore, to more comprehensively demonstrate the extraction results, two strategies are employed when statistically analyzing the results. The first strategy is relatively lenient; a prediction is considered correct if the predicted entity exists in the labeled entity list. This is denoted as the Soft strategy. The second strategy is more stringent, requiring that the number of predicted entities match the number of labeled entities, building upon the first strategy. This is denoted as the Hard strategy.

[0142] The actual test results are shown in Tables 1, 2, 3, and 4 below. Pre-Soft represents Precision under the Soft strategy, Re-Soft represents Recall under the Soft strategy, and F1-Soft represents the F1 score under the Soft strategy. Pre-Hard represents Precision under the Hard strategy, Re-Hard represents Recall under the Hard strategy, and F1-Hard represents the F1 score under the Hard strategy.

[0143] Table 1 shows the results of extracting reaction parameters from chemical reaction texts in the Reaxys database using a large language model, without incorporating information from the chemical reaction diagrams into the chemical reaction parameter extraction process.

[0144]

[0145] Table 1

[0146] Table 2 shows the results of extracting reaction parameters from chemical reaction texts from the Reaxys database by incorporating information from the chemical reaction diagrams into the chemical reaction parameter extraction process. This was achieved using a large language model that incorporates information about the reaction substrates and products into the prompts.

[0147]

[0148] Table 2

[0149] Table 3 shows the results of extracting reaction parameters from chemical reaction texts in organic synthetic chemistry journals using a large language model, without incorporating information from the chemical reaction diagrams into the chemical reaction parameter extraction process.

[0150]

[0151] Table 3

[0152] Table 4 shows the results of extracting reaction parameters from chemical reaction texts from organic synthetic chemistry journals by incorporating information from chemical reaction diagrams into the chemical reaction parameter extraction process. This was achieved using a large language model that incorporates information about reaction substrates and products into prompts.

[0153]

[0154] Table 4

[0155] Comparing the results in Tables 1 and 2, and Tables 3 and 4, it can be seen that the information on the reaction substrates and products explicitly shown in the chemical reaction diagrams helps the large language model correctly identify the reaction substrates, products, catalysts, solvents, and general reagents in the chemical reaction text. Furthermore, this effect is even more pronounced in general chemical journal literature. General chemical journal literature is more complex than the Reaxys database, with longer text content and more interfering information. The information extracted from the chemical reaction diagrams provides more relevant reaction information for the reaction parameter extraction process, helping the large language model to better determine reaction parameters. Thus, the chemical synthesis literature information extraction method provided by this invention not only accurately extracts relevant information about substrates and products but also significantly improves the accuracy of extracting catalysts, solvents, and general reagents.

[0156] In summary, the chemical synthesis literature information extraction method provided by this invention achieves a more comprehensive extraction and display of chemical reaction diagrams, solving the problems of incomplete extraction and omission of key information in existing methods, and providing more sufficient data support for downstream tasks. Furthermore, it innovatively proposes a method for extracting reaction parameters from chemical reaction texts that incorporates chemical reaction diagram information, utilizing a modal fusion strategy to assist large language models in determining reaction parameters, thereby improving the accuracy of determination.

[0157] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0158] Those skilled in the art will understand that information, signals, and data can be represented using any of the various techniques and skills available. For example, the data, instructions, commands, information, signals, bits, symbols, and chips described throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0159] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0160] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0161] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0162] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0163] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for extracting literature information on chemical synthesis, characterized in that, Including the following steps: Obtain schematic diagrams and texts of chemical reactions from chemical literature; The target entity in the chemical reaction diagram is detected using a target detection model. The reaction equation detection model is used to identify the reaction equation information in the chemical reaction diagram. The constituent entities of the reaction equation information include the reaction substrate, reaction conditions, and reaction products. The target entity and the constituent entities of the reaction equation information are fused and filtered to obtain reaction data and the remaining target entities after filtering. Based on the reaction data of the molecular structure type, determine the name of the compound corresponding to the molecular structure; The compound name is matched against the chemical reaction text, and the successfully matched compound name and corresponding reaction information are incorporated into the prompt words of the large language model; and The reaction parameters are extracted from the chemical reaction text using the large language model. The extraction results are then combined with the reaction data and the remaining filtered target entities to determine the complete chemical reaction parameters.

2. The method for extracting chemical synthesis literature information as described in claim 1, characterized in that, The pre-training process of the target detection model includes the following steps: Obtain the pre-trained dataset; Based on the defined type of the target entity, the schematic diagrams in the pre-training dataset are labeled with data. as well as The object detection model is trained using the labeled pre-trained dataset, and the object detection model is built based on YOLO11.

3. The method for extracting chemical synthesis literature information as described in claim 1, characterized in that, The target entity and the constituent entities of the reaction equation information are fused and filtered based on the bounding box. The fusion and filtering steps include: In response to the fact that the type of the constituent entity is a molecular structure, the bounding box of the target entity of type molecular structure is traversed based on the bounding box of the constituent entity. When the intersection-union ratio between the bounding box of the constituent entity and the bounding box of the target entity is greater than a preset value, the target entity is removed; Calculate the distance between the bounding box of the target entity (of type identifier) ​​and the bounding box of the constituent entities; The constituent entities are identified and bound to the nearest target entity of type identifier; and Repeat the above steps until all constituent entities of all molecular structure types have been traversed.

4. The method for extracting chemical synthesis literature information as described in claim 3, characterized in that, The step of calculating the distance between the bounding box of the target entity (type identifier) ​​and the bounding box of the constituent entities includes: Expand the bounding box range outward based on the bounding box of the constituent entities; and In response to the target entity of type identifier having its bounding box center point located within a new bounding box, the distance between the target entity of type identifier and the constituent entities is determined based on the bounding box center point.

5. The method for extracting chemical synthesis literature information as described in claim 3, characterized in that, The step of fusing and filtering the constituent entities of the target entity and the reaction equation information based on the bounding box includes: In response to the fact that the type of the constituent entity is text, the bounding box of the target entity of type text is traversed based on the bounding box of the constituent entity; When the intersection-union ratio between the bounding box of the constituent entity and the bounding box of the target entity is greater than a preset value, the target entity is removed; The constituent entities are subjected to OCR text recognition and classification using a large language model; and Repeat the above steps until all the constituent entities of all text types have been traversed.

6. The method for extracting chemical synthesis literature information as described in claim 1, characterized in that, The step of determining the compound name corresponding to the molecular structure based on the reaction data of the molecular structure type includes: Convert the molecular structure entities in the reaction data of the molecular structure type to SMILES representation; and According to SMILES, the corresponding compound name is determined in a compound name lookup database, and the corresponding compound name includes synonymous compound names.

7. The method for extracting chemical synthesis literature information as described in claim 6, characterized in that, The compound name lookup database was built based on the PubChem public dataset.

8. The method for extracting chemical synthesis literature information as described in claim 1, characterized in that, The complete chemical reaction parameters include the reaction substrate, reaction products, solvent, catalyst, general reagents, reaction temperature, reaction time and / or yield / proportion.

9. A chemical synthesis literature information extraction system, characterized in that, include: Memory, on which computer instructions are stored; as well as A processor, connected to the memory, and configured to execute computer instructions stored in the memory to implement the chemical synthesis literature information extraction method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, the chemical synthesis literature information extraction method as described in any one of claims 1 to 8 is implemented.

Images