Chemical structure recognition method and recognition system
By generating image segmentation and recognition datasets, the model is trained to identify atoms and hypertext in chemical structure images, solving the problem of inefficiency in extracting chemical structure formulas from PDF documents in existing technologies, and achieving high accuracy and efficiency in chemical structure recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INFINITE INTELLIGENCE PHARMACEUTICAL TECHNOLOGY CO LTD
- Filing Date
- 2022-10-26
- Publication Date
- 2026-06-05
AI Technical Summary
Current technology cannot quickly extract and convert chemical structural formulas from PDF documents into machine-readable formats, resulting in low efficiency for researchers in identifying and storing compound images in literature.
By generating image segmentation and image recognition datasets, image segmentation and image recognition models are trained to identify chemical atoms and hypertext in chemical structure images, and finally chemical structural formulas conforming to SMILES or InChI specifications are constructed.
It improves the accuracy and efficiency of extracting chemical structures from PDF documents, ensures the performance of the image recognition model, and generates highly accurate chemical structures.
Smart Images
Figure CN115631507B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of chemical structure recognition technology, specifically relating to a chemical structure recognition method and system. Background Technology
[0002] In the fields of chemistry and drug discovery, there is a vast amount of literature, including journals and patents. Accurate identification and storage of compound images from these documents would facilitate retrieval and analysis for researchers, significantly improving research efficiency. Given the immense value of converting 2D characterization images of compounds from journals and patents into machine-readable formats, the academic community has conducted extensive research, and several open-source projects, such as molvec, offer molecular image recognition capabilities. However, existing technologies cannot systematically solve the problem of quickly extracting molecular images from PDF documents such as patents, journals, and e-books with a single click, and then converting these images into a machine-readable format for storage. This invention proposes a chemical structure recognition method and system to address this issue. Summary of the Invention
[0003] This invention generates image segmentation datasets and image recognition datasets based on historical literature, which are used to train image segmentation models and image recognition models, respectively. The image segmentation model identifies image detection regions containing chemical structures in images and extracts the corresponding chemical structure images. The image recognition model identifies chemical atoms and hypertext in the chemical structure images, and finally obtains chemical structure formulas in machine-readable format.
[0004] To achieve the aforementioned objectives, a chemical structure identification method is provided, which mainly includes the following steps:
[0005] Based on historical literature, original datasets of images containing chemical structures in different styles are obtained, and an image segmentation dataset is generated based on the original dataset, thereby generating an image recognition dataset based on the image segmentation dataset.
[0006] For literature documents that require chemical structure identification, a PDF file parsing tool is used to convert the PDF documents into several images to be identified. An image segmentation model is trained using the image segmentation dataset. The trained image segmentation model identifies chemical structures in several images to be identified, determines the image detection region containing the chemical structure, and extracts the chemical structure image corresponding to the image detection region.
[0007] Based on the image recognition dataset, an image recognition learning dataset and an image recognition test dataset are generated respectively. The image recognition learning dataset is used to train an image recognition model, and the image recognition test dataset is used to test the performance of the trained image recognition model. When the test is passed, the trained image recognition model can identify the chemical atoms and hypertext in the chemical structure image.
[0008] Based on the identification results of chemical atoms and hypertext from the chemical structure image, chemical atoms and hypertext are further identified and distinguished from the identification results to infer and construct a chemical molecular diagram. Then, chemical structural formulas conforming to SMILES or InChI specifications are parsed and output. After the chemical structural formulas are generated from the several images to be identified, the accuracy of the generated chemical structural formulas is evaluated.
[0009] As a preferred embodiment of the present invention, generating an image segmentation dataset based on the original dataset, and then further generating an image recognition dataset based on the image segmentation dataset, includes the following steps:
[0010] The first machine learning model is used to process images containing chemical structures of different styles in the original dataset to identify chemical structures in the images, determine image detection regions containing chemical structures, and extract images corresponding to the image detection regions to form the image segmentation dataset.
[0011] The image segmentation dataset is labeled manually to distinguish between correct and incorrect images, and a preset number of labeled images are obtained. The labeled images are then used as teacher data for machine learning to generate a second machine learning model for images of different styles in the image segmentation dataset.
[0012] The correct images are extracted from the unlabeled images of different styles in the image segmentation dataset using a second machine learning model, and all the correct images are combined to form the image recognition dataset.
[0013] As a preferred embodiment of the present invention, a correct image refers to an image that does not include a background with features similar to those of a chemical structure and that contains the complete chemical structure; an incorrect image refers to an image that includes a background with features similar to those of a chemical structure, or an image in which the chemical structure is partially hidden.
[0014] As a preferred embodiment of the present invention, the number of images in the image segmentation dataset that are manually labeled is much smaller than the number of unlabeled images in the image segmentation dataset.
[0015] As a preferred embodiment of the present invention, generating an image recognition learning dataset and an image recognition test dataset based on the image recognition dataset includes the following steps:
[0016] The image recognition dataset is divided into two parts: an image recognition learning dataset and an image recognition test dataset.
[0017] Extract the data features from the image recognition learning dataset and the data features from the image recognition test dataset, respectively.
[0018] Calculate the similarity between the data features of the image recognition learning dataset and the data features of the image recognition test dataset;
[0019] Determine whether the similarity between the data features of the image recognition learning dataset and the data features of the image recognition test dataset is greater than a preset similarity threshold. If it is greater, determine that the image recognition learning dataset is suitable as the learning dataset for the image recognition model; otherwise, continue to the next step.
[0020] Repeat the above steps until the similarity between the data features of the corresponding image recognition learning dataset and the data features of the corresponding image recognition test dataset is greater than the preset similarity threshold.
[0021] As a preferred embodiment of the present invention, generating an image recognition learning dataset and an image recognition test dataset based on the image recognition dataset further includes the following steps:
[0022] The image recognition dataset is divided into two parts: an image recognition learning dataset and an image recognition test dataset.
[0023] Extract the data features of the image recognition learning dataset, the image recognition test dataset, and the image recognition dataset, respectively;
[0024] Calculate the similarity between the data features of the image recognition learning dataset and the data features of the image recognition test dataset. If the similarity is greater than a preset similarity threshold, continue to the next step; otherwise, jump to the last step.
[0025] Calculate the similarity between the data features of the image recognition learning dataset and the data features of the image recognition dataset. If the similarity is greater than a preset similarity threshold, continue to the next step; otherwise, jump to the last step.
[0026] Calculate the similarity between the data features of the image recognition test dataset and the data features of the image recognition dataset. If the similarity is greater than a preset similarity threshold, continue to the next step; otherwise, jump to the last step.
[0027] Once it is determined that the image recognition learning dataset is suitable as the learning dataset for the image recognition model, the process ends.
[0028] The image recognition learning dataset and the image recognition test dataset are regenerated based on the image recognition dataset until the above judgment conditions are met simultaneously.
[0029] This invention also provides a chemical structure identification system, which mainly includes the following modules:
[0030] The dataset generation module is used to obtain raw datasets containing chemical structures in different styles based on historical literature, and to generate an image segmentation dataset based on the raw dataset, and further generate an image recognition dataset based on the image segmentation dataset.
[0031] The detection module is used to use a PDF file parsing tool to convert the PDF documents into several images to be identified for the literature documents that need to be identified by chemical structure recognition. The image segmentation model is trained through the image segmentation dataset. The trained image segmentation model identifies chemical structures in several images to be identified, determines the image detection region containing the chemical structure, and extracts the chemical structure image corresponding to the image detection region.
[0032] The recognition module is used to generate an image recognition learning dataset and an image recognition test dataset based on the image recognition dataset, and to train an image recognition model using the image recognition learning dataset. It also uses the image recognition test dataset to test the performance of the trained image recognition model. When the test is passed, the trained image recognition model identifies the chemical atoms and hypertext in the chemical structure image.
[0033] The chemical formula generation module is used to identify chemical atoms and hypertext from chemical structure images, and further identify and distinguish chemical atoms and hypertext from the identification results to infer and construct chemical molecular diagrams. Then, it parses and outputs chemical structural formulas that conform to SMILES or InChI specifications. It is also used to evaluate the accuracy of the generated chemical structural formulas after generating chemical structural formulas from several images to be identified.
[0034] Compared with the prior art, the beneficial effects of the present invention are at least as follows:
[0035] 1. This invention first obtains a raw dataset containing images of chemical structures based on historical literature, and generates an image segmentation dataset and an image recognition dataset based on the raw dataset; secondly, for the literature that needs to be identified for chemical structure recognition, the PDF format literature is converted into several images to be recognized, and chemical structures are identified and extracted from these images; thirdly, based on the image recognition dataset, an image recognition learning dataset and an image recognition test dataset are generated respectively, and the chemical atoms and hypertext in the chemical structure images are identified through the image recognition model; finally, based on the chemical atoms and hypertext, a chemical molecular diagram is constructed by reasoning, and the chemical structural formula conforming to the SMILES or InChI standard is parsed and output.
[0036] 2. This invention solves the problem that existing technologies cannot quickly extract machine-readable chemical structural formulas from PDF documents. Furthermore, this invention generates an image recognition dataset based on an image segmentation dataset, ensuring that the image recognition dataset contains the correct images. At the same time, it ensures that a suitable image recognition learning dataset can be generated from the image recognition dataset, thereby improving the accuracy of using image recognition models to identify chemical atoms and hypertext, and thus improving the accuracy of the final generated chemical structural formula. Attached Figure Description
[0037] Figure 1 This is a flowchart illustrating the steps of a chemical structure identification method according to the present invention;
[0038] Figure 2 A flowchart illustrating the steps of a method for generating a suitable image recognition learning dataset according to the present invention;
[0039] Figure 3 This is a structural diagram of a chemical structure recognition system according to the present invention. Detailed Implementation
[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0041] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of this application, a first script may be referred to as a second script, and similarly, a second script may be referred to as a first script.
[0042] To address the problem that existing technologies cannot systematically solve the issue of quickly extracting molecular images from PDF documents such as patents, journals, and e-books with a single click, and converting these images into a machine-readable format for storage, the inventors have provided a solution such as... Figure 1 The chemical structure identification method shown is mainly achieved by performing the following steps:
[0043] Step 1: Based on historical literature, obtain raw datasets of images containing chemical structures in different styles, and generate an image segmentation dataset based on the raw dataset. Then, generate an image recognition dataset based on the image segmentation dataset.
[0044] Step 2: For the literature materials that need to be identified by chemical structure, use a PDF file parsing tool to convert the PDF documents into several images to be identified. Train the image segmentation model using the above image segmentation dataset. The trained image segmentation model identifies chemical structures in several images to be identified, determines the image detection region containing the chemical structure, and extracts the chemical structure image corresponding to the above image detection region.
[0045] Step 3: Based on the above image recognition dataset, generate an image recognition learning dataset and an image recognition test dataset respectively. Use the above image recognition learning dataset to train the image recognition model, and use the above image recognition test dataset to test the performance of the trained image recognition model. When the test is passed, use the trained image recognition model to identify the chemical atoms and hypertext in the above chemical structure image.
[0046] Based on the identification results of chemical atoms and hypertext from the above chemical structure images, chemical atoms and hypertext are further identified and distinguished from the identification results to infer and construct chemical molecular diagrams. Then, chemical structural formulas conforming to SMILES or InChI specifications are parsed and output. After the chemical structural formulas are generated from the above images to be identified, the accuracy of the generated chemical structural formulas is evaluated.
[0047] Specifically, in steps one through four above, firstly, based on various journals, patents, and other literature, an original dataset that fits the real situation is generated. Furthermore, an image segmentation dataset and an image recognition dataset are generated from the original dataset. Secondly, an image segmentation model is trained using the image segmentation dataset. This model is used to identify chemical structures in images, determine image detection regions containing chemical structures, and extract the corresponding chemical structure images. Next, a suitable image recognition learning dataset is generated based on the image recognition dataset to train the image recognition model. The image recognition model is then used to identify the chemical atoms and hypertext that make up the chemical structure from the chemical structure images. Finally, based on the chemical atoms and hypertext, a machine-readable chemical structure formula is generated, for example, a chemical structure formula conforming to the SMILES or InChI specifications.
[0048] Furthermore, an image segmentation dataset is generated based on the original dataset, and then an image recognition dataset is generated based on the image segmentation dataset, including the following steps:
[0049] The first step is to use the first machine learning model to process images containing chemical structures of different styles in the above original dataset in order to identify chemical structures in the images, determine the image detection regions containing chemical structures, and extract the images corresponding to the image detection regions to form the above image segmentation dataset.
[0050] The second step involves manually labeling a predetermined number of images in the image segmentation dataset to distinguish between correct and incorrect images, and obtaining a predetermined number of labeled images. These labeled images are then used as teacher data for machine learning, generating a second machine learning model for images of different styles in the image segmentation dataset.
[0051] The third step is to extract the correct images from the unlabeled images of different styles in the above image segmentation dataset using the second machine learning model, and to form the above image recognition dataset from all the correct images.
[0052] Furthermore, the above-mentioned correct image refers to an image that does not include a background with features similar to those of the chemical structure and that contains the complete chemical structure; the above-mentioned incorrect image refers to an image that includes a background with features similar to those of the chemical structure, or an image in which the chemical structure is partially hidden.
[0053] Specifically, to identify and generate machine-readable chemical structural formulas from PDF documents, it is generally necessary to convert the PDF documents containing chemical structures into image format. This allows for the identification of chemical structures within the image, determining their locations (i.e., labeling the image detection regions containing the chemical structures). Then, by identifying chemical atoms and hypertext from the images corresponding to these detection regions, a machine-readable chemical structural formula can be generated. Accurately labeling these detection regions among a large number of images containing chemical structures is crucial to obtaining the correct images corresponding to them. Training the image recognition model with incorrect images will negatively impact its performance, leading to reduced accuracy or even failure to generate machine-readable chemical structural formulas. Therefore, steps one through three are described above to ensure the correct images are obtained.
[0054] In steps one through three above, the image detection regions are first labeled using a traditional first machine learning model to generate an image segmentation dataset. This first machine learning model has been pre-trained and can identify chemical structures in the images to be recognized. It can also draw a bounding box around the chemical structure using methods such as bounding box annotation. This bounding box corresponds to the image detection region. Since the accuracy of labeling image detection regions using the first machine learning model is often lower than that relying on manual labeling, the image segmentation dataset generally contains erroneous images. To ensure the performance of the image recognition model, erroneous images should be removed from the image segmentation dataset. Next, some images in the image segmentation dataset are manually labeled. This manual labeling method ensures the generation of correct images corresponding to the image detection regions. These images are then used to train a second machine learning model for different styles of images in the image segmentation dataset. The trained second machine learning model can classify correct and incorrect images, exhibiting better classification performance compared to using the same second machine learning model for all styles of images. Finally, for unlabeled images of different styles in the image segmentation dataset, the corresponding second machine learning model is used for classification, and erroneous images are removed, resulting in an image recognition dataset composed of all correct images.
[0055] Furthermore, the number of images in the aforementioned image segmentation dataset that rely on manual labeling is far less than the number of unlabeled images in the aforementioned image segmentation dataset. For example, if 10% of the images in the aforementioned image segmentation dataset are manually labeled, this not only generates a second machine model to automatically classify the remaining images in the aforementioned image segmentation dataset, but also reduces the cost of generating the aforementioned image recognition dataset.
[0056] Furthermore, based on the image recognition dataset, an image recognition learning dataset and an image recognition test dataset are generated, including the following steps:
[0057] The first step is to divide the above image recognition dataset into two parts: an image recognition learning dataset and an image recognition test dataset.
[0058] The second step is to extract the data features of the image recognition learning dataset and the image recognition test dataset, respectively.
[0059] The third step is to calculate the similarity between the data features of the above image recognition learning dataset and the data features of the above image recognition test dataset.
[0060] Step 4: Determine whether the similarity between the data features of the above image recognition learning dataset and the data features of the above image recognition test dataset is greater than a preset similarity threshold. If it is greater, determine that the above image recognition learning dataset is suitable as the learning dataset for the above image recognition model; otherwise, continue to the next step.
[0061] Step 5: Repeat the above steps until the similarity between the data features of the corresponding image recognition learning dataset and the data features of the corresponding image recognition test dataset is greater than the preset similarity threshold.
[0062] Specifically, the inventors considered that one of the main factors affecting the performance of the above image recognition model is the image recognition learning dataset. If the image recognition learning dataset has a data offset problem, that is, if the attribute features of the data in the image recognition learning dataset cannot fully represent the attribute features of the data in the above image recognition dataset, then the generalization ability of the image recognition model generated from the image recognition learning dataset will be reduced. In order to solve this problem, the image recognition learning dataset should maintain the same data distribution as the image recognition test dataset or the image recognition dataset.
[0063] In steps one through five above, the image recognition dataset is first divided into an image recognition learning dataset and an image recognition test dataset, using methods such as K-fold verification and hold-out verification. Next, data features are extracted from both the learning and test datasets. These features can be generated by obtaining the probability density function, probability distribution function, and likelihood function of the learning and test datasets. Then, the similarity between the data features of the learning and test datasets is calculated. If the similarity value is greater than a similarity threshold, the learning dataset is considered to be free of data bias; otherwise, it is considered to have a data bias. Finally, the learning dataset is regenerated until it is free of data bias. This method, based on obtaining the image recognition dataset from the image segmentation dataset, further ensures that a suitable learning dataset can be derived from the image recognition dataset, thereby guaranteeing the performance of the image recognition model.
[0064] Further, see references such as Figure 2 As shown, generating an image recognition learning dataset and an image recognition test dataset based on the aforementioned image recognition dataset also includes the following steps:
[0065] The first step is to divide the above image recognition dataset into two parts: an image recognition learning dataset and an image recognition test dataset.
[0066] The second step is to extract the data features of the above image recognition learning dataset, the above image recognition test dataset, and the above image recognition dataset, respectively.
[0067] The third step is to calculate the similarity between the data features of the above image recognition learning dataset and the data features of the above image recognition test dataset. If the similarity is greater than the preset similarity threshold, continue to the next step; otherwise, skip to the last step.
[0068] Step 4: Calculate the similarity between the data features of the above image recognition learning dataset and the data features of the above image recognition dataset. If the similarity is greater than the preset similarity threshold, continue to the next step; otherwise, jump to the last step.
[0069] Step 5: Calculate the similarity between the data features of the above image recognition test dataset and the data features of the above image recognition dataset. If the similarity is greater than the preset similarity threshold, continue to the next step; otherwise, jump to the last step.
[0070] Step 6: Determine whether the above image recognition learning dataset is suitable as the learning dataset for the above image recognition model, and end the step;
[0071] Step 7: Regenerate the image recognition learning dataset and the image recognition test dataset based on the image recognition dataset above, until the above judgment conditions are met simultaneously.
[0072] Specifically, a suitable image recognition learning dataset can be generated through steps one through seven above to train the image recognition model. First, the image recognition dataset is divided into an image recognition learning dataset and an image recognition test dataset. Data features are extracted from each dataset. Second, if the similarity between the data features of the image recognition learning dataset and the data features of the image recognition test dataset is greater than a similarity threshold, it indicates that the image recognition learning dataset does not have a data bias problem, and the next step is continued. Otherwise, the first step is returned to regenerate the image recognition learning dataset. Third, if the similarity between the data features of the image recognition learning dataset and the data features of the image recognition dataset is also greater than a similarity threshold, it indicates that the image recognition learning dataset does not have a data bias problem, and the next step is continued. Otherwise, the first step is returned to regenerate the image recognition learning dataset. Finally, if the similarity between the data features of the image recognition test dataset and the data features of the image recognition dataset continues to be greater than a similarity threshold, it indicates that the image recognition learning dataset does not have a data bias problem. Otherwise, the first step is returned to regenerate the image recognition learning dataset. This method can improve the accuracy of using image recognition models to identify chemical atoms and hypertext from chemical structure images.
[0073] Furthermore, when the aforementioned image recognition learning dataset has a data offset problem, a new image recognition learning dataset can be generated by randomly selecting the same number of data points from both the image recognition learning dataset and the image test learning dataset and then swapping the data. At the same time, it is necessary to ensure that the content of the image recognition learning dataset generated this time is not completely consistent with the content of the previously generated image recognition learning dataset.
[0074] For reference Figure 3 As shown, the present invention also provides a chemical structure identification system to implement the chemical structure identification method described above. Specifically, the functions of each module are described as follows:
[0075] The dataset generation module is used to obtain raw datasets containing chemical structures in different styles based on historical literature, and to generate an image segmentation dataset based on the raw datasets, and further generate an image recognition dataset based on the image segmentation datasets.
[0076] The detection module is used to use a PDF file parsing tool to convert the PDF documents into several images to be identified. The image segmentation model is trained using the image segmentation dataset. The trained image segmentation model identifies chemical structures in several images to be identified, determines the image detection region containing the chemical structure, and extracts the chemical structure image corresponding to the image detection region.
[0077] The recognition module is used to generate an image recognition learning dataset and an image recognition test dataset based on the above image recognition dataset, and to train an image recognition model using the above image recognition learning dataset. It also uses the above image recognition test dataset to test the performance of the trained image recognition model. When the test is passed, the trained image recognition model identifies the chemical atoms and hypertext in the above chemical structure image.
[0078] The chemical formula generation module is used to identify chemical atoms and hypertext from chemical structure images, and further identify and distinguish chemical atoms and hypertext from the identification results to infer and construct chemical molecular diagrams. Then, it parses and outputs chemical structural formulas that conform to SMILES or InChI specifications. It is also used to evaluate the accuracy of the generated chemical structural formulas after generating chemical structural formulas from several images to be identified.
[0079] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.
[0080] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.
[0081] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0082] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for identifying chemical structures, characterized in that, The steps include the following: Based on historical literature, original datasets of images containing chemical structures in different styles are obtained, and an image segmentation dataset is generated based on the original dataset, thereby generating an image recognition dataset based on the image segmentation dataset. Generating an image segmentation dataset from the original dataset, and then further generating an image recognition dataset from the image segmentation dataset, includes the following steps: The first machine learning model is used to process images containing chemical structures of different styles in the original dataset to identify chemical structures in the images, determine image detection regions containing chemical structures, and extract images corresponding to the image detection regions to form the image segmentation dataset. The image segmentation dataset is manually labeled to distinguish between correct and incorrect images, and a predetermined number of labeled images are obtained. A correct image is one that does not include a background with features similar to those of a chemical structure and that contains the complete chemical structure; an incorrect image is one that includes a background with features similar to those of a chemical structure, or one in which the chemical structure is partially hidden. The labeled images are also used as teacher data for machine learning to generate a second machine learning model for images of different styles in the image segmentation dataset. The correct images are extracted from the unlabeled images of different styles in the image segmentation dataset using a second machine learning model, and all the correct images are combined to form the image recognition dataset. For literature documents that require chemical structure identification, a PDF file parsing tool is used to convert the PDF documents into several images to be identified. An image segmentation model is trained using the image segmentation dataset. The trained image segmentation model identifies chemical structures in several images to be identified, determines the image detection region containing the chemical structure, and extracts the chemical structure image corresponding to the image detection region. Based on the image recognition dataset, an image recognition learning dataset and an image recognition test dataset are generated respectively. The image recognition learning dataset is used to train an image recognition model, and the image recognition test dataset is used to test the performance of the trained image recognition model. When the test is passed, the trained image recognition model can identify the chemical atoms and hypertext in the chemical structure image. Based on the identification results of chemical atoms and hypertext from the chemical structure image, chemical atoms and hypertext are further identified and distinguished from the identification results to infer and construct a chemical molecular diagram. Then, chemical structural formulas conforming to SMILES or InChI specifications are parsed and output. After the chemical structural formulas are generated from the several images to be identified, the accuracy of the generated chemical structural formulas is evaluated.
2. The chemical structure identification method according to claim 1, characterized in that, The number of images in the image segmentation dataset that rely on manual labeling is much smaller than the number of unlabeled images in the image segmentation dataset.
3. The chemical structure identification method according to claim 1, characterized in that, The process of generating an image recognition learning dataset and an image recognition test dataset based on the image recognition dataset includes the following steps: The image recognition dataset is divided into two parts: an image recognition learning dataset and an image recognition test dataset. Extract the data features from the image recognition learning dataset and the data features from the image recognition test dataset, respectively. Calculate the similarity between the data features of the image recognition learning dataset and the data features of the image recognition test dataset; Determine whether the similarity between the data features of the image recognition learning dataset and the data features of the image recognition test dataset is greater than a preset similarity threshold. If it is greater, determine that the image recognition learning dataset is suitable as the learning dataset for the image recognition model; otherwise, continue to the next step. Repeat the above steps until the similarity between the data features of the corresponding image recognition learning dataset and the data features of the corresponding image recognition test dataset is greater than the preset similarity threshold.
4. The chemical structure identification method according to claim 3, characterized in that, The process of generating an image recognition learning dataset and an image recognition test dataset based on the image recognition dataset also includes the following steps: The image recognition dataset is divided into two parts: an image recognition learning dataset and an image recognition test dataset. Extract the data features of the image recognition learning dataset, the image recognition test dataset, and the image recognition dataset, respectively; Calculate the similarity between the data features of the image recognition learning dataset and the data features of the image recognition test dataset. If the similarity is greater than a preset similarity threshold, continue to the next step; otherwise, jump to the last step. Calculate the similarity between the data features of the image recognition learning dataset and the data features of the image recognition dataset. If the similarity is greater than a preset similarity threshold, continue to the next step; otherwise, jump to the last step. Calculate the similarity between the data features of the image recognition test dataset and the data features of the image recognition dataset. If the similarity is greater than a preset similarity threshold, continue to the next step; otherwise, jump to the last step. Once it is determined that the image recognition learning dataset is suitable as the learning dataset for the image recognition model, the process ends. The image recognition learning dataset and the image recognition test dataset are regenerated based on the image recognition dataset until the above judgment conditions are met simultaneously.
5. A chemical structure recognition system for implementing the method as described in any one of claims 1-4, characterized in that, Includes the following modules: The dataset generation module is used to obtain raw datasets containing chemical structures in different styles based on historical literature, and to generate an image segmentation dataset based on the raw dataset, and further generate an image recognition dataset based on the image segmentation dataset. The detection module is used to use a PDF file parsing tool to convert the PDF documents into several images to be identified for the literature documents that need to be identified by chemical structure recognition. The image segmentation model is trained through the image segmentation dataset. The trained image segmentation model identifies chemical structures in several images to be identified, determines the image detection region containing the chemical structure, and extracts the chemical structure image corresponding to the image detection region. The recognition module is used to generate an image recognition learning dataset and an image recognition test dataset based on the image recognition dataset, and to train an image recognition model using the image recognition learning dataset. It also uses the image recognition test dataset to test the performance of the trained image recognition model. When the test is passed, the trained image recognition model identifies the chemical atoms and hypertext in the chemical structure image. The chemical formula generation module is used to identify chemical atoms and hypertext from chemical structure images, and further identify and distinguish chemical atoms and hypertext from the identification results to infer and construct chemical molecular diagrams. Then, it parses and outputs chemical structural formulas that conform to SMILES or InChI specifications. It is also used to evaluate the accuracy of the generated chemical structural formulas after generating chemical structural formulas from several images to be identified.