Compound database processing system
By splitting the compound database into a basic information library, a property library and a fingerprint library, and based on a non-relational database and a PostgreSQL database, the problem of waste of redundant fields and low retrieval performance of existing compound databases is solved, and efficient data storage and retrieval is achieved.
Patent Information
- Application Number
- CN202411646119.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-18
- Publication Date
- 2025-03-21
- Estimated Expiration
- 2044-11-18
AI Technical Summary
Existing compound databases have problems with waste of redundant fields, reduced data storage flexibility and search performance when constructing, especially when computed resource occupancy is high when performing similarity comparisons and search sorting.
The compound database is split into three sub-banks: the compound basic information database, the compound property database and the compound fingerprint database, which are implemented based on the distributed non-relational database and the PostgreSQL database respectively. This solution avoids pre-stating and configuring the maximum property field set through dynamic scaling of data record lengths, improving data storage flexibility and retrieval performance.
It reduces the storage resource waste rate, improves data storage flexibility and retrieval performance, and reduces the amount of calculation, reduces the computing resource occupancy, improves the search efficiency, and enhances the diversity of retrieval functions.
Smart Images

Figure CN119669184A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to a compound database processing system. Background Art
[0002] In many scientific and technological fields (such as semiconductors, new energy, nanotechnology, etc.), it is necessary to build a large-scale compound database for scientific and technological workers to improve research and development efficiency; the compounds mentioned here include compounds such as solvents, solutions, electrolytes of battery electrolytes, positive / negative electrode material compounds of batteries, crystal material compounds, etc. This type of compound database usually uses a single compound as the basic data unit and stores all unit information related to it. The unit information mentioned here should at least include the chemical formula of the compound molecule, the SMILES sequence of the compound molecule, and all physical / chemical properties of the compound molecule (such as melting point, boiling point, mass, atomic number, effective charge, volume, density, oxidation-reduction energy, etc.).
[0003] At present, most of the construction methods of this type of compound database are based on relational databases (such as SQL databases, MySQL databases, etc.). However, we have found in development practice that this conventional database construction method has some problems: 1) Due to differences in compound structure and application fields, not all compounds have the same data structure, especially in terms of physical / chemical properties. If the database is to be constructed based on a relational database, all possible parameters of all compounds must be counted and a maximum range of data field sets must be constructed based on the statistical results. This will not only cause a large amount of redundant fields to be wasted, reduce the flexibility of data storage, but also reduce the data retrieval performance; 2) When comparing the similarity of such a large number of field records, a large amount of calculation will be generated. If the database is required to support similarity-based retrieval and sorting functions, it will inevitably consume a lot of computing resources to ensure retrieval efficiency. In other words, under conventional computing resource configuration, it is difficult for this compound database based on a relational database to implement similarity-based retrieval and sorting functions.
[0004] In order to reduce the waste of storage resources, improve the flexibility of data storage, improve data retrieval performance, and enhance the diversity of retrieval functions, we propose a new compound database design scheme; this scheme splits the compound database into three sub-databases: compound basic information library, compound property library and compound fingerprint library, among which: 1) Based on a distributed non-relational database (such as MongoDB database, Redis database, SequoiaDB database, etc.), a compound basic information library is constructed, and a compound basic information record is created for each compound in the compound basic information library. The compound basic information record is only used to store the common characteristic information of the compound (such as chemical formula and SMILES sequence); because the common characteristic information is available to all compounds, the compound basic information library of the new scheme will not only not cause redundant field waste, but also improve data retrieval performance when dealing with brief information retrieval; 2) Based on a distributed non-relational database, a compound property library is constructed, and a corresponding compound property table is created for each compound in the compound property library, and in each When a physical / chemical property of a compound is obtained, a corresponding compound property record is added to the property table; because the data record length of the non-relational database can be dynamically expanded, there is no need to pre-count all property types of all compounds, nor is there a need to forcibly configure a maximum property field set for each data record, so the compound property library of the new scheme can not only avoid the waste problem caused by redundant fields, but also improve data storage flexibility and data retrieval performance; 3) Build a compound fingerprint library based on the PostgreSQL database, and create a corresponding compound fingerprint record for each compound in the compound property library. The compound fingerprint record is used to store the molecular fingerprint information of the compound (such as molecular fingerprints calculated based on algorithms such as torsionbv, mfp2, ffp2, etc.), and these molecular fingerprint information can be directly used for similarity comparison; when retrieval and sorting based on similarity is required, using the compound fingerprint library of the new scheme as a retrieval reference can reduce the amount of calculation, reduce the occupancy rate of computing resources, and improve retrieval efficiency.
[0005] To realize this new solution, the following three types of design work need to be completed: database data structure design, database creation / update mechanism design and database retrieval mechanism design. Among them, database data structure design and creation / update mechanism design are also the technical problems that the present invention needs to solve. Summary of the invention
[0006] The purpose of the present invention is to provide a compound database processing system in view of the defects of the prior art, the system comprising: a first access gateway, a first queue module, a first queue management module, a first data update module, a second data update module, a third data update module, a first registered user database, a first data source authorization database and a first compound database. Among them, the first access gateway is used to update the first queue of the first queue module according to each received service application; the first queue management module is used to update the second, third and fourth queues of the first queue module according to the first queue of the first queue module, the first registered user database and the first data source authorization database, and regularly perform batch deletion of processed messages in the first, second, third and fourth queues; the first data update module is used to update the compound basic information library, compound property library and compound fingerprint library of the first compound database according to the second queue; the second data update module is used to update the compound property library according to the compound basic information library and the third queue; the third data update module is used to update the compound fingerprint library according to the compound basic information library and the fourth queue; the first registered user database Used to store multiple first registered user data records; the first data source authorization database is used to store multiple first data source authorization data records; the compound basic information library and the compound property library are each implemented based on a type of distributed non-relational database (MongoDB database, Redis database, SequoiaDB database), and the compound fingerprint library is implemented based on the PostgreSQL database; the compound basic information library is used to store multiple compound basic information records; the compound property library is used to store multiple compound property tables, each compound property table includes multiple compound property records; the compound fingerprint library is used to store multiple compound fingerprint records; each compound basic information record corresponds to a compound molecule, and the compound property table / compound fingerprint record corresponds to the compound basic information record one by one. The first compound database provided by the present invention can reduce the storage resource waste rate, improve data storage flexibility, and improve data retrieval performance; when it is necessary to search and sort based on similarity, if the search is performed based on the first compound database provided by the present invention, it can achieve the purpose of reducing the amount of calculation, reducing the occupancy rate of computing resources, improving retrieval efficiency, and enhancing the diversity of retrieval functions.
[0007] To achieve the above object, an embodiment of the present invention provides a compound database processing system, the system comprising: a first access gateway, a first queue module, a first queue management module, a first data update module, a second data update module, a third data update module, a first registered user database, a first data source authorization database and a first compound database;
[0008] The first queue module is connected to the first access gateway, the first queue management module, the first data update module, the second data update module and the third data update module respectively; the first queue management module is connected to the first registered user database and the first data source authorization database respectively; the first compound database is connected to the first data update module, the second data update module and the third data update module respectively;
[0009] The first access gateway is used for, when receiving a service application, taking the service application received at that time as the corresponding first service application; and performing message publishing processing on the first queue of the first queue module according to the current first service application;
[0010] The first queue module is used to store four message queues, namely the first queue, the second queue, the third queue and the fourth queue; the message publisher of the first queue is the first access gateway, and the message subscriber is the first queue management module; the message publishers of the second, third and fourth queues are all the first queue management module; the message subscribers of the second, third and fourth queues are the corresponding first, second and third data update modules;
[0011] The first queue management module is used to perform message publishing processing on the second, third and fourth queues according to the first queue, the first registered user database and the first data source authorization database; the first queue management module is also used to regularly perform processed message deletion processing on the first, second, third and fourth queues;
[0012] The first data updating module is used to perform data updating processing on the compound basic information library, compound property library and compound fingerprint library of the first compound database according to the second queue;
[0013] The second data updating module is used to perform data updating processing on the compound property library according to the compound basic information library and the third queue;
[0014] The third data updating module is used to perform data updating processing on the compound fingerprint library according to the compound basic information library and the fourth queue;
[0015] The first registered user database is used to store a plurality of first registered user data records;
[0016] The first data source authorization database is used to store a plurality of first data source authorization data records;
[0017] The first compound database includes the compound basic information library, the compound property library and the compound fingerprint library; the compound basic information library and the compound property library are each implemented based on a type of distributed non-relational database; the distributed non-relational database includes at least MongoDB database, Redis database, SequoiaDB database; the compound fingerprint library is implemented based on PostgreSQL database;
[0018] The compound basic information database is used to store multiple compound basic information records; the compound basic information record includes a first compound identification field, a first property table identification field, a first chemical formula field, a first SMILES sequence field, a first creator / updater field, a first creation time field, a first update time field and a first data source field; each of the compound basic information records corresponds to a compound molecule;
[0019] The compound property library is used to store multiple compound property tables; the compound property table includes multiple compound property records; the compound property record includes a second compound identification field, a second chemical formula field, a second SMILES sequence field, a first property name field, a first property attribute sequence field, a first property version field, a first creator field, a second creation time field, and a second data source field; the compound property table corresponds to the compound basic information record one by one;
[0020] The compound fingerprint library is used to store multiple compound fingerprint records; the compound fingerprint record includes a third compound identification field, a second property table identification field, a third chemical formula field, a third SMILES sequence field, a first chemical element sequence field, a first molecular fingerprint field, a second molecular fingerprint field, a third molecular fingerprint field, a second creator / updater field, a third creation time field, a second update time field and a third data source field; the compound fingerprint record corresponds one-to-one to the compound basic information record.
[0021] Preferably, the first service application includes a first user identification, a first user token and a first database instruction;
[0022] The first database instruction includes a first instruction type and a first instruction parameter;
[0023] The first instruction type includes basic information update instructions, property update instructions and fingerprint update instructions;
[0024] When the first instruction type is a basic information update instruction, the corresponding first instruction parameters include a first chemical formula, a first SMILES sequence and a first data source;
[0025] When the first instruction type is a property update instruction, the corresponding first instruction parameters include a second chemical formula, a first property name, a first property attribute sequence, a first property version, and a second data source; the first property attribute sequence consists of one or more first attributes; the first attribute includes an attribute name and an attribute value;
[0026] When the first instruction type is a fingerprint update instruction, the corresponding first instruction parameters include a third chemical formula, a first chemical element sequence, a first molecular fingerprint, a second molecular fingerprint, a third molecular fingerprint and a third data source; the first chemical element sequence is a chemical element sequence corresponding to the third chemical formula, and is composed of a plurality of first chemical elements; the first molecular fingerprint is a molecular fingerprint calculated based on the TorsionBV algorithm, the second molecular fingerprint is a molecular fingerprint calculated based on the MFP2 algorithm, and the third molecular fingerprint is a molecular fingerprint calculated based on the FFP2 algorithm;
[0027] The first queue includes a plurality of first queue messages; the first queue message includes the first user identifier, the first user token, the first database instruction, the first publishing time and the first processing status; the first processing status includes unprocessed and processed;
[0028] The second queue includes a plurality of second queue messages; the second queue messages include the first chemical formula, the first SMILES sequence, the first data source, the first creator information, the second publishing time, and the second processing status; the second processing status includes unprocessed and processed;
[0029] The third queue includes a plurality of third queue messages; the third queue messages include the second chemical formula, the first property name, the first property attribute sequence, the first property version, the second data source, the second creator information, the third publishing time and the third processing status; the third processing status includes unprocessed and processed;
[0030] The fourth queue includes a plurality of fourth queue messages; the fourth queue messages include the third chemical formula, the first chemical element sequence, the first molecular fingerprint, the second molecular fingerprint, the third molecular fingerprint, the third data source, third creator information, a fourth publishing time and a fourth processing status; the fourth processing status includes unprocessed and processed;
[0031] The first registered user database is implemented based on a type of relational database; the relational database includes at least an SQL database and a MySQL database; the first registered user data record includes at least a first user identification field, a first user token field, a first token validity period field and a first user information field;
[0032] The first data source authorization database is implemented based on a type of relational database; the first data source authorization data record at least includes a first data source field and a first authorization validity period field.
[0033] Preferably, the first access gateway is specifically used to extract the corresponding first user identifier, the first user token and the first database instruction from the current first business application when performing message publishing processing on the first queue of the first queue module according to the current first business application; and use the current time as the corresponding first publishing time; and set a corresponding first processing status to unprocessed; and form a corresponding first queue message composed of the obtained first user identifier, the first user token, the first database instruction, the first publishing time and the first processing status and publish it to the first queue.
[0034] Preferably, the first queue management module is specifically used for, when performing message publishing processing on the second, third and fourth queues according to the first queue, the first registered user database and the first data source authorization database,
[0035] The total number of the first queue messages in the first queue whose first processing status is unprocessed is counted in real time, and the corresponding total number of currently unprocessed messages is refreshed based on the statistical result; and when the total number of currently unprocessed messages is greater than zero, the first queue message in the first queue with the earliest time and the first processing status being unprocessed is taken as the corresponding current queue message;
[0036] and extracting the corresponding first user identifier, the first user token and the first database instruction from the current queue message as the corresponding current user identifier, the current user token and the current database instruction; and taking the first instruction type of the current database instruction as the corresponding current instruction type, and taking the first, second or third data source corresponding to the current instruction type in the current database instruction as the corresponding current data source; and taking the current time as the corresponding current user time; and taking the first registered user data record in the first registered user database whose first user identifier field matches the current user identifier, whose first user token field matches the current user token and whose first token validity period field satisfies the current user time as the corresponding current registered user data record; and taking the first data source authorization data record in the first data source authorization database whose first data source field matches the current data source and whose first authorization validity period field satisfies the current user time as the corresponding current data source authorization data record; and identifying whether both the current registered user data record and the current data source authorization data record are not empty, and if so, setting the corresponding current verification result as verification passed, and otherwise setting the corresponding current verification result as verification failed;
[0037] And when the current verification result is verification passed, identifying the current instruction type;
[0038] If the current instruction type is a basic information update instruction, the corresponding first chemical formula, the first SMILES sequence and the first data source are extracted from the current database instruction; and the first user information field of the current registered user data record is extracted as the corresponding first creator information; and the current time is used as the corresponding second publishing time; and a corresponding second processing state is set to unprocessed; and a corresponding second queue message composed of the first chemical formula, the first SMILES sequence, the first data source, the first creator information, the second publishing time and the second processing state obtained this time is published to the second queue; and the first processing state of the current queue message is set to processed;
[0039] If the current instruction type is a property update instruction, the corresponding second chemical formula, the first property name, the first property attribute sequence, the first property version and the second data source are extracted from the current database instruction; and the first user information field of the current registered user data record is extracted as the corresponding second creator information; and the current time is used as the corresponding third publishing time; and a corresponding third processing state is set to unprocessed; and a corresponding third queue message is composed of the second chemical formula, the first property name, the first property attribute sequence, the first property version, the second data source, the second creator information, the third publishing time and the third processing state obtained this time, and is published to the third queue; and the first processing state of the current queue message is set to processed;
[0040] If the current instruction type is a fingerprint update instruction, the corresponding third chemical formula, the first chemical element sequence, the first molecular fingerprint, the second molecular fingerprint, the third molecular fingerprint and the third data source are extracted from the current database instruction; and the first user information field of the current registered user data record is extracted as the corresponding third creator information; and the current time is used as the corresponding fourth release time; and a corresponding fourth processing status is set to unprocessed; and the third chemical formula, the first chemical element sequence, the first molecular fingerprint, the second molecular fingerprint, the third molecular fingerprint, the third data source, the third creator information, the fourth release time and the fourth processing status obtained this time form a corresponding fourth queue message and publish it to the fourth queue; and the first processing status of the current queue message is set to processed.
[0041] Preferably, the first queue management module is specifically used to periodically delete the first, second, third and fourth queue messages whose corresponding first, second, third and fourth processing status are processed in the first, second, third and fourth queues according to a preset first time frequency when the processed messages in the first, second, third and fourth queues are periodically deleted.
[0042] Preferably, the first data updating module is specifically used for performing data updating processing on the compound basic information library, compound property library and compound fingerprint library of the first compound database according to the second queue,
[0043] The total number of the second queue messages whose second processing status is unprocessed in the second queue is counted in real time, and the corresponding total number of currently unprocessed messages is refreshed based on the statistical results; and when the total number of currently unprocessed messages is greater than zero, the second queue message whose earliest time in the second queue and whose second processing status is unprocessed is taken as the corresponding current queue message; and the first chemical formula, the first SMILES sequence, the first data source and the first creator information of the current queue message are taken as the corresponding current chemical formula, current SMILES sequence, current data source and current creator; and the compound basic information record whose first chemical formula field in the compound basic information library matches the current chemical formula is taken as the corresponding current basic information record;
[0044] and identifying whether the current basic information record is empty;
[0045] If the current basic information record is empty, then a new compound basic information record with all fields being empty is added to the compound basic information library as the corresponding current newly added basic information record; and a unique identifier is assigned to the current newly added basic information record as the corresponding current compound identifier; and an empty compound property table is added to the compound property library as the corresponding current newly added property table, and a unique identifier is assigned to the current newly added property table as the corresponding current property table identifier; and a new compound fingerprint record with all fields being empty is added to the compound fingerprint library as the corresponding current newly added fingerprint record; and the current time is used as the corresponding current creation time; and the first compound identification field, the first property table identification field, the first chemical formula field, and the first SMILES sequence of the currently newly added basic information record are assigned to the first compound identification field, the first property table identification field, the first chemical formula field, and the first SMILES sequence field, the first creator / updater field, the first creation time field and the first data source field are set to the corresponding current compound identifier, the current property table identifier, the current chemical formula, the current SMILES sequence, the current creator, the current creation time and the current data source; and the third compound identifier field, the second property table identifier field, the third chemical formula field, the third SMILES sequence field, the second creator / updater field, the third creation time field and the third data source field of the current newly added fingerprint record are set to the corresponding current compound identifier, the current property table identifier, the current chemical formula, the current SMILES sequence, the current creator, the current creation time and the current data source; and the second processing status of the current queue message is set to processed;
[0046] If the current basic information record is not empty, the current time is used as the corresponding current update time; and the first SMILES sequence field, the first creator / updater field, the first update time field and the first data source field of the current basic information record are reset to the corresponding current SMILES sequence, the current creator, the current update time and the current data source; and the second processing status of the current queue message is set to processed.
[0047] Preferably, the second data updating module is specifically used for, when performing data updating processing on the compound property library according to the compound basic information library and the third queue,
[0048] The total number of the third queue messages whose third processing status is unprocessed in the third queue is counted in real time, and the corresponding total number of currently unprocessed messages is refreshed based on the statistical result; and when the total number of currently unprocessed messages is greater than zero, the third queue message whose earliest time in the third queue and whose third processing status is unprocessed is taken as the corresponding current queue message; and the second chemical formula, the first property name, the first property attribute sequence, the first property version, the second data source and the second creator information of the current queue message are taken as the corresponding current chemical formula, current property name, current property attribute sequence, current property version, current data source and current creator;
[0049] and taking the compound basic information record whose first chemical formula field in the compound basic information library matches the current chemical formula as the corresponding current basic information record; and taking the first compound identification field, the first property table identification field and the first SMILES sequence field of the current basic information record as the corresponding current compound identification, current property table identification and current SMILES sequence;
[0050] And the compound property table corresponding to the current property table identifier in the compound property library is used as the corresponding current property table; and when the current property table is not empty, a compound property record with all fields being empty is added to the current property table as the corresponding current newly added property record; and the current time is used as the corresponding current creation time; and the second compound identification field, the second chemical formula field, the second SMILES sequence field, the first property name field, the first property attribute sequence field, the first property version field, the first creator field, the second creation time field and the second data source field of the current newly added property record are set to the corresponding current compound identification, the current chemical formula, the current SMILES sequence, the current property name, the current property attribute sequence, the current property version, the current creator, the current creation time and the current data source;
[0051] And the third processing status of the current queue message is set to processed.
[0052] Preferably, the third data updating module is specifically used for, when performing data updating processing on the compound fingerprint library according to the compound basic information library and the fourth queue,
[0053] The total number of the fourth queue messages whose fourth processing status is unprocessed in the fourth queue is counted in real time, and the corresponding total number of currently unprocessed messages is refreshed based on the statistical result; and when the total number of currently unprocessed messages is greater than zero, the fourth queue message whose earliest time in the fourth queue and whose fourth processing status is unprocessed is taken as the corresponding current queue message; and the third chemical formula, the first chemical element sequence, the first molecular fingerprint, the second molecular fingerprint, the third molecular fingerprint, the third data source, and the third creator information of the current queue message are taken as the corresponding current chemical formula, current chemical element sequence, current molecular fingerprint A, current molecular fingerprint B, current molecular fingerprint C, current data source, and current updater;
[0054] and taking the compound basic information record whose first chemical formula field in the compound basic information library matches the current chemical formula as the corresponding current basic information record; and taking the first compound identification field, the first property table identification field and the first SMILES sequence field of the current basic information record as the corresponding current compound identification, current property table identification and current SMILES sequence;
[0055] and taking the compound fingerprint record whose third compound identification field in the compound fingerprint library matches the current compound identification as the corresponding current fingerprint record; and when the current fingerprint record is not empty, taking the current time as the corresponding current update time; and resetting the second property table identification field, the third chemical formula field, the third SMILES sequence field, the first chemical element sequence field, the first molecular fingerprint field, the second molecular fingerprint field, the third molecular fingerprint field, the second creator / updater field, the second update time field and the third data source field of the current fingerprint record to the corresponding current property table identification, the current chemical formula, the current SMILES sequence, the current chemical element sequence, the current molecular fingerprint A, the current molecular fingerprint B, the current molecular fingerprint C, the current updater, the current update time and the current data source;
[0056] And the fourth processing status of the current queue message is set to processed.
[0057] An embodiment of the present invention provides a compound database processing system, which includes: a first access gateway, a first queue module, a first queue management module, a first data update module, a second data update module, a third data update module, a first registered user database, a first data source authorization database and a first compound database. Among them, the first access gateway is used to update the first queue of the first queue module according to each received service application; the first queue management module is used to update the second, third and fourth queues of the first queue module according to the first queue of the first queue module, the first registered user database and the first data source authorization database, and regularly delete the processed messages in the first, second, third and fourth queues in batches; the first data update module is used to update the compound basic information library, compound property library and compound fingerprint library of the first compound database according to the second queue; the second data update module is used to update the compound property library according to the compound basic information library and the third queue; the third data update module is used to update the compound fingerprint library according to the compound basic information library and the fourth queue; the first registered user database Used to store multiple first registered user data records; the first data source authorization database is used to store multiple first data source authorization data records; the compound basic information database and the compound property database are each implemented based on a type of distributed non-relational database (MongoDB database, Redis database, SequoiaDB database), and the compound fingerprint database is implemented based on the PostgreSQL database; the compound basic information database is used to store multiple compound basic information records; the compound property database is used to store multiple compound property tables, each compound property table includes multiple compound property records; the compound fingerprint database is used to store multiple compound fingerprint records; each compound basic information record corresponds to a compound molecule, and the compound property table / compound fingerprint record corresponds to the compound basic information record one by one. The first compound database provided by the embodiment of the present invention reduces the waste rate of storage resources, improves data storage flexibility and data retrieval performance; when it is necessary to search and sort based on similarity, if the search is performed based on the first compound database provided by the embodiment of the present invention, it not only reduces the amount of calculation, reduces the occupancy rate of computing resources, improves the retrieval efficiency, but also enhances the diversity of retrieval functions. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] Figure 1 A module structure diagram of a compound database processing system provided by an embodiment of the present invention;
[0059] Figure 2 A data structure diagram of a first registered user database provided by an embodiment of the present invention;
[0060] Figure 3 A data structure diagram of a first data source authorization database provided by an embodiment of the present invention;
[0061] Figure 4 This is a data structure diagram of the first compound database provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0062] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0063] The embodiment of the present invention provides a compound database processing system, such as Figure 1 As shown in the module structure diagram of a compound database processing system provided in an embodiment of the present invention, the system mainly includes: a first access gateway 1, a first queue module 2, a first queue management module 3, a first data update module 4, a second data update module 5, a third data update module 6, a first registered user database 7, a first data source authorization database 8 and a first compound database 9.
[0064] The connection relationship between the components of the embodiment of the present invention is: the first queue module 2 is connected to the first access gateway 1, the first queue management module 3, the first data update module 4, the second data update module 5 and the third data update module 6 respectively; the first queue management module 3 is connected to the first registered user database 7 and the first data source authorization database 8 respectively; the first compound database 9 is connected to the first data update module 4, the second data update module 5 and the third data update module 6 respectively.
[0065] (I) First access gateway 1:
[0066] The first access gateway 1 of the embodiment of the present invention is any type of port / interface, service / microservice, device, equipment or server that can implement the gateway forwarding function. The first access gateway 1 is used to receive a service application each time, and use the service application received at that time as the corresponding first service application; and perform message publishing processing on the first queue of the first queue module 2 according to the current first service application.
[0067] Here, the first service application of the embodiment of the present invention includes a first user identifier, a first user token and a first database instruction; the first database instruction includes a first instruction type and a first instruction parameter; the first instruction type includes a basic information update instruction, a property update instruction and a fingerprint update instruction; 1) when the first instruction type is a basic information update instruction, the corresponding first instruction parameter includes a first chemical formula, a first SMILES sequence and a first data source; 2) when the first instruction type is a property update instruction, the corresponding first instruction parameter includes a second chemical formula, a first property name, a first property attribute sequence, a first property version and a second data source; the first property attribute sequence consists of one or more first attributes; the first attribute includes an attribute name and an attribute value; 3) when the first instruction type is a fingerprint update instruction, the corresponding first instruction parameter includes a third chemical formula, a first chemical element sequence, a first molecular fingerprint, a second molecular fingerprint, a third molecular fingerprint and a third data source; the first chemical element sequence is a chemical element sequence corresponding to the third chemical formula, consisting of multiple first chemical elements; the first molecular fingerprint is a molecular fingerprint calculated based on the torsionbv (Topological-Torsion Bit Fingerprint) algorithm, and the second molecular fingerprint is a molecular fingerprint calculated based on the mfp2 (Morgan Bit Fingerprint with Default Radius 2) The molecular fingerprint calculated by the algorithm. The third molecular fingerprint is the molecular fingerprint calculated based on the ffp2 (Morgan FCFP Bit Fingerprint with Default Radius 2) algorithm.
[0068] In a specific implementation method of an embodiment of the present invention, the first access gateway 1 is specifically used to, when performing message publishing processing on the first queue of the first queue module 2 according to the current first business application: extract the corresponding first user identifier, the first user token and the first database instruction from the current first business application; and use the current time as the corresponding first publishing time; and set a corresponding first processing status to unprocessed; and a corresponding first queue message composed of the obtained first user identifier, first user token, first database instruction, first publishing time and first processing status is published to the first queue.
[0069] (II) First queue module 2:
[0070] The first queue module 2 of the embodiment of the present invention is any type of service / microservice, device, equipment, server, system or platform that provides message queue storage / management. The first queue module 2 is used to store four message queues, namely the first queue, the second queue, the third queue and the fourth queue; wherein the message publisher of the first queue is the first access gateway 1, and the message subscriber is the first queue management module 3; the message publishers of the second, third and fourth queues are all the first queue management module 3; the message subscribers of the second, third and fourth queues are the corresponding first data update module 4, the second data update module 5 and the third data update module 6.
[0071] The first queue of the embodiment of the present invention includes multiple first queue messages; wherein the first queue message includes a first user identifier, a first user token, a first database instruction, a first publishing time and a first processing status; the first processing status includes unprocessed and processed.
[0072] The second queue of the embodiment of the present invention includes multiple second queue messages; wherein the second queue messages include a first chemical formula, a first SMILES sequence, a first data source, first creator information, a second publishing time and a second processing status; the second processing status includes unprocessed and processed.
[0073] The third queue of an embodiment of the present invention includes multiple third queue messages; wherein the third queue messages include a second chemical formula, a first property name, a first property attribute sequence, a first property version, a second data source, second creator information, a third publishing time and a third processing status; the third processing status includes unprocessed and processed.
[0074] The fourth queue of an embodiment of the present invention includes multiple fourth queue messages; wherein the fourth queue messages include a third chemical formula, a first chemical element sequence, a first molecular fingerprint, a second molecular fingerprint, a third molecular fingerprint, a third data source, third creator information, a fourth publishing time and a fourth processing status; the fourth processing status includes unprocessed and processed.
[0075] (III) First queue management module 3:
[0076] The first queue management module 3 of the embodiment of the present invention is any type of service / microservice, device, equipment, server, system or platform that can implement the following queue management module functions. The first queue management module 3 is used to perform message publishing processing on the second, third and fourth queues according to the first queue, the first registered user database 7 and the first data source authorization database 8. The first queue management module 3 is also used to regularly delete processed messages on the first, second, third and fourth queues.
[0077] In another specific implementation of the embodiment of the present invention, the first queue management module 3 is specifically used to: when performing message publishing processing on the second, third and fourth queues according to the first queue, the first registered user database 7 and the first data source authorization database 8:
[0078] Step A1, real-time statistics are performed on the total number of first queue messages in the first queue whose first processing status is unprocessed, and the corresponding current total number of unprocessed messages is refreshed based on the statistical results; and when the current total number of unprocessed messages is greater than zero, the first queue message in the first queue with the earliest time and the first processing status being unprocessed is used as the corresponding current queue message;
[0079] Step A2, and extract the corresponding first user identification, first user token and first database instruction from the current queue message as the corresponding current user identification, current user token and current database instruction; and use the first instruction type of the current database instruction as the corresponding current instruction type, and use the first, second or third data source corresponding to the current instruction type in the current database instruction as the corresponding current data source; and use the current time as the corresponding current user time; and use the first registered user data record in the first registered user database 7 where the first user identification field matches the current user identification and the first user token field matches the current user token and the first token validity period field meets the current user time as the corresponding current registered user data record; and use the first data source authorization data record in the first data source authorization database 8 where the first data source field matches the current data source and the first authorization validity period field meets the current user time as the corresponding current data source authorization data record; and identify whether the current registered user data record and the current data source authorization data record are both not empty, and if so, set the corresponding current verification result as verification passed, and if otherwise, set the corresponding current verification result as verification failed;
[0080] Step A3, when the current verification result is verification passed, identify the current instruction type;
[0081] Step A4, if the current instruction type is a basic information update instruction, extract the corresponding first chemical formula, first SMILES sequence and first data source from the current database instruction; extract the first user information field of the current registered user data record as the corresponding first creator information; use the current time as the corresponding second publishing time; set a corresponding second processing state as unprocessed; and form a corresponding second queue message with the first chemical formula, first SMILES sequence, first data source, first creator information, second publishing time and second processing state obtained this time, and publish it to the second queue; and set the first processing state of the current queue message to be processed;
[0082] Step A5, if the current instruction type is a property update instruction, extract the corresponding second chemical formula, first property name, first property attribute sequence, first property version and second data source from the current database instruction; extract the first user information field of the current registered user data record as the corresponding second creator information; use the current time as the corresponding third release time; set a corresponding third processing state as unprocessed; and form a corresponding third queue message with the second chemical formula, first property name, first property attribute sequence, first property version, second data source, second creator information, third release time and third processing state obtained this time, and publish it to the third queue; set the first processing state of the current queue message to processed;
[0083] Step A6, if the current instruction type is a fingerprint update instruction, then extract the corresponding third chemical formula, first chemical element sequence, first molecular fingerprint, second molecular fingerprint, third molecular fingerprint and third data source from the current database instruction; and extract the first user information field of the current registered user data record as the corresponding third creator information; and use the current time as the corresponding fourth release time; and set a corresponding fourth processing state to unprocessed; and form a corresponding fourth queue message composed of the third chemical formula, first chemical element sequence, first molecular fingerprint, second molecular fingerprint, third molecular fingerprint, third data source, third creator information, fourth release time and fourth processing state obtained this time to be released to the fourth queue; and set the first processing state of the current queue message to be processed.
[0084] In another specific implementation of the embodiment of the present invention, the first queue management module 3 is specifically used to periodically delete the first, second, third and fourth queue messages whose corresponding first, second, third and fourth processing states are processed in the first, second, third and fourth queues according to a preset first time frequency when periodically deleting the processed messages in the first, second, third and fourth queues. Here, the first time frequency is a preset time frequency parameter.
[0085] (IV) First data updating module 4:
[0086] The first data update module 4 of the embodiment of the present invention is any type of service / microservice, device, equipment, server, system or platform that can implement the following update function. The first data update module 4 is used to perform data update processing on the compound basic information library 91, the compound property library 92 and the compound fingerprint library 93 of the first compound database 9 according to the second queue.
[0087] In another specific implementation of the embodiment of the present invention, the first data updating module 4 is specifically used for performing data updating processing on the compound basic information library 91, the compound property library 92 and the compound fingerprint library 93 of the first compound database 9 according to the second queue:
[0088] Step B1, real-time statistics are performed on the total number of second queue messages in the second queue whose second processing status is unprocessed, and the corresponding total number of currently unprocessed messages is refreshed based on the statistical results; and when the total number of currently unprocessed messages is greater than zero, the second queue message with the earliest time and the second processing status being unprocessed in the second queue is taken as the corresponding current queue message; and the first chemical formula, the first SMILES sequence, the first data source and the first creator information of the current queue message are taken as the corresponding current chemical formula, the current SMILES sequence, the current data source and the current creator; and the compound basic information record whose first chemical formula field in the compound basic information library 91 matches the current chemical formula is taken as the corresponding current basic information record;
[0089] Step B2, identifying whether the current basic information record is empty;
[0090] Step B3, if the current basic information record is empty, then add a new compound basic information record in which all fields are empty fields in the compound basic information library 91 as the corresponding current newly added basic information record; and assign a unique identifier to the current newly added basic information record as the corresponding current compound identifier; and add an empty compound property table in the compound property library 92 as the corresponding current newly added property table, and assign a unique identifier to the current newly added property table as the corresponding current property table identifier; and add a new compound fingerprint record in which all fields are empty fields in the compound fingerprint library 93 as the corresponding current newly added fingerprint record; and use the current time as the corresponding current creation time; and assign the first compound identification field and the first property table identification field of the current newly added basic information record to the compound fingerprint library 93; , the first chemical formula field, the first SMILES sequence field, the first creator / updater field, the first creation time field and the first data source field are set to the corresponding current compound identifier, the current property table identifier, the current chemical formula, the current SMILES sequence, the current creator, the current creation time and the current data source; and the third compound identifier field, the second property table identifier field, the third chemical formula field, the third SMILES sequence field, the second creator / updater field, the third creation time field and the third data source field of the current newly added fingerprint record are set to the corresponding current compound identifier, the current property table identifier, the current chemical formula, the current SMILES sequence, the current creator, the current creation time and the current data source; and the second processing status of the current queue message is set to processed;
[0091] Step B4, if the current basic information record is not empty, the current time is used as the corresponding current update time; and the first SMILES sequence field, the first creator / updater field, the first update time field and the first data source field of the current basic information record are reset to the corresponding current SMILES sequence, current creator, current update time and current data source; and the second processing status of the current queue message is set to processed.
[0092] (V) Second data updating module 5:
[0093] The second data update module 5 of the embodiment of the present invention is any type of service / microservice, device, equipment, server, system or platform that can implement the following update function. The second data update module 5 is used to perform data update processing on the compound property library 92 according to the compound basic information library 91 and the third queue.
[0094] In another specific implementation of the embodiment of the present invention, the second data updating module 5 is specifically used for performing data updating processing on the compound property library 92 according to the compound basic information library 91 and the third queue:
[0095] Step C1, real-time statistics are performed on the total number of third queue messages in the third queue whose third processing status is unprocessed, and the corresponding total number of currently unprocessed messages is refreshed based on the statistical results; and when the total number of currently unprocessed messages is greater than zero, the third queue message with the earliest time and the third processing status being unprocessed in the third queue is taken as the corresponding current queue message; and the second chemical formula, the first property name, the first property attribute sequence, the first property version, the second data source and the second creator information of the current queue message are taken as the corresponding current chemical formula, the current property name, the current property attribute sequence, the current property version, the current data source and the current creator;
[0096] Step C2, and taking the compound basic information record whose first chemical formula field in the compound basic information library 91 matches the current chemical formula as the corresponding current basic information record; and taking the first compound identification field, the first property table identification field and the first SMILES sequence field of the current basic information record as the corresponding current compound identification, the current property table identification and the current SMILES sequence;
[0097] Step C3, and use the compound property table corresponding to the current property table identifier in the compound property library 92 as the corresponding current property table; and when the current property table is not empty, add a compound property record in which all fields are empty fields in the current property table as the corresponding current newly added property record; and use the current time as the corresponding current creation time; and set the second compound identification field, the second chemical formula field, the second SMILES sequence field, the first property name field, the first property attribute sequence field, the first property version field, the first creator field, the second creation time field and the second data source field of the current newly added property record to the corresponding current compound identification, current chemical formula, current SMILES sequence, current property name, current property attribute sequence, current property version, current creator, current creation time current data source;
[0098] Step C4, and set the third processing status of the current queue message to processed.
[0099] (VI) Third data updating module 6:
[0100] The third data update module 6 of the embodiment of the present invention is any type of service / microservice, device, equipment, server, system or platform that can implement the following update function. The third data update module 6 is used to perform data update processing on the compound fingerprint library 93 according to the compound basic information library 91 and the fourth queue.
[0101] In another specific implementation of the embodiment of the present invention, the third data updating module 6 is specifically used for performing data updating processing on the compound fingerprint library 93 according to the compound basic information library 91 and the fourth queue:
[0102] Step D1, real-time statistics are performed on the total number of fourth queue messages in the fourth queue whose fourth processing status is unprocessed, and the corresponding total number of currently unprocessed messages is refreshed based on the statistical results; and when the total number of currently unprocessed messages is greater than zero, the earliest fourth queue message in the fourth queue with the fourth processing status being unprocessed is used as the corresponding current queue message; and the third chemical formula, the first chemical element sequence, the first molecular fingerprint, the second molecular fingerprint, the third molecular fingerprint, the third data source, and the third creator information of the current queue message are used as the corresponding current chemical formula, the current chemical element sequence, the current molecular fingerprint A, the current molecular fingerprint B, the current molecular fingerprint C, the current data source, and the current updater;
[0103] Step D2, and use the compound basic information record whose first chemical formula field in the compound basic information library 91 matches the current chemical formula as the corresponding current basic information record; and use the first compound identification field, the first property table identification field and the first SMILES sequence field of the current basic information record as the corresponding current compound identification, the current property table identification and the current SMILES sequence;
[0104] Step D3, and use the compound fingerprint record whose third compound identification field in the compound fingerprint library 93 matches the current compound identification as the corresponding current fingerprint record; and when the current fingerprint record is not empty, use the current time as the corresponding current update time; and reset the second property table identification field, the third chemical formula field, the third SMILES sequence field, the first chemical element sequence field, the first molecular fingerprint field, the second molecular fingerprint field, the third molecular fingerprint field, the second creator / updater field, the second update time field and the third data source field of the current fingerprint record to the corresponding current property table identification, current chemical formula, current SMILES sequence, current chemical element sequence, current molecular fingerprint A, current molecular fingerprint B, current molecular fingerprint C, current updater, current update time and current data source;
[0105] Step D4, and set the fourth processing status of the current queue message to processed.
[0106] (VII) First registered user database 7:
[0107] The first registered user database 7 of the embodiment of the present invention is implemented based on a type of relational database, and the relational database mentioned here includes at least an SQL database and a MySQL database. The first registered user database 7 is used to store multiple first registered user data records; wherein the first registered user data record includes at least a first user identification field, a first user token field, a first token validity period field, and a first user information field, such as Figure 2 The data structure diagram of the first registered user database provided in the embodiment of the present invention is shown.
[0108] (VIII) First data source authorization database 8:
[0109] The first data source authorization database 8 of the embodiment of the present invention is implemented based on a type of relational database. The first data source authorization database 8 is used to store multiple first data source authorization data records; wherein the first data source authorization data record at least includes a first data source field and a first authorization validity period field, such as Figure 3 The data structure diagram of the first data source authorization database provided by the embodiment of the present invention is shown.
[0110] (IX) First Compound Database 9:
[0111] The first compound database 9 of the embodiment of the present invention includes a compound basic information library 91, a compound property library 92 and a compound fingerprint library 93. The compound basic information library 91 and the compound property library 92 are each implemented based on a type of distributed non-relational database, and the distributed non-relational database mentioned here includes at least MongoDB database, Redis database, and SequoiaDB database; the compound fingerprint library 93 is implemented based on a PostgreSQL database.
[0112] like Figure 4 As shown in the data structure diagram of the first compound database provided in an embodiment of the present invention, the compound basic information library 91 of the embodiment of the present invention is used to store multiple compound basic information records; wherein the compound basic information record includes a first compound identification field, a first property table identification field, a first chemical formula field, a first SMILES sequence field, a first creator / updater field, a first creation time field, a first update time field and a first data source field; each compound basic information record corresponds to a compound molecule.
[0113] like Figure 4 As shown, the compound property library 92 of the embodiment of the present invention is used to store multiple compound property tables; wherein the compound property table includes multiple compound property records; each compound property record includes a second compound identification field, a second chemical formula field, a second SMILES sequence field, a first property name field, a first property attribute sequence field, a first property version field, a first creator field, a second creation time field and a second data source field; the compound property table corresponds one-to-one to the compound basic information record.
[0114] like Figure 4 As shown, the compound fingerprint library 93 of the embodiment of the present invention is used to store multiple compound fingerprint records; wherein the compound fingerprint record includes a third compound identification field, a second property table identification field, a third chemical formula field, a third SMILES sequence field, a first chemical element sequence field, a first molecular fingerprint field, a second molecular fingerprint field, a third molecular fingerprint field, a second creator / updater field, a third creation time field, a second update time field and a third data source field; the compound fingerprint record corresponds one-to-one to the compound basic information record.
[0115] It should be noted that it should be understood that the division of the various modules of the above system is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the first data update module can be a separately established processing element, or it can be integrated in a chip of the above-mentioned device. In addition, it can also be stored in the memory of the above-mentioned device in the form of program code, and called and executed by a processing element of the above-mentioned system. The function of the above-mentioned module is determined. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each method step of the aforementioned method or each module processing step of the aforementioned system can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.
[0116] For example, the modules of the above system may be one or more integrated circuits configured to implement the aforementioned method, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module of the above system is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
[0117] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the above method embodiments are generated. The above computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.) methods. The above-mentioned computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The above-mentioned available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.
[0118] An embodiment of the present invention provides a compound database processing system, which includes: a first access gateway, a first queue module, a first queue management module, a first data update module, a second data update module, a third data update module, a first registered user database, a first data source authorization database and a first compound database. Among them, the first access gateway is used to update the first queue of the first queue module according to each received service application; the first queue management module is used to update the second, third and fourth queues of the first queue module according to the first queue of the first queue module, the first registered user database and the first data source authorization database, and regularly delete the processed messages in the first, second, third and fourth queues in batches; the first data update module is used to update the compound basic information library, compound property library and compound fingerprint library of the first compound database according to the second queue; the second data update module is used to update the compound property library according to the compound basic information library and the third queue; the third data update module is used to update the compound fingerprint library according to the compound basic information library and the fourth queue; the first registered user database Used to store multiple first registered user data records; the first data source authorization database is used to store multiple first data source authorization data records; the compound basic information database and the compound property database are each implemented based on a type of distributed non-relational database (MongoDB database, Redis database, SequoiaDB database), and the compound fingerprint database is implemented based on the PostgreSQL database; the compound basic information database is used to store multiple compound basic information records; the compound property database is used to store multiple compound property tables, each compound property table includes multiple compound property records; the compound fingerprint database is used to store multiple compound fingerprint records; each compound basic information record corresponds to a compound molecule, and the compound property table / compound fingerprint record corresponds to the compound basic information record one by one. The first compound database provided by the embodiment of the present invention reduces the waste rate of storage resources, improves data storage flexibility and data retrieval performance; when it is necessary to search and sort based on similarity, if the search is performed based on the first compound database provided by the embodiment of the present invention, it not only reduces the amount of calculation, reduces the occupancy rate of computing resources, improves the retrieval efficiency, but also enhances the diversity of retrieval functions.
[0119] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0120] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0121] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A compound database processing system, characterized in that: The system comprises: a first access gateway, a first queue module, a first queue management module, a first data update module, a second data update module, a third data update module, a first registered user database, a first data source authorization database and a first compound database; The first queue module is connected to the first access gateway, the first queue management module, the first data update module, the second data update module and the third data update module respectively; the first queue management module is connected to the first registered user database and the first data source authorization database respectively; the first compound database is connected to the first data update module, the second data update module and the third data update module respectively; The first access gateway is used for, when receiving a service application, taking the service application received at that time as the corresponding first service application; and performing message publishing processing on the first queue of the first queue module according to the current first service application; The first queue module is used to store four message queues, namely the first queue, the second queue, the third queue and the fourth queue; the message publisher of the first queue is the first access gateway, and the message subscriber is the first queue management module; the message publishers of the second, third and fourth queues are all the first queue management module; the message subscribers of the second, third and fourth queues are the corresponding first, second and third data update modules; The first queue management module is used to perform message publishing processing on the second, third and fourth queues according to the first queue, the first registered user database and the first data source authorization database; the first queue management module is also used to regularly perform processed message deletion processing on the first, second, third and fourth queues; The first data updating module is used to perform data updating processing on the compound basic information library, compound property library and compound fingerprint library of the first compound database according to the second queue; The second data updating module is used to perform data updating processing on the compound property library according to the compound basic information library and the third queue; The third data updating module is used to perform data updating processing on the compound fingerprint library according to the compound basic information library and the fourth queue; The first registered user database is used to store a plurality of first registered user data records; The first data source authorization database is used to store a plurality of first data source authorization data records; The first compound database includes the compound basic information library, the compound property library and the compound fingerprint library; the compound basic information library and the compound property library are each implemented based on a type of distributed non-relational database; the distributed non-relational database includes at least MongoDB database, Redis database, SequoiaDB database; the compound fingerprint library is implemented based on PostgreSQL database; The compound basic information database is used to store multiple compound basic information records; the compound basic information record includes a first compound identification field, a first property table identification field, a first chemical formula field, a first SMILES sequence field, a first creator / updater field, a first creation time field, a first update time field and a first data source field; each of the compound basic information records corresponds to a compound molecule; The compound property library is used to store multiple compound property tables; the compound property table includes multiple compound property records; the compound property record includes a second compound identification field, a second chemical formula field, a second SMILES sequence field, a first property name field, a first property attribute sequence field, a first property version field, a first creator field, a second creation time field, and a second data source field; the compound property table corresponds to the compound basic information record one by one; The compound fingerprint library is used to store multiple compound fingerprint records; the compound fingerprint record includes a third compound identification field, a second property table identification field, a third chemical formula field, a third SMILES sequence field, a first chemical element sequence field, a first molecular fingerprint field, a second molecular fingerprint field, a third molecular fingerprint field, a second creator / updater field, a third creation time field, a second update time field and a third data source field; the compound fingerprint record corresponds one-to-one to the compound basic information record.
2. The compound database processing system according to claim 1, characterized in that: The first service application includes a first user identification, a first user token and a first database instruction; The first database instruction includes a first instruction type and a first instruction parameter; The first instruction type includes basic information update instructions, property update instructions and fingerprint update instructions; When the first instruction type is a basic information update instruction, the corresponding first instruction parameters include a first chemical formula, a first SMILES sequence and a first data source; When the first instruction type is a property update instruction, the corresponding first instruction parameters include a second chemical formula, a first property name, a first property attribute sequence, a first property version, and a second data source; the first property attribute sequence consists of one or more first attributes; the first attribute includes an attribute name and an attribute value; When the first instruction type is a fingerprint update instruction, the corresponding first instruction parameters include a third chemical formula, a first chemical element sequence, a first molecular fingerprint, a second molecular fingerprint, a third molecular fingerprint and a third data source; the first chemical element sequence is a chemical element sequence corresponding to the third chemical formula, and is composed of a plurality of first chemical elements; the first molecular fingerprint is a molecular fingerprint calculated based on a torsionbv algorithm, the second molecular fingerprint is a molecular fingerprint calculated based on an mfp2 algorithm, and the third molecular fingerprint is a molecular fingerprint calculated based on an ffp2 algorithm; The first queue includes a plurality of first queue messages; the first queue message includes the first user identifier, the first user token, the first database instruction, the first publishing time and the first processing status; the first processing status includes unprocessed and processed; The second queue includes a plurality of second queue messages; the second queue messages include the first chemical formula, the first SMILES sequence, the first data source, the first creator information, the second publishing time, and the second processing status; the second processing status includes unprocessed and processed; The third queue includes a plurality of third queue messages; the third queue messages include the second chemical formula, the first property name, the first property attribute sequence, the first property version, the second data source, the second creator information, the third publishing time and the third processing status; the third processing status includes unprocessed and processed; The fourth queue includes a plurality of fourth queue messages; the fourth queue messages include the third chemical formula, the first chemical element sequence, the first molecular fingerprint, the second molecular fingerprint, the third molecular fingerprint, the third data source, third creator information, a fourth publishing time and a fourth processing status; the fourth processing status includes unprocessed and processed; The first registered user database is implemented based on a type of relational database; the relational database includes at least an SQL database and a MySQL database; the first registered user data record includes at least a first user identification field, a first user token field, a first token validity period field and a first user information field; The first data source authorization database is implemented based on a type of relational database; the first data source authorization data record at least includes a first data source field and a first authorization validity period field.
3. The compound database processing system according to claim 2, characterized in that: The first access gateway is specifically configured to extract the corresponding first user identifier, the first user token and the first database instruction from the current first service application when performing message publishing processing on the first queue of the first queue module according to the current first service application; and taking the current time as the corresponding first publishing time; and setting a corresponding first processing state to unprocessed; A corresponding first queue message is composed of the obtained first user identifier, the first user token, the first database instruction, the first publishing time and the first processing status and is published to the first queue.
4. The compound database processing system according to claim 2, characterized in that: The first queue management module is specifically used for performing message publishing processing on the second, third and fourth queues according to the first queue, the first registered user database and the first data source authorization database. The total number of the first queue messages in the first queue whose first processing status is unprocessed is counted in real time, and the corresponding total number of currently unprocessed messages is refreshed based on the statistical result; and when the total number of currently unprocessed messages is greater than zero, the first queue message in the first queue with the earliest time and the first processing status being unprocessed is taken as the corresponding current queue message; and extracting the corresponding first user identifier, the first user token and the first database instruction from the current queue message as the corresponding current user identifier, the current user token and the current database instruction; and taking the first instruction type of the current database instruction as the corresponding current instruction type, and taking the first, second or third data source corresponding to the current instruction type in the current database instruction as the corresponding current data source; And use the current time as the corresponding current user time; And take the first registered user data record in the first registered user database whose first user identification field matches the current user identification, whose first user token field matches the current user token, and whose first token validity period field satisfies the current user time as the corresponding current registered user data record; and take the first data source authorization data record in the first data source authorization database whose first data source field matches the current data source and whose first authorization validity period field satisfies the current user time as the corresponding current data source authorization data record; and identify whether both the current registered user data record and the current data source authorization data record are both not empty, and if so, set the corresponding current verification result as verification passed, and otherwise set the corresponding current verification result as verification failed; And when the current verification result is verification passed, identifying the current instruction type; If the current instruction type is a basic information update instruction, extracting the corresponding first chemical formula, the first SMILES sequence and the first data source from the current database instruction; and extracting the first user information field of the currently registered user data record as the corresponding first creator information; and taking the current time as the corresponding second publishing time; and setting a corresponding second processing state to unprocessed; The first chemical formula, the first SMILES sequence, the first data source, the first creator information, the second publishing time and the second processing status obtained this time form a corresponding second queue message and publish it to the second queue; and the first processing status of the current queue message is set to processed; If the current instruction type is a property update instruction, extracting the corresponding second chemical formula, the first property name, the first property attribute sequence, the first property version and the second data source from the current database instruction; and extracting the first user information field of the currently registered user data record as the corresponding second creator information; and taking the current time as the corresponding third publishing time; and setting a corresponding third processing state to unprocessed; The second chemical formula, the first property name, the first property attribute sequence, the first property version, the second data source, the second creator information, the third publishing time and the third processing status obtained this time form a corresponding third queue message and publish it to the third queue; and the first processing status of the current queue message is set to processed; If the current instruction type is a fingerprint update instruction, extracting the corresponding third chemical formula, the first chemical element sequence, the first molecular fingerprint, the second molecular fingerprint, the third molecular fingerprint and the third data source from the current database instruction; and extracting the first user information field of the currently registered user data record as the corresponding third creator information; and taking the current time as the corresponding fourth publishing time; and setting a corresponding fourth processing state to unprocessed; The third chemical formula, the first chemical element sequence, the first molecular fingerprint, the second molecular fingerprint, the third molecular fingerprint, the third data source, the third creator information, the fourth publishing time and the fourth processing status obtained this time form a corresponding fourth queue message and publish it to the fourth queue; and the first processing status of the current queue message is set to processed.
5. The compound database processing system according to claim 2, characterized in that: The first queue management module is specifically used to regularly delete the first, second, third and fourth queue messages whose corresponding first, second, third and fourth processing status are processed in the first, second, third and fourth queues according to a preset first time frequency when regularly deleting processed messages in the first, second, third and fourth queues.
6. The compound database processing system according to claim 2, characterized in that: The first data updating module is specifically used for performing data updating processing on the compound basic information library, compound property library and compound fingerprint library of the first compound database according to the second queue. The total number of the second queue messages in the second queue whose second processing status is unprocessed is counted in real time, and the corresponding total number of currently unprocessed messages is refreshed based on the statistical result; and when the total number of currently unprocessed messages is greater than zero, the second queue message in the second queue with the earliest time and the second processing status being unprocessed is taken as the corresponding current queue message; and taking the first chemical formula, the first SMILES sequence, the first data source and the first creator information of the current queue message as the corresponding current chemical formula, current SMILES sequence, current data source and current creator; and taking the compound basic information record whose first chemical formula field in the compound basic information database matches the current chemical formula as the corresponding current basic information record; and identifying whether the current basic information record is empty; If the current basic information record is empty, then a new compound basic information record with all fields being empty is added to the compound basic information database as the corresponding current newly added basic information record; and a unique identifier is assigned to the current newly added basic information record as the corresponding current compound identifier; Adding an empty compound property table in the compound property library as the corresponding current newly added property table, and assigning a unique identifier to the current newly added property table as the corresponding current property table identifier; and adding a compound fingerprint record in which all fields are empty fields in the compound fingerprint library as the corresponding current newly added fingerprint record; And use the current time as the corresponding current creation time; and setting the first compound identification field, the first property table identification field, the first chemical formula field, the first SMILES sequence field, the first creator / updater field, the first creation time field, and the first data source field of the currently newly added basic information record to the corresponding current compound identification, the current property table identification, the current chemical formula, the current SMILES sequence, the current creator, the current creation time, and the current data source; and setting the third compound identification field, the second property table identification field, the third chemical formula field, the third SMILES sequence field, the second creator / updater field, the third creation time field and the third data source field of the current newly added fingerprint record to the corresponding current compound identification, the current property table identification, the current chemical formula, the current SMILES sequence, the current creator, the current creation time and the current data source; and setting the second processing status of the current queue message to processed; If the current basic information record is not empty, the current time is used as the corresponding current update time; And reset the first SMILES sequence field, the first creator / updater field, the first update time field and the first data source field of the current basic information record to the corresponding current SMILES sequence, the current creator, the current update time and the current data source; and set the second processing status of the current queue message to processed.
7. The compound database processing system according to claim 2, characterized in that: The second data updating module is specifically used for performing data updating processing on the compound property library according to the compound basic information library and the third queue. The total number of the third queue messages in the third queue whose third processing status is unprocessed is counted in real time, and the corresponding total number of currently unprocessed messages is refreshed based on the statistical result; and when the total number of currently unprocessed messages is greater than zero, the third queue message in the third queue with the earliest time and the third processing status being unprocessed is taken as the corresponding current queue message; and taking the second chemical formula, the first property name, the first property attribute sequence, the first property version, the second data source and the second creator information of the current queue message as the corresponding current chemical formula, current property name, current property attribute sequence, current property version, current data source and current creator; and taking the compound basic information record whose first chemical formula field in the compound basic information library matches the current chemical formula as the corresponding current basic information record; and taking the first compound identification field, the first property table identification field and the first SMILES sequence field of the current basic information record as the corresponding current compound identification, current property table identification and current SMILES sequence; and taking the compound property table corresponding to the current property table identifier in the compound property library as the corresponding current property table; When the current property table is not empty, a new property record of the compound with all fields being empty is added to the current property table as the corresponding current newly added property record; And use the current time as the corresponding current creation time; And set the second compound identification field, the second chemical formula field, the second SMILES sequence field, the first property name field, the first property attribute sequence field, the first property version field, the first creator field, the second creation time field and the second data source field of the current newly added property record to the corresponding current compound identification, the current chemical formula, the current SMILES sequence, the current property name, the current property attribute sequence, the current property version, the current creator, the current creation time and the current data source; And the third processing status of the current queue message is set to processed.
8. The compound database processing system according to claim 2, characterized in that: The third data updating module is specifically used for performing data updating processing on the compound fingerprint library according to the compound basic information library and the fourth queue. The total number of the fourth queue messages in the fourth queue whose fourth processing status is unprocessed is counted in real time, and the corresponding total number of currently unprocessed messages is refreshed based on the statistical result; and when the total number of currently unprocessed messages is greater than zero, the fourth queue message in the fourth queue with the earliest time and the fourth processing status being unprocessed is taken as the corresponding current queue message; and taking the third chemical formula, the first chemical element sequence, the first molecular fingerprint, the second molecular fingerprint, the third molecular fingerprint, the third data source, and the third creator information of the current queue message as the corresponding current chemical formula, current chemical element sequence, current molecular fingerprint A, current molecular fingerprint B, current molecular fingerprint C, current data source, and current updater; and taking the compound basic information record whose first chemical formula field in the compound basic information library matches the current chemical formula as the corresponding current basic information record; and taking the first compound identification field, the first property table identification field and the first SMILES sequence field of the current basic information record as the corresponding current compound identification, current property table identification and current SMILES sequence; and taking the compound fingerprint record whose third compound identification field in the compound fingerprint library matches the current compound identification as the corresponding current fingerprint record; and when the current fingerprint record is not empty, taking the current time as the corresponding current update time; and resetting the second property table identification field, the third chemical formula field, the third SMILES sequence field, the first chemical element sequence field, the first molecular fingerprint field, the second molecular fingerprint field, the third molecular fingerprint field, the second creator / updater field, the second update time field and the third data source field of the current fingerprint record to the corresponding current property table identification, the current chemical formula, the current SMILES sequence, the current chemical element sequence, the current molecular fingerprint A, the current molecular fingerprint B, the current molecular fingerprint C, the current updater, the current update time and the current data source; And the fourth processing status of the current queue message is set to processed.
Citation Information
Patent Citations
Molecular formula database expansion method
CN110400607A
Metabolite recognition system based on molecular fingerprint prediction and application method thereof
CN112735532A
Molecular structure database construction method and device, electronic equipment and storage medium
CN114464273A
Database access system of compound database
CN119669284A
Searching of chemical structures
US12020781B1