System for generating candidate ideas
The system addresses the limitations of biased interpolative search by using a VAE to generate diverse and high-quality candidate ideas, balancing novelty and quality through a multivariate space analysis.
Patent Information
- Application Number
- JP2021045937
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-19
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2041-03-19
AI Technical Summary
Current artificial intelligence technologies for generating candidate ideas rely heavily on interpolative search methods that are biased by initial conditions, leading to a lack of diversity and poor search efficiency, as they often suggest similar ideas to past trials without considering novelty or quality.
A system that utilizes a variational autoencoder (VAE) to convert idea vectors into reduced vectors in a multivariate space, maintaining similarity with initial vectors and quality indices, allowing for the generation of novel and high-quality candidate ideas through gradient-based search in the reduced space.
Enables the generation of candidate ideas that balance novelty and quality, improving the efficiency and diversity of idea generation by considering both past results and desired qualities.
Smart Images

Figure 0007594951000001 
Figure 0007594951000002 
Figure 0007594951000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a system for generating candidate ideas. [Background technology]
[0002] A mechanism that aims to improve system efficiency and intellectual productivity by quantifying and quantitatively analyzing observational data from the real world in cyberspace is called a cyber-physical system (CPS). In recent years, the maturation of sensing technologies such as the Internet of Things (IoT) and computing technologies capable of collecting and analyzing large amounts of data has made it possible to link cyberspace and physical space more closely.
[0003] In particular, system optimization using CPS is achieved by alternating between a phase of proposing improvements to the current situation and a phase of observing what actually happens when those improvements are implemented. By evaluating and feeding back the results of the observations, i.e. the quality of the ideas, the aim is to propose more effective improvement measures in the next proposal phase.
[0004] In recent years, a technique has been proposed that uses an interpolative search method using artificial intelligence technology in this planning phase to present ideas with a high probability of improvement as candidate measures in advance. For example, Bayesian optimization is a method for determining the next idea to be tried based on an expected value calculated from known information. Non-Patent Document 1 discloses a method using Bayesian optimization to search for candidate compounds to be next tested in material development. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2020-95452 A [Non-patent literature]
[0006] [Non-Patent Document 1] R. Griffiths et al., “Constrained Bayesian optimization for automatic chemical design using variational autoencoders,” Chemical Science, 11(2), 577-586 (2020) Summary of the Invention [Problem to be solved by the invention]
[0007] Current artificial intelligence technology mainly uses interpolative search methods based on known information. This is a search method that is highly dependent on initial values. In other words, if past trial conditions are biased, the next candidate ideas to be tried will be too similar to past trials, resulting in a lack of idea diversity.
[0008] For example, if an idea discovered through trial and error has a high effect on improving the current situation and has a high economic impact, the profits can be fully recovered even if some costs are incurred in trying out the idea. In such cases, it seems more efficient to try out ideas that have not been tried before, rather than using an interpolation search method. However, a simple random search does not take into account the results of past trials, so this also has the problem of low search efficiency.
[0009] Therefore, when searching for the next candidate ideas whose quality should be evaluated, a technique is desired that simultaneously considers both the novelty of the candidate ideas and the quality of the candidate ideas predicted based on known data. [Means for solving the problem]
[0010] One aspect of the present invention is a system for generating candidate ideas to be evaluated next, comprising one or more processors and one or more storage devices for storing programs executed by the one or more processors, wherein the one or more processors convert a first input idea vector representing an idea into a first reduced vector, generate one or more second reduced vectors in a multivariate space to which the first reduced vector belongs based on the value of a first predetermined component of the first reduced vector in the multivariate space, and generate one or more first output idea vectors representing candidate ideas to be presented to a user from each of the one or more second reduced vectors, wherein the multivariate space is configured to maintain a similarity between the input idea vector from which the reduced vector is generated and the output idea vector generated from the reduced vector, and a similarity between the first predetermined component of the reduced vector and the first predetermined index value. Effect of the Invention
[0011] According to one aspect of the present invention, candidate ideas can be generated to be evaluated in subsequent trials based on novelty and quality of the candidate ideas. [Brief description of the drawings]
[0012] [Figure 1] 1 illustrates an example of a logical configuration of a candidate idea suggestion system according to an embodiment of the present specification. [Diagram 2] 1 shows an example of the hardware configuration of a candidate idea generating device, a data storage device, and a user interface. [Diagram 3] 4 shows an example of parameter data included in the parameter database. [Figure 4] 1 shows an example of the structure of an idea expression database. [Diagram 5] 1 shows an example of the structure of an idea vector database. [Figure 6] 13 is a flowchart of an example process of an idea vectorization module. [Figure 7] An example of generating a structural formula vector, which is an idea vector, from a chemical structural formula, which is an idea expression, will be shown. [Figure 8] 1 shows an example of the configuration of a reduced vector database. [Figure 9] 13 is a flowchart illustrating an example of a process of an idea reduction module. [Figure 10] Here is an example of the configuration of a variational autoencoder. [Figure 11] 13 is a flowchart illustrating an example of processing of a reduced space search module. [Figure 12] 1 shows an example of the structure of a candidate idea database. [Figure 13] 13 is a flowchart of an example process of a candidate idea generation module. [Figure 14A] 13 shows an example of a user interface (screen) of an idea generation assistance application according to the second embodiment. [Figure 14B] 13 shows an example of a user interface (screen) of an idea generation assistance application according to the second embodiment. [Figure 15] An example of the evaluation result table is shown below. [Figure 16] An example of a word vector is shown below. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] In the following, when necessary for convenience, the description will be divided into multiple sections or examples, but unless otherwise specified, they are not unrelated to each other, and one is related to the other as a partial or complete modification, detail, supplementary explanation, etc. Furthermore, in the following, when the number of elements, etc. (including the number, numerical value, amount, range, etc.) is mentioned, it is not limited to the specific number, and may be more or less than the specific number, unless otherwise specified or clearly limited in principle to a specific number, etc.
[0014] This system may be a physical computer system (one or more physical computers) or a system built on a group of computing resources (multiple computing resources) such as a cloud infrastructure. The computer system or the group of computing resources includes one or more interface devices (including, for example, a communication device and an input / output device), one or more storage devices (including, for example, a memory (main memory) and an auxiliary storage device), and one or more processors.
[0015] When a function is realized by a program being executed by a processor, the defined processing is performed using a storage device and / or an interface device, etc. as appropriate, and therefore the function may be at least a part of the processor. Processing described with a function as the subject may be processing performed by a processor or a system having the processor. The program may be installed from a program source. The program source may be, for example, a program distribution computer or a computer-readable storage medium (for example, a computer-readable non-transitory storage medium). The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions. [Summary]
[0016] In the following, a technique for generating candidate ideas to be evaluated for quality in the next trial is disclosed. Ideas can be expressed in various expression formats. A chemical structural formula and experimental conditions in material development are one form of ideas to be experimented on next. The expression format of the chemical structural formula can be, for example, a character string written according to certain grammatical rules or a matrix. An example of the grammatical rules is SMILES (Simplified Molecular Input Line Entry System). The embodiment described below uses SMILES as an example of grammatical rules for describing a chemical structural formula.
[0017] A report (document) in R&D strategy planning is a form of idea that indicates the next R&D guidelines. The sentences contained in the document, or the vector representation obtained after morphological analysis of the sentences, are also forms of ideas.
[0018] 1 shows an example of a logical configuration of a candidate idea proposal system according to an embodiment of the present specification. The candidate idea proposal system includes a candidate idea generation device M01, a data storage device M02, and a user interface M05. These devices can communicate with each other via a network. . weather The candidate idea generating device M01 generates candidate ideas to be proposed to a user. The candidate idea generating device M01 includes an idea vectorizing module P01, an idea contracting module P02, a contracted space searching module P03, and a candidate idea generating module P04.
[0019] The data storage device M02 stores data referenced or generated by the candidate idea generation device M01. In the configuration example of Fig. 1, the data storage device M02 stores various databases. Specifically, an idea expression database DB01, an idea vector database DB02, a contracted vector database DB03, a candidate idea database DB04, a model database DB05, and a parameter database DB06 are stored. The data storage device M02 further manages the databases DB01 to DB06. It includes a data management module P06.
[0020] The user interface M05 allows the user to input data and presents information to the user. The user interface M05 includes a display unit P07 and an input reception unit P08. The display unit P07 presents, for example, information generated by the candidate idea generation device M01 to the user. The input reception unit P08 displays, for example, information generated by the candidate idea generation device M01 when the candidate idea generation device M01 is performing processing. conduct The information required for this is received from the user.
[0021] The idea vectorization module P01 of the candidate idea generating device M01 converts the idea expression expressed in a predetermined format into a real vector, that is, an idea vector, according to a predetermined method.
[0022] The idea reduction module P02 converts the idea vector from the idea vectorization module P01 into another real vector, that is, a reduced vector. The idea reduction module P02 converts the similarity of one or more index values set in advance by the user so that it corresponds to a distance in a multivariate space formed by the reduced vector. For example, the idea reduction module P02 converts the multivariate space formed by the reduced vector so that the similarity of the input idea vector and the similarity of the index value of the quality of the idea are correlated.
[0023] The idea reduction module P02 may include, for example, one variational autoencoder (VAE). A VAE is a type of autoencoder, and is a deep generative model consisting of two neural networks: an encoder and a decoder. The encoder converts an input (vector) into a real vector. The space to which the real vector belongs is called a latent space, and is assumed to follow a predetermined distribution, for example, a normal distribution. The decoder inversely converts the real vector and outputs a vector of the same dimension as the input.
[0024] A typical encoder and decoder are trained (learned) so that the input and output are equal. Being able to reconstruct the input from the intermediate output real vector means that sufficient features of the input are reflected in the real vector. The dimension of the latent space is set to be smaller than the dimension of the input. Therefore, the encoder can extract the features of the input and compress the dimension of the input.
[0025] The intermediate output vector is called a latent variable or latent vector, and is an abstract representation of features extracted from the input idea vector. Therefore, the latent vector of the VAE is a contracted vector. The latent vector is assumed to follow a predetermined distribution, for example, a Gaussian distribution. Therefore, the decoder can restore the input vector with high accuracy, for example, when it receives a vector to which noise has been added. In this way, the VAE has high robustness as a generative model.
[0026] In the learning (training) process in which the VAE constructs a latent space, the parameters of the neural network are sequentially optimized so that the output value (loss) of the loss function is reduced. In general, the neural network is trained to reduce the error between the input and the output, that is, the reconstruction error. In one embodiment of this specification, a loss function is defined in which another term is added to the reconstruction error.
[0027] First, a term indicating the similarity between one or more predetermined components in the latent vector and a predetermined index value, for example, an index value representing the quality of an idea, is included in the loss function. The term indicating the similarity may be, for example, a term indicating correlation or a term indicating Euclidean distance from a certain data. For example, a term that reduces the loss as the value indicating correlation increases is added to the loss function. The latent space of a VAE trained using this loss function is a multivariate space in which both the similarity between the input idea vector and the corresponding output vector and the similarity between the predetermined components in the latent vector and the predetermined index value are maintained.
[0028] In addition, by adding a term to the loss function according to the search purpose of the candidate ideas, it is possible to have the VAE construct a latent space that reflects the desired similarity. For example, if the index value (true value) of the actual quality of candidate ideas proposed by this system in the past is known, the error between the quality of the idea predicted at the time of proposal (predicted value) and the true value of the index value can be added to the loss function. This makes it possible to construct a latent space that reflects the similarity between the predicted value and the true value, making it possible to avoid areas where predictions are likely to be significantly off.
[0029] Furthermore, instead of true values obtained through experiments, predicted values determined by a machine learning system trained on a different data set, or theoretical values calculated by theoretical calculations based on physics or chemistry may be used.
[0030] The contracted space search module P03 can find a more desirable contracted vector by searching the vicinity of a target value, for example, a contracted vector corresponding to a known idea of the highest quality at the time of search, based on a gradient method in a multivariate space formed by contracted vectors. As described above, a predetermined component in a latent vector is correlated with an index value of the quality of an idea, so that a search based on a gradient method along that axis can be realized by changing the predetermined component in a predetermined manner. For example, a predetermined value is added or subtracted from the predetermined component.
[0031] The candidate idea generation module P04 receives the reduced vectors found by the reduced space search module P03 and converts them into an output format previously set by the user. For example, the reduced vectors are first converted back into idea vectors and then into idea expressions. This allows the reduced vectors to be restored to the same expression format as the input idea expressions, thereby converting them into an expression format that the user can understand.
[0032] An embodiment of the present specification mainly performs the following steps: First, the system receives user settings and data and executes the idea vectorization module P01. Then, the system trains the VAE that constitutes the idea reduction module P02. After the VAE training is completed, the system generates candidate ideas to be evaluated for quality next by the reduced space search module P03 and the subsequent candidate idea generation module P04. The generated candidates are presented to the user via the user interface M05.
[0033] Training of the VAE must be performed first when the system is put into operation. For subsequent searches, the training step may be omitted. Also, when new trial results of an idea are input, additional training may be performed using the data.
[0034] By operating this system, it is possible to obtain the true value of the index value of the quality of ideas proposed in the past. At this time, the user can input the true value to this system via the user interface M05. The data management module P06 has a function of searching for stored ideas and a function of accepting and storing the input of the actual index value of the quality of the candidate ideas. FIG. 2 shows an example of the hardware configuration of the candidate idea generation device M01, the data storage device M02, and the user interface M05. The candidate idea generation device M01 includes a processor U111 having computing performance, and a DRAM U112 that provides a volatile temporary storage area for storing the programs and data executed by the processor U111.
[0035] The candidate idea generation device M01 further includes a communication device U113 that performs data communication with other devices including other devices in this system, and an auxiliary storage device U114 that provides a permanent information storage area using a HDD (Hard Disk Drive), flash memory, etc. The candidate idea generation device M01 also includes an input device U115 that accepts operations from a user, and a monitor U116 (an example of an output device) that presents the output results of each process to the user.
[0036] For example, the candidate idea generating device M01 stores programs such as an idea vectorization module P01, an idea contraction module P02, a contracted space search module P03, and a candidate idea generating module P04. The programs executed by the processor U111 and data to be processed are loaded from the auxiliary storage device U114 to the DRAM U112.
[0037] The hardware elements constituting the data storage device M02 may be the same as those of the candidate idea generation device M01. For example, the data storage device M02 stores programs such as the various databases DB01 to DB06 and the data management module P06 shown in Fig. 1, and executes the data management module P06. The hardware elements constituting the user interface M05 may be the same as those of the candidate idea generation device M01. For example, the user interface M05 executes the display unit P07 and the input receiving unit P08.
[0038] In addition, the functions that are divided among the multiple devices may be integrated into one device, or the functions of the multiple devices may be distributed among even more devices. Thus, the candidate idea proposal system includes one or more storage devices and one or more processors. EXAMPLES
[0039] In the first embodiment, the structural formula of the candidate compound to be next experimented is searched and presented. The user can set the parameter database DB06 via the input device of any device. The parameter database DB06 stores data required for the processing of the candidate idea generating device M01.
[0040] The parameter database DB06 may include, for example, information that the idea vectorization module P01 refers to in order to generate an idea vector. As an example, the parameter database DB06 may include vocabulary information for converting a chemical structural formula expressed in SMILES notation into a matrix. The parameter database DB06 may also include configuration information of the idea reduction module P02. Specifically, the parameter database DB06 describes the structure and learning parameters of the idea reduction module P02.
[0041] FIG. 3 shows an example of parameter data included in the parameter database DB06. For example, "TableNames" in section 61 indicates a table that stores vocabulary information for converting a chemical structural formula expressed in SMILES notation into a matrix. "TargetProperties" in section 62 indicates parameters for characteristic values (target variables) that indicate the quality of a compound. Specifically, characteristic values for mwt and lopP are shown. These characteristic values are referenced in the learning of the idea reduction module P02 and in the search for reduced vectors by the reduced space search module P03.
[0042] "VAEInitialParams" in section 63 indicates the parameters of the network configuration of the idea reduction module P02. In this example, the idea reduction module P02 includes a VAE, so the configuration parameters of the VAE are shown. For example, "CorrDimIndex" specifies which dimension is to be correlated, and "CorrDir" specifies whether the bias should be positive or negative.
[0043] Section 64 "SearchParams" indicates parameters referenced in the search for condensed vectors by the condensed space search module P03. In this example, conditions for filtering candidate condensed vectors to be presented are shown.
[0044] The idea expression database DB01 will be explained. A user can store data in the idea expression database DB01 via the user interface M05. The idea expression database DB01 includes information on past experiment conditions and results. The idea expression database DB01 stores ideas that have been tried so far. What ideas have been tried and what KPIs have been obtained are stored in correspondence with each other. For example, in material development, this applies to experimental data.
[0045] FIG. 4 shows an example of the configuration of the idea expression database DB01. In the example shown in FIG. 4, the ID column T1C1 indicates the identifier of a record. The SuggestID column T1C21 indicates the identifier of a candidate idea previously proposed by this system. This column T1C21 is linked to the SuggestID column T4C1 of the candidate idea database DB04. In the SuggestID column T1C21, for experiments that have not been presented in the past, a blank column or the symbol "-" indicating a blank column is entered. In this way, the idea expression database DB01 stores both previously proposed materials (experimental results) and unproposed materials (experimental results).
[0046] The TimeStamp column T1C2 indicates the date the experiment was performed. The mwt column T1C3 indicates the molecular weight. In this example, the molecular weight is one of the objective variables to be improved, and a larger value is desirable. The logP column T1C4 indicates the water partition coefficient. In this example, logP is one of the objective variables to be improved, and a smaller value is desirable. The SMILES column T1C5 indicates the chemical structure expressed in SMILES notation. The catalyst column T1C6 indicates the catalyst, and is one of the experimental conditions. The temp column T1C7 indicates the synthesis temperature, and is one of the experimental conditions.
[0047] The idea expression database DB01 includes columns of estimated values of some of the properties of chemical compounds. As described below, the VAE of the idea reduction module P02 takes as input the property values of chemical compounds and outputs predicted values of those property values.
[0048] Specifically, the pred_mwt column T1C8 indicates the predicted mwt value of candidate ideas previously proposed by the system. "pred_logP" is the predicted logP value of candidate ideas previously proposed by the system. The "pred_catalyst" column T1C10 indicates the predicted catalyst value when the system previously proposed the idea as a candidate. The "pred_temp" column T1C11 indicates the predicted temp value of candidate ideas previously proposed by the system. In columns T1C8 to T1C11 including pred_, for experiments not previously presented, a blank or the symbol "-" indicating a blank is entered.
[0049] As described above, the configuration example shown in Fig. 4 describes identification information such as ID and TimeStamp (experiment date), structural formula information of organic compounds called SMILES, process condition information such as catalyst and temp, and KPIs (objective variables) such as mwt and logP. One or more KPIs (objective variables) can be set. By setting multiple KPIs, candidate ideas based on those KPIs can be proposed.
[0050] Next, the process of the idea vectorization module P01 will be described. The idea vectorization module P01 generates an idea vector database DB02 from an idea expression database DB01.
[0051] 5 shows an example of the configuration of the idea vector database DB02. The idea vector database DB02 includes an idea_vector column T2C5 in addition to the data of the idea expression database DB01. The idea_vector column T2C5 shows vectors of idea expressions, in this example, vectors of chemical structural formulas expressed in SMILES notation.
[0052] 6 is a flowchart of a process example of the idea vectorization module P01. The idea vectorization module P01 reads parameter data from the parameter database DB06. Then, it reads idea expression data from the idea expression database DB01 (S101).
[0053] Next, the idea vectorization module P01 converts the chemical structural formula expressed in SMILES notation into a vector (matrix) using the vocabulary information described in the parameter data. Note that matrices and vectors are examples of methods for expressing arrays, and the same array can be expressed as a vector and a matrix. This conversion method can use, for example, a method widely known as one-hot encoding. According to this method, chemical structural formulas and vectors have a one-to-one correspondence, making inverse conversion possible.
[0054] Next, the idea vectorization module P01 duplicates the idea expression database DB01, adds an idea_vector column T2C5, stores the corresponding converted vectors, and writes it out as an idea vector database DB02.
[0055] Figure 7 shows an example of generating a structural formula vector, which is an idea vector, from a chemical structural formula, which is an idea representation. The idea vectorization module P01 converts a chemical structural formula expressed in SMILES notation into a vector by one-hot encoding.
[0056] Next, the process of the idea reduction module P02 will be described. The idea reduction module P02 generates a reduced vector database DB03 from the idea vector database DB02.
[0057] 8 shows an example of the configuration of the contracted vector database DB03. The contracted vector database DB03 includes an abst_vector column T3C5 in addition to the data of the idea vector database DB02. The abst_vector column T3C5 stores contracted vectors of idea vectors. In this example, the idea vector is a vector that represents a chemical structural formula expressed in SMILES notation.
[0058] 9 is a flowchart showing an example of processing of the idea reduction module P02. The idea reduction module P02 first reads parameter data from the parameter database DB06 (S151). Furthermore, the idea reduction module P02 reads model data from the model database DB05 (S152). Furthermore, the idea reduction module P02 reads idea vector data from the idea vector database DB02 (S153).
[0059] Next, the idea reduction module P02 instantiates the VAE model (S154). If a trained model is saved, the trained model is instantiated. If a trained VAE is not saved, the idea reduction module P02 initializes the (neural network) VAE using the initial values described in the parameter data.
[0060] FIG. 10 shows an example of the configuration of a VAE. The encoder of the VAE can be composed of, for example, an outer encoder 211 and an inner encoder 212 at the subsequent stage. The outer encoder 211 can be composed of multiple one-dimensional convolution layers, and the inner encoder 212 can be composed of multiple fully connected layers. The encoder receives, as input data 201, a vector 202 of a chemical structure formula that can be expressed by an M×N dimensional matrix, and a vector 203 in which explanatory variables other than SMILES are concatenated. The encoder of the VAE converts the input data 201 into an L×1 dimensional vector 214 (hereinafter, simply referred to as an L-dimensional vector). This vector 214 is a contracted vector in the latent space 213.
[0061] The encoder generates a vector V1 by concatenating the output vector of the final layer of the outer encoder 211 (convolutional layer) with a vector that concatenates explanatory variables other than SMILES contained in the idea vector database DB02. The encoder inputs the vector V1 to the inner encoder 212 (fully connected layer) following the final layer of the convolutional layer to generate an L-dimensional vector 214.
[0062] In this example, the KPIs mwt and logP and the experimental conditions catalyst and temp are included in vector 203 (vector V1) and input to the inner encoder 212 of the encoder. In this way, it is possible to encode the property values and the experimental conditions as well. In other configuration examples, if the compound property estimates and the like are not required, explanatory variables other than SMILES do not need to be included in the input data.
[0063] The decoder can be composed of, for example, an inner decoder 215 and an outer decoder 216 at the subsequent stage. The inner decoder 215 can be composed of a plurality of fully connected layers, and the outer decoder 216 can be composed of an RNN (Recurrent Neural Network). The decoder receives an L-dimensional vector 214 as an input, and performs inverse conversion into a vector 222 representing a chemical structural formula corresponding to an M×N-dimensional matrix, and a vector 223 consisting of explanatory variables other than the chemical structural formula. The output chemical structural formula 222 can be slightly different from the input chemical structural formula. In addition to the chemical structural formula 222, the output data 221 also includes predicted values 223 of explanatory variables other than the input chemical structural formula.
[0064] Because the encoder also encodes explanatory variables other than the SMILES transformed matrix, the output vector V2 of the final fully connected layer is split. This allows the restored explanatory variables, such as mwt, logP, catalyst, and temp (predicted values), to be obtained. The connection positions of V1 and the split positions of V2 are the same.
[0065] One of the divided parts is input to the outer decoder 216, which outputs a vector (matrix) 222 indicating the chemical structure. The other part is output as a predicted value 223 of an input variable other than the chemical structure. Note that Fig. 10 shows an example of a model for the idea contraction module P02 to form a contracted vector, and any model can be used depending on the design.
[0066] Returning to FIG. 9, the steps following step S155 are a recursive process (learning process) based on batch processing commonly used in deep learning. In other words, the number of pieces of learning data specified by the batch size are processed at a time. The time until all batches are processed once is called one epoch, and actual learning is repeated for multiple epochs. The process performed in one batch will be described below.
[0067] First, the idea reduction module P02 inputs the idea vectors to be trained into the VAE for the number of batches (S155). The reconstruction error U0 is calculated from the error between the encoder input and the decoder output (S156). The calculation method for the reconstruction error U0 is arbitrary, and may be, for example, the average value of the error between the VAE input and output for each sample included in the batch. As in this example, when the VAE is trained not only on SMILES but also on characteristic values (mwt, logP) and experimental conditions (catalyst, temp), the reconstruction error including these parts is calculated.
[0068] Next, the idea contraction module P02 calculates a correlation coefficient between a predetermined component in the idea contraction vector and the objective variable (KPI) (S157). For example, the idea contraction module P02 calculates a correlation coefficient R0 between a predetermined first component of the encoder output vector (contracted vector) and mwt. Next, the idea contraction module P02 calculates a correlation coefficient R1 between a predetermined second component of the encoder output vector (contracted vector) and logP.
[0069] The correlation between which objective variable and which component is to be calculated is specified by "CorrDimIndex" in the parameter data. The method of calculating the correlation is arbitrary. For example, a kernel function that calculates the correlation may be defined, a correlation coefficient calculated based on the covariance and the average may be used, or the sum of products of the correlation coefficient and weights calculated by a different method may be used.
[0070] Next, the idea reduction module P02 calculates the error (difference) between the true value and the predicted value of the characteristic value. The true value is the input to the VAE, and the predicted value is the output. For example, the idea reduction module P02 calculates the difference between the value of the mwt column T1C3 and the value of the pred_mwt column T1C8 of the idea vector database DB02, and the difference between the value of the logP column T1C4 and the value of the pred_logP column T1C9, and calculates the sum W0 of them (S158).
[0071] The sum W0 is, for example, the sum of the differences in the characteristic values of multiple samples. The sum of the differences in the values of different characteristics may be a simple sum, or may be a sum after multiplying by a weight specified by the parameter data. Note that the calculation of the error W0 of the characteristic values (mwt, logP) may be omitted. In this configuration, the input data and output data of the VAE may not include the characteristic values (KPI). The experimental conditions can be omitted because the error is not included in the loss function.
[0072] Next, the idea reduction module P02 calculates the loss from the sum of the reconstruction error U0, the correlation coefficient R0, the correlation coefficient R1, and the difference W0 between the true value and the predicted value of the objective variable. The neural network learns to reduce the loss, so the signs are set appropriately. For example, the signs of R0 and R1 are set in advance by the parameter data "CorrDir".
[0073] The idea contraction module P02 updates the VAE parameters to optimize them based on the loss (S159). In this example, the parameters of the VAE are updated so that the reconstruction error U0 and the characteristic value error W0 become smaller and the correlations R1 and R2 become larger. The sign of the variable in the loss function is determined by design, and for example, the sign may be determined so that the loss function becomes smaller when the correlation becomes smaller.
[0074] Next, the idea contraction module P02 determines whether the learning is completed (S160). In this example, the end flag is set when the learning process is completed for a predetermined number of epochs. If not (S160: NO), the process returns to step S155 and continues.
[0075] After the learning process is completed (S160: YES), the idea reduction module P02 writes and stores the parameters of the learned model in the model database DB05 (S161).
[0076] Next, the idea reduction module P02 inputs all idea vectors again to the trained VAE and converts them into reduced vectors by obtaining the output of the encoder (S162).
[0077] Next, the idea contraction module P02 adds an abst_vector column T3C5 to the configuration of the idea vector database DB02, stores the contracted vector, and writes out this table as a contracted vector database DB03 (S163).
[0078] Next, the process of the contracted space search module P03 will be described. Fig. 11 is a flowchart showing an example of the process of the contracted space search module P03. The contracted space search module P03 first reads parameter data from the parameter database DB06 (S201). Next, the contracted space search module P03 reads the contracted vector database DB03 (S202).
[0079] Next, the contracted space search module P03 extracts a contracted vector that is the search center from the contracted vector database DB03 based on the search conditions described in the parameter data (S203). In the example shown in Fig. 3, the target value of each objective variable is described in "TargetValue" in section 62 of the parameter data.
[0080] Therefore, in the contracted vector database DB03, samples (records) with mwt close to 1.6 and logP close to -1.5 are extracted. The predetermined range indicating the extent of closeness to be extracted is specified by "FilteringRange" in the parameter data. The number to be extracted is arbitrary and may be specified by the parameter data. Records with at least one of mwt and logP close to the target value may be extracted, or records with a criterion different from the target value, for example, specified by the user, may be extracted.
[0081] Next, the contracted space search module P03 generates a new contracted vector by adding a small real number as a predetermined bias to the first and second components of the extracted contracted vector (S204). The value of the objective variable of the new contracted vector is slightly different from the value of the objective variable of the original contracted vector. The number of bias sets consisting of biases of one or more components given to each contracted vector may be one or more than one. By adding a different bias set to each contracted vector, multiple new contracted vectors can be generated from each contracted vector.
[0082] The bias added to each component may be a preset value, and one or different biases may be added to each extracted condensed vector. The set of biases added may be common or different for different components. The extracted condensed vectors and the newly generated condensed vectors based on the biases are candidates for compounds (ideas) to be presented to the user.
[0083] Next, the contracted space search module P03 extracts all or some of the records in which a valid value is stored in the SuggestID column T1C21 of the contracted vector database DB03. These are records that have been previously suggested.
[0084] The contracted space search module P03 calculates a similarity between each contracted vector of the extracted or generated contracted vectors and each contracted vector of the extracted proposed record. The similarity is expressed, for example, as a Euclidean distance. In one example, the values of components 0 and 1 are excluded from the similarity calculation. These values may be included in the similarity calculation. Furthermore, the contracted space search module P03 uses the calculated similarity to filter the extracted contracted vector and the newly generated contracted vector based on the target value (S205).
[0085] In this example, the novelty of the candidate ideas is emphasized, and filtering is performed so that reduced vectors that are more than a certain threshold away from all previously proposed ideas remain. The threshold is specified by "threshold" in the parameter data. This allows the selection of compounds (ideas) whose objective variables (KPIs) are close to the target value and are far from previously proposed compounds (ideas).
[0086] Next, the contracted space search module P03 refers to the TimeStamp column T1C2 of the contracted vector database DB03, and extracts records within a certain period from the current date and time.
[0087] The contracted space search module P03 calculates the similarity between the contracted vector included in each extracted record and each candidate contracted vector selected by the above filtering. The similarity can be expressed, for example, by Euclidean distance as described above. The calculation of the similarity may be the same or different between the presented idea and the recent idea. The contracted space search module P03 applies filtering using the similarity to select a part from the candidate contracted vectors (S206).
[0088] In this example, emphasis is placed on the chronological novelty of the candidate ideas, and filtering is performed so that candidate contraction vectors that are more than a threshold away from all records within a certain period from the current date and time remain. The threshold can be specified by "threshold_Time" in the parameter data, and is set to 30 days in this example. Note that filtering may be performed only for the similarity to previously presented ideas or the similarity to recent ideas, or these filtering may be performed independently.
[0089] Next, the contracted space search module P03 passes the candidate contracted vectors remaining after filtering to the candidate idea generation module P04 (S207).
[0090] The process of the candidate idea generation module P04 will be described below. The candidate idea generation module P04 generates candidate ideas and stores them in the candidate idea database DB04.
[0091] Fig. 12 shows an example of the configuration of the candidate idea database DB04. In the example shown in Fig. 12, the SuggestID column T4C1 shows the ID of the proposed idea. The SuggestDate column T4C2 shows the date of suggestion. The pred_mwt column T4C3 shows the predicted value of mwt of the candidate idea. The pred_logP column T4C4 shows the predicted value of logP of the candidate ideas previously proposed by the system.
[0092] The SMILES column T4C5 shows the SMILES notation of candidate compounds (candidate ideas) previously proposed by the system as candidate ideas. The pred_catalyst column T4C6 shows the predicted value of catalyst when previously proposed by the system as a candidate idea. The pred_temp column T4C7 shows the predicted value of temp of candidate ideas previously proposed by the system. Columns T41C to T4C2 are each associated with the columns of the same name in the idea expression database DB01, and the values of the same proposals are the same.
[0093] 13 is a flowchart of a process example of the candidate idea generation module P04. The candidate idea generation module P04 first reads parameter data (S251), and then receives a contracted vector, which is an abstract representation of a candidate idea, from the contracted space search module P03 (S252).
[0094] Next, the candidate idea generation module P04 reads the trained model from the model database DB05 (S253). The model read here is the same as the VAE used in the idea reduction module P02.
[0095] Next, the candidate idea generation module P04 inputs the reduced vector from the reduced space search module P03 into the decoder of the trained VAE to obtain a matrix that is an idea vector (S254). In this example, the idea reduction module P02 encodes the characteristics mwt and logP and the experimental conditions catalyst and temp, which are explanatory variables other than SMILES. Therefore, the candidate idea generation module P04 obtains the outputs of mwt and logP, and catalyst and temp by dividing the intermediate output vector V2. If only SMILES is encoded, this process is not performed.
[0096] Next, the candidate idea generation module P04 converts the inversely converted idea vector into an idea expression according to a predetermined method (S255). In this example, the decoded matrix is one-hot decoded to obtain a chemical structure formula in SMILES notation, which is an idea expression.
[0097] Next, the candidate idea generation module P04 compiles the decoded SMILES, characteristic values, and experimental conditions, and stores them as a new record in the candidate idea database DB04. At this time, it adds a unique identifier to the SuggestID column T4T1C, and the execution date and time to the SuggestDate column T4C2.
[0098] The candidate idea generating module P04 displays the updated data added to the candidate idea database DB04 via the display unit P07 of the user interface M05, and presents candidate ideas to the user. EXAMPLES
[0099] In the following, an embodiment of a candidate idea generation system based on document analysis technology will be disclosed as Example 2. Various documents such as patent documents, academic papers, and internal company reports, as well as sentences that are components of each document, are also forms of expressing ideas. This system can generate and present new candidate ideas based on these known ideas, thereby supporting the user's idea generation.
[0100] For example, consider a system that helps people organize their ideas. The process by which people organize their ideas can be seen as the process of verbalizing abstract images. Furthermore, an idea that has been successfully verbalized in a way that anyone can understand can be considered an organized idea. In other words, consider the following system that appropriately supports the selection of words to explain the implicit ideas that users have in their minds.
[0101] 14A and 14B show examples of a user interface (screen) of an idea generation assistance application according to Example 2. After a user operation on a screen H10A shown in Fig. 14A, a user operation on a screen H10B shown in Fig. 14B is executed.
[0102] In the screen H10A, the object H13 shows the keywords "in your possession." In the object H13, four kinds of keywords are arranged as flashcards. These are the most likely words currently used to explain the user's ideas.
[0103] The system presents new words (word cards) in the "New Words" object H12. The user first evaluates whether the current cards are sufficient to verbalize the idea, and inputs the result into the system by selecting the button indicated by "Explanation of current cards" H11, for example, on a 5-point scale.
[0104] The user then compares the handheld vocabulary with the new vocabulary, selects the vocabulary card that is least relevant to his / her idea, and discards the selected vocabulary card by pressing the "Next" button H14. The system replaces the vocabulary cards as necessary, and then displays the next screen H10B shown in FIG. 14B.
[0105] The system displays the new flashcard in the "New Word" object H12 on the screen H10B shown in FIG. 14B. Object H13 shows the "Your Inventory" keyword. As with the operation described with reference to FIG. 14A, the user evaluates whether the current in-hand cards are sufficient for verbalizing the idea, and inputs the evaluation into the system by selecting the button indicated by "Explanation of Current In-Hand Cards" H11 on a 5-point scale. The user compares the in-hand words with the new words, selects the flashcard that is least relevant to his / her idea, and presses the "Next" button H14 to discard the selected flashcard. The system replaces the flashcards as necessary, and then displays the next screen (not shown).
[0106] The on-hand word group and the user's evaluation of the word group are recorded for each step and stored in an evaluation result table. Figure 15 shows an example of the configuration of the evaluation result table T60. The step column T6C1 shows the identifier of the step of the user's evaluation.
[0107] The word column T6C2 shows the hand-held word group for each evaluation step. The vector representing this word group is an example of a word vector. This word vector uses 0 and 1 to indicate whether a word is included in the hand-held word group for the evaluation step.
[0108] The evaluation index column T6C3 shows the evaluation result of the evaluation step. The evaluation index column T6C3 is composed of a difference evaluation column T6C5 and an absolute evaluation column T6C6. The user evaluation of each evaluation step entered on the evaluation screen is stored in the absolute evaluation column T6C6. The difference between the absolute evaluation of the word vector of the previous step and the absolute evaluation of the word vector of the next step is stored in the difference evaluation column T6C5.
[0109] By repeating the evaluation of the word groups described with reference to Figures 14A and 14B, the user can have a collection of word cards suitable for explaining his / her ideas, and can further verbalize his / her ideas.
[0110] In order to realize the above operation, the idea vectorization module P01 in the second embodiment receives an idea expression with a single sentence or multiple sentences as one unit and converts it into an idea vector, which is a real vector. Methods for converting such natural language expressions into real vectors are widely known, for example, Word2Vec and Doc2Vec.
[0111] Alternatively, a method is known in which highly related vocabulary is extracted based on the frequency of occurrence or co-occurrence probability of words appearing in a document, and then converted into a real vector as a feature. Idea vectors like these that are directly linked to words are called word vectors. Figure 16 shows an example of a word vector T70. The document to be converted is one record (one word vector), and the one word vector uses 0 and 1 to indicate whether a word is included in the document.
[0112] In the initial operation stage, the system searches for the closest word group, relying only on the similarity (distance) between word vectors, and determines the next new word. For example, the three records in Figure 16 are word vector groups with only the component values differing. The currently available word group is the first record, and the system searches for similar word vectors and extracts the second and third records. The system randomly selects the third record from among them, and presents the difference, "resolution," as the next new word object H12.
[0113] When the system is operated a certain number of times, pairs of word vectors and user evaluations, i.e., many records in FIG. 15 are accumulated. The evaluation result table T60 in FIG. 15 is an example of the idea vector database DB02. The idea reduction module P02 in the second embodiment may use a VAE, as in the first embodiment. The idea reduction module P02 takes word vectors as input, and trains a network using a loss function that strengthens the correlation between a predetermined component of a reduced vector obtained by an encoder and a user evaluation value.
[0114] Then, similar to the first embodiment, the contracted space search module P03 can search for useful word vectors that contribute to the organization of ideas. Since new word vectors can be obtained by the candidate idea generation module, it is possible to support the organization of thoughts according to the user's habits and interests.
[0115] Instead of user evaluation, any index value related to the quality of the idea may be used, such as an evaluation value for the document used as the basis for generating the word vector, or evaluations from other users.
[0116] Furthermore, it is possible to integrate idea vector databases from multiple users. By incorporating other people's thought processes as information and having the idea reduction module learn from them, it becomes possible to transition from one thought to another inspired by other people's ideas, that is, to support a change in thinking.
[0117] The present invention is not limited to the above-mentioned embodiment, but includes various modified examples. For example, the above-mentioned embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those including all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.
[0118] In addition, the above-mentioned configurations, functions, processing units, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card or SD card.
[0119] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]
[0120] 201 Input Data 211 Outer Encoder 212 Inner Encoder 213 Latent space 214 Latent Vector 215 Inner Decoder 216 Outer Decoder 221 Output Data DB01 Idea Expression Database DB02 Idea Vector Database DB03 Reduced Vector Database DB04 Candidate Idea Database DB05 Model Database DB06 Parameter Database M01 Candidate idea generator M02 Data storage device M05 User Interface P01 Idea Vectorization Module P02 Idea Reduction Module P03 Reduced space search module P04 Candidate idea generation module P06 Data Management Module P07 Display section P08 Input reception section U111 Processor U113 Communication equipment U114 Auxiliary storage device U115 Input device U116 Monitor
Claims
1. A system for generating candidate ideas, comprising: one or more processors; one or more storage devices for storing programs executed by the one or more processors; The one or more processors: Transforming the first input idea vector into a first condensed vector belonging to a multivariate space using the trained model; extracting one or more second condensed vectors from the multivariate space based on values of a first predetermined component in the multivariate space of the first condensed vector; outputting one or more first output idea vectors representing candidate ideas based on each of the one or more second reduced vectors using the trained model; The system, wherein the trained model has been trained using an objective function including a term representing the similarity between an input idea vector and an output idea vector of a contracted vector, and a term representing the similarity between the first specified component and a first specified index value.
2. 2. The system of claim 1, the one or more processors extract the one or more second condensed vectors based on the first and second predetermined components of the first condensed vector; The system, wherein the objective function further includes a term representing a similarity between the second predetermined component and a second predetermined index value.
3. 2. The system of claim 1, the similarity between the first predetermined component and the first predetermined index value indicates a correlation between the first predetermined component and the first predetermined index value, The system, wherein the similarity between the input idea vector and the output idea vector is expressed as a Euclidean distance.
4. 2. The system of claim 1, The one or more processors may apply one or more predetermined biases to the first predetermined component of the first condensed vector in extracting the one or more second condensed vectors.
5. 2. The system of claim 1, The one or more processors select ideas from a plurality of ideas that have a difference between the first predetermined indicator value and a target value within a predetermined range, and generate the first input idea vector from the selected ideas.
6. The system of claim 1, the one or more second condensed vectors are a plurality of second condensed vectors; In extracting the plurality of second condensed vectors, the one or more processors apply a different bias to the first predetermined component of the first condensed vector for each of the plurality of second condensed vectors.
7. The system of claim 1, The one or more storage devices manage proposed ideas; the one or more second condensed vectors are a plurality of second condensed vectors; The one or more processors select a second condensed vector of a candidate idea to suggest to a user based on a similarity between the condensed vector in the multivariate space of the proposed idea and the plurality of second condensed vectors.
8. The system of claim 1, The idea includes at least one of a material or a phrase; The one or more processors convert the one or more first output idea vectors into a representation representing the ideas for display.
9. A method for generating candidate ideas, comprising: the system converts a first input idea vector into a first condensed vector in a multivariate space using a trained model; the system extracts one or more second condensed vectors from the multivariate space based on values of a first predetermined component in the multivariate space of the first condensed vector; the system outputs one or more first output idea vectors representing candidate ideas based on the one or more second reduced vectors using a trained model; A method in which the trained model has been trained using an objective function including a term representing the similarity between an input idea vector and an output idea vector of a contracted vector, and a term representing the similarity between the first predetermined component and a first predetermined index value.
Citation Information
Patent Citations
Thinking support device, thinking support method, and thinking support program
JP2005063312A
Vocabulary extraction support system and vocabulary extraction support method
JP2020095452A
Optimization device, optimization system, optimization method, and optimization program
JP2020187642A
Deep semantic content selection
US20180336490A1
