Inference device

Abstract

To achieve the optimization of an output value with high accuracy even if the accuracy of an output value from a neural network is low.SOLUTION: According to an embodiment, an inference device includes at least one memory and at least one processor. The at least one processor uses a neural network to output scores, applies back propagation to the neural network to calculate a differential value of the scores, uses an index indicating the uncertainty of the scores, the differential value of the scores and the scores to set a search condition for an optimal solution of the scores, and determines the optimal solution of the scores by a gradient method using the search condition.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] TECHNICAL FIELD An embodiment of the present disclosure relates to an inference device. [Background technology]

[0002] Conventionally, various optimization methods have been known. For example, when a gradient method using a line search is used as an optimization method, convergence is expected. Specifically, optimization methods called gradient methods, such as the quasi-Newton method, use the value to be optimized (hereinafter referred to as the target value) as well as the gradient (derivative, hereinafter referred to as the differential value) of the target value to perform efficient optimization.

[0003] However, if the precision of the floating-point numbers related to the target values ​​is low, or if the calculations performed before optimization are not deterministic, the target values ​​may contain uncertainty. In such cases, the original quasi-Newton method may not work correctly.

[0004] On the other hand, when a trained neural network is used to calculate the target value, the differential value of the target value may be calculated by backpropagation through the neural network. In this case, it is known that the accuracy of the differential value is higher than that of the target value even if the accuracy of the floating-point number related to the target value is low. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] So Takamoto, Chikashi Shinagawa, Daisuke Motoki, Kosuke Nakago, Wenwen Li, Iori Kurata, Taku Watanabe, Yoshihiro Yayama, Hiroki Iriguchi, Yusuke Asano, Tasuku Onodera, Takafumi Ishii, Takao Kudo, Hideki Ono, Ryohto Sawada, Ryuichiro Ishitani, Marc Ong, Taiki Yamaguchi, Toshiki Kataoka, Akihide Hayashi, Nontawat Charoenphakdee, Takeshi Ibuka, “Towards universal neural network potential for material discovery applicable to arbitrary combinations of 45 elements” Nature Communications volume 13, Article number: 2991 (2022), URL: https: / / www.nature.com / articles / s41467-022-30687-9 [Non-patent document 2] Fletcher, Roger (1987), Practical methods of optimization (2nd ed.), New York: John Wiley & Sons, ISBN 978-0-471-91547-8 Summary of the Invention [Problem to be solved by the invention]

[0006] The problem to be solved by the present disclosure is to realize accurate optimization of output values ​​from a neural network even when the accuracy of the output values ​​is low. [Means for solving the problem]

[0007] An inference device according to an embodiment includes at least one memory and at least one processor, which outputs a score using a neural network, calculates a differential value of the score by applying backpropagation to the neural network, sets search conditions for an optimal solution of the score using an index indicating the uncertainty of the score, the differential value of the score, and the score, and determines the optimal solution of the score by a gradient method using the search conditions. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an inference device according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of functional blocks in a processor according to the embodiment. [Figure 3] FIG. 3 is a flowchart illustrating an example of a procedure for the optimization process according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of functional blocks in a processor in the learning device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, the embodiments will be described in detail with reference to the drawings.

[0010] (Embodiment) FIG. 1 is a block diagram showing an example of the hardware configuration of an inference device 1 according to this embodiment. As shown in FIG. 1, the inference device 1 may be connected to an external device 9A via a communication network 5. The inference device 1 may also include an external device 9B connected via a device interface 39. The inference device 1 may receive, for example, information indicating a physical system to be inferred. Examples of information indicating a physical system to be inferred include the structure of a substance made up of multiple atoms (atomic coordinates, atomic bonding state, etc.), a structure such as a building (structure coordinates, stress, etc.), a fluid (position, viscosity, flow velocity, etc. of virtual particles), and information regarding a closed area related to global illumination (light source, position of walls, position of placed objects).

[0011] For the sake of concreteness, the information indicating the physical system to be inferred is assumed to be information indicating an atomic structure. In this case, the inference device 1 may input a notation indicating the structure of a substance composed of multiple atoms input by the user. The substance is, for example, a molecule. Note that the substance is not limited to molecules and may be various crystals, etc. The notation is, for example, SMILES (Simplified Molecular Input Line Entry System) notation related to the substance and input by the user. SMILES notation represents, for example, information about a specific molecule (information about atoms and how they are connected) according to certain rules. For example, SMILES notation is granular information, such as that in the case of methane, one C (carbon) is connected to four H (hydrogen).

[0012] Note that the notation is not limited to SMILES notation, and any other known notation may be used as long as it can uniquely identify a substance. For the sake of concreteness, it is assumed below that information input by a user via an input device (described later) is information corresponding to SMILES notation (hereinafter referred to as SMILES information).

[0013] The inference device 1 includes a computer 30 and an external device 9B connected to the computer 30 via a device interface 39. The computer 30 includes, as an example, a processor 31, a main storage device (memory) 33, an auxiliary storage device (memory) 35, a network interface 37, and a device interface 39. The inference device 1 may be realized as a computer 30 in which the processor 31, the main storage device 33, the auxiliary storage device 35, the network interface 37, and the device interface 39 are connected via a bus 41.

[0014] Although the computer 30 shown in FIG. 1 includes one of each component, it may also include multiple of the same component. Furthermore, while FIG. 1 shows a single computer 30, the software may be installed on multiple computers, with each of the multiple computers executing the same or different parts of the software. In this case, a distributed computing configuration may be used in which each computer communicates via a network interface 37 or the like to execute processing. In other words, the inference device 1 in this embodiment may be configured as a system in which one or more computers execute instructions stored in one or more storage devices to realize various functions described below. Furthermore, information transmitted from a terminal may be processed by one or more computers provided on the cloud, and the processing results may be transmitted to a terminal such as a display device (display unit) corresponding to the external device 9B.

[0015] The various calculations of the inference device 1 in this embodiment may be executed in parallel using one or more processors, or using multiple computers via a network. Furthermore, the various calculations may be distributed to multiple processor cores within a processor and executed in parallel. Furthermore, some or all of the processes, means, etc. disclosed herein may be executed by at least one of a processor and a storage device provided on a cloud that can communicate with the computer 30 via a network. Thus, the various functions described below in this embodiment may be implemented in the form of parallel computing using one or more computers.

[0016] The processor 31 may be an electronic circuit (such as a processing circuit, processing circuitry, CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit)) including a control device and an arithmetic device of the computer 30. The processor 31 may also be a semiconductor device including a dedicated processing circuit. The processor 31 is not limited to an electronic circuit using electronic logic elements, but may also be realized by an optical circuit using optical logic elements. The processor 31 may also include an arithmetic function based on quantum computing.

[0017] The processor 31 performs arithmetic processing based on data and software (programs) input from each device, etc., configured internally of the computer 30, and can output the arithmetic results and control signals to each device, etc. The processor 31 may control each component constituting the computer 30 by executing the OS (Operating System) of the computer 30, applications, etc.

[0018] The inference device 1 in this embodiment may be realized by one or more processors 31. Here, the processor 31 may refer to one or more electronic circuits arranged on one chip, or may refer to one or more electronic circuits arranged on two or more chips or two or more devices. When multiple electronic circuits are used, the electronic circuits may communicate with each other via wire or wirelessly.

[0019] The main memory device 33 is a memory device that stores instructions executed by the processor 31 and various data, and information stored in the main memory device 33 is read by the processor 31. The auxiliary memory device 35 is a memory device other than the main memory device 33. Note that these memory devices refer to any electronic component that can store electronic information, and may be semiconductor memory. The semiconductor memory may be either volatile memory or non-volatile memory. The memory device for saving various data used in the inference device 1 according to this embodiment may be realized by the main memory device 33 or the auxiliary memory device 35, or may be realized by an internal memory built into the processor 31. For example, the memory unit in this embodiment may be realized by the main memory device 33 or the auxiliary memory device 35.

[0020] Multiple processors may be connected (coupled) to one storage device (memory), or a single processor 31 may be connected. Multiple storage devices (memories) may be connected (coupled) to one processor. When the inference device 1 in this embodiment is configured with at least one storage device (memory) and multiple processors connected (coupled) to this at least one storage device (memory), it may include a configuration in which at least one of the multiple processors is connected (coupled) to at least one storage device (memory). This configuration may also be realized by storage devices (memories) and processors 31 included in multiple computers. Furthermore, it may include a configuration in which the storage device (memory) is integrated with the processor 31 (for example, a cache memory including an L1 cache and an L2 cache).

[0021] The network interface 37 is an interface for connecting to the communication network 5 wirelessly or via a wire. The network interface 37 may be an appropriate interface, such as one that conforms to an existing communication standard. Information may be exchanged with an external device 9A connected via the communication network 5 through the network interface 37. The communication network 5 may be any one of a WAN (Wide Area Network), a LAN (Local Area Network), a PAN (Personal Area Network), etc., or a combination thereof, as long as information is exchanged between the computer 30 and the external device 9A. An example of a WAN is the Internet, an example of a LAN is IEEE802.11 or Ethernet (registered trademark), and an example of a PAN is Bluetooth (registered trademark) or NFC (Near Field Communication), etc.

[0022] The device interface 39 is an interface such as a USB (Universal Serial Bus) that directly connects to an output device such as a display device, an input device, and an external device 9 B. The output device may also have a speaker that outputs sound and the like.

[0023] The external device 9A is a device connected to the computer 30 via a network. The external device 9B is a device connected to the computer 30 directly.

[0024] The external device 9A or the external device 9B may be, for example, an input device (input unit). The input device is, for example, a device such as a camera, a microphone, a motion capture device, various sensors, a keyboard, a mouse, or a touch panel, and provides acquired information to the computer 30. The external device 9A or the external device 9B may also be a device equipped with an input unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0025] Furthermore, the external device 9A or the external device 9B may be, for example, an output device (output unit). The output device may be, for example, a display device (display unit) such as an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube), a PDP (Plasma Display Panel), or an organic EL (Electro Luminescence) panel, or may be a speaker that outputs sound or the like. Furthermore, the external device 9A or the external device 9B may be a device that includes an output device, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0026] Furthermore, the external device 9A or the external device 9B may be a storage device (memory). For example, the external device 9A may be a network storage or the like, and the external device 9B may be a storage such as an HDD.

[0027] Furthermore, the external device 9A or the external device 9B may be a device having some of the functions of the components of the inference device 1 in this embodiment. In other words, the computer 30 may transmit or receive some or all of the processing results of the external device 9A or the external device 9B.

[0028] 2 is a diagram illustrating an example of functional blocks in the processor 31. The processor 31 has, for example, a calculation unit 311, a setting unit 313, and an optimization unit 315 as functions realized by the processor 31. The functions realized by the calculation unit 311, the setting unit 313, and the optimization unit 315 are each stored as a program in, for example, the main storage device 33 or the auxiliary storage device 35. The processor 31 can realize the functions related to the calculation unit 311, the setting unit 313, and the optimization unit 315 by reading and executing the program stored in the main storage device 33 or the auxiliary storage device 35. The calculation unit 311 may be referred to as an arithmetic unit. Furthermore, the calculation unit 311 and the optimization unit 315 may be collectively referred to as a calculation unit.

[0029] The calculation unit 311 may generate a three-dimensional atomic structure based on SMILES information (hereinafter referred to as SMILES information) input via an input device. The atomic structure corresponds to an atomic arrangement in which multiple atoms of a substance are three-dimensionally arranged, as indicated in the SMEILES notation. The calculation unit 311 generates the atomic structure by inputting the SMEILES notation into a neural network (hereinafter referred to as a neural network potential (NNP)) that approximates potential energy, which is a function of atomic coordinates. For example, the NNP corresponds to a neural network that performs a physical simulation using the atomic structure as information indicating a physical system to be inferred, and outputs an energy value related to the atomic structure as an output value. In addition to the energy value itself, a value obtained by performing arithmetic operations such as multiplying the energy value output from the neural network by a certain value, or a value obtained by performing other operations on the energy value, may also be used. Such values, including the output value of the neural network, are referred to as a score.

[0030] The process of generating atomic structures based on SMILES information can be performed using known techniques as appropriate, and therefore a detailed description thereof will be omitted. NNP has high versatility and can generate accurate energy values ​​for various atomic structures. NNP may also be referred to as a trained model or a trained neural network. That is, the neural network used by the calculation unit 311 may be a trained NNP. The trained model is not limited to NNP, and other trained neural networks may also be used.

[0031] The calculation unit 311 may input information indicating a physical system to be inferred into a physics simulation using a neural network and calculate an output value from the neural network. For example, the calculation unit 311 may input the generated atomic structure into a trained neural network (NNP) to generate an energy value (output value) corresponding to the atomic structure. The output value is output from the trained neural network as, for example, a scalar function. The calculation unit 311 may store the generated energy value in the main storage device 33 or the auxiliary storage device 35. The trained neural network may be trained in advance and stored in the main storage device 33 or the auxiliary storage device 35. Since known techniques can be used as appropriate to generate an energy value using the neural network (NNP) using the atomic structure, a description thereof will be omitted.

[0032] Furthermore, the calculation unit 311 may calculate the derivative of the output value (hereinafter referred to as the derivative value) by applying backpropagation to the neural network. For example, the calculation unit 311 calculates the derivative value corresponding to the output value by backpropagating the neural network using the output value. If the output value is a scalar function with coordinates as arguments, the derivative value corresponds to the coordinate derivative of the scalar function. Specifically, if the output value is an energy value, the calculation unit 311 may calculate the force corresponding to the energy value by backpropagating the neural network using the energy value. The calculation unit 311 may store the calculated derivative value (the derivative of the output value, the force) in the main storage device 33 or the auxiliary storage device 35. Since a known method can be appropriately used to calculate the force (the derivative of the energy value) by backpropagation of the neural network using atomic structures (NNP), a description thereof will be omitted. Note that an example of calculating the derivative value using the output value of the neural network as a score will be described below, but it is also possible to calculate a score from the output value of the neural network and then calculate the derivative value of the score.

[0033] The setting unit 313 may set search conditions for an optimal solution for the output value using an index indicating the uncertainty of the output value, a derivative of the output value, and the output value. The index indicating the uncertainty of the output value may be set in advance depending on information indicating the physical system to be inferred and the precision of the floating-point number used to calculate the output value. Specifically, the index may be set in advance based on the precision of the floating-point number used to calculate the output value, the characteristics of the neural network, the dimension of the output value, etc., and stored in the main storage device 33 or the auxiliary storage device 35. In other words, the index is a parameter determined by the user based on the expected uncertainty. The index indicating the uncertainty of the output value may also be referred to as noise, error, etc.

[0034] For example, the setting unit 313 sets the search condition by adding the above index to the output value in the Armijo condition. The Armijo condition is a condition used when searching for an objective function that achieves a maximum or minimum value using a gradient method. For example, if the information indicating the physical system to be inferred is an atomic structure and the neural network is an NNP, the objective function is a scalar function indicating energy. In this case, the search condition is used to search for a scalar function that minimizes the energy value (in other words, to optimize the energy for the atomic structure).

[0035] As the gradient method, for example, a quasi-Newton method using a line search is used. As the quasi-Newton method, for example, a BFGS (Broyden-Fletcher-Goldfarb-Shanno) method may be used. Since the quasi-Newton method, the BFGS method, etc. are known techniques, a description thereof will be omitted. The Almiho condition is expressed, for example, by the following equation (1).

[0036] f(x k +αp k )≦f(x k )+c1α∇f k T p k ···(1)

[0037] In equation (1), f corresponds to a scalar function that is the output value from the neural network. k corresponds to the search direction in which the scalar function f is searched for to reach its minimum value. k corresponds to the argument (position) of the scalar function f (energy according to position when the neural network is NNP). c1 in equation (1) is a value between 0 and 1 and may be set in advance.

[0038] The setting unit 313 may set the search condition by adding an index to the Almiho condition that uses the differential of the output value and the output value. When the index is expressed as ε, the setting unit 313 may add the index ε to the right side of the Almiho condition in equation (1) to set the search condition shown in the following equation (2).

[0039] f(x k +αp k )≦(f(x k )+ε)+c1α∇f k T p k ···(2)

[0040] The setting unit 313 may store the set search condition (Equation (2)) in the main storage device 33 or the auxiliary storage device 35. Adding an index ε to Equation (1), which is the Almjo condition, corresponds to relaxing the Almjo condition. The search condition is not limited to Equation (2), and as an application example of this embodiment, for example, the search condition may further include a higher-order derivative (such as a second-order derivative or a third-order derivative) of the output value f. Furthermore, in addition to Equation (2), the Wolf condition shown in the following Equation (3) may also be set.

[0041] ∇f(x k +αp k ) T p k ≧c2∇f k T p k ···(3)

[0042] In formula (3), c2 may be set in advance and is a value between c1 and 1. The definitions of the other symbols in formula (3) are the same as in formulas (1) and (2).

[0043] For example, in the distribution of output values ​​(e.g., energy distribution), single-precision floating-point numbers (FP32) can exhibit variations ten times greater than double-precision floating-point numbers (FP64). For these reasons, while it is possible to optimize the energy distribution with double-precision floating-point numbers (FP64), it is difficult to do so with single-precision floating-point numbers (FP32).

[0044] On the other hand, it is known that the distribution of the differential value ∇f of the output value calculated by backpropagation to a neural network (force distribution) is comparable to that of double-single-precision floating-point numbers (FP64) even when using single-precision floating-point numbers (FP32). In other words, when the output value from a neural network contains numerical calculation errors equivalent to uncertainty, it has been experimentally shown that the accuracy of the gradient information (differential of the output value) calculated by the backpropagation method for the neural network is relatively higher than the accuracy of the output value.

[0045] The search conditions set by the setting unit 313 indicate that the limitations imposed by the output value are relaxed in the linear search, as shown in formula (2). In other words, the search conditions indicate that the derivative value is more reliable than the output value, that is, that a high-precision derivative value is more important than a low-precision output value. More specifically, the search conditions shown in formula (2) indicate that a difference of a certain width (f+ε) or more is treated as a significant difference in the linear search.

[0046] The optimization unit 315 determines an optimal solution for the output value by applying a gradient method using the search conditions to the output value. For example, the optimization unit 315 executes the BFGS method on the output value using the search conditions to calculate the optimal value for the output value. Specifically, for example, if the information indicating the physical system to be inferred is an atomic structure and the neural network is an NNP, the optimization unit 315 searches for a scalar function f that minimizes the energy value as the output value. In this way, the optimization unit 315 optimizes the output value, i.e., minimizes the energy.

[0047] The above describes the configuration of the inference device 1. Below, the process of optimizing the output value by the inference device 1 (hereinafter referred to as optimization process) will be described with reference to FIG.

[0048] FIG. 3 is a flowchart illustrating an example of a procedure for the optimization process.

[0049] (Optimization process) (Step S301) The calculation unit 311 may input information indicating a physical system to be inferred to the neural network and calculate an output value from the trained neural network. Specifically, the calculation unit 311 may input SMILES information to the NNP to generate an atomic structure. Then, the calculation unit 311 may input the generated atomic structure to the NNP to calculate an energy value as an output value. The distribution of energy values ​​corresponds to a scalar function f. The calculation unit 311 may store the calculated distribution of energy values ​​in the main storage device 33 or the auxiliary storage device 35 in association with the generated atomic structure.

[0050] (Step S302) The calculation unit 311 may use the output value to apply backpropagation to the neural network to calculate the derivative of the output value (derivative value: ∇f). The calculation unit 311 may store the calculated derivative value in the main storage device 33 or the auxiliary storage device 35 in association with the generated atomic structure.

[0051] (Step S303) The setting unit 313 may set search conditions for an optimal solution for the output value using the index ε, the differential value ∇f, and the output value f. Specifically, the setting unit 313 may read out the preset index ε, the differential value ∇f, and the output value f from the main storage device 33 or the auxiliary storage device 35. Next, the setting unit 313 may set the search conditions, Equation (2), by adding the index ε to Equation (1) indicating the Almiho condition. Alternatively, the setting unit 313 may set Equation (3) indicating the Wolff condition.

[0052] (Step S304) The optimization unit 315 may optimize the output value by applying a gradient line search using the set search conditions to the output value. That is, the optimization unit 315 may calculate, for example, the minimum output value f(x) by performing a line search using the search conditions on the output value f. A known procedure can be applied to the calculation method for the line search, so a description thereof will be omitted.

[0053] From the above, the inference device 1 according to this embodiment may input information indicating the physical system to be inferred into a physical simulation using a neural network to calculate an output value f from the neural network, or may apply backpropagation to the neural network to calculate a derivative f' of the output value f, or may set a search condition (equation (2)) for an optimal solution of the output value f using an index ε indicating the uncertainty of the output value f, the derivative f' of the output value f, and the output value f, or may determine an optimal solution of the output value f by applying a gradient method using the search condition (equation (2)) to the output value f.

[0054] Furthermore, in the inference device 1 according to this embodiment, the search conditions may be set by adding the index ε to the output value f in the Almiho condition. Furthermore, in the inference device 1 according to this embodiment, the index ε may be set in advance according to information indicating the physical system to be inferred and the precision of floating-point numbers used to calculate the output value f. Furthermore, in the inference device 1 according to this embodiment, the neural network may be a trained neural network potential (NNP). In the inference device 1 according to this embodiment, the gradient method using the search conditions may be a line search. Furthermore, in the inference device 1 according to this embodiment, the information indicating the physical system to be inferred may be information indicating an atomic structure. Furthermore, in the inference device 1 according to this embodiment, the output value may be expressed by a scalar function.

[0055] FIG. 4 is a diagram showing an example of the range AC of the Armiho conditions and the range SC of the search conditions in a line search. In FIG. 4, the range SC of the search conditions is wider than the range AC of the Armiho conditions due to the index ε. As shown in FIG. 4, when performing a line search, the step size may be determined independently after the search direction is determined. According to the inference device 1 of this embodiment, the line search is guaranteed to steadily progress toward a stable point, thereby improving the robustness of optimization compared to other algorithms that lack convergence proof and are always at risk of divergence.

[0056] For example, in single-precision floating-point numbers (FP32), the output value contains more uncertainty than the differential value, in other words, the output value contains more noise than the differential value, so optimization calculations using the BFGS method to which the Almiho condition (Equation (1)) is applied may fail. However, with the inference device 1 according to this embodiment, it is possible to perform a line search by taking advantage of the fact that the differential f' of the output value calculated by backpropagation to the neural network is more accurate than the output value f.

[0057] Specifically, according to the inference device 1 of this embodiment, in order to trust the differential f' of the output value more than the output value f, a search condition may be set in which an uncertainty index ε is added to the output value f in the Armiho condition. The search condition (Equation (2)) may be set by relaxing the Armiho condition (Equation (1)) and adding an index ε to the right side of the Armiho condition (Equation (1)) to allow a certain increase due to the uncertainty of the output value f. In a line search using the search condition (Equation (2)), the value (f + ε) obtained by adding the uncertainty index ε to the output value f is treated as a significant difference in the line search.

[0058] According to the inference device 1 of this embodiment, when a line search in the gradient method is performed using search conditions on a scalar function f, which is an output value from a neural network, the gradient of the function (the derivative f' of the output value) is 0 at a local minimum point, and information on the gradient of the scalar function f (the derivative f' of the output value) is used to find and determine the local minimum point. Therefore, according to the inference device 1 of this embodiment, even if the value of the scalar function f contains uncertainty (noise), information on the gradient of the scalar function f (the derivative f' of the output value) is reliable, making it possible to search for the local minimum point.

[0059] For these reasons, according to the inference device 1 of this embodiment, even if the value of the function (output value) output from the neural network contains uncertainty, the search does not fail and the optimization calculation can be completed normally. In other words, according to the inference device 1 of this embodiment, even if the accuracy of the output value from the neural network is low, the output value can be optimized with high accuracy. For example, by setting search conditions in accordance with the properties of the neural network, i.e., in accordance with the uncertainty of the output value output from the neural network, it is possible to realize the optimization calculation with the calculation accuracy of double single-precision floating-point numbers (FP64) even when calculation is performed using single-precision floating-point numbers (FP32).

[0060] As described above, according to the inference device 1 of this embodiment, when an output value is output using a neural network and the derivative of the output value is calculated by backpropagation through the neural network, the use of floating-point numbers, which have a high calculation speed, enables optimization calculation of the output value in a short time with high accuracy. Note that, as an application example of this embodiment, the technical features of this embodiment can be applied when an output value is output using a neural network and the derivative of the output value is calculated by backpropagation through the neural network. Note that the optimization process according to this embodiment is not limited to the BFGS method, and can also be performed in other gradient methods by modifying the conditions using the output value from the neural network to allow an increase in the index ε, which indicates the uncertainty of the output value.

[0061] When the technical ideas of the embodiments are realized by an inference method, the inference method may input information indicating a physical system to be inferred into a physical simulation using a neural network to calculate an output value from the neural network, may apply backpropagation to the neural network to calculate the derivative of the output value, may set search conditions for an optimal solution of the output value using an index ε indicating the uncertainty of the output value, the derivative of the output value, and the output value, and may determine the optimal solution of the output value by applying a gradient method using the search conditions. The procedure and effects of the optimization process related to the inference method are similar to those described in the embodiments, and therefore will not be described again.

[0062] When the technical ideas in the embodiments are realized by an inference program, the inference program may input information indicating a physical system to be inferred into a physical simulation using a neural network in a computer to calculate an output value from the neural network, or may apply backpropagation to the neural network to calculate the derivative of the output value, or may set search conditions for an optimal solution to the output value using an index ε indicating the uncertainty of the output value, the derivative of the output value, and the output value, and may apply a gradient method using the search conditions to the output value to realize the determination of the optimal solution for the output value.

[0063] For example, the optimization process can be realized by installing the inference program in a computer in a server device or any of various simulation devices that execute physical simulations using a neural network and expanding the program in memory. In this case, the program that enables a computer to execute the inference method can also be stored and distributed in a storage medium such as a magnetic disk (such as a hard disk), an optical disk (such as a CD-ROM or DVD), or a semiconductor memory. The procedure and effect of the optimization process using the inference program are the same as those in the embodiment, so a description thereof will be omitted.

[0064] Some or all of the devices in the above-described embodiments may be configured as hardware, or may be configured as information processing software (programs) executed by a CPU, a GPU, or the like. In the case of software information processing, software that realizes at least some of the functions of each device in the above-described embodiments may be stored on a non-transitory storage medium (non-transitory computer-readable medium) such as a flexible disk, a CD-ROM (Compact Disc-Read Only Memory), or a USB memory, and the software information processing may be executed by loading the software into the computer 30. The software may also be downloaded via the communication network 5. Furthermore, the software may be implemented in a circuit such as an ASIC or FPGA, so that the information processing is executed by hardware.

[0065] The type of storage medium that stores the software is not limited. The storage medium is not limited to removable media such as magnetic disks or optical disks, but may be fixed storage media such as hard disks or memory. The storage medium may be provided inside the computer or outside the computer.

[0066] In this specification (including the claims), when the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used, it includes any of a, b, c, ab, ac, bc, or abc. It may also include multiple instances of any element, such as aa, abb, aabbcc, etc. Furthermore, it also includes the addition of elements other than the enumerated elements (a, b, and c), such as having d, as in abcd.

[0067] In this specification (including the claims), when expressions such as "using data as input / based on / according to / in response to" (including similar expressions) are used, unless otherwise specified, this includes cases where various data itself is used as input, or where various data that has been processed in some way (e.g., noise-added, normalized, intermediate representation of various data, etc.) is used as input. Furthermore, when a statement is made that a result is obtained "based on / according to / in response to data," this includes cases where the result is obtained based solely on the data in question, as well as cases where the result is obtained as a result of being influenced by other data, factors, conditions, and / or states other than the data in question. Furthermore, when a statement is made that "data is output," this includes cases where various data itself is used as output, or where various data that has been processed in some way (e.g., noise-added, normalized, intermediate representation of various data, etc.) is output, unless otherwise specified.

[0068] When the terms "connected" and "coupled" are used in this specification (including the claims), they are intended as open-ended terms that encompass any of direct connection / coupling, indirect connection / coupling, electrically connection / coupling, communicatively connection / coupling, functionally connection / coupling, and physically connection / coupling. These terms should be interpreted appropriately according to the context in which they are used, but any form of connection / coupling that is not intentionally or naturally excluded should be interpreted as being included in these terms without limitation.

[0069] In this specification (including the claims), the expression "A configured to B" may include the physical structure of element A having a configuration capable of performing operation B, and the permanent or temporary setting / configuration of element A being configured / set to actually perform operation B. For example, if element A is a general-purpose processor, it is sufficient that the processor has a hardware configuration capable of performing operation B, and is configured to actually perform operation B by setting a permanent or temporary program (instruction). Also, if element A is a dedicated processor or dedicated arithmetic circuit, it is sufficient that the circuit structure of the processor is implemented to actually perform operation B, regardless of whether control instructions and data are actually attached.

[0070] When used in this specification (including the claims), terms implying containing or possessing (e.g., "comprising / including" and "having") are intended to be open-ended terms that include containing or possessing things other than the object designated by the object of the term. When the object of such a term implies no quantity or a singular number (e.g., an article such as "a" or "an"), the expression should be construed as not being limited to a specific number.

[0071] In this specification (including the claims), even if expressions such as "one or more" or "at least one" are used in some places and expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") are used in other places, the latter expressions are not intended to mean "one." In general, expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") should be interpreted as not necessarily being limited to a specific number.

[0072] In this specification, when a particular advantage / result is described as being obtained from a particular configuration of an embodiment, it should be understood that the same advantage / result can also be obtained from one or more other embodiments having the same configuration, unless otherwise stated. However, it should be understood that the presence or absence of the effect generally depends on various factors, conditions, and / or states, etc., and that the effect is not necessarily obtained by the configuration. The effect is merely obtained by the configuration described in the embodiment when various factors, conditions, and / or states, etc. are satisfied, and the effect does not necessarily occur in a claimed invention that defines the same or a similar configuration.

[0073] When used in this specification (including the claims), terms such as "maximize" include finding a global maximum, finding an approximation of a global maximum, finding a local maximum, and finding an approximation of a local maximum, and should be interpreted appropriately according to the context in which the term is used. It also includes finding approximations of these maxima probabilistically or heuristically. Similarly, when used in this specification (including the claims), terms such as "minimize" include finding a global minimum, finding an approximation of a global minimum, finding a local minimum, and finding an approximation of a local minimum, and should be interpreted appropriately according to the context in which the term is used. It also includes finding approximations of these minima probabilistically or heuristically. Similarly, when used in this specification (including the claims), terms such as "optimize" include finding a global optimum, finding an approximation of a global optimum, finding a local optimum, and finding an approximation of a local optimum, and should be interpreted appropriately according to the context in which the term is used. It also includes finding approximations of these optima probabilistically or heuristically.

[0074] In this specification (including claims), when multiple pieces of hardware perform a predetermined process, the pieces of hardware may cooperate to perform the predetermined process, or some of the hardware may perform all of the predetermined process. Furthermore, some of the hardware may perform part of the predetermined process, and other hardware may perform the rest of the predetermined process. In this specification (including claims), when an expression such as "one or more pieces of hardware perform a first process, and the one or more pieces of hardware perform a second process" is used, the hardware performing the first process and the hardware performing the second process may be the same or different. In other words, it is sufficient that the hardware performing the first process and the hardware performing the second process are included in the one or more pieces of hardware. Note that hardware may include an electronic circuit or a device including an electronic circuit.

[0075] In this specification (including the claims), when multiple storage devices (memories) store data, each of the multiple storage devices (memories) may store only a portion of the data, or may store the entire data.

[0076] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the individual embodiments described above. Various additions, modifications, substitutions, partial deletions, etc. are possible within the scope of the conceptual idea and spirit of the present invention derived from the content defined in the claims and their equivalents. For example, in all of the above-described embodiments, when numerical values ​​or formulas are used in the explanation, they are shown as examples and are not limited to these. Furthermore, the order of each operation in the embodiments is shown as an example and is not limited to these.

[0077] With respect to the above-described embodiments, the following supplementary notes are disclosed as one aspect and optional features of the invention. (Appendix 1) At least one memory; at least one processor, The at least one processor: A neural network is used to output a score, Applying backpropagation to the neural network to calculate a differential value of the score; setting search conditions for an optimal solution for the score using an index indicating the uncertainty of the score, a differential value of the score, and the score; determining an optimal solution for the score by a gradient method using the search conditions; Reasoning device. (Appendix 2) The search condition is f(x k +αp k )≦f(x k )+c1α∇f k T p k The index is set by adding the index to the right side of the Armijo condition: 10. The inference apparatus of claim 1. (Appendix 3) The search conditions are set by the Wolf conditions in addition to the Armijo conditions. 10. The inference device of claim 2. (Appendix 4) The output of the score is performed by inputting information indicating a physical system that is an inference target of the inference device to the neural network. 4. An inference device according to any one of appendices 1 to 3. (Appendix 5) the index is set according to information indicating an inference target of the inference device and the precision of floating-point numbers used in calculating the score; 5. An inference device according to any one of appendices 1 to 4. (Appendix 6) The gradient method using the search conditions is a linear search. 6. An inference device according to any one of appendices 1 to 5. (Appendix 7) The information indicating the physical system to be inferred is information on an atomic structure. 10. The inference apparatus of claim 4. (Appendix 8) The score is expressed as a scalar function: 8. An inference device according to any one of appendices 1 to 7. (Appendix 9) The search condition further includes a higher order derivative of the score. 9. An inference device according to any one of appendices 1 to 8. (Appendix 10) The neural network is a trained neural network potential. 10. An inference device according to any one of appendices 1 to 9. [Explanation of symbols]

[0078] 1 Reasoning device 5. Communication Network 9A external device 9B External device 30 Computer 31 processors 33 Main memory 35 Auxiliary storage device 37 Network Interface 39 Device Interfaces 41 Bus 311 Calculation Unit 313 Settings 315 Optimization Department

Claims

1. at least one memory; at least one processor, The at least one processor A neural network is used to output a score, Applying backpropagation to the neural network to calculate a differential value of the score; setting search conditions for an optimal solution for the score using an index indicating the uncertainty of the score, a differential value of the score, and the score; determining an optimal solution for the score by a gradient method using the search conditions; Reasoning device.

2. The search conditions are: f(x ｋ +αp ｋ )≦f(x ｋ )+c １ α∇f ｋ Ｔ p ｋ The index is set by adding the index to the right side of the Armijo condition: The inference device according to claim 1 .

3. The search conditions are set by the Wolf conditions in addition to the Armijo conditions. The inference device according to claim 2 .

4. The output of the score is performed by inputting information indicating a physical system that is an inference target of the inference device to the neural network. The inference device according to claim 1 .

5. the index is set according to information indicating an inference target of the inference device and the precision of floating-point numbers used in calculating the score; The inference device according to claim 1 .

6. The gradient method using the search conditions is a linear search. The inference device according to claim 1 .

7. The information indicating the physical system to be inferred is information on an atomic structure. The inference device according to claim 4.

8. The score is expressed as a scalar function: The inference device according to claim 1 .

9. The search condition further includes a higher order derivative of the score. The inference device according to claim 1 .

10. The neural network is a trained neural network potential. An inference device according to any one of claims 1 to 9.

Images