Apparatus and method for evaluating the reliability of a model

Abstract

This evaluates the reliability of the output of models or artificial intelligence, such as generative AI or LLMs. [Solution] The system comprises: γ1 for calculating a value C1 relating to the reliability / confidence of at least the input u or output y of the model / AI / artificial intelligence α based on the input u or the output y of the model / AI / artificial intelligence α when the input u is given; γ2 for calculating a value C2 relating to the reliability / confidence of at least the input u or output y based on the internal calculation value x of the model / AI / artificial intelligence α when the input u is given; and γ3 for calculating a value C3 relating to the reliability / confidence of at least the input u or output y based on at least the C1 or C2.

Description

Technical Field

[0001] The present invention relates to a model reliability evaluation apparatus and method, and particularly to an apparatus and method for evaluating the reliability of the output of artificial intelligence.

Background Art

[0002] As the background art of this technology, there is Japanese Patent Application Laid-Open No. 2022-109410 (Patent Document 1). This document describes "inputting first data into a first machine learning model to obtain first output data, and inputting the first data into a second machine learning model generated by machine learning based on the difference between the second output data obtained by inputting second data into the first machine learning model and the correct label of the second data, and the second data, to obtain third output data and a confidence interval related to the third output data, and presenting a combination of the first output data, the third output data, and the confidence interval, and causing a computer to execute the process. (See [Claim 1])."

[0003] Also, there is Japanese Patent Application Laid-Open No. 2020-046905 (Patent Document 2). This document describes "a control device using artificial intelligence, comprising: a determination unit that determines whether the operating state of the control device is normal or abnormal based on whether at least one of the input value, internal calculation value, and output value of the control device is within a predetermined range; and a determination unit that determines the predetermined range based on the control parameters of the control device during learning of the value of the performance parameter that determines the performance of the control device. (See [Claim 1])."

[0004] There is also arXiv:2309.15217 (Non-Patent Document 1). This document states, "With RAGAS, we put forward a suite of metrics which can be used to evaluate these different dimensions without having to rely on ground truth human annotations. We posit that such a framework can crucially contribute to faster evaluation cycles of RAG architectures, which is especially important given the fast adoption of LLMs. (See Abstract)."

[0005] There is also arXiv:2404.08679 (Non-Patent Literature 2). This document states, "In our paper, we propose that pretrained base models can function as a repository of prior knowledge for OOD data relative to in-distribution data, effectively acting as OOD proxy distributions. Guided by this insight, we discover that the likelihood ratio between the base model and its finetuned counterpart serves as an effective criterion for detecting OOD data. Moreover, for LLM-based question-answering (QA) systems, the same likelihood ratio excels in detecting OOD questions. By identifying and rejecting these OOD questions, we can greatly enhance the robustness of current QA systems. (See Introduction)" [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2022-109410 [Patent Document 2] Japanese Patent Publication No. 2020-046905 [Non-patent literature]

[0007] [Non-Patent Document 1] arXiv:2309.15217 [Non-Patent Document 2] arXiv:2404.08679 [Overview of the project] [Problems that the invention aims to solve]

[0008] However, the aforementioned prior art (Patent Document 1) evaluates the reliability of the output of a machine learning model based solely on the input and output of the machine learning model, and does not evaluate it based on internal information of the model or artificial intelligence.

[0009] Furthermore, the prior art (Patent Document 2) determines an abnormality in the control device based on whether or not a control parameter, which includes at least one of the input value, internal calculation value, and output value of the control device, is within a predetermined range. The determination method based on input and output and the determination method based on internal calculation value are the same method, and accuracy is not improved by using different methods.

[0010] Furthermore, the prior art (Non-Patent Literature 1) evaluates the validity of the output of artificial intelligence (generative AI) based solely on the input and output of the artificial intelligence (generative AI), and does not evaluate based on the internal information of the artificial intelligence (generative AI). It does not have the function of compensating for the shortcomings of evaluation methods based solely on input and output.

[0011] Furthermore, the prior art (Non-Patent Literature 2) evaluates the validity of the output of artificial intelligence (generating AI) based solely on the internal calculation values ​​of the artificial intelligence (generating AI), and does not evaluate based on the input and output information of the artificial intelligence (generating AI). It does not have the function of compensating for the shortcomings of an evaluation method based solely on internal calculation values. [Means for solving the problem]

[0012] To solve the above problems, for example, the configuration described in the claims is adopted. This application includes several means for solving the above problems, for example, Means γ1 for calculating a value C1 related to at least the reliability / confidence level of the input u or the output y of the model / AI / artificial intelligence α based on at least the input u or the output y of the model / AI / artificial intelligence α when the input u is received; Means γ2 for calculating a value C2 related to at least the reliability / confidence level of the input u or the output y based on the internal calculation value x of the model / AI / artificial intelligence α when the input u is received; Means γ3 for calculating a value C3 related to at least the reliability / confidence level of the input u or the output y based on at least the C1 or the C2; and is provided with the above.

[0013] Also, for example, the model / AI / artificial intelligence α is an LLM or a Transformer or a state space model (SSM)-based model. This is the feature.

[0014] Also, for example, the input u is a prompt generated based on information obtained by referring to a database or the Internet. This is the feature.

[0015] Also, for example, the calculation means γ1 calculates the value C1 related to the reliability / confidence level based on at least the information (context) used at the time of generating the input u or the input u (prompt) or the output (output). This is the feature.

[0016] Also, for example, is provided with an internal calculation value x extraction means. This is the feature.

[0017] Also, for example, the model / AI / artificial intelligence α is a Transformer, The internal calculation value x extraction means includes at least the embedding vector of the input or the output of the transformer or the output of each intermediate layer or the softmax value of the output layer or the attention vector of each intermediate layer or the magnitude of the attention vector and is a means for extracting any one of them. It is characterized by this.

[0018] Also, for example, the model / AI / artificial intelligence α is Mamba, the internal calculation value x extraction means includes at least h(t) (implicit latent state), or the values of each element of the matrices A￣ (the upper line of A), B￣ (the upper line of B), and C, or Δt (step size) and is a means for extracting them. It is characterized by this.

[0019] Also, for example, the calculation means γ2 includes at least whether the internal calculation value x is within a predetermined range, or the difference of the internal calculation value x from the predetermined range and calculates the value C2 related to the reliability / confidence based on this. It is characterized by this.

[0020] Also, for example, the calculation means γ3 based on the value C1 related to the reliability / confidence and the value C2 related to the reliability / confidence, with λ as a parameter C3 = λ·C1+(1 - λ)C2 The value calculated using this method is defined as the confidence / reliability value C3. It is characterized by the following:

[0021] Also, for example, The calculation means γ3 is, Based on the aforementioned confidence / confidence values ​​C1 and C2, C_tmp = λ·C1 + (1-λ)C2 Calculate, When C_tmp is less than or equal to a predetermined value Kc, a notification is issued. It is characterized by the following:

[0022] Also, for example, When the difference between the aforementioned reliability / confidence value C1 and the aforementioned reliability / confidence value C2 is large, or, When the confidence / confidence value C1 is less than or equal to a predetermined value and the confidence / confidence value C2 is greater than or equal to a predetermined value, or When the confidence / confidence value C1 is greater than or equal to a predetermined value and the confidence / confidence value C2 is less than or equal to a predetermined value, To report It is characterized by the following:

[0023] Also, for example, The calculation means γ2 is The system includes means for calculating a confidence / confidence value C2_k for each word or token in at least the input u or output y, When C2_k is less than or equal to a predetermined value, A new input u is calculated based on the word or token corresponding to C2_k. It is characterized by the following:

[0024] Also, for example, Based on the new input u, the model α calculates a new output y. Along with, The calculation means γ1 calculates a value C1 related to reliability / confidence. It is characterized by the following:

[0025] Also, for example, Display at least one confidence / reliability value C1, or one confidence / reliability value C2, or one confidence / reliability value C3. It is characterized by the following:

[0026] Also, for example, Display the confidence / reliability value C1 or C2, and the confidence / reliability value C3. It is characterized by the following: [Effects of the Invention]

[0027] The present invention includes means γ1 for calculating a confidence level value C1 of at least the input u or output y of model / AI / artificial intelligence α based on the input u or output y of model / AI / artificial intelligence α when the input u is given; means γ2 for calculating a confidence level value C2 of at least the input u or output y based on the internal calculation value x of model / AI / artificial intelligence α when the input u is given; and means γ3 for calculating a confidence level value C3 of at least the input u or output y based on at least the C1 or C2.

[0028] When calculating confidence / confidence values ​​based on input and output, for example, the input and output are generally highly readable, so the interpretability of the confidence / confidence values ​​is also high. Furthermore, it is possible to evaluate the validity of the output by referring to the internet or databases, so the information used to calculate the confidence / confidence values ​​is also relatively fresh. However, there are security challenges when referring to external information. In addition, since information about the inference process between the input and output is not used, the reliability of that process cannot be guaranteed.

[0029] On the other hand, when calculating confidence / confidence values ​​based on internal calculations, the inputs and outputs are generally not highly readable, resulting in low interpretability of the confidence / confidence values. Furthermore, it is generally difficult to evaluate the validity of the output by referring to the internet or databases, and updating the information used to calculate confidence / confidence values ​​requires retraining the model, making it difficult to maintain the freshness of the information. However, since it does not refer to external information, it can be said to have high security. Moreover, since it uses internal calculations, which are information from the inference process, the reliability of that process is relatively easy to guarantee.

[0030] As described above, according to the present invention, a value relating to the reliability / confidence level of the input or output of a model / AI is calculated based on both an evaluation based on the input and output of the model / AI / artificial intelligence and an evaluation based on internally calculated values ​​that are different from those. Therefore, the reliability / confidence level of the input or output of the model / AI / artificial intelligence is evaluated by utilizing the advantages of both methods while compensating for their respective disadvantages. Consequently, it becomes possible to ensure a high level of reliability for the model / AI / artificial intelligence. [Brief explanation of the drawing]

[0031] [Figure 1] Block diagram showing the basic configuration of an apparatus / method for evaluating the reliability of a model / AI / artificial intelligence. [Figure 2] Diagram showing the overall configuration in Examples 1-3 and 5-6. [Figure 3] System diagrams of the reliability evaluation devices for the models in Examples 1-6 [Figure 4] Input u creation unit in Examples 1-6 [Figure 5] Figure showing Model α in Examples 1-5 [Figure 6] This figure shows the calculation process of the C1 calculation unit γ1 related to the reliability / confidence level in Examples 1 to 6. [Figure 7] Examples 1-5: Internal calculation value x extraction means [Figure 8]This figure shows the calculation process of the C2 calculation unit γ2 related to the confidence level / confidence level in Examples 1-3 and 5. [Figure 9] This figure shows the calculation process of the C3 calculation unit γ3 related to the confidence level / confidence level in Examples 1, 4 to 6. [Figure 10] This figure shows the calculation process of the C3 calculation unit γ3 related to the confidence level / confidence level in Example 2. [Figure 11] This figure shows the calculation process of the C3 calculation unit γ3 related to the confidence level / confidence level in Example 3. [Figure 12] Diagram showing the overall configuration in Example 4 [Figure 13] This figure shows the calculation process of the C2 calculation unit γ2 related to the confidence level / confidence level in Example 4. [Figure 14] Diagram showing the overall configuration in Example 5 [Figure 15] Figure showing the calculation process of the display target calculation means in Example 5. [Figure 16] Figure (1) showing Model α in Example 6 [Figure 17] Figure (2) showing Model α in Example 6 [Figure 18] Means for extracting the internal calculation value x in Example 6 (Part 1) [Figure 19] Means for extracting the internal calculation value x in Example 6 (Part 2) [Modes for carrying out the invention]

[0032] Several embodiments will be described below with reference to the drawings. [Examples]

[0033] In this embodiment, A means γ1 for calculating a value C1 relating to the confidence level of at least the input u or output y of the model / AI / artificial intelligence α, based on at least the input u or output y of the model / AI / artificial intelligence α given the input u, A means γ2 for calculating a value C2 relating to the confidence level of at least the input u or the output y, based on the internal calculation value x of the model / AI / artificial intelligence α when the input u is given, means γ3 for calculating a value C3 relating to the reliability / confidence level of at least the input u or the output y, based at least C1 or C2, and It is equipped with.

[0034] Also, The aforementioned model / AI / artificial intelligence α is It is based on LLM, Transformer, or SSM (State Space Model) models. It is characterized by the following:

[0035] Also, The aforementioned input u is This prompt is generated based on information obtained from a database or the internet. It is characterized by the following:

[0036] Also, The calculation means γ1 is The confidence level value C1 is calculated based at least on the information (context) used when generating the input u, or on the input u (prompt) or output (output). It is characterized by the following:

[0037] Also, Means for extracting the internal calculation value x are provided It is characterized by the following:

[0038] Also, The aforementioned Model / AI / Artificial Intelligence α is a Transformer, The internal calculation value x extraction means includes at least, The embedding vector of the input or output of the transformer or Output of each intermediate layer or Output layer softmax value or The attention vector of each intermediate layer or the magnitude of that attention vector. This is a means of extracting one of the following. It is characterized by the following:

[0039] Also, The calculation means γ2 includes at least, Whether the internal calculation value x is within a predetermined range, or, The difference between the internal calculation value x and the predetermined range Based on this, the confidence / confidence value C2 is calculated. It is characterized by the following:

[0040] Also, The calculation means γ3 is, Based on the aforementioned confidence / confidence values ​​C1 and C2, λ is a parameter C3 = λ·C1 + (1-λ)C2 The value calculated using this method is defined as the confidence / reliability value C3. It is characterized by the following:

[0041] The above configuration is shown below.

[0042] Figure 2 is a block diagram showing the overall configuration in this embodiment. The input u creation unit 6 creates the input u for model α(2). Model α(2) calculates the output y based on the input u (here referred to as text / prompt). The confidence / confidence value C1 calculation unit γ1(3) calculates the confidence / confidence value C1 based on the input u and the output y. The internal calculation value x extraction means 7 extracts the internal calculation value x of model α(2). The confidence / confidence value C2 calculation unit γ2(4) calculates the confidence / confidence value C2 based on the internal state x. The confidence / confidence value C3 calculation unit γ3(5) calculates the confidence / confidence value C3 based on the confidence / confidence value C1 and the confidence / confidence value C2.

[0043] Figure 3 is a system diagram of a device that implements the processing shown in Figure 2. The device is equipped with an input circuit 26 that processes external signals. External signals here refer to, for example, the input u mentioned above. These external signals pass through the input circuit 26 and become input signals, which are sent to the input / output port 27. Each input information sent to the input / output port 27 is written to the RAM 24 via the data bus 25, or stored in the storage device 21. The ROM 23 or storage device 21 contains the processing described later, which is executed by the CPU 22. At that time, calculations are performed using the values ​​written to the RAM 24 or storage device 21 as appropriate. Of the calculation results, the information (values) to be sent to the outside is sent to the input / output port 27 via the data bus 25 and sent to the output circuit 28 as an output signal. The output circuit 28 outputs a signal to the outside. The signal to the outside here refers to the output y mentioned above.

[0044] In other words, the reliability / confidence value calculation unit C1 γ1(3), the reliability / confidence value calculation unit C2 γ2(4), and the reliability / confidence value calculation unit C3 γ3(5) shown in Figures 1 and 2, as well as the input u creation unit 6 and the internal calculation value x extraction means 7 shown in Figure 2, are all realized by the CPU 22 shown in Figure 3 executing the processes written to the ROM 23 or storage device 21. Similarly, the various means and each functional unit described later are also realized by the CPU 22 shown in Figure 3 executing the processes written to the ROM 23 or storage device 21.

[0045] The details of each process are explained below.

[0046] <Input u creation section (Figure 4)> This process performs calculations on the input u. Specifically, this is shown in Figure 4. The input u is created by referring to a database or the internet. More specifically, the generated input u0 (generally text) is used to perform calculations on the input u using RAG (Retrieval Augmented Generation). There is a lot of literature on RAG, so it will not be described in detail here. The input u0 can be created by a human or automatically generated by some method.

[0047] <Model α (Figure 5)> This process calculates the output y. Specifically, this is shown in Figure 5. Model α(2) is a decoder-based model of a transformer. There is a lot of literature on transformers and their decoders, so they will not be described in detail here. Note that the embedding process, direct terms (residual paths), and regression parts are not relevant to this embodiment and have been omitted.

[0048] The input u (here, a sentence / prompt) is converted into n tokens (h1, h2, ...hn) through a tokenizer. Based on the tokens, each layer from the 1st to the sth layer is processed using attention layers and FNN layers, and a probability Pk (k: 1, 2, ...f) is calculated based on the output of the sth layer, where f is the number of candidate tokens. The output token number j is determined from P1 to Pf, and based on token number j, the tokenizer outputs hj, which is then used as output y. This process is repeated recursively, and the output sentence is constructed by concatenating the outputs y.

[0049] <Calculation unit γ1 for reliability / confidence level C1 (Figure 6)> This process calculates a confidence / reliability value C1. Specifically, this is shown in Figure 6. Based on the input u and output y, the confidence / reliability value C1 is calculated using RAGAS (Retrieval Augmented Generation Assessment). There is a lot of literature on RAGAS, so it will not be described in detail here. As the value of C1, it is best to use Faithfulness, Answer Relevancy, Context Precision, Context Recall, and Context Relevancy calculated by RAGAS, but other quantified values ​​that are output may also be used.

[0050] <Internal calculation value x extraction means (Figure 7)> This process involves calculating (extracting) the internal calculation value X of model α(2). Specifically, this is shown in Figure 7. Model α(2) is a decoder-based model of a transformer.

[0051] As described in the explanation of Model α(2) (Figure 5), the input u is used to produce the output y. At this time, the internal calculation values ​​X: x1, x2, ..., xn of the r-th layer are extracted. Here, in this embodiment, xk is the internal calculation value corresponding to the k-th token in the input u. Alternatively, the internal calculation values ​​(xn+1, xn+2, ..., xn+m) when the output y is regressed and a new output y is calculated may be targeted. X may be a single value (x1) or multiple values ​​(x1, x2, ..., xn(, xn+1, xn+2, ..., xn+m)). The choice of which r-th layer and which internal calculation values ​​to target should be determined empirically, but other guidelines may be followed if available. Multiple layers may also be targeted. It has been pointed out that as the layers get deeper (as they move further away from the input), the processing shifts from global to detailed, so such insights may be taken into consideration. Furthermore, the internal calculation values ​​include the self-attention vector of the attention layer, the magnitude of that vector, and the output of each layer. Since there are variables that clearly correspond to the k-th token, it is also a good idea to choose such variables as internal variables. Alternatively, Pk could be used.

[0052] <Calculation unit γ2 for reliability / confidence level C2 (Figure 8)> In this process, the difference D is calculated based on the internal state X and the region representative value z. Specifically, the following calculations are performed as shown in Figure 8.

[0053] - Dk=||z_in-xk|| 2 (k:1,2,···,n) Perform the calculation.

[0054] - Let D be the average value of Dk.

[0055] The regional representative value z is a representative value of the region with high confidence / reliability. For example, it is conceivable to use a value calculated by some method as the representative value of the region where the data used to train model α(2) exists. This method could be the center vector of the training data, or it could be the center vector of each cluster after clustering the training data. Various methods are possible.

[0056] Furthermore, Dk can be any value that represents the distance between vectors, such as the cosine distance or Mahalanobis distance between z_in and xk.

[0057] Additionally, D can be a value that represents Dk, such as the maximum value of Dk.

[0058] Next, based on the difference D, the confidence / confidence value C2 is calculated via the function f(D). As shown in Figure 8, the function f(D) has the characteristic that C2 decreases as the value of D increases. The function f(D) may be set empirically or according to the required specifications using a table lookup format, or a suitable mathematical formula may be used if one is obtained.

[0059] <Value C3 calculation unit γ3 related to reliability / confidence level (Figure 9)> This process involves calculating (extracting) the confidence / confidence value C3. Specifically, this is shown in Figure 9. Based on the confidence / confidence values ​​C1 and C2, it is calculated using the following formula.

[0060] C3 = (λ × C1) + (1 - λ) × C2

[0061] The weighting coefficient λ may be determined in advance according to some guidelines, or it may be variable so that the user can determine it as appropriate during the operation of the device shown in this embodiment.

[0062] In this configuration, a confidence level value C1 is calculated based on the input u and output y of model α(2), a confidence level value C2 is calculated based on the internal state x of model α(2), and a confidence level value C3 is calculated based on C1 and C2. In this embodiment, C3 is calculated by weighting C1 and C2, so it is possible to adjust which of C1 or C2 is given more weight depending on the situation in which this configuration is used. Furthermore, it is also possible to automatically adjust the weight coefficients by adding a mechanism that adjusts the weight coefficients so that some objective function / evaluation function is minimized or maximized.

[0063] As described above, according to the configuration shown in this embodiment, the reliability / confidence level of the input or output of model α(2) is calculated based on both an evaluation based on the input and output of model α(2) and an evaluation based on different internal calculation values. This allows the reliability / confidence level of the input or output of model α(2) to be evaluated by utilizing the advantages of both methods while compensating for their respective disadvantages. Therefore, it is possible to ensure a high level of reliability for model α(2). [Examples]

[0064] In this embodiment, A means γ1 for calculating a value C1 relating to the confidence level of at least the input u or output y of the model / AI / artificial intelligence α, based on at least the input u or output y of the model / AI / artificial intelligence α given the input u, A means γ2 for calculating a value C2 relating to the confidence level of at least the input u or the output y, based on the internal calculation value x of the model / AI / artificial intelligence α when the input u is given, means γ3 for calculating a value C3 relating to the reliability / confidence level of at least the input u or the output y, based at least C1 or C2, and It is equipped with.

[0065] Also, The aforementioned model / AI / artificial intelligence α is It is based on LLM, Transformer, or SSM (State Space Model) models. It is characterized by the following:

[0066] Also, The aforementioned input u is This prompt is generated based on information obtained from a database or the internet. It is characterized by the following:

[0067] Also, The calculation means γ1 is The confidence level value C1 is calculated based at least on the information (context) used when generating the input u, or on the input u (prompt) or output (output). It is characterized by the following:

[0068] Also, Means for extracting the internal calculation value x are provided It is characterized by the following:

[0069] Also, The aforementioned Model / AI / Artificial Intelligence α is a Transformer, The internal calculation value x extraction means includes at least, The embedding vector of the input or output of the transformer or Output of each intermediate layer or Output layer softmax value or The attention vector of each intermediate layer or the magnitude of that attention vector. This is a means of extracting one of the following. It is characterized by the following:

[0070] Also, The calculation means γ2 includes at least, Whether the internal calculation value x is within a predetermined range, or, The difference between the internal calculation value x and the predetermined range Based on this, the confidence / confidence value C2 is calculated. It is characterized by the following:

[0071] Also, The calculation means γ3 is, Based on the aforementioned confidence / confidence values ​​C1 and C2, C_tmp = λ·C1 + (1-λ)C2 Calculate, When C_tmp is less than or equal to a predetermined value Kc, a notification is issued. It is characterized by the following:

[0072] The above configuration is shown below.

[0073] Figure 2 is a block diagram showing the overall configuration in this embodiment, but it is the same as in Embodiment 1, so it will not be described in detail.

[0074] Figure 3 is a system diagram of the device that implements the process shown in Figure 2, but it is the same as in Example 1, so it will not be described in detail.

[0075] <Input u creation section (Figure 4)> This process performs calculations on the input u. Specifically, this is shown in Figure 4, but since it is the same as in Example 1, it will not be described in detail.

[0076] <Model α (Figure 5)> This process calculates the output y. Specifically, it is shown in Figure 5, but since it is the same as in Example 1, it will not be described in detail.

[0077] <Calculation unit γ1 for reliability / confidence level C1 (Figure 6)> This process calculates the confidence / reliability value C1. Specifically, as shown in Figure 6, it is the same as in Example 1 and will not be described in detail.

[0078] <Internal calculation value x extraction means (Figure 7)> This process involves calculating (extracting) the internal calculation value X of model α(2). Specifically, this is shown in Figure 7, but since it is the same as in Example 1, it will not be described in detail.

[0079] <Calculation unit γ2 for reliability / confidence level C2 (Figure 8)> In this process, the difference D is calculated based on the internal state X and the region representative value z. Specifically, this is shown in Figure 8, but it is the same as in Example 1, so it will not be described in detail.

[0080] <Value C3 calculation unit γ3 related to reliability / confidence level (Figure 10)> This process involves calculating (extracting) the confidence / confidence value C3. Specifically, this is shown in Figure 10. Based on the confidence / confidence values ​​C1 and C2, it is calculated using the following formula.

[0081] C_tmp = λ·C1 + (1-λ)·C2

[0082] When C_tmp ≤ K1c, let C3 = 1.

[0083] Otherwise, set C3 = 0.

[0084] Furthermore, λ and K1c may be determined in advance according to some guidelines, or they may be variable so that the user can determine them as appropriate during the operation of the device shown in this embodiment.

[0085] In this configuration, a confidence level value C1 is calculated based on the input u and output y of model α(2), a confidence level value C2 is calculated based on the internal state x of model α(2), and a confidence level value C3 is calculated based on C1 and C2. In this embodiment, when the weighted values ​​of C1 and C2 are less than or equal to a predetermined value, this is notified by a predetermined method, such as displaying a message on the screen, so that the confidence level can be evaluated in an easy-to-understand manner for the user. Furthermore, since the weight coefficients and thresholds can be adjusted according to the situation, flexible operation is possible.

[0086] As described above, according to the configuration shown in this embodiment, the reliability / confidence level of the input or output of model α(2) is calculated based on both an evaluation based on the input and output of model α(2) and an evaluation based on different internal calculation values. This allows for the evaluation of the reliability / confidence level of the input or output of model α(2) by utilizing the advantages of both methods while compensating for their respective disadvantages. Therefore, it is possible to ensure high reliability of model α(2) in a user-friendly and flexible manner. [Examples]

[0087] In this embodiment, A means γ1 for calculating a value C1 relating to the confidence level of at least the input u or output y of the model / AI / artificial intelligence α, based on at least the input u or output y of the model / AI / artificial intelligence α given the input u, A means γ2 for calculating a value C2 relating to the confidence level of at least the input u or the output y, based on the internal calculation value x of the model / AI / artificial intelligence α when the input u is given, means γ3 for calculating a value C3 relating to the reliability / confidence level of at least the input u or the output y, based at least C1 or C2, and It is equipped with.

[0088] Also, The aforementioned model / AI / artificial intelligence α is It is based on LLM, Transformer, or SSM (State Space Model) models. It is characterized by the following:

[0089] Also, The aforementioned input u is This prompt is generated based on information obtained from a database or the internet. It is characterized by the following:

[0090] Also, The calculation means γ1 is The confidence level value C1 is calculated based at least on the information (context) used when generating the input u, or on the input u (prompt) or output (output). It is characterized by the following:

[0091] Also, Means for extracting the internal calculation value x are provided It is characterized by the following:

[0092] Also, The aforementioned Model / AI / Artificial Intelligence α is a Transformer, The internal calculation value x extraction means includes at least, The embedding vector of the input or output of the transformer or Output of each intermediate layer or Output layer softmax value or The attention vector of each intermediate layer or the magnitude of that attention vector. This is a means of extracting one of the following. It is characterized by the following:

[0093] Also, The calculation means γ2 includes at least, Whether the internal calculation value x is within a predetermined range, or, The difference between the internal calculation value x and the predetermined range Based on this, the confidence / confidence value C2 is calculated. It is characterized by the following:

[0094] Also, When the difference between the aforementioned reliability / confidence value C1 and the aforementioned reliability / confidence value C2 is large, or, When the confidence / confidence value C1 is less than or equal to a predetermined value and the confidence / confidence value C2 is greater than or equal to a predetermined value, or When the confidence / confidence value C1 is greater than or equal to a predetermined value and the confidence / confidence value C2 is less than or equal to a predetermined value, To report It is characterized by the following:

[0095] The above configuration is shown below.

[0096] Figure 2 is a block diagram showing the overall configuration in this embodiment, but it is the same as in Embodiment 1, so it will not be described in detail.

[0097] Figure 3 is a system diagram of the device that implements the process shown in Figure 2, but it is the same as in Example 1, so it will not be described in detail.

[0098] <Input u creation section (Figure 4)> This process performs calculations on the input u. Specifically, this is shown in Figure 4, but since it is the same as in Example 1, it will not be described in detail.

[0099] <Model α (Figure 5)> This process calculates the output y. Specifically, it is shown in Figure 5, but since it is the same as in Example 1, it will not be described in detail.

[0100] <Calculation unit γ1 for reliability / confidence level C1 (Figure 6)> This process calculates the confidence / reliability value C1. Specifically, as shown in Figure 6, it is the same as in Example 1 and will not be described in detail.

[0101] <Internal calculation value x extraction means (Figure 7)> This process involves calculating (extracting) the internal calculation value X of model α(2). Specifically, this is shown in Figure 7, but since it is the same as in Example 1, it will not be described in detail.

[0102] <Calculation unit γ2 for reliability / confidence level C2 (Figure 8)> In this process, the difference D is calculated based on the internal state X and the region representative value z. Specifically, this is shown in Figure 8, but it is the same as in Example 1, so it will not be described in detail.

[0103] <Value C3 calculation unit γ3 related to reliability / confidence level (Figure 11)> This process involves calculating (extracting) the confidence / confidence value C3. Specifically, this is shown in Figure 11. Based on the confidence / confidence values ​​C1 and C2, it is calculated using the following formula.

[0104] When |C1-C2|≧K2c, let C3=1.

[0105] Otherwise, set C3 = 0.

[0106] Furthermore, K2c may be determined in advance according to some guidelines, or it may be made variable so that the user can determine it as appropriate during the operation of the device shown in this embodiment.

[0107] Alternatively, the process could be changed to "set C3 to 1 when C1 is less than or equal to a predetermined value and C2 is greater than or equal to a predetermined value, or when C1 is greater than or equal to a predetermined value and C2 is less than or equal to a predetermined value."

[0108] Alternatively, weighting can be applied by setting |C1-λ·C2|≧K2c or |λ·C1-C2|≧K2c.

[0109] Alternatively, C3 can be determined by individually evaluating C1 and C2, as shown below.

[0110] If C1 ≤ K4c or C2 ≤ K5c, then C3 = 1.

[0111] Otherwise, set C3 = 0.

[0112] In this configuration, a confidence level value C1 is calculated based on the input u and output y of model α(2), a confidence level value C2 is calculated based on the internal state x of model α(2), and a confidence level value C3 is calculated based on C1 and C2. In this embodiment, when the difference between C1 and C2 is large, that is, when there is a difference in judgment between the confidence level value C1 calculation unit γ1(3) and the confidence level value C2 calculation unit γ2(4), this is notified by a predetermined method, such as displaying it on the screen, so that it is possible to detect that there is some kind of problem (possibility of low confidence level) when the judgments of the two differ. In addition, since the threshold can be adjusted according to the situation, flexible operation is possible.

[0113] As described above, according to the configuration shown in this embodiment, the reliability / confidence level of the input or output of model α(2) is calculated based on both an evaluation based on the input and output of model α(2) and an evaluation based on different internal calculation values. This allows for the evaluation of the reliability / confidence level of the input or output of model α(2) by utilizing the advantages of both methods while compensating for their respective disadvantages. Therefore, it is possible to ensure high reliability of model α(2) in a user-friendly and flexible manner. [Examples]

[0114] In this embodiment, A means γ1 for calculating a value C1 relating to the confidence level of at least the input u or output y of the model / AI / artificial intelligence α, based on at least the input u or output y of the model / AI / artificial intelligence α given the input u, A means γ2 for calculating a value C2 relating to the confidence level of at least the input u or the output y, based on the internal calculation value x of the model / AI / artificial intelligence α when the input u is given, means γ3 for calculating a value C3 relating to the reliability / confidence level of at least the input u or the output y, based at least C1 or C2, and It is equipped with.

[0115] Also, The aforementioned model / AI / artificial intelligence α is It is based on LLM, Transformer, or SSM (State Space Model) models. It is characterized by the following:

[0116] Also, The aforementioned input u is This prompt is generated based on information obtained from a database or the internet. It is characterized by the following:

[0117] Also, The calculation means γ1 is The confidence level value C1 is calculated based at least on the information (context) used when generating the input u, or on the input u (prompt) or output (output). It is characterized by the following:

[0118] Also, Means for extracting the internal calculation value x are provided It is characterized by the following:

[0119] Also, The aforementioned Model / AI / Artificial Intelligence α is a Transformer, The internal calculation value x extraction means includes at least, The embedding vector of the input or output of the transformer or Output of each intermediate layer or Output layer softmax value or The attention vector of each intermediate layer or the magnitude of that attention vector. This is a means of extracting one of the following. It is characterized by the following:

[0120] Also, The calculation means γ2 includes at least, Whether the internal calculation value x is within a predetermined range, or, The difference between the internal calculation value x and the predetermined range Based on this, the confidence / confidence value C2 is calculated. It is characterized by the following:

[0121] Also, The calculation means γ3 is, Based on the aforementioned confidence / confidence values ​​C1 and C2, λ is a parameter C3 = λ·C1 + (1-λ)C2 The value calculated using this method is defined as the confidence / reliability value C3. It is characterized by the following:

[0122] Also, The calculation means γ2 is The system includes means for calculating a confidence / confidence value C2_k for each word or token in at least the input u or output y, When C2_k is less than or equal to a predetermined value, A new input u is calculated based on the word or token corresponding to C2_k. It is characterized by the following:

[0123] Also, Based on the new input u, the model α calculates a new output y. Along with, The calculation means γ1 calculates a value C1 related to reliability / confidence. It is characterized by the following:

[0124] The above configuration is shown below.

[0125] Figure 12 is a block diagram showing the overall configuration in this embodiment. The confidence / confidence value C2 calculation unit γ2(4) calculates the input u0, and the input u creation unit 6 uses this input u0 to create the input u. Everything else is the same as in Embodiment 1, so it will not be described in detail.

[0126] Figure 3 is a system diagram of the device that implements the process shown in Figure 2, but it is the same as in Example 1, so it will not be described in detail.

[0127] <Input u creation section (Figure 4)> This process performs calculations on the input u. Specifically, this is shown in Figure 4, but since it is the same as in Example 1, it will not be described in detail.

[0128] <Model α (Figure 5)> This process calculates the output y. Specifically, it is shown in Figure 5, but since it is the same as in Example 1, it will not be described in detail.

[0129] <Calculation unit γ1 for reliability / confidence level C1 (Figure 6)> This process calculates the confidence / reliability value C1. Specifically, as shown in Figure 6, it is the same as in Example 1 and will not be described in detail.

[0130] <Internal calculation value x extraction means (Figure 7)> This process involves calculating (extracting) the internal calculation value X of model α(2). Specifically, this is shown in Figure 7, but since it is the same as in Example 1, it will not be described in detail.

[0131] <Value C2 calculation unit γ2 related to reliability / confidence level (Figure 13)> In this process, the difference D is calculated based on the internal state X and the region representative value z. Specifically, the following calculations are performed as shown in Figure 13.

[0132] - Dk=||z_in-xk|| 2 (k:1,2,···,n) Perform the calculation.

[0133] - Let D be the average value of Dk.

[0134] The regional representative value z is a representative value of the region with high confidence / reliability. For example, it is conceivable to use a value calculated by some method as the representative value of the region where the data used to train model α(2) exists. This method could be the center vector of the training data, or it could be the center vector of each cluster after clustering the training data. Various methods are possible.

[0135] Furthermore, Dk can be any value that represents the distance between vectors, such as the cosine distance or Mahalanobis distance between z_in and xk.

[0136] Additionally, D can be a value that represents Dk, such as the maximum value of Dk.

[0137] Next, based on the difference D, the confidence / confidence value C2 is calculated via the function f(D). As shown in Figure 8, the function f(D) has the characteristic that C2 decreases as the value of D increases. The function f(D) may be set empirically or according to the required specifications using a table lookup format, or a suitable mathematical formula may be used if one is obtained.

[0138] Furthermore, based on the difference Dk, tokens corresponding to k such that Dk ≥ K are converted into words, and these words are used as input u.

[0139] <Value C3 calculation unit γ3 related to reliability / confidence level (Figure 9)> This process involves calculating (extracting) the confidence / confidence value C3. Specifically, this is shown in Figure 9, but since it is the same as in Example 1, it will not be described in detail.

[0140] In this configuration, a confidence / confidence value C1 is calculated based on the input u and output y of model α(2), a confidence / confidence value C2 is calculated based on the internal state x of model α(2), and a confidence / confidence value C3 is calculated based on C1 and C2. In this embodiment, a token (word) with a low confidence / confidence detected by the confidence / confidence value C2 calculation unit γ2(4) is used as a new input u0, an input u is created from input u0 using RAG, the confidence / confidence is calculated again using the confidence / confidence value C1 calculation unit γ1(3) and the confidence / confidence value C2 calculation unit γ2(4) based on this input u, and then the confidence / confidence value C3 is calculated using C1 and C3 by the confidence / confidence value C3 calculation unit γ3(5). This allows for a focused re-evaluation of tokens (words) with low reliability / confidence levels, which is expected to increase their overall trustworthiness.

[0141] As described above, according to the configuration shown in this embodiment, the reliability / confidence level of the input or output of model α(2) is calculated based on both an evaluation based on the input and output of model α(2) and an evaluation based on different internal calculation values. This allows the reliability / confidence level of the input or output of model α(2) to be evaluated by utilizing the advantages of both methods while compensating for their respective disadvantages. Therefore, it becomes possible to ensure even higher reliability for model α(2). [Examples]

[0142] In this embodiment, A means γ1 for calculating a value C1 relating to the confidence level of at least the input u or output y of the model / AI / artificial intelligence α, based on at least the input u or output y of the model / AI / artificial intelligence α given the input u, A means γ2 for calculating a value C2 relating to the confidence level of at least the input u or the output y, based on the internal calculation value x of the model / AI / artificial intelligence α when the input u is given, means γ3 for calculating a value C3 relating to the reliability / confidence level of at least the input u or the output y, based at least C1 or C2, and It is equipped with.

[0143] Also, The aforementioned model / AI / artificial intelligence α is It is based on LLM, Transformer, or SSM (State Space Model) models. It is characterized by the following:

[0144] Also, The aforementioned input u is This prompt is generated based on information obtained from a database or the internet. It is characterized by the following:

[0145] Also, The calculation means γ1 is The confidence level value C1 is calculated based at least on the information (context) used when generating the input u, or on the input u (prompt) or output (output). It is characterized by the following:

[0146] Also, Means for extracting the internal calculation value x are provided It is characterized by the following:

[0147] Also, The aforementioned Model / AI / Artificial Intelligence α is a Transformer, The internal calculation value x extraction means includes at least, The embedding vector of the input or output of the transformer or Output of each intermediate layer or Output layer softmax value or The attention vector of each intermediate layer or the magnitude of that attention vector. This is a means of extracting one of the following. It is characterized by the following:

[0148] Also, The calculation means γ2 includes at least, Whether the internal calculation value x is within a predetermined range, or, The difference between the internal calculation value x and the predetermined range Based on this, the confidence / confidence value C2 is calculated. It is characterized by the following:

[0149] Also, The calculation means γ3 is, Based on the aforementioned confidence / confidence values ​​C1 and C2, λ is a parameter C3 = λ·C1 + (1-λ)C2 The value calculated using this method is defined as the confidence / reliability value C3. It is characterized by the following:

[0150] Also, Display at least one confidence / reliability value C1, or one confidence / reliability value C2, or one confidence / reliability value C3. It is characterized by the following:

[0151] Also, Display the confidence / reliability value C1 or C2, and the confidence / reliability value C3. It is characterized by the following:

[0152] The above configuration is shown below.

[0153] Figure 14 is a block diagram showing the overall configuration in this embodiment. The display target calculation means 8 performs display processing for the reliability / confidence value C1, or the reliability / confidence value C2, or the reliability / confidence value C3. Everything else is the same as in Embodiment 1 and will not be described in detail.

[0154] Figure 3 is a system diagram of the device that implements the process shown in Figure 2, but it is the same as in Example 1, so it will not be described in detail.

[0155] <Input u creation section (Figure 4)> This process performs calculations on the input u. Specifically, this is shown in Figure 4, but since it is the same as in Example 1, it will not be described in detail.

[0156] <Model α (Figure 5)> This process calculates the output y. Specifically, it is shown in Figure 5, but since it is the same as in Example 1, it will not be described in detail.

[0157] <Calculation unit γ1 for reliability / confidence level C1 (Figure 6)> This process calculates the confidence / reliability value C1. Specifically, as shown in Figure 6, it is the same as in Example 1 and will not be described in detail.

[0158] <Internal calculation value x extraction means (Figure 7)> This process involves calculating (extracting) the internal calculation value X of model α(2). Specifically, this is shown in Figure 7, but since it is the same as in Example 1, it will not be described in detail.

[0159] <Calculation unit γ2 for reliability / confidence level C2 (Figure 8)> In this process, the difference D is calculated based on the internal state X and the region representative value z. Specifically, this is shown in Figure 8, but it is the same as in Example 1, so it will not be described in detail.

[0160] <Value C3 calculation unit γ3 related to reliability / confidence level (Figure 9)> This process involves calculating (extracting) the confidence / confidence value C3. Specifically, this is shown in Figure 9, but since it is the same as in Example 1, it will not be described in detail.

[0161] <Display Countermeasure Calculation Means (Figure 15)> This process performs calculations for the information to be displayed. Specifically, this is shown in Figure 15. The numerical values ​​of confidence / reliability C1, confidence / reliability C2, and confidence / reliability C3 are displayed, or shown using a gauge or similar.

[0162] In this configuration, a confidence level value C1 is calculated based on the input u and output y of model α(2), a confidence level value C2 is calculated based on the internal state x of model α(2), and a confidence level value C3 is calculated based on C1 and C2. Furthermore, C1, C2, and C3 are visually displayed on the screen.

[0163] As described above, according to the configuration shown in this embodiment, the reliability / confidence level of the input or output of model α(2) is calculated based on both an evaluation based on the input and output of model α(2) and an evaluation based on different internal calculation values. This utilizes the advantages of both methods while compensating for their respective disadvantages to evaluate the reliability / confidence level of the input or output of model α(2). Furthermore, each reliability / confidence level value is visually displayed on the screen, making it easy for the user to understand.

[0164] Therefore, it ensures high reliability for model α(2) and allows users to understand the reliability / confidence level in an easy-to-understand manner. [Examples]

[0165] In this embodiment, A means γ1 for calculating a value C1 relating to the confidence level of at least the input u or output y of the model / AI / artificial intelligence α, based on at least the input u or output y of the model / AI / artificial intelligence α given the input u, means γ2 for calculating a value C2 related to at least the reliability / confidence of the input u or the output y based on the internal calculation value x of the model / AI / artificial intelligence α at the time of the input u; means γ3 for calculating a value C3 related to at least the reliability / confidence of the input u or the output y based on at least the C1 or the C2; It is provided with.

[0166] Also, the model / AI / artificial intelligence α is an LLM or a Transformer or a state space model (SSM)-based model characterized by this.

[0167] Also, the input u is a prompt generated based on information obtained by referring to a database or the Internet characterized by this.

[0168] Also, the calculating means γ1 calculates the value C1 related to the reliability / confidence based on at least the information (context) used at the time of generating the input u or the input u (prompt) or the output (output) characterized by this.

[0169] Also, characterized by being provided with an internal calculation value x extraction means characterized by this.

[0170] Also, the model / AI / artificial intelligence α is Mamba, the internal calculation value x extraction means includes at least h(t) (implicit latent state), or the values of each element of the matrices A￣ (the upper line of A), B￣ (the upper line of B), and C, or Δt (step size) A means of extracting It is characterized by the following:

[0171] Also, The calculation means γ2 includes at least, Whether the internal calculation value x is within a predetermined range, or, The difference between the internal calculation value x and the predetermined range Based on this, the confidence / confidence value C2 is calculated. It is characterized by the following:

[0172] Also, The calculation means γ3 is, Based on the aforementioned confidence / confidence values ​​C1 and C2, λ is a parameter C3 = λ·C1 + (1-λ)C2 The value calculated using this method is defined as the confidence / reliability value C3. It is characterized by the following:

[0173] The above configuration is shown below.

[0174] Figure 2 is a block diagram showing the overall configuration in this embodiment, but it is the same as in Embodiment 1, so it will not be described in detail.

[0175] Figure 3 is a system diagram of the device that implements the process shown in Figure 2, but it is the same as in Example 1, so it will not be described in detail.

[0176] <Input u creation section (Figure 4)> This process performs calculations on the input u. Specifically, this is shown in Figure 4, but since it is the same as in Example 1, it will not be described in detail.

[0177] <Model α (Figures 16 and 17)> In this process, the operation of the output y is performed. Specifically, it is shown in FIG. 16 or FIG. 17. In FIG. 16, the model α(2) is Mamba. In FIG. 17, the model α(2) is Mamba2. Since there are many documents about the Mamba series, it will not be elaborated here. Note that the embedding process, regression part, etc. are omitted because they are not related to this embodiment.

[0178] In both Mamba and Mamba2, the input u (here it is a sentence / prompt) is converted into n tokens (h1, h2, ··· hn) through a tokenizer. Based on the tokens, each process with the SSM (State Space Model) of each layer from the first layer to the s-th layer as the core is performed, and the probability Pk (k: 1, 2, ··· f) is calculated based on the output of the s-th layer. Here, f is the number of token candidates. The token number j to be output from P1 to Pf is determined, and hj is output by the tokenizer based on the token number j, and hj is taken as the output y. This is recursively repeated, and the output sentence is constructed by concatenating the output y.

[0179] <Value C1 calculation unit γ1 related to reliability / confidence (FIG. 6)> In this process, the value C1 related to reliability / confidence is calculated. Specifically, it is shown in FIG. 6, but it is the same as that in Embodiment 1, so it will not be elaborated here.

[0180] <Internal operation value x extraction means (FIGS. 18 and 19)> In this process, the operation (extraction) of the internal operation value X of the model α(2) is performed. Specifically, it is shown in FIGS. 18 and 19. In FIG. 16, the model α(2) is Mamba. In FIG. 17, the model α(2) is Mamba2.

[0181] As described in the explanation of Model α(2) (Figures 16 and 17), an output y is generated from an input u. The internal calculation values ​​X: x1, x2, ..., xn of the r-th layer at this time are extracted. Here, in this embodiment, xk is the internal calculation value corresponding to the k-th token in the input u. Alternatively, the internal calculation values ​​(xn+1, xn+2, ..., xn+m) when the output y is regressed and a new output y is calculated may be targeted. X may be a single value (x1) or multiple values ​​(x1, x2, ..., xn(, xn+1, xn+2, ..., x+m)). It is best to decide the target r-th layer and which internal calculation values ​​of the r-th layer to choose based on experience, but if there are other guidelines, they may be followed. Multiple layers may also be targeted. It has been pointed out that as the layers get deeper (as you move further away from the input), the processing shifts from global to detailed, so such insights may be taken into consideration. Furthermore, the internal calculation values ​​in both Mamba and Mamba2 are either h(t) (implicit latent state) in SSM, the values ​​of each element in matrices A￣ (upper line of A), B￣ (upper line of B), and C, or Δt (step size). Alternatively, a combination of these internal calculation values ​​may be used.

[0182] <Calculation unit γ2 for reliability / confidence level C2 (Figure 8)> In this process, the difference D is calculated based on the internal state X and the region representative value z. Specifically, this is shown in Figure 8, but it is the same as in Example 1, so it will not be described in detail.

[0183] <Value C3 calculation unit γ3 related to reliability / confidence level (Figure 9)> This process involves calculating (extracting) the confidence / confidence value C3. Specifically, this is shown in Figure 9, but since it is the same as in Example 1, it will not be described in detail.

[0184] In this configuration, a confidence / reliability value C1 of model α(2) is calculated based on the input u and output y of model α(2), a confidence / reliability value C2 of model α(2) is calculated based on the internal state x of model α(2), and a confidence / reliability value C3 of model α(2) is calculated based on C1 and C2. Furthermore, model α(2) is from the Mamba series. The Mamba series is a model that is said to be capable of faster processing than transformers.

[0185] As described above, according to the configuration shown in this embodiment, the confidence level of the input or output of model α(2) is calculated based on both an evaluation based on the input and output of model α(2) and an evaluation based on different internal calculation values. This allows for the evaluation of the confidence level of the input or output of model α(2) by utilizing the advantages of both methods while compensating for their respective disadvantages. Furthermore, using Mamba in model α(2) enables faster processing.

[0186] Therefore, it becomes possible to ensure high reliability for model α(2) while also speeding up processing.

[0187] Although several embodiments have been described above, these are merely illustrative examples for explaining the present invention and are not intended to limit the scope of the invention to these embodiments only. The present invention can be carried out in various other forms.

[0188] Furthermore, in the above explanation, "RAM24" is one or more memory devices (hereinafter also simply referred to as "memory") which are an example of one or more storage devices, and is typically a main memory device. At least one memory device in the memory may be a volatile memory device or a non-volatile memory device.

[0189] Furthermore, in the above explanation, "storage device 21" may be one or more persistent storage devices, which are examples of one or more storage devices. Persistent storage devices are typically non-volatile storage devices (e.g., auxiliary storage devices), and specifically, they may be, for example, HDDs (Hard Disk Drives), SSDs (Solid State Drives), NVMe (Non-Volatile Memory Express) drives, or SCMs (Storage Class Memory).

[0190] Furthermore, in the above explanation, "storage device 21" may include memory.

[0191] Furthermore, in the above explanation, "CPU22" may refer to one or more processor devices (hereinafter also simply as "processors"). At least one processor device may typically be a microprocessor device such as a CPU (Central Processing Unit)22, but it may also be other types of processor devices such as a GPU (Graphics Processing Unit). At least one processor device may be single-core or multi-core. At least one processor device may be a processor core. At least one processor device may be a broad-sense processor device such as a circuit that is a collection of gate arrays defined by a hardware description language that performs some or all of the processing (e.g., FPGA (Field-Programmable Gate Array), CPLD (Complex Programmable Logic Device), or ASIC (Application Specific Integrated Circuit)).

[0192] Furthermore, in the above explanation, functions may be described using expressions such as "xxx means" or "xxx part," but a function may be realized by the execution of one or more computer programs by a processor, by one or more hardware circuits (e.g., FPGA or ASIC), or by a combination thereof. When a function is realized by the execution of a program by a processor, the defined processing is carried out using the storage device 21 and / or interface device as appropriate, so the function may be at least a part of the processor. The processing described with a function as the subject may also be processing performed by the processor or the reliability evaluation device 1 having that processor. The program may be installed from a program source. The program source may be, for example, a program distribution computer or a computer-readable recording medium (e.g., a non-temporary recording medium). The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.

[0193] Furthermore, in the above explanation, the process is sometimes described using "program" as the subject. However, the process described using "program" as the subject may also be a process performed by the processor or the reliability evaluation device 1 having that processor. Also, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

[0194] Furthermore, in the above explanation, "reliability evaluation device 1" may be a system composed of one or more physical computers (for example, an on-premise system), or a system implemented on a group of physical computing resources (for example, a cloud infrastructure) (for example, a cloud computing system). "Displaying" information by reliability evaluation device 1 may mean displaying the information on a display device owned by a computer, or it may mean that the computer transmits the information to a display computer (in the latter case, the display computer displays the information). [Explanation of symbols]

[0195] 1. Reliability evaluation device Model α 3. Calculation unit γ1 for the value related to reliability / confidence level C1 4. Value calculation unit γ2 related to reliability / confidence level C2 5. Value related to reliability / confidence level: C3 calculation unit γ3 6. Input u creation section 7. Means for extracting the internal calculation value x 8. Display target calculation means 21 Storage device for reliability evaluation equipment 22. CPU of the reliability evaluation device 23. ROM of a reliability evaluation device 24. RAM of the reliability evaluation device 25. Data bus of reliability evaluation device 26 Input circuit of reliability evaluation device 27 Input / Output Ports of Reliability Evaluation Device 28 Output circuit of reliability evaluation device

Claims

1. A reliability evaluation device for evaluating the reliability of a model, A computer having at least a processor and memory, A means γ1 for calculating a value C1 relating to the confidence level or confidence level of at least the input u or output y of model α, which is at least one of a model, AI, or artificial intelligence, based on the input u or the output y of model α for the input u, A means γ2 for calculating a value C2 relating to the reliability or confidence level of at least the input u or the output y, based on the internal calculation value x of the model α when the input u is given, means γ3 for calculating a value C3 relating to the reliability or confidence level of at least the input u or the output y, based at least on C1 or C2, A reliability evaluation device equipped with the following features.

2. In claim 1, The aforementioned model α is, It is an LLM, Transformer, or SSM (State Space Model) based model. A reliability evaluation device characterized by the following features.

3. In claim 1, The aforementioned input u is This prompt is generated based on information obtained from a database or the internet. A reliability evaluation device characterized by the following features.

4. In claim 1, The calculation means γ1 is The confidence level or confidence value C1 is calculated based at least on the information used when generating the input u, or on the input u or output. A reliability evaluation device characterized by the following features.

5. In claim 1, Equipped with means for extracting internal calculation value x A reliability evaluation device characterized by the following features.

6. In claim 1, The aforementioned Model α is a Transformer, The internal calculation value x extraction means includes at least, The embedding vector of the input or output of the transformer or Output of each intermediate layer or Output layer softmax value or The attention vector of each intermediate layer or the magnitude of that attention vector. This is a means of extracting one of the following. A reliability evaluation device characterized by the following features.

7. In claim 1, The aforementioned model α is Mamba, The internal calculation value x extraction means includes at least, h(t) (implicit latent state), or, The values ​​of each element in matrices A￣ (upper line of A), B￣ (upper line of B), and C, or, Δt (step size) A means of extracting A reliability evaluation device characterized by the following features.

8. In claim 1, The calculation means γ2 includes at least, Whether the internal calculation value x is within a predetermined range, or, The difference between the internal calculation value x and the predetermined range Based on this, the value C2 relating to the confidence level or level of confidence is calculated. A reliability evaluation device characterized by the following features.

9. In claim 1, The calculation means γ3 is Based on the aforementioned reliability or confidence value C1 and the aforementioned reliability or confidence value C2, λ is a parameter C3=λ・C1+(1-λ)C2 The value calculated using this method is defined as the confidence level or level of confidence value C3. A reliability evaluation device characterized by the following features.

10. In claim 1, The calculation means γ3 is Based on the aforementioned reliability or confidence value C1 and the aforementioned reliability or confidence value C2, C_tmp=λ・C1+(1−λ)C2 Calculate, When C_tmp is less than or equal to a predetermined value Kc, a notification is issued. A reliability evaluation device characterized by the following features.

11. In claim 1, When the difference between the aforementioned reliability or confidence value C1 and the aforementioned reliability or confidence value C2 is large, or, When the value C1 relating to the reliability or confidence level is less than or equal to a predetermined value and the value C2 relating to the reliability or confidence level is greater than or equal to a predetermined value, or When the value C1 relating to the reliability or confidence level is greater than or equal to a predetermined value and the value C2 relating to the reliability or confidence level is less than or equal to a predetermined value, To report A reliability evaluation device characterized by the following features.

12. In claim 1, The calculation means γ2 is The system includes means for calculating a confidence level or confidence value C2_k for each word or token in at least the input u or output y, When C2_k is less than or equal to a predetermined value, A new input u is calculated based on the word or token corresponding to C2_k. A reliability evaluation device characterized by the following features.

13. In claim 12, Based on the new input u, the model α calculates a new output y. Along with, The calculation means γ1 calculates a value C1 related to reliability or confidence level. A reliability evaluation device characterized by the following features.

14. In claim 1, Display at least one confidence level value C1, or one confidence level value C2, or one confidence level value C3. A reliability evaluation device characterized by the following features.

15. In claim 14, The values ​​C1 or C2 relating to the reliability or confidence level, and C3 relating to the reliability or confidence level are displayed. A reliability evaluation device characterized by the following features.

16. A reliability evaluation method for evaluating the reliability of a model, In a computer having at least a processor and memory, The aforementioned processor, By means γ1, based on the input u to model α, which is at least one of a model, AI, or artificial intelligence, or the output y of model α when the input u is given, a value C1 relating to the confidence level or confidence level of at least the input u or the output y is calculated. By means γ2, based on the internal calculation value x of model α when the input u is given, a value C2 relating to the reliability or confidence level of at least the input u or the output y is calculated, By means γ3, a value C3 relating to the reliability or confidence level of at least the input u or the output y is calculated based on at least C1 or C2. Reliability evaluation methods.

Images