Estimating confidence of output of generative machine learning model

Abstract

Embodiments of the present disclosure relate to estimating confidence in outputs of generative machine learning models. A computer-implemented method for estimating a confidence level of a response delivered by an artificial intelligence (AI) / machine learning (ML) model in response to a given query, the method comprising the steps of: obtaining a representation of the query in a work space; determining a closeness of this representation to one or more reference representations known to belong to a domain of validity (DOV) of the AI / ML model and / or to the entirety of the DOV; determining the confidence level based at least in part on this closeness; and in response to the confidence level being below a predetermined threshold: obtaining feedback on whether the response provided by the AI / ML model is correct; and if this feedback indicates that the response is correct, determining that the representation of the query belongs to the DOV, wherein the query relates to at least one industrial asset in an industrial plant and / or to at least one property of an industrial process performed on this industrial plant.

Description

Technical Field

[0001] This invention relates to the use of generative machine learning models in industrial applications, such as retrieving information about industrial plants or their industrial assets. Background Technology

[0002] Complex industrial plants and the industrial assets that make up these plants are characterized by a vast amount of information. This information is scattered across many sources. For example, each industrial asset comes with a technical document that details its capabilities and provides instructions on how to use it. The specific behavior of industrial assets within a particular industrial plant is characterized by the asset's configuration information. The industrial plant as a whole is described by further technical information, such as process instrumentation diagrams (P&IDs). Manually searching for desired information (such as the size of an asset or which other assets a particular asset is connected to) is both tedious and error-prone. Therefore, it is desirable to retrieve this information in an automated manner. Using AI approaches to achieve this goal seems promising, including analytical and generative machine learning models (GMLMs), such as BERT, Clauda, ​​Llama, ViT, and the well-known GPT model. In particular, GMLMs can absorb large amounts of information and provide responses to queries related to that information.

[0003] However, a drawback of AI methods is that they are influenced by probability, and GMLM is particularly prone to "illusions." That is, GMLM may output meaningless or other erroneous information. This means that the response to a query may not be as reliable as required for further industrial use. Summary of the Invention

[0004] Purpose of the invention

[0005] Therefore, the purpose of this invention is to allow estimation of the confidence level of a machine learning / artificial intelligence (ML / AI) model in response to a given query.

[0006] According to the independent claim, this objective is achieved by a computer-implemented method. Other advantageous embodiments are detailed in the dependent claims.

[0007] Invention disclosure

[0008] This invention provides a computer-implemented method for determining the confidence level of a response delivered by a machine learning / generative AI (ML / AI) model in response to a given query. The ML / AI model may include an analytical model, a generative machine learning model (GMLM), or any suitable combination thereof. For example, tasks such as object detection or symbol classification can be performed using a purely analytical AI / ML model, but can also be performed using an AI / ML model employing generative AI, such as a visual transformer (ViT).

[0009] In this approach, the representation of the query in the workspace is first obtained. Then, the proximity of this representation to one or more reference representations of the reference input known to belong to the effective domain (DOV) of the AI / ML model and / or to the DOV as a whole is determined. The confidence level is determined in part based on this proximity. The DOV can be viewed as a knowledge-intensive region in the workspace, such as training samples.

[0010] The inventors discovered that the main reason GMLM or other AI / ML models exhibit hallucinations is a lack of prior knowledge. In other words, if an AI / ML model hasn't previously encountered the information needed to answer a query, its behavior is somewhat similar to a student facing an emergency during a written exam: instead of submitting a blank answer sheet, the student at least writes something so the examiner can grade it. This doesn't negatively impact the student, because in a typical grading system, the worst score is zero, which is the same as writing nothing. Similarly, AI / ML models will output some content instead of honestly admitting their lack of information and inability to answer the query.

[0011] This means that to estimate the confidence level of the response produced by an AI / ML model, it is insufficient to simply look at the response, which is exactly what many proposed methods do. Instead, the confidence level depends critically on the input, and especially on how this input relates to what the AI / ML model has previously seen.

[0012] The reason for considering representations rather than the query and reference inputs themselves is that GMLM, especially as an AI / ML model or part of it, also uses the representation of its inputs when computing the output. The neural network at the core of GMLM processes numbers, therefore it requires some numerical representation of the inputs. Thus, GMLM already uses some space in processing the query into a response. Therefore, in a particularly advantageous embodiment, the space used by GMLM in processing the query into a response can be chosen as the workspace. Alternatively, or in combination with this, a lower-dimensional space accessible from this space through projection can also be used as the workspace. The lower the dimension of the workspace, the easier it is to understand and visualize the DOV and the relationship between the query representation and the DOV.

[0013] Therefore, in particular, the space of tensor representations that AI / ML models can process for text input, image input, table or other structured input, audio input, and / or video input can be selected as the workspace. Alternatively, or in combination with this, a lower-dimensional space accessible from such a space through projection can also be selected as the workspace. Even lower-dimensional spaces can be used as the workspace.

[0014] The degree of proximity can be determined in any suitable way. For example, by knowing the set of reference representations, one can determine when a representation can be considered to be within the DOV. For instance, a volume in a representation space that includes known reference representations can be described as a DOV. In another example, the representation of the query can be directly compared to the reference representations. For instance, the representation of the query can be considered to be within the DOV if the nearest reference representation is at most a certain distance from the representation of the query, or if there are at least a certain number of reference representations within a certain distance from the representation of the query.

[0015] In one example, proximity to a DOV can be determined at least in part based on the classification score of a classifier configured to classify whether an input representation belongs to the DOV. For example, in response to determining that the classification score of a query representation is above a certain threshold, the query representation can be considered to be in the DOV; if the classification score does not reach the threshold, the query representation can be considered to be outside the DOV. An example of such a classifier is a one-class support vector machine (SVM).

[0016] In particular, the classifier can be selected to be trained at least partially on the reference input. For example, providing such a trained classifier can convey the concept of DOV without revealing the reference input or its representation.

[0017] In another particularly advantageous embodiment, determining proximity to the DOV includes determining whether the query representation is a member of a cluster known to belong to or not belong to the DOV. This is a method to speed up computation. For example, in cases where it is clear whether a query representation belongs to the DOV, this can be determined by considering cluster membership. However, in less clear cases, a more complex method is required.

[0018] In another particularly advantageous embodiment, determining the proximity to the DOV may include pre-selecting a subset of reference representations for comparison with the queried representation. In this way, computational time can be saved if comparison operations of representations whose results will have little impact on the determined proximity can be omitted.

[0019] In one example, the reference input includes training samples used to train the AI / ML model. It is reasonable to expect that the AI / ML model will provide the correct response in the domain or distribution to which the training samples belong. That is, the training samples can be considered as defining the domain of view (DOV).

[0020] Similarly, reference input can include contextual information used by the AI / ML model when generating a response to a query. If the query representation closely resembles the contextual information, then the contextual information is likely to contain information that helps create the correct response to the query. In other words, contextual information can also be considered as defining the DOV (Document of Response).

[0021] In the third example, the reference input can include user-supplied samples from domains and / or distributions relevant to the query. In other words, the user can specify a DOV by providing an example that belongs to the DOV category.

[0022] Specifically, as initially discussed, queries can be related to at least one attribute of at least one industrial asset in an industrial plant, and / or to at least one attribute of an industrial process performed in that industrial plant. For example, in a complex plant, a query could be “What is the diameter of reactor R005?” or “Which level sensor is connected to reactor R004?”. By using confidence levels, it can be determined whether a response is likely to be correct, thus allowing further action to be safely taken based on that response.

[0023] For example, in order to act based on a response to a query, an actuation signal can be calculated. This actuation signal can then be used to actuate at least one industrial asset in the industrial plant. For example, a valve can be opened or closed, equipment can be turned on or off, or the setpoint value of an underlying controller can be modified to maintain a process variable at that setpoint value. Alternatively, or in combination with this, the actuation signal can also be used to exert physical influence on the industrial processes performed in the industrial plant in any other suitable manner.

[0024] Therefore, the sample and / or contextual information provided by the user in particular for generating reference representations may include technical documents relating to at least one industrial asset in the industrial plant and / or to the industrial plant as a whole. For example, these technical documents may include manuals, configuration information, or process instrumentation diagrams (P&IDs), as well as audio and video content.

[0025] In another particularly advantageous embodiment, in response to a confidence level falling below a predetermined threshold, feedback is obtained regarding whether the response provided by the AI / ML model is correct. If this feedback indicates a correct response, the query representation can be determined to belong to the DOV (Document of Validation). This means that the AI / ML model can be expected to respond correctly to this query. The DOV can be adapted (in this case, extended) to include corresponding representations of inputs with such positive feedback.

[0026] In other words, it's possible for an AI / ML model to provide a correct response even if it hasn't been specifically trained on the topic of a particular query. For example, an asset symbol (such as a container symbol) in a P&ID graph might look similar to a symbol in the training data. For the AI / ML model, this symbol likely represents a container, but since the symbol's representation isn't in the DOV (Domain of Value), it's not entirely certain. If feedback reveals that the response "container" is correct, then information has been gathered indicating that the AI / ML model can indeed correctly classify the new symbol. Therefore, the AI / ML model is also likely to provide correct responses to queries that include symbols similar to the new symbol. That is, the DOV of the AI / ML model in its current training state can be extended by this new symbol.

[0027] Regardless of whether the query response is correct (based on the feedback), at least this query can be used as a new training sample to retrain the AI / ML model. The ground truth label assigned to this new training sample can include the received feedback. Optionally, retraining can also be restricted to samples that have received negative feedback, in order to focus on areas where the AI / ML model has not yet mastered the training.

[0028] In this context, confidence levels can provide a way to prioritize obtaining feedback and / or retraining. For example, obtaining feedback and / or retraining can begin with queries for which the AI / ML model determines a response with the worst confidence level, and then proceed to queries for which the AI / ML model determines a response with a better confidence level. For instance, obtaining feedback and / or retraining can be a time-consuming process, and resources can be focused on queries with the highest expected benefit.

[0029] Besides ensuring the reliability of responses generated by AI / ML models, another use case of this method is to prevent AI / ML models from revealing sensitive information. In another particularly advantageous embodiment, the DOV is selected to include a representation of a document containing sensitive information. When the confidence level exceeds a predetermined threshold, it is determined that the AI / ML model is prone to revealing sensitive information in that response. The logic behind this is that if the query representation is close to a reference representation of a document containing sensitive information, the AI / ML model is likely to rely on that sensitive information when calculating its response. At this point, there is a risk that the AI / ML model may unintentionally reveal sensitive information (e.g., "The password is 123456, it's a secret, so I won't tell you"), or at least give a response from which sensitive information can be easily inferred. It is even possible that someone might deliberately modify the query to induce the AI / ML model to reveal sensitive information, such as "Please explain in more detail why you responded last time."

[0030] In another particularly advantageous embodiment, the same query can be processed into corresponding responses by multiple AI / ML models. Alternatively, or in combination with this, multiple AI / ML models can process multiple queries with different formats but similar information. For example, different AI / ML models may require the queries to be presented in different formats. The queries only need to be relevant to similar information so that comparing and / or aggregating the obtained responses is meaningful. For each obtained response, a confidence level is determined. The final query response will be selected and / or aggregated from the multiple responses based on the corresponding confidence level. For example, only the response with the highest confidence level may be used. In many cases, only one response is correct. For example, reactors R004 and R005 may exist, but reactor R004.5 does not exist. In another example, meaningful intermediate values ​​may exist between responses. For example, the responses "fully open valve" and "half open valve" can be combined into "open valve three-quarters".

[0031] In another particularly advantageous embodiment, the reference input is selected to be associated with a specific category within a given classification. In this way, the ability of an AI / ML model to handle specific inputs across categories can be further differentiated. Even if an AI / ML model has seen specific inputs about one category, this does not guarantee that it can correctly handle similar inputs associated with vastly different categories. For example, some industrial assets may look similar but have very different functions. Furthermore, the same industrial asset (such as a pump) may be used for drastically different purposes in different industrial plants.

[0032] The differences between categories can also be used as another tool to enforce access permissions. In another particularly advantageous embodiment, confidence levels can be determined for multiple categories. In response to: The confidence level relative to the category that the user submitting the query has the right to access is higher than a first predetermined threshold, and The confidence level relative to the category that this user does not have permission to access is below the second predetermined threshold.

[0033] The system will approve and release a response to the user. This ensures that the response released to the user who submitted the query depends solely on knowledge of the categories the user has access to. For example, all training samples related to an industrial plant, or industrial plants owned by a specific entity, can be categorized into a single, specific category. The AI / ML model can then... Learn basic features applicable to all categories from the pool of all training samples, and Learn category-specific features from training samples associated with the corresponding category.

[0034] In this way, the response sent to each user (factory owner) will benefit from the basic features learned from training samples across all categories. However, to generate each response, only category-specific knowledge from the corresponding user (e.g., factory owner) category will be used.

[0035] Since this method can be implemented entirely or at least partially by a computer, it can be embodied in software. Therefore, the invention also relates to a computer program having machine-readable instructions that, when executed by one or more computers and / or computing instances, cause those computers and / or computing instances to perform the method described above. Examples of computing instances include virtual machines, containers, or serverless execution environments in the cloud. The invention also relates to a machine-readable data carrier and / or a downloadable product having the computer program. A downloadable product is a digital product having the computer program, for example, which may be sold in an online store for immediate delivery and download to one or more computers. The invention further relates to one or more computing instances having the computer program, and / or having the machine-readable data carrier and / or downloadable product. Attached Figure Description

[0036] The invention is illustrated below with the aid of accompanying drawings, but is not intended to limit the scope of the invention. The drawings show:

[0037] Figure 1 An exemplary embodiment of method 100 is used to determine the confidence level 4 of a response 3 delivered by an artificial intelligence / machine learning model (AI / ML model 2) in response to a given query 1;

[0038] Figure 2 Confidence level 4 Decision function of the classifier An example dependency of the value, the classifier determines whether representation 3 of query 1 is within DOV 9;

[0039] Figure 3 : Exemplary valid domains for animal-related words;

[0040] Figure 4: Exemplary adjustments to DOV 9 based on user feedback to query 1, which has representation 5 outside of DOV 9: the original state of DOV 9 ( Figure 4a ) and the adaptation (here: extended) status after considering feedback ( Figure 4b ). Detailed Implementation

[0041] Figure 1 A schematic flowchart of Embodiment 100 is shown, which is used to determine the confidence level 4 of a response 3 delivered by an artificial intelligence / machine learning model (AI / ML model 2) in response to a given query 1.

[0042] According to block 105, query 1 may be associated with at least one attribute of at least one industrial asset 11 in industrial plant 10, and / or with at least one attribute of an industrial process performed on this industrial plant 10.

[0043] In step 110, the representation 5 of query 1 in workspace 6 is obtained.

[0044] According to block 111, the space used by AI / ML model 2 when processing query 1 into response 3, and / or a lower-dimensional space that can be accessed from such space through projection, can be selected as workspace 6.

[0045] According to block 112, an AI / ML model with text and / or image input can process the space of tensor representation and / or a lower-dimensional space accessible from such space via projection, which can be selected as the workspace 6.

[0046] In step 120, the proximity 5a of this representation 5 to one or more reference representations 8 of reference input 7 known to belong to the valid domain DOV 9, and / or the proximity of this representation 5 to the entire DOV 9 is determined.

[0047] According to block 121, reference input 7 may include: Training samples used to train AI / ML model 2, and / or Samples supplied by users from the domain and / or distribution related to query 1, and / or The contextual information used by AI / ML model 2 when generating response 3 to query 1.

[0048] Specifically, according to block 121a, the sample and / or contextual information supplied by the user may include technical documents relating to at least one industrial asset 11 in the industrial plant 10 and / or the industrial plant 10 as a whole.

[0049] According to block 122, the proximity 5a to DOV 9 can be determined at least in part based on the classification score of a classifier configured to determine whether the representation of the input component belongs to DOV 9. Specifically, according to block 122a, the classifier can be selected to be trained at least in part on the reference input 7.

[0050] According to block 123, determining the proximity 5a to DOV 9 may include determining whether the representation 5 of query 1 is a member of a cluster known to belong to or not belong to DOV 9.

[0051] According to block 124, determining the proximity 5a to DOV 9 may include pre-selecting a subset of reference representation 8 for comparison with the representation of the query.

[0052] According to block 125, DOV 9125 was chosen to include a representation of a document containing sensitive information. As will be discussed later, this allows for the examination of whether the AI / ML model could potentially disclose this sensitive information to the user who issued query 1.

[0053] According to block 126, the same query 3 can be processed into corresponding responses 3 by multiple AI / ML models 2 so that these responses 3 can be evaluated with the help of their respective confidence levels 4.

[0054] According to block 127, reference input 7 can be selected to involve a specific category in a given classification, thereby allowing the determined confidence level 4 to be category-specific.

[0055] In step 130, confidence level 4 is determined at least in part based on the proximity 5a determined in step 120.

[0056] According to block 131, after the same query 3 has been processed into corresponding responses 3 by multiple AI / ML models 2 according to block 126, a confidence level 4 can be determined for each such response 3. According to block 132, the final response 3 to query 1 can then be selected and / or aggregated from the multiple responses 3 based on the corresponding confidence level 4. .

[0057] According to block 133, confidence level 4 can be determined for multiple categories. According to block 134, it can then be checked whether the confidence level 4 for categories that the user submitting query 1 has the right to access is higher than a first predetermined threshold. According to block 135, it can also be checked sequentially or simultaneously whether the confidence level 4 for categories that this user does not have the right to access is lower than a second predetermined threshold. If both conditions are met (i.e., both true values ​​are 1), then according to block 136, it can be approved to issue response 3 to the user.

[0058] In step 140, an actuation signal 12 can be calculated from the AI / ML model 2 based on the response 3 calculated from query 1. In step 150, at least one industrial asset 11 in the industrial plant 10 can be actuated, and / or other physical effects can be applied to the industrial process 10a performed by the industrial plant 10, based at least in part on the actuation signal 12. Optionally, this process can be gated based on a confidence level 4 by any suitable conditions. That is, if the confidence level 4 is below a predetermined acceptance criterion and / or meets a predetermined rejection criterion, the calculation and / or use of the actuation signal 12 can be suppressed.

[0059] In step 160, it can be determined whether confidence level 4 is below a predetermined threshold. If this is true (true value 1), then in step 170, feedback 3a can be obtained regarding whether the response 3 provided by AI / ML model 2 is correct. In step 180, it can be checked whether this feedback 3a indicates that the response is correct. If this is true (true value 1), then in step 190, it can be determined that the representation 5 of query 3 belongs to DOV 9. The result is extended DOV 9. .

[0060] In any case, in step 200, the AI / ML model 2 can be retrained using query 1 as a new training sample. The ground truth label assigned to this new training sample can include feedback 3a regarding whether the response 3 provided by the AI / ML model 2 to this query 1 is correct.

[0061] In step 210, in the use case where DOV 9 is selected according to block 125 to include a representation of a document with sensitive information, it can be determined whether the confidence level 4 regarding this DOV 9 exceeds a predetermined threshold. If this is true (the true value is 1), it can be determined in step 120 that the AI / ML model 2 has a tendency to disclose sensitive information in response 3.

[0062] Figure 2 This demonstrates how to base decisions on the classifier's decision function. The value determines the confidence level of query 1 (also known as 4). The classifier determines whether the representation 5 of query 1 is within DOV 9. Figure 2In the example shown, the classifier is a one-class support vector machine (SVM) that has been trained on a reference representation 8 generated from reference input 7. Reference input 7 is an input that is known or at least reasonably expected to produce a correct response by the AI / ML model 2. Therefore, reference representation 8 is known to belong to DOV 9.

[0063] If the decision function of the classifier is used The output is greater than or equal to the threshold. (In the case of a single-class SVM, such a threshold is 0), then the projection of user query 1 into the embedding space of AI / ML model 2, which is used as the working space 6. 5 belongs to the embeddings of the AI / ML model training data or the embeddings of the user-defined document (which is used as reference representation 8) DOV 9. As a result, the confidence level of the response 3 of AI / ML model 2. 4 should be close to 1. On the other hand, if a user queries the projection of 1 into the embedded space 6... If DOV 5 is not an embedding of the generative AI model training data or an embedding of a user-defined document (this embedding is used as reference representation 8), then the decision function of the classifier used is... The output will be less than the threshold. In this context, the confidence level of the generative AI model's response... It will be less than 1. When the decision function of the classifier used... The output is significantly smaller than the threshold. hour, It will be very close to 0.

[0064] Confidence level It is based on the decision function of the classifier used. The output is used for calculation, where Use the following confidence scoring function To calculate: , , in It is the embedded input of the user query, and It is the decision function of the classifier used. and ∈ yes The tuning parameters. If the classifier's decision function... The value is equal to Then the confidence level 4 equals If the classifier's decision function equal Then the confidence level 4 equals This represents the maximum achievable confidence level. A decision function is required based on the embeddings of the classifier's training data for the generative AI model or the embeddings of a user-defined document (which serves as a reference representation). The value determines the parameter This parameter also reflects how much we trust the generalizability of generative AI models. (Function) It is the sigmoid function, where the confidence level is... The reachable value range is: .exist Figure 2 The function given in In the graphical representation, the tuning parameters are: , and .

[0065] To visualize the computed DOV of a generative AI model, feature extraction methods such as Principal Component Analysis (PCA), t-distributed random neighborhood embeddings (t-SNE), and Linear Discriminant Analysis (LDA) can be used to first project the embeddings of the generative AI model's training data or user-defined document embeddings into a lower-dimensional space. The boundaries of the low-dimensional embeddings are determined by training a classifier such as a one-class SVM. In this case, the confidence level of the generative AI model... The calculation is as follows: , , Where the function Projecting the raw embeddings of generative AI model training data or the embeddings of user-defined documents into a lower-dimensional space enables better visualization of AI / ML models in DOV 9.

[0066] After deploying the AI ​​system, users can verify whether the AI ​​system's output (response) 3 is correct, even though user query / input 1 does not belong to the estimated DOV 9 of AI / ML model 2. In this case, user feedback will be taken into account, and the DOV 9 of AI / ML model 2 will be adapted / extended to include the embedding (representation) 5 of the tested user input 1. Therefore, model 2 will have a high confidence level for future inputs similar to user query 1. Thus, this can be considered an online adaptation of DOV.

[0067] Figure 3A simple case study illustrates the implementation and advantages of the proposed DOV-based method. We use a GloVe-based pre-trained word vector dictionary, generated by the Global Word Vector Representation (GloVe) machine learning algorithm, to obtain word vector representations. This dictionary includes embeddings of 400,000 words, and our goal is to estimate the effective domain (DOV) representing the word embeddings of available animal names.

[0068] In this example, we first use the t-SNE algorithm to reduce the dimensionality of the word embeddings from 50D to 2D to enable visualization of the estimated DOV. In this case, a single-class SVM with an RBF kernel is used to estimate the DOV by determining the boundaries of the regions densely covered by the 2D projection of the animal word embeddings. Along with the training data, user query 1 is also embedded using t-SNE and projected into the same 2D space. In this case study, we randomly select user query 1 from 400,000 available words in a GloVe-based dictionary. Then, we obtain the corresponding embeddings. 5. The embedding has been computed and stored in the dictionary. In this case, the confidence level of AI / ML model 2 is... 4 is calculated as follows: , , in It is used to reduce user queries t-SNE function of dimension. The tuning parameters were selected as follows: , and The RBF kernel used in a single-class SVM is composed of... Defined, where the tuning parameter γ is chosen as .

[0069] We used several test words that were both animal-related and animal-unrelated to the determined DOV value. Figure 3 The estimated DOV 9 of a set of animal names based on the GloVe dictionary, along with the confidence levels corresponding to the test words, are shown in two-dimensional form d1 and d2. All animal names: Tiger 401 Mazda 402 Dog 403, 404 for rats Bat 405, Elephant 406, Sheep 407 Niu 408 Rat 409, and Lion 410, Their representations (5) using circle markings all have confidence scores above 0.99 because they fall within the estimated DOV of 9. Other terms: Employee 301 Employer 302, Worker 303 304 for the elderly People 305, Children 306, Child 307, Adults 308 Human 309 310 people Patient 311, Men 312 Ronaldo 313, and Maxwell 314, For those related to humans, their representations are marked with squares and have extremely low confidence scores because they are far from the estimated DOV of the object names.

[0070] Figure 4 illustrates the ability of the developed scheme to estimate the effective domain (DOV 9) of AI / ML model 2 in another case study, which uses images (e.g., symbols in P&ID documents) as input. This demonstrates the generality of this DOV estimation method, which can be used to determine the trust region of an ML model with any kind of input.

[0071] In this example, we estimate the DOV of a ResNet-18 model. A pre-trained version of ResNet-18 is a convolutional neural network developed to classify images into 1000 object categories. In this case, we consider a fine-tuned ResNet-18 trained to perform classification of symbols in customer-specific P&ID documents. Users can then use this AI model to automatically classify symbols in a specific P&ID document.

[0072] Using the proposed method, the DOV 9 of Model 2 is estimated by first reducing the dimensionality of the embeddings in the Res-Net-18 model. We employ the t-SNE method to reduce the embedding dimension from 512 to 2. Then, a single-class SVM with an RBF kernel is trained using the low-dimensional projection of embedding 5 to estimate the boundary of the low-dimensional projection of embedding 5. In this case, the confidence level of AI Model 2 is... 4. Embedding based on corresponding images The calculation is as follows: , The tuning parameters were selected as follows: , and The RBF kernel used in a single-class SVM is defined as... , where γ is chosen as .

[0073] Figure 4a The estimated DOV 9 of Model 2 is shown, along with eight different P&ID symbols, which are in Figure 4a The positions in the matrix correspond to their representations in the reduced-dimensional embedding space consisting of two-dimensional d1 and d2. Figure 4a In the example shown, DOV 9 is not a single continuous region; rather, it consists of several disjoint regions. This indicates that 5 belongs to the following category: Control valve 501, Condenser 502, Heat exchanger 503, Compressor 504, Lamp 505, Container 506, and Pump 507.

[0074] The five symbols (with transparent background) representing 5 belong to the DOV of the fine-tuned Res-Net-18 model, while the remaining symbols (with shaded background) representing 5 do not belong to DOV 9 of model 2 because these symbols were not included in the training data for model 2. The proposed confidence scoring function gives a high confidence level (4, ) to P&ID symbols belonging to DOV 9 of this model. ), while giving low confidence scores (4, ) to other symbols that did not appear in the training dataset of Model 2. Therefore, on the one hand, this tool is very helpful in supporting users to assess how confident they are in the model's classification results.

[0075] On the other hand, these out-of-distribution data points (marked with a shaded background) may explicitly trigger efficient and effective feedback from human experts (e.g., by adding additional labels to these data points). Figure 4a This feedback is also illustrated in the diagram. Based on expert feedback, response 3 of Model 2 is marked with an upward arrow (↑) if it is judged to be correct, and with a downward arrow (↓) if it is judged to be incorrect.

[0076] The classification of two symbols was incorrect. However, the classification of the container image (labeled V) was correct, even though this symbol was not included in the training data. This is because this container symbol is very similar to those included in the training data. As a result, based on user feedback, additional data points provided by these experts can be considered to effectively retrain the model.

[0077] Figure 4b The results of the DOV 9 adaptation (extended here) for AI / ML Model 2 are shown. This adaptation is based on the feedback that AI / ML Model 2 correctly responded to container images (labeled V) that looked different from those observed during training. Confidence level 4. With Figure 4a The same method has been determined.

[0078] Since the container image V has already been included in DOV 9, this image now has a very high confidence level. To achieve this effect, it is not even necessary to include this image in another training dataset used for fine-tuning the AI / ML model 2. This saves computation time during fine-tuning or other retraining processes.

[0079] List of reference numerals in the attached diagram:

[0080] 1. Queries submitted by users

[0081] 2. Artificial Intelligence / Machine Learning Models (AI / ML Models)

[0082] 2 Retraining status of AI / ML model 2

[0083] 3. AI / ML Model 2's response to Query 1

[0084] 3 The final response aggregated / selected from multiple responses 3

[0085] 3a User feedback on response 3

[0086] 4,γ: Confidence level for response 3

[0087] 5. Representation of Query 1 in workspace 6

[0088] 6. Workspace, for example, embedded space

[0089] 7. Known AI / ML model 2: Familiar reference input

[0090] 8. Given the reference representation of reference input 7 in DOV 9.

[0091] 9. Valid Domain (DOV)

[0092] 10 Industrial plants

[0093] 10a Industrial processes executed in industrial plant 10

[0094] Industrial assets in Industrial Plant 10

[0095] 12 Actuation Signal

[0096] 100 Methods for determining confidence level 4

[0097] 105 Select queries related to Industrial Plant 10, Process 10a, and Asset 11.

[0098] 110 Retrieves the representation of query 5 as 1

[0099] The space used in Model 2, or a lower-dimensional space, is selected as the workspace.

[0100] 112 Select the space of the tensor representation of the AI / ML model as the working space 6

[0101] 120 Determine the degree of proximity to DOV 9 and / or reference representation 8. 5a

[0102] 121 Select a specific type of reference input 7

[0103] 121a Select technical documents as reference input 7

[0104] 122 Determining proximity based on classification scores 5a

[0105] 122a Select a classifier trained on reference input 7

[0106] 123 Using clustering to determine proximity 5a

[0107] 124 Pre-select a subset of reference representation 8

[0108] 125 Select DOVs related to sensitive information 9

[0109] 126 Query processing is handled by multiple AI / ML models 2

[0110] 127 Select category-specific reference input 7

[0111] 130. Determining the confidence level based on proximity (5a) 4

[0112] 131 Determine the confidence level of responses from multiple AI / ML models 2 4

[0113] 132 Determine the final response 3 from multiple responses 3

[0114] 133 Determining confidence levels relative to multiple categories 4

[0115] 134. Check if the confidence level of 4 for the "Access Normal" category is high.

[0116] 135. Check if the confidence level of 4 for the "abnormal access" category is low.

[0117] 136 Approved to release response to user 3

[0118] 140 Calculate the actuation signal 12

[0119] 150. Actuation signal 12 is used to actuate plant 10, asset 10, and process 10a.

[0120] 160. Determine whether confidence level 4 is below the threshold.

[0121] 170 Get Feedback 3a

[0122] 180 Determine if feedback 3a indicates a correct response.

[0123] 190. Determines that query 3 indicates 5 belongs to DOV 9.

[0124] 200 Retraining AI / ML Models 2

[0125] 210 Determine whether the confidence level of 4 for sensitive DOV 9 is higher than the threshold.

[0126] 220 Determine if there is a tendency to leak sensitive information.

[0127] 301 employees (example outside of DOV 9)

[0128] 302 Employer (Example outside of DOV 9)

[0129] 303 Workers (Examples outside of DOV 9)

[0130] 304 Older adults (Examples outside of DOV 9)

[0131] 305 People (Examples outside of DOV 9)

[0132] 306 Children (Examples outside of DOV 9)

[0133] 307 Children (Examples outside of DOV 9)

[0134] 308 Adults (Examples outside of DOV 9)

[0135] 309 Humans (Examples outside of DOV 9)

[0136] 310 people (examples outside of DOV 9)

[0137] Patient 311 (Examples outside of DOV 9)

[0138] 312 Men (Examples outside of DOV 9)

[0139] 313 Ronaldo (an example outside of DOV 9)

[0140] 314 Max (An example outside of DOV 9)

[0141] 401 Tiger (Example in DOV 9)

[0142] 402 Horses (Example in DOV 9)

[0143] 403 Dog (Example in DOV 9)

[0144] 404 Mouse (Example in DOV 9)

[0145] 405 Bat (Example in DOV 9)

[0146] 406 Elephant (Example in DOV 9)

[0147] 407 Sheep (Example in DOV 9)

[0148] 408 Cows (Example in DOV 9)

[0149] 409 Mice (Example in DOV 9)

[0150] 410 Lion (Example in DOV 9)

[0151] 501 Control Valve Category

[0152] 502 Condenser Category

[0153] 503 Heat Exchanger Category

[0154] 504 Compressor Category

[0155] 505 lamp category

[0156] 506 Container Category

[0157] 507 Pump Category

[0158] V container image

Claims

1. A computer-implemented method (100) for estimating the confidence level (4) of a response (3) delivered by an artificial intelligence (AI) / machine learning (ML) model (2) in response to a given query (1), the method comprising the steps of: Obtain (110) the representation (5) of the query (1) in the workspace (6); Determine (120) the proximity (5a) of the representation (5) to one or more reference representations (8) of the reference input (7) of the known effective domain DOV (9) of the AI / ML model (2) and / or the proximity (5a) of the representation (5) to the DOV (9) as a whole. The confidence level (4) is determined (130) at least in part based on the proximity (5a); And in response to the confidence level (4) being below a predetermined threshold: Obtain feedback (3a) on whether the response (3) provided by the AI / ML model (2) is correct; and If the feedback (3a) indicates that the response (3) is correct, then the representation (5) of the query (3) is determined to belong to the DOV (9). The query (1) is associated with at least one attribute of an industrial asset (11) in the industrial plant (10) and / or an industrial process performed on the industrial plant (10).

2. The method (100) according to claim 1, wherein the reference input (7) comprises (121): The training samples used to train the AI / ML model (2), and / or Samples supplied by users from the domain and / or distribution associated with the query (1), and / or The AI / ML model (2) uses contextual information when generating the response (3) to the query (1).

3. The method (100) according to any one of claims 1 to 2, wherein the user-supplied sample and / or the context information includes (121a) technical documents relating to at least one industrial asset (11) in the industrial plant (10) and / or to the industrial plant (10) as a whole.

4. The method (100) according to any one of claims 1 to 3, further comprising: The actuation signal (12) is calculated (140) based on the response (3); as well as Based on the actuation signal (12), actuate (150) at least one industrial asset (11) in the industrial plant (10) and / or exert physical influence on the industrial process (10a) performed by the industrial plant (10).

5. The method (100) according to any one of claims 1 to 4, wherein the space used by the AI / ML model (2) during processing the query (1) into the response (3), and / or a lower-dimensional space (111) accessible from such space by projection is selected as the workspace (6).

6. The method (100) according to any one of claims 1 to 5, wherein the space of the AI / ML model that can process the text input, image input, table or other structured input, audio input, and / or video input, and / or a lower-dimensional space (112) accessible from such space by projection is selected as the workspace (6).

7. The method (100) according to any one of claims 1 to 6, wherein the degree of proximity (5a) to the DOV (9) is determined (122) at least in part based on the classification score of a classifier configured to classify whether the input representation belongs to the DOV (9).

8. The method (100) according to claim 7, wherein the classifier is selected (122a) to be trained at least partially on the reference input (7).

9. The method (100) according to any one of claims 1 to 8, wherein determining the degree of proximity (5a) to the DOV (9) comprises: Determine (123) whether the representation (5) of the query (1) is a member of a cluster known to belong to or not belong to the DOV (9).

10. The method (100) according to any one of claims 1 to 9, wherein determining the degree of proximity (5a) to the DOV (9) comprises: A subset of the reference representation (8) is pre-selected (124) for comparison with the representation of the query.

11. The method (100) according to any one of claims 1 to 10, further comprising: The AI / ML model (2) is retrained (200) using the query (1) as a new training sample, wherein the ground truth label assigned to the new training sample includes feedback (3a) on whether the response (3) provided by the AI / ML model (2) is correct.

12. The method (100) according to any one of claims 1 to 11, wherein: The DOV (9) is selected (125) as a representation that includes documents containing sensitive information, and In response to the confidence level (4) exceeding a predetermined threshold (210), it is determined (220) that the AI / ML model (2) has a tendency to disclose sensitive information in the response (3).

13. The method (100) according to any one of claims 1 to 12, wherein: The same query (1) and / or multiple queries (1) with similar information but different formats are processed (126) by multiple AI / ML models (2) into corresponding responses (3); The confidence level (4) is determined (131) for each such response (3); and The final response (3) to the query (1) Based on the corresponding confidence level (4), the responses (3) are selected and / or aggregated (132) from the plurality of responses (3).

14. The method (100) according to any one of claims 1 to 13, wherein the reference input (7) is selected (127) to be associated with a particular category of a given classification.

15. The method (100) according to any one of claims 1 to 14, wherein: The confidence level (4) is determined relative to multiple categories (133); and In response that the confidence level (4) relative to the category that the user has the right to access is higher than (134) a first predetermined threshold, and the confidence level (4) relative to the category that the user does not have the right to access is lower than (135) a second predetermined threshold, the response (3) is issued to the user as approved.

16. A computer program comprising machine-readable instructions that, when executed by one or more computers and / or computing instances, cause the one or more computers to perform the method (100) according to any one of claims 1 to 15.

17. A non-transitory machine-readable data carrier and / or download product having the computer program according to claim 16.

18. One or more computers and / or computing instances having a computer program as described in claim 16 and / or a machine-readable data carrier and / or downloadable product as described in claim 17.

Images