Out-of-domain detection for improved ai performance

Abstract

Out-of-domain detection for improved AI performance. A system and method are provided for determining whether input data is outside the domain of an AI-based (artificial intelligence) system. The system receives input data for the AI-based system. It models the in-domain and out-of-domain feature spaces of the AI-based system. The in-domain feature space corresponds to the features of the data to which the AI-based system is trained for classification. The out-of-domain feature space corresponds to the features of the data to which the AI-based system is not trained for classification. Probability distribution functions are generated for the input data and for the data to which the AI-based system is trained for classification in the in-domain and out-of-domain feature spaces, respectively. Based on the probability distribution functions for the input data and the data to which the AI-based system is trained for classification, the system determines whether the input data is outside the domain of the AI-based system.

Description

Technical Field

[0001] This invention generally relates to improved AI (artificial intelligence) performance, and particularly to out-of-domain detection for improved AI performance. Background Technology

[0002] AI (Artificial Intelligence) based systems are often deployed to automate tasks that are typically performed manually by users. For example, in medical imaging analysis, AI-based systems can be deployed for the detection, quantification, and clinical decision-making of patient-based medical images. Real-world deployments of AI-based systems increasingly encounter scenarios not presented in the training dataset, leading to poor performance. Some examples of such scenarios include the analysis of medical images with varying degrees of quality or in which objects of interest appear in different locations, orientations, scales, or in all or part of the view. While improving and expanding the training dataset or employing data augmentation techniques may help improve the robustness of AI-based systems, they cannot be practically trained to solve every scenario they might encounter. One solution is to have users manually review the results of the AI-based system. However, this would require users to review all the results of the AI-based system at the cost of full automation. Summary of the Invention

[0003] According to one or more embodiments, systems and methods for out-of-domain detection are provided to detect input data of an AI-based system that is outside the distribution of training data used to train the AI-based system. Input data for inputting into the AI-based system is received. An in-domain feature space and an out-of-domain feature space of the AI-based system are modeled. The in-domain feature space corresponds to the features of data for which the AI-based system is trained to classify. The out-of-domain feature space corresponds to the features of data for which the AI-based system is not trained to classify. Probability distribution functions are generated for the input data and for the data for which the AI-based system is trained to classify in the in-domain and out-of-domain feature spaces, respectively. Based on the probability distribution functions for the input data and for the data for which the AI-based system is trained to classify, it is determined whether the input data is outside the domain of the AI-based system.

[0004] In one embodiment, an intra-domain feature space is computed based on one or more intra-domain linear projection matrices of the AI-based system. One or more orthogonal linear projection matrices for the out-of-domain feature space are computed based on one or more intra-domain linear projection matrices of the AI-based system. An out-of-domain feature space is computed based on one or more orthogonal linear projection matrices for the out-of-domain feature space. In one embodiment, a probability distribution function is generated using a Gaussian process model or a combination of a Gaussian probability distribution model suitable for the available data.

[0005] In one embodiment, in response to determining that the input data is outside the domain of the AI-based system, a notification is transmitted to the user to review the prediction generated by the AI-based system based on the input data. In another embodiment, in response to determining that the input data is outside the domain of the AI-based system, the input data is annotated, and the AI-based system is trained based on the annotated input data. In yet another embodiment, in response to determining that the input data is not outside the domain of the AI-based system, the AI-based system generates a prediction based on the input data.

[0006] In one embodiment, receiving, modeling, generating, and determining are performed by modules combined with the AI-based system, and the modules combined with the AI-based system generate predictions based on input data and determination of whether the input data is outside the domain of the AI-based system.

[0007] In one embodiment, one of several algorithms for the AI-based system is selected based on the determination (e.g., by selecting an algorithm in which the input data is within a domain). The AI-based system can be used for medical imaging analysis.

[0008] These and other advantages of the present invention will be apparent to those skilled in the art from the following detailed description and accompanying drawings. Attached Figure Description

[0009] Figure 1 A method for detecting out-of-domain input data according to one or more embodiments is shown;

[0010] Figure 2 An exemplary model of the feature space of an AI-based system according to one or more embodiments is shown;

[0011] Figure 3 An image showing the probability of data being classified into a domain according to one or more embodiments;

[0012] Figure 4 The workflow for implementing an out-of-domain detector module separate from an AI-based system, according to one or more embodiments, is illustrated.

[0013] Figure 5The workflow for implementing an out-of-domain detector module in combination with an AI-based system, according to one or more embodiments, is illustrated.

[0014] Figure 6 The workflow for implementing an out-of-domain detector module according to one or more embodiments is shown, which determines whether input data is outside the domain of the AI-based system based on both input data and predictions generated by the AI-based system based on the input data.

[0015] Figure 7 A workflow for annotating input data according to one or more embodiments is illustrated, the input data being used to train an AI-based system based on whether the input data is outside the domain of the AI-based system, as determined by an out-of-domain detector module.

[0016] Figure 8 An exemplary artificial neural network that can be used to implement one or more embodiments is shown;

[0017] Figure 9 A convolutional neural network that can be used to implement one or more embodiments is shown; and

[0018] Figure 10 A high-level block diagram of a computer that can be used to implement one or more embodiments is shown. Detailed Implementation

[0019] This invention generally relates to methods and systems for out-of-domain detection for improved AI (artificial intelligence) performance. Embodiments of the invention are described herein to provide a visual understanding of such methods and systems. Digital images typically consist of digital representations of one or more objects (or shapes). The digital representations of objects are generally described herein based on the identification and manipulation of the objects. Such manipulation is a virtual manipulation performed in the memory or other circuitry / hardware of a computer system. Therefore, it is to be understood that embodiments of the invention can be performed within a computer system using data stored within the computer system.

[0020] The embodiments described herein provide an out-of-domain detector for detecting whether input data to be fed into an AI-based system is within the distribution of training data on which the AI-based system is trained. When applied to out-of-domain input data, the AI-based system may exhibit relatively poor performance. Therefore, out-of-domain input data can be flagged to notify the user that the AI-based system is outside the system's domain. In some embodiments, out-of-domain input data can also be used to retrain the AI-based system to increase robustness and improve performance.

[0021] Figure 1A method 100 for detecting out-of-domain input data according to one or more embodiments is illustrated. The steps of method 100 can be performed by one or more suitable computing devices, such as, for example... Figure 10 Computer 1002.

[0022] exist Figure 1 At step 102, input data is received for input into the AI-based system. In one example, the AI-based system is used to perform medical imaging analysis tasks, such as detection, quantification, and clinical decision-making. The AI-based system can be any suitable system for generating output based on the input data. In one embodiment, the AI-based system includes a machine learning-based system (e.g., a neural network).

[0023] The input data can be any suitable data. In one embodiment, the input data includes medical images of the patient. The medical images can have any suitable modality, such as, for example, CT (computational computed tomography), MRI (magnetic resonance imaging), ultrasound, X-ray, or any other medical imaging modality or combination of medical imaging modalities. The medical images can be 2D (two-dimensional) images and / or 3D (three-dimensional) volumes, and can include a single input medical image or multiple input medical images. In another embodiment, the input data includes the patient's clinical data, such as, for example, medical history, demographics, laboratory results, etc.

[0024] Input data can be received by loading previously acquired data from the storage device or memory of a computer system (e.g., an EMR (Electronic Medical Record) system or a PACS (Picture Archiving and Communication System)), or by receiving data that has already been transmitted from a remote computer system. When the input data includes medical images, the medical images can be received directly from the image acquisition device during the acquisition process.

[0025] exist Figure 1 In step 104, the in-domain feature space and the out-of-domain feature space of the AI-based system are modeled. The in-domain feature space corresponds to the features of the data that the AI-based system is trained to classify, while the out-of-domain feature space corresponds to the features of the data that the AI-based system is not trained to classify.

[0026] Figure 2An exemplary model 200 of the feature space of an AI-based system according to one or more embodiments is shown. Model 200 includes an in-domain feature space x 202 and an out-of-domain feature space y 204. The in-domain feature space x 202 corresponds to the features of data that the AI-based system is trained to classify. Therefore, the AI-based system uses the in-domain feature space x 202 to distinguish data of category 1 206 and data of category 2 208. The out-of-domain feature space y 204 corresponds to the features of data that the AI-based system is not trained to classify (e.g., OoD (out-of-domain) data 210). The out-of-domain feature space y 204 is in an orthogonal space (relative to the in-domain feature space x 202) or a null space and is not visible to the AI-based system. Out-of-domain data projected onto category 1 206 and / or category 2 208 (e.g., OoD data 210) cannot be distinguished by the AI-based system into category 1 206 or category 2 208.

[0027] In one embodiment, the AI-based system includes a machine learning-based network (e.g., a neural network) that includes a linear projection matrix. A One or more projection layers (e.g., fully connected and / or convolutional layers) implemented. Projection matrix. A The values / weights are typically learned offline during the training phase so that the resulting projected feature values ​​best represent the features relevant to the current task. Therefore, the in-domain feature space x²0² is modeled or computed as:

[0028] x=Az

[0029] Where x is the feature space 202 within the domain, A Let z be the linear projection matrix of the AI-based system for the in-domain feature space x 202, and z be the input vector representing the original space. The out-of-domain feature space y 204 is then determined by finding the projection matrix. A ⊥ Modeled or calculated as:

[0030]

[0031] in A ⊥ It is a linear projection matrix for the 204 feature space outside the domain. A T It is a projection matrix A The transpose of , and (∙) # This represents the pseudo-inverse of the matrix. The out-of-domain feature space y 204 is modeled or computed as:

[0032] .

[0033] Then, the machine learning-based network is configured to carry both the in-domain feature space x202 and the out-of-domain feature space y204. This can be achieved by constructing a network with orthogonal projection matrices. A ⊥ This is achieved through an additional network with defined weights. By modeling the data distribution in both the in-domain feature space x and the orthogonal feature space y, we can distinguish out-of-domain data that is "invisible" to the in-domain feature space.

[0034] Alternatively, A The extraterritorial feature space y 204 can be obtained from... A Obtain the corresponding basis vectors from the singular value decomposition (by... (Representation) to project

[0035]

[0036] Where U and V are the left and right singular vectors, respectively, S is the diagonal matrix of singular values, and the subscripts R and N represent the range and null space, respectively.

[0037] In one embodiment, if multiple matrices are used for computation in the intra-domain feature space, the out-of-domain transformation can consist of one or more projections, provided that the features of the projections lie in the out-of-domain feature space y 204 (i.e., the null space) of the intra-domain features. This can be achieved either by replacing the final transformation with only orthogonal projections or by combining the projected features of any orthogonal projections from the sequence of orthogonal projections corresponding to the intra-domain transformation.

[0038] exist Figure 1 At step 106, probability distribution functions are generated for the input data and for the data trained on the AI-based system for classification in both the in-domain and out-of-domain feature spaces. In one example, in Figure 2 In model 200, a probability distribution function 216 is generated for OoD data 210, which can descriptively represent the input data. Probability distribution functions 212 and 214 are also generated for category 1 206 and category 2 208, respectively, representing data that the AI-based system is trained to classify. Probability distribution functions 202, 214, and 216 are generated in both the in-domain feature space x 202 and the out-of-domain feature space y 204.

[0039] After the neural network training phase, a probability distribution function can be generated using any suitable method by modeling the features in the training data. The model of the data distribution can be a Gaussian probability distribution model, a nonparametric density estimate, or a mixture or combination of Gaussian processes. The model parameters are adapted to the in-domain / out-of-domain features computed by the modified neural network. In one embodiment, a Gaussian process model is used to generate the probability distribution function, which may or may not have inducing points in the feature space.

[0040] exist Figure 1 At step 108, based on the probability distribution functions for the input data and the data trained by the AI-based system for classification, it is determined whether the input data is outside the domain of the AI-based system. Input data outside the domain of the AI-based system indicates that the input data corresponds to data that the AI-based system was not trained to classify. Input data not outside the domain (i.e., within the domain) indicates that the input data corresponds to data that the AI-based system was trained to classify.

[0041] The probability or score of the input data belonging to the distribution within the domain is determined by the distribution in the domain. Figure 1 The probability of the projected features of the fitted domain probability distribution function modeled at step 106 is given. A high probability indicates that the input data is most similar to the training (in-domain) distribution, while a low probability indicates that the input data is dissimilar to the training distribution. The determination of whether the input data is outside the domain of the AI-based system can be represented in any suitable format. For example, the determination of whether the input data is outside the domain of the AI-based system is represented as an out-of-domain score. The classification (e.g., yes, no, or uncertain) can be determined by comparing the out-of-domain score to one or more thresholds.

[0042] exist Figure 1 At step 110, a determination of whether the input data is outside the domain of the AI-based system is output. For example, this can be achieved by displaying the determination on a display device of the computer system, storing the determination on a memory or storage device of the computer system, or transmitting the determination to a remote computer system.

[0043] AI-based systems may exhibit relatively poor performance with out-of-domain input data. Therefore, in one embodiment, in response to determining that the input data is outside the domain of the AI-based system, a notification is transmitted to the user to manually review the predictions generated by the AI-based system based on the input data.

[0044] In one embodiment, when the AI-based system is initially trained, method 100 may be combined with spectral normalization or other regularization techniques. In one embodiment, supervised training may be applied to find a discriminative feature space on which additional out-of-domain detector models can be built.

[0045] Figure 3 Image 300 illustrates the probability of data being classified as in-domain according to one or more embodiments. In image 300, region 308 represents a low probability of being classified as in-domain, while region 310 represents a high probability of being classified as in-domain. Image 302 illustrates the probabilities of the original classifier in the final classifier output. While the original classifier distinguishes the two classes well, it does not separate out-of-domain input data well. Image 304 illustrates the probabilities of a Gaussian process trained in the in-domain feature space of the second linear layer. Image 306 illustrates the probabilities of a Gaussian process trained in both the in-domain and out-of-domain feature spaces of the second linear classifier. Image 306 shows the separation between out-of-domain and in-domain data much better than Image 304.

[0046] In one embodiment, the extra-domain detector module can be based on... Figure 1 Method 100 (or any other suitable method) is used to implement this and in various configurations of AI-based systems, as described below. Figure 4-6 As stated above.

[0047] Figure 4 A workflow 400 for implementing an out-of-domain detector module separate from an AI-based system, according to one or more embodiments, is illustrated. In workflow 400, input data 402 is input into the out-of-domain detector module 404. In one example, the input data 402 is... Figure 1 The input data received at step 102. The out-of-domain detector module 404 determines whether the input data 402 is outside the domain of the AI-based system 408. In one embodiment, the out-of-domain detector module 404 determines whether the input data 402 is outside the domain of the AI-based system 408. Figure 1Method 100 determines whether input data 402 is outside the domain of the AI-based system 408. In another embodiment, an out-of-domain detector module 404 determines whether input data 402 is outside the domain of the AI-based system 408, for example, by normalizing the flow to explicitly model the distribution of the training data used to train the AI-based system 408 and a priori checking whether the input data is inside or outside the domain of the AI-based system 408. The out-of-domain detector module 404 may use any other suitable method to determine whether input data 402 is outside the domain of the AI-based system 408. If input data 402 is not outside the domain at decision box 406 (i.e., input data 402 is inside the domain), then input data 402 is fed into the AI-based system 408, and the AI-based system 408 generates a prediction 410 as output. If input data 402 is outside the domain at decision box 406, then a notification 412 is generated to notify the user that input data 402 is outside the domain. In response, the user can approve the input data 402 into the AI-based system 408 and manually review the prediction 410, or the user can annotate the input data 402 and retrain the AI-based system 408 with the annotated input data.

[0048] Figure 5 A workflow 500 for implementing an out-of-domain detector module in combination with an AI-based system, according to one or more embodiments, is illustrated. In workflow 500, input data 502 is fed into the AI-based system / out-of-domain detector module 504. In one example, input data 502 is... Figure 1 The input data received at step 102. The AI-based system / outside-domain detector module 504 is a combination of an AI-based system and an outside-domain detector module in a single module. The outside-domain detector module implemented by the AI-based system / outside-domain detector module 504 determines whether the input data 502 is outside the domain of the AI-based system implemented by the AI-based system / outside-domain detector module 504. In one embodiment, according to Figure 1In method 100, the out-of-domain detector module determines whether input data 502 is outside the domain of the AI-based system. In another embodiment, the out-of-domain detector module determines whether input data 502 is outside the domain of the AI-based system based on an energy-based or entropy-based method on the output softmax layer of a deep learning network implemented by the AI-based system. A higher energy level means that input data 502 is outside the domain of the AI-based system. The out-of-domain detector module can use any other suitable method to determine whether input data 502 is outside the domain of the AI-based system. The AI-based system / out-of-domain detector module 504 generates both a prediction 506 and a determination of whether input data 502 is outside the domain of the AI-based system as outputs. If input data 502 is not outside the domain at decision box 508 (i.e., input data 502 is inside the domain), prediction 506 is output as final prediction 510. If input data 502 is outside the domain at decision box 508, notification 512 is generated to notify the user that input data 502 is outside the domain. In response, users can manually review prediction 506, or they can annotate input data 502 and retrain the AI-based system with the annotated input data.

[0049] In one embodiment, the out-of-domain detector module / out-of-domain detector module 504 of the AI-based system determines whether input data 502 is outside the domain of the AI-based system by training a set of networks with different architectures and stopping at different times. The network is evaluated based on the input data 502 to assess the discrepancy. The determination of a pair of input data 502 / prediction 506 can be a function of the discrepancy. Cases where the determination is relatively high (e.g., based on a comparison of the out-of-domain score with one or more thresholds) can be reverted to having relatively low uncertainty and classified as not outside the domain.

[0050] Figure 6 A workflow 600 for implementing an out-of-domain detector module according to one or more embodiments is shown, which determines whether input data is outside the domain of an AI-based system based on both input data and a prediction generated by the AI-based system from the input data. In workflow 600, input data 602 is input to an AI-based system 604, which generates a prediction 606 as output. In one example, input data 602 is... Figure 1 The input data received at step 102.

[0051] The out-of-domain detector module 608 receives both input data 602 and prediction 606 as input. For input data 602 and for the pair of input data 602 / prediction 606, the out-of-domain detector module 608 generates a joint determination of whether input data 602 is out of domain (e.g., in the form of an out-of-domain score). There may be out-of-domain input data that the AI-based system 604 can still generalize well. Therefore, by generating the determination of input data 602 and the determination of the pair of input data 602 / prediction 606, the out-of-domain detector module 608 measures the degree to which prediction 606 is out of domain, thereby providing a filter for content that the user may have to review. In one embodiment, the out-of-domain detector module 608... Figure 1 Method 100 determines whether input data 602 is outside the domain of the AI-based system 604. The out-of-domain detector module 608 can use any other suitable method to determine whether input data 602 is outside the domain of the AI-based system 604.

[0052] If input data 602 at decision box 610 is not outside the domain (i.e., input data 602 is within the domain), then prediction 606 is output as the final prediction 612. If input data 602 at decision box 610 is outside the domain, then notification 614 is generated to inform the user that input data 602 is outside the domain. In response, the user can manually review prediction 606, or the user can annotate input data 602 and retrain the AI-based system with the annotated input data.

[0053] In one embodiment, out-of-domain scores (e.g., from...) Figure 4 404, an external detector module Figure 5 AI-based system / outside detector module 504 or Figure 6 The out-of-domain detector module 608 can be used to derive confidence intervals for measurements extracted from predictions. For example, in a system segmenting the left ventricle from 2D echocardiography, the out-of-domain score can be directly correlated with differences in the measurements of ejection fraction. One approach to this would be to calculate the differences based on multiple predictions used to compute the out-of-domain score. Alternatively, the correlation between the out-of-domain score and the differences in the measurements can be learned during the training phase.

[0054] Figure 7 A workflow 700 for annotating input data according to one or more embodiments is illustrated, the input data being used to train an AI-based system based on whether the input data is outside the domain of the AI-based system, as determined by an out-of-domain detector module. Out-of-domain input data is an indicator of the performance gap of the AI-based system in real-world scenarios. Determining whether the input data is outside the domain of the AI-based system allows the input data to be labeled and used to improve the performance of the AI-based system.

[0055] In workflow 700, at step 702, the AI-based system and the out-of-domain detector module are trained. In one embodiment, this can be based on... Figure 1 Method 100 implements the out-of-domain detector module. At step 704, unannotated input data is fed into the AI-based system and / or the out-of-domain detector module, and the unannotated out-of-domain input data is selected for annotation (e.g., by comparing the out-of-domain score with one or more thresholds). At step 706, the unannotated input data is annotated. At step 702, the annotated input data can be fed back to train the AI-based system and the out-of-domain detector module.

[0056] In one embodiment, an out-of-domain detector module can operate continuously alongside an AI-based system on the clinical side to collect out-of-domain input data. This out-of-domain input data can be used to identify patterns of failure (e.g., pediatric echocardiography versus adult echocardiography). Over time, a batch of out-of-domain input data can be used to directly train the AI-based system. When out-of-domain input data is received and detected, or during the receipt and detection of out-of-domain input data, it can be used for on-site online learning. The out-of-domain detector module can be useful in situations involving federated learning, where data privacy regulations prevent external entities from thoroughly analyzing local site data. Out-of-domain input data can be used to automatically select low-performance input data and train the AI-based system.

[0057] In one embodiment, the out-of-domain detector module can be used to select the most suitable algorithm for the AI-based system from a variety of AI algorithms to run on given input data. For example, a left ventricular ejection fraction calculation algorithm can be trained on normal and pathological patients. However, it is not unimportant whether the detection of incoming 2D echoes is normal. An algorithm trained on normal patients can be applied first, and if the out-of-domain detector module identifies the input data as out of domain, then an algorithm trained on pathological patients can then be applied.

[0058] The embodiments described herein relate to the claimed system and the claimed method. Features, advantages, or alternative embodiments described herein can be assigned to other claimed objects, and vice versa. In other words, the claims concerning the system can be modified using features described or claimed in the context of the method. In this case, the functional features of the method are implemented by the target unit providing the system.

[0059] Furthermore, some embodiments described herein are related to methods and systems for utilizing trained machine learning-based networks (or models), and to methods and systems for training machine learning-based networks. Features, advantages, or alternative embodiments described herein may be assigned to other claimed objects, and vice versa. In other words, the claims for methods and systems for training machine learning-based networks can be modified with features described or claimed in the context of methods and systems for utilizing trained machine learning-based networks, and vice versa.

[0060] Specifically, the trained machine learning-based networks used in the embodiments described herein can be adapted by methods and systems for training machine learning-based networks. Furthermore, the input data for the trained machine learning-based network can include advantageous features and embodiments of the training input data, and vice versa. Similarly, the output data for the trained machine learning network can include advantageous features and embodiments of the output training data, and vice versa.

[0061] Typically, trained machine learning-based networks mimic human cognitive functions associated with other human thought processes. In particular, by training on training data, trained machine learning-based networks can adapt to new environments and detect and infer patterns.

[0062] Typically, the parameters of a machine learning-based network can be adapted through training. Specifically, supervised training, semi-supervised training, unsupervised training, reinforcement learning, and / or active learning can be used. Furthermore, representation learning (an alternative term is "feature learning") can be used. In particular, the parameters of a trained machine learning-based network can be iteratively adapted through several training steps.

[0063] Specifically, the trained machine learning-based network can include neural networks, support vector machines, decision trees, and / or Bayesian networks, and / or the trained machine learning-based network can be based on k-means clustering, Q-learning, genetic algorithms, and / or association rules. Specifically, the neural network can be a deep neural network, a convolutional neural network, or a convolutional deep neural network. Furthermore, the neural network can be an adversarial network, a deep adversarial network, and / or a generative adversarial network.

[0064] Figure 8 An embodiment of an artificial neural network 800 according to one or more embodiments is shown. Alternative terms for "artificial neural network" are "neural network," "artificial neural network," or "neural network." The artificial neural network 800 can be used to implement the machine learning networks described herein, such as those that can be used to implement... Figure 1 AI-based systems Figure 4The extra-domain detector module 404 and the AI-based system 408 Figure 5 AI-based system / outside-domain detector module 504 Figure 6 AI-based system 604 and out-of-domain detector module 608, and Figure 7 AI-based systems and machine learning-based networks for out-of-domain detector modules.

[0065] The artificial neural network 800 includes nodes 802-822 and edges 832, 834, ..., 836, where each edge 832, 834, ..., 836 is a directed connection from a first node 802-822 to a second node 802-822. Typically, the first node 802-822 and the second node 802-822 are different nodes, but it is also possible that the first node 802-822 and the second node 802-822 are the same. For example, in... Figure 8 In the diagram, edge 832 is a directed connection from node 802 to node 806, and edge 834 is a directed connection from node 804 to node 806. Edges 832, 834, ..., 836 from the first node 802-822 to the second node 802-822 are also represented as "ingoing edges" of the second node 802-822 and "outgoing edges" of the first node 802-822.

[0066] In this embodiment, nodes 802-822 of the artificial neural network 800 can be arranged in layers 824-830, wherein the layers can include an inherent order introduced by edges 832, 834, ..., 836 between nodes 802-822. Specifically, edges 832, 834, ..., 836 may exist only between layers of adjacent nodes. Figure 8 In the illustrated embodiment, input layer 824 includes only nodes 802 and 804 without any incoming edges, output layer 830 includes only node 822 without any outgoing edges, and hidden layers 826 and 828 are located between input layer 824 and output layer 830. Typically, the number of hidden layers 826 and 828 can be arbitrarily chosen. The number of nodes 802 and 804 in input layer 824 is typically related to the number of input values ​​of neural network 800, and the number of nodes 822 in output layer 830 is typically related to the number of output values ​​of neural network 800.

[0067] Specifically, (real) numbers can be assigned as values ​​to each node 802-822 of the neural network 800. Here, x (n) iThis represents the value of the i-th node 802-822 in the n-th layer 824-830. The values ​​of nodes 802-822 in the input layer 824 are equivalent to the input values ​​of the neural network 800, and the value of node 822 in the output layer 830 is equivalent to the output value of the neural network 800. Furthermore, each edge 832, 834, ..., 836 may include a weight that is a real number, specifically a real number within the interval [-1, 1] or the interval [0, 1]. Here, w... (m,n) i,j This represents the weight of the edge between the i-th node 802-822 in layer m (824-830) and the j-th node 802-822 in layer n (824-830). Furthermore, for the weight w... (n,n+1) i,j The abbreviation w was defined. (n) i,j .

[0068] Specifically, to calculate the output value of neural network 800, the input values ​​are propagated through the neural network. Specifically, the values ​​of nodes 802-822 in the (n+1)th layer 824-830 can be calculated based on the values ​​of nodes 802-822 in the nth layer 824-830 using the following formula.

[0069] .

[0070] In this paper, the function f is the transfer function (another term is "activation function"). Known transfer functions are step functions, sigmoid functions (e.g., logic functions, generalized logic functions, hyperbolic tangent functions, arctangent functions, error functions, smooth step functions), or rectifier functions. Transfer functions are primarily used for normalization purposes.

[0071] Specifically, the value is propagated layer by layer through a neural network, wherein the value of the input layer 824 is given by the input of the neural network 800, wherein the value of the first hidden layer 826 can be calculated based on the value of the input layer 824 of the neural network, wherein the value of the second hidden layer 828 can be calculated based on the value of the first hidden layer 826, and so on.

[0072] To set the value w of the edge (m,n) i,j Training data is required to train the neural network 800. Specifically, the training data includes training input data and training output data (denoted as t). i For the training step, the neural network 800 is applied to the training input data to generate computational output data. Specifically, the training data and the computational output data include multiple values, the number of which is equal to the number of nodes in the output layer.

[0073] Specifically, the comparison between the calculated output data and the training data is used to recursively adapt the weights within the neural network (backpropagation algorithm). Specifically, the weights vary according to the following formula:

[0074]

[0075] Where γ is the learning rate, and if the (n+1)th layer is not the output layer, then it is based on δ. (n+1) j , number δ (n) j It can be recursively computed as

[0076] ,

[0077] And if the (n+1)th layer is the output layer 830, then

[0078] ,

[0079] Where f' is the first derivative of the activation function, and y (n+1) j It is the comparison training value of the j-th node of the output layer 830.

[0080] Figure 9 A convolutional neural network 900 according to one or more embodiments is illustrated. The convolutional neural network 900 can be used to implement the machine learning networks described herein, such as those that can be used to implement… Figure 1 AI-based systems Figure 4 The extra-domain detector module 404 and the AI-based system 408 Figure 5 AI-based system / outside-domain detector module 504 Figure 6 AI-based system 604 and out-of-domain detector module 608, and Figure 7 AI-based systems and machine learning-based networks for out-of-domain detector modules.

[0081] exist Figure 9 In the illustrated embodiment, the convolutional neural network 900 includes an input layer 902, a convolutional layer 904, a pooling layer 906, a fully connected layer 908, and an output layer 910. Alternatively, the convolutional neural network 900 may include several convolutional layers 904, several pooling layers 906, and several fully connected layers 908, as well as other types of layers. The order of the layers can be chosen arbitrarily; typically, the fully connected layer 908 is used as the last layer before the output layer 910.

[0082] Specifically, within the convolutional neural network 900, nodes 912-920 of layer 902-910 can be considered as arranged as a d-dimensional matrix or as an d-dimensional image. In particular, in the two-dimensional case, the values ​​of nodes 912-920 indexed by i and j in the nth layer 902-910 can be represented as x. (n) [i,j] However, the arrangement of nodes 912-920 in layer 902-910 has no effect on the computation performed within the convolutional neural network 900 itself, because these are given only by the structure and weights of the edges.

[0083] Specifically, convolutional layer 904 is characterized by the structure and weights of the incoming edges that form the convolution operation based on a specific number of kernels. Specifically, the structure and weights of the incoming edges are chosen such that the value x of node 914 of convolutional layer 904... (n) k The value x of node 912, calculated based on the previous layer 902, is... (n-1) convolution x (n) k = K k * x (n-1) Convolution * In the two-dimensional case, it is defined as

[0084] .

[0085] Here, the k-th kernel K k It is a d-dimensional matrix (a two-dimensional matrix in this embodiment), which is typically small compared to the number of nodes 912-918 (e.g., a 3×3 or 5×5 matrix). Specifically, this means that the weights of the incoming edges are not independent, but are chosen such that they produce the convolution equation. Specifically, for a kernel that is a 3×3 matrix, there are only 9 independent weights (each entry in the kernel matrix corresponds to one independent weight), regardless of the number of nodes 912-920 in the corresponding layers 902-910. Specifically, for convolutional layer 904, the number of nodes 914 in the convolutional layer is equivalent to the number of nodes 912 in the previous layer 902 multiplied by the number of kernels.

[0086] If the nodes 912 of the previous layer 902 were arranged as a d-dimensional matrix, then using multiple kernels can be interpreted as adding another dimension (denoted as the "depth" dimension) so that the nodes 914 of the convolutional layer 904 are arranged as a (d+1)-dimensional matrix. If the nodes 912 of the previous layer 902 were already arranged as a (d+1)-dimensional matrix including the depth dimension, then using multiple kernels can be interpreted as extending along the depth dimension so that the nodes 914 of the convolutional layer 904 are also arranged as a (d+1)-dimensional matrix, where the size of the (d+1)-dimensional matrix relative to the depth dimension is a multiple of the number of kernels in the previous layer 902.

[0087] The advantage of using convolutional layer 904 is that it can take advantage of the spatial local correlation of the input data by implementing local connection patterns between nodes in neighboring layers, in particular by each node connecting only to small regions of nodes in previous layers.

[0088] exist Figure 9 In the illustrated embodiment, the input layer 902 comprises 36 nodes 912 arranged in a two-dimensional 6×6 matrix. The convolutional layer 904 comprises 72 nodes 914 arranged in two two-dimensional 6×6 matrices, each of which is the result of the convolution of the input layer values ​​with the kernel. Equivalently, the nodes 914 of the convolutional layer 904 can be interpreted as arranged in a three-dimensional 6×6×2 matrix, where the last dimension is the depth dimension.

[0089] Pooling layer 906 can be characterized by the structure and weights of the incoming edges and the activation function of its nodes 916, which form pooling operations based on a nonlinear pooling function f. For example, in the two-dimensional case, the value x of node 916 of pooling layer 906 is... (n) It can be based on the value x of node 914 in the previous layer 904. (n-1) To calculate as

[0090] .

[0091] In other words, by using pooling layer 906, the number of nodes 914 and 916 can be reduced by replacing the number d1·d2 of adjacent nodes 914 in the previous layer 904 with a single node 916, which is calculated as a function of the number of adjacent nodes in the pooling layer. Specifically, the pooling function f can be a maximum function, an average function, or an L2 norm function. Specifically, for pooling layer 906, the weights of the incoming edges are fixed and not modified through training.

[0092] The advantage of using pooling layer 906 is that it reduces the number of nodes 914 and 916 and the number of parameters. This leads to a reduction in the amount of computation in the network and controls overfitting.

[0093] exist Figure 9 In the embodiment shown, pooling layer 906 is max pooling, which replaces four adjacent nodes with only one node, the maximum of the values ​​of the four adjacent nodes. Max pooling is applied to each d-dimensional matrix of the previous layer; in this embodiment, max pooling is applied to each of two two-dimensional matrices, thereby reducing the number of nodes from 72 to 18.

[0094] The fully connected layer 908 can be characterized by the fact that most, in particular all, edges exist between nodes 916 of the previous layer 906 and nodes 918 of the fully connected layer 908, and the weight of each of these edges can be adjusted individually.

[0095] In this embodiment, the nodes 916 of the preceding layer 906 of the fully connected layer 908 are displayed as a two-dimensional matrix, and additionally by nodes that are displayed as unrelated nodes (indicated as a row of nodes, where the number of nodes is reduced for better presentation). In this embodiment, the number of nodes 918 in the fully connected layer 908 is equal to the number of nodes 916 in the preceding layer 906. Alternatively, the number of nodes 916 and 918 may be different.

[0096] Furthermore, in this embodiment, the value of node 920 in output layer 910 is determined by applying the Softmax function to the value of node 918 in the previous layer 908. By applying the Softmax function, the sum of the values ​​of all nodes 920 in output layer 910 is 1, and all values ​​of all nodes 920 in the output layer are real numbers between 0 and 1.

[0097] The convolutional neural network 900 may also include ReLU (Rectified Linear Unit) layers or activation layers with nonlinear transfer functions. Specifically, the number and structure of nodes in the ReLU layer are identical to those in the previous layer. In particular, the value of each node in the ReLU layer is computed by applying a rectified function to the value of the corresponding node in the previous layer.

[0098] The inputs and outputs of different convolutional neural network blocks can be wired using summation (residual / dense neural networks), element-wise multiplication (attention), or other differentiable operators. Therefore, if the entire pipeline is differentiable, the convolutional neural network architecture can be nested rather than sequential.

[0099] Specifically, a convolutional neural network (CNN) 900 can be trained based on the backpropagation algorithm. To prevent overfitting, regularization methods can be used, such as dropping nodes 912-920, random pooling, using artificial data, and weight decay based on L1 or L2 norm or maximum norm constraints. Different loss functions can be combined to train the same neural network to reflect the joint training objective. A subset of neural network parameters can be excluded from the optimization to retain weights pre-trained on another dataset.

[0100] The systems, apparatus, and methods described herein can be implemented using digital circuitry or using one or more computers employing well-known computer processors, memory units, storage devices, computer software, and other components. Typically, a computer includes a processor for executing instructions and one or more memories for storing instructions and data. A computer may also include or be coupled to one or more mass storage devices, such as one or more disks, internal hard disks and removable disks, magneto-optical disks, optical disks, etc.

[0101] The systems, apparatus, and methods described herein can be implemented using computers operating in a client-server relationship. Typically, in such a system, the client computer is located remotely from the server computer and interacts via a network. The client-server relationship can be defined and controlled by computer programs running on the respective client and server computers.

[0102] The systems, apparatus, and methods described herein can be implemented in a network-based cloud computing system. In such a system, a server or other processor connected to the network communicates with one or more client computers via the network. For example, a client computer may communicate with the server via a web browser application residing and operating on the client computer. The client computer may store data on the server and access the data via the network. The client computer may transmit requests for data or online services to the server via the network. The server may perform the requested service and provide data to one or more client computers(s). The server may also transmit data suitable for causing the client computer to perform specific functions, such as performing calculations, displaying specified data on a screen, etc. For example, the server may transmit requests suitable for causing the client computer to perform one or more steps or functions of the methods and workflows described herein, including... Figure 1 Or one or more of steps or functions from 4-7. (Including) Figure 1 Some steps or functions of the methods and workflows described herein, such as steps 4-7, may be performed by a server or by another processor in a network-based cloud computing system. Figure 1Some steps or functions of the methods and workflows described herein, or steps 4-7, may be performed by a client computer in a web-based cloud computing system. Figure 1 The steps or functions of the methods and workflows described herein, or steps 4-7, may be performed by a server and / or by a client computer in a web-based cloud computing system in any combination.

[0103] The systems, apparatuses, and methods described herein can be implemented using a computer program product tangibly implemented in an information carrier (e.g., in a non-transitory machine-readable storage device) for execution by a programmable processor; and include Figure 1 The methods and workflow steps described herein, or one or more of the steps or functions in 4-7, may be implemented using one or more computer programs executable by such a processor. A computer program is a collection of computer program instructions that can be used directly or indirectly in a computer to perform a specific activity or produce a specific result. Computer programs can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0104] Figure 10 A high-level block diagram of an example computer 1002, which can be used to implement the systems, apparatus, and methods described herein, is depicted. Computer 1002 includes a processor 1004 and a memory 1010 operatively coupled to a data storage device 1012. The processor 1004 controls the overall operation of computer 1002 by executing computer program instructions that define such operations. The computer program instructions may be stored in the data storage device 1012 or other computer-readable medium and loaded into memory 1010 when execution of the computer program instructions is desired. Therefore, Figure 1 The methods and workflow steps or functions described in 4-7 may be defined by computer program instructions stored in memory 1010 and / or data storage device 1012, and controlled by processor 1004 that executes the computer program instructions. For example, the computer program instructions may be implemented as computer-executable code programmed by a person skilled in the art to perform... Figure 1 Or the methods and workflow steps or functions of 4-7. Therefore, by executing computer program instructions, processor 1004 performs... Figure 1Alternatively, methods and workflow steps or functions as described in 4-7. Computer 1002 may also include one or more network interfaces 1006 for communicating with other devices via a network. Computer 1002 may also include one or more input / output devices 1008 (e.g., monitor, keyboard, mouse, speaker, buttons, etc.) that enable a user to interact with computer 1002.

[0105] Processor 1004 may include both general-purpose and special-purpose microprocessors, and may be the sole processor of computer 1002 or one of multiple processors. For example, processor 1004 may include one or more central processing units (CPUs). Processor 1004, data storage device 1012, and / or memory 1010 may include one or more application-specific integrated circuits (ASICs) and / or one or more field-programmable gate arrays (FPGAs), supplemented or incorporated into one or more ASICs and / or one or more FPGAs.

[0106] Data storage device 1012 and memory 1010 each include a tangible, non-transitory, computer-readable storage medium. Data storage device 1012 and memory 1010 may each include high-speed random access memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), dual data rate synchronous dynamic random access memory (DDR RAM), or other random access solid-state memory devices, and may include non-volatile memory, such as one or more disk storage devices (such as internal hard disks and removable disks), magneto-optical disk storage devices, optical disk storage devices, flash memory devices, semiconductor memory devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM), digital universal disc read-only memory (DVD-ROM), or other non-volatile solid-state storage devices.

[0107] Input / output device 1008 may include peripheral devices such as printers, scanners, displays, etc. For example, input / output device 1008 may include a display device such as a cathode ray tube (CRT) or liquid crystal display (LCD) monitor for displaying information to a user, a keyboard, and a pointing device such as a mouse or trackball, through which the user can provide input to computer 1002.

[0108] Image acquisition device 1014 can be connected to computer 1002 to input image data (e.g., medical images) to computer 1002. It is possible to implement image acquisition device 1014 and computer 1002 as a single device. It is also possible for image acquisition device 1014 and computer 1002 to communicate wirelessly via a network. In a possible embodiment, computer 1002 may be remotely located relative to image acquisition device 1014.

[0109] Any or all of the systems and apparatuses discussed herein can be implemented using one or more computers (such as computer 1002).

[0110] Those skilled in the art will recognize that the implementation of an actual computer or computer system may have other structures and may also include other components, and Figure 10 It is a high-level representation of some components of such a computer for illustrative purposes.

[0111] The foregoing detailed description is to be understood as illustrative and exemplary in each aspect, and not restrictive, and the scope of the invention disclosed herein is not determined by the detailed description, but by the claims as interpreted in the full breadth permitted by patent law. It is to be understood that the embodiments shown and described herein are merely illustrative of the principles of the invention, and various modifications can be made by those skilled in the art without departing from the scope and spirit of the invention. Various other combinations of features can be implemented by those skilled in the art without departing from the scope and spirit of the invention.

Claims

1. A method comprising: Receive input data for input into an AI-based system; Model the in-domain feature space and the out-of-domain feature space of the AI-based system. The in-domain feature space corresponds to the features of the data that the AI-based system is trained to classify, and the out-of-domain feature space corresponds to the features of the data that the AI-based system is not trained to classify. Generate probability distribution functions for the input data and for the data trained to classify the AI-based system in both the in-domain and out-domain feature spaces; as well as Whether the input data is outside the domain of the AI-based system is determined based on the probability distribution function for the input data and the data trained to classify the AI-based system. The modeling of the intra-domain feature space and the extra-domain feature space of AI-based systems includes: The in-domain feature space is computed based on one or more in-domain linear projection matrices of the AI-based system. Calculate one or more orthogonal linear projection matrices for the out-of-domain feature space based on one or more in-domain linear projection matrices of the AI-based system; and The out-of-domain feature space is computed based on one or more orthogonal linear projection matrices for the out-of-domain feature space; Furthermore, the probability distribution functions generated for the input data and for the data trained to classify the AI-based system in the intra-domain feature space and the extra-domain feature space include: The probability distribution function is generated using a combination of Gaussian process models or Gaussian probability distribution models suitable for the available data.

2. The method according to claim 1, further comprising: In response to determining that the input data is outside the domain of the AI-based system, a notification is transmitted to the user to review the prediction generated by the AI-based system based on the input data.

3. The method according to claim 1, further comprising: In response to determining that the input data is outside the domain of the AI-based system: Comment on the input data; as well as Training AI-based systems based on annotated input data.

4. The method according to claim 1, further comprising: In response to the determination that the input data is outside the domain of the AI-based system, the AI-based system generates a prediction based on the input data.

5. The method according to claim 1, wherein, Receiving, modeling, generating, and determining are performed by modules combined with the AI-based system, and these modules generate predictions based on the input data and the determination of whether the input data is outside the domain of the AI-based system.

6. The method of claim 1, further comprising: Based on the determination, one of several algorithms for the AI-based system is selected.

7. The method of claim 1, wherein the AI-based system is used for medical imaging analysis.

8. An apparatus comprising: A device for receiving input data for input into an artificial intelligence (AI) based system; An apparatus for modeling the in-domain feature space and the out-of-domain feature space of an AI-based system, wherein the in-domain feature space corresponds to the features of data for which the AI-based system is trained to classify, and the out-of-domain feature space corresponds to the features of data for which the AI-based system is not trained to classify. Apparatus for generating probability distribution functions for input data and for data trained to classify AI-based systems in both in-domain and out-of-domain feature spaces; as well as A device for determining whether input data is outside the domain of an AI-based system based on a probability distribution function for the input data and the data trained to classify the AI-based system. The apparatus used for modeling the in-domain feature space and the out-of-domain feature space of AI-based systems includes: A device for calculating a domain feature space based on one or more domain linear projection matrices of an AI-based system; A means for calculating one or more orthogonal linear projection matrices for an out-of-domain feature space based on one or more in-domain linear projection matrices of an AI-based system; and A device for calculating an out-of-domain feature space based on one or more orthogonal linear projection matrices for the out-of-domain feature space; And the means for generating probability distribution functions for the input data and for the data trained to classify the AI-based system in the in-domain feature space and the out-of-domain feature space includes: A device for generating a probability distribution function using a Gaussian process model or a combination of Gaussian probability distribution models suitable for available data.

9. The apparatus according to claim 8, further comprising: A means for transmitting a notification to a user to review a prediction generated by the AI-based system based on the input data in response to determining that the input data is outside the domain of the AI-based system.

10. The apparatus according to claim 8, wherein, Receiving, modeling, generating, and determining are performed by modules combined with the AI-based system, and these modules generate predictions based on the input data and the determination of whether the input data is outside the domain of the AI-based system.

11. A non-transitory computer-readable medium storing computer program instructions, which, when executed by a processor, cause the processor to perform operations, said operations including: Receive input data for input into an AI-based system; Model the in-domain feature space and the out-of-domain feature space of the AI-based system. The in-domain feature space corresponds to the features of the data that the AI-based system is trained to classify, and the out-of-domain feature space corresponds to the features of the data that the AI-based system is not trained to classify. Generate probability distribution functions for the input data and for the data trained to classify the AI-based system in both the in-domain and out-domain feature spaces; as well as Whether the input data is outside the domain of the AI-based system is determined based on the probability distribution function for the input data and the data trained to classify the AI-based system. The modeling of the intra-domain feature space and the extra-domain feature space of AI-based systems includes: The in-domain feature space is computed based on one or more in-domain linear projection matrices of the AI-based system. Calculate one or more orthogonal linear projection matrices for the out-of-domain feature space based on one or more in-domain linear projection matrices of the AI-based system; and The out-of-domain feature space is computed based on one or more orthogonal linear projection matrices for the out-of-domain feature space; Furthermore, the probability distribution functions generated for the input data and for the data trained to classify the AI-based system in the intra-domain feature space and the extra-domain feature space include: The probability distribution function is generated using a combination of Gaussian process models or Gaussian probability distribution models suitable for the available data.

12. The non-transitory computer-readable medium of claim 11, wherein the operation further comprises: In response to determining that the input data is outside the domain of the AI-based system: A device for annotating input data; as well as A device for training AI-based systems using annotated input data.

13. The non-transitory computer-readable medium of claim 11, wherein the operation further comprises: In response to the determination that the input data is outside the domain of the AI-based system, the AI-based system generates a prediction based on the input data.

14. The non-transitory computer-readable medium of claim 11, wherein the operation further comprises: Based on the determination, one of several algorithms for the AI-based system is selected.

15. The non-transitory computer-readable medium of claim 11, wherein the AI-based system is used for medical imaging analysis.

Images