Method of training sub-module and preventing capturing ai module

Abstract

This disclosure presents a method for training a submodule (14) and preventing capture of an AI module (12). Input data received from an input interface (11) is sent to the AI ​​module (12) via an interceptor module (18), and the AI ​​module computes first output data by executing a first model (M). The submodule (14) in the AI ​​system (10) trained using method steps (200) processes the input data to identify attack vectors from the input data. The submodule (14) executes the first model (M) and at least a second model. The first model (M) and the second model have first and second sets of network parameters and hyperparameters, respectively. The identification information of the attack vectors is sent to an information gain module (16).

Description

[0001] Full instruction manual:

[0002] The following description defines and defines the nature of the invention and the manner in which it is carried out. Technical Field

[0003] This disclosure relates to a method for training a submodule in an AI system and a method for preventing the capture of AI modules in an AI system. Background Technology

[0004] With the advent of data science, data processing and decision-making systems are implemented using artificial intelligence (AI) modules. These AI modules employ various techniques, such as machine learning, neural networks, and deep learning. Most AI-based systems receive and process large amounts of data to train AI models. The trained AI models then generate outputs based on use cases requested by the user. Typically, AI systems are used in fields such as computer vision, speech recognition, natural language processing, audio recognition, healthcare, autonomous driving, manufacturing, and robotics, where they process data to generate desired outputs based on certain rules / intelligence acquired through training.

[0005] To process input and produce the desired output, AI systems employ various models / algorithms, which are trained using training data. Once trained, the AI ​​system uses these models to analyze real-time data and generate appropriate results. The models can be fine-tuned in real-time based on the results. The models within an AI system form the core of the system. Numerous efforts, resources (tangible and intangible), and knowledge are invested in developing these models.

[0006] It is possible that an adversary might attempt to capture / copy / extract the model from an AI system. An adversary could use various techniques to capture the model. One simple technique employed by an adversary involves iteratively sending different queries to the AI ​​system using their own test data. This test data can be designed to extract internal information about the model's performance within the AI ​​system. The adversary uses the generated results to train their own model. By iteratively performing these steps, it is possible to capture the model's internals and build similar models using similar logic. This would pose a significant challenge to the original developers of the AI ​​system. This challenge could manifest as business flaws, loss of confidential information, wasted time spent on development, loss of intellectual property, and loss of future revenue.

[0007] Several methods are known in the prior art for identifying such attacks by adversaries and protecting the models used in AI systems. Prior art US 20190095629A1, "Protecting Cognitive Systems from Model Stealing Attacks," discloses one such method. It discloses a method in which input data is processed by applying a trained model to generate an output vector having values ​​for each of a plurality of predefined classes. A query engine modifies the output vector by inserting a query into a function associated with generating the output vector, thereby generating a modified output vector. The modified output vector is then output. The query engine modifies one or more values ​​to disguise the trained configuration of the trained model logic while maintaining the accuracy of the classification of the input data. Attached Figure Description

[0008] Embodiments of the present invention are described with reference to the following figures:

[0009] Figure 1 Describing AI systems;

[0010] Figure 2 Describe the sub-modules of the AI ​​system;

[0011] Figure 3 The method steps for training submodules in an AI system are shown; and

[0012] Figure 4 The method steps for preventing the capture of AI modules in an AI system are shown. Detailed Implementation

[0013] Understanding artificial intelligence (AI) technologies and AI-based systems, or aspects of AI systems, is important. This disclosure covers two aspects of AI systems. The first aspect relates to the training of submodules within an AI system, while the second aspect relates to preventing the capture of AI modules within an AI system.

[0014] Some important aspects of AI technology and AI systems can be explained as follows. Depending on the architecture of the implementation, an AI system can include many components. One such component is an AI module. Referring to the AI ​​module of this disclosure, it can be interpreted as a component that runs a model. A model can be defined as a reference or inference set of data, using different forms of correlation matrices. Using these models and the data from them, correlations can be established between different types of data to achieve a certain logical understanding of the data. Those skilled in the art will be familiar with different types of AI models, such as linear regression, Naive Bayes classifiers, support vector machines, neural networks, etc. It should be understood that this disclosure is not specific to the type of model executed in an AI module, but can be applied to any AI module regardless of the AI ​​model executed. Those skilled in the art will also understand that an AI module can be implemented as a set of software instructions, a combination of software and hardware, or any combination thereof.

[0015] Some typical tasks performed by AI systems are classification, clustering, and regression. Most classification tasks rely on labeled datasets; that is, the dataset is manually labeled so that the neural network learns the correlation between the labels and the data. This is called supervised learning. Some typical applications of classification are: face recognition, object identification, pose recognition, and speech recognition. Clustering, or grouping, detects similarities in inputs. Clustering learning techniques do not require labels to detect similarities. Learning without labels is called unsupervised learning. Most of the data in the world is unlabeled. A general rule of machine learning is that the more data an algorithm can be trained on, the more accurate it will be. Therefore, unsupervised learning models / algorithms have the potential to produce accurate models as the size of the training dataset increases.

[0016] Since the AI ​​module forms the core of an AI system, it needs to be protected from attacks. Attackers attempt to compromise the model within the AI ​​module and steal information from it. Attacks are launched using attack vectors. In computing technology, a vector can be defined as a method in which malicious code / virus data is used to propagate itself, such as infecting a computer, computer system, or computer network. Similarly, an attack vector is defined as a path or means by which a hacker can gain access to a computer or network to deliver payloads or malicious results. One type of model-stealing attack uses an attack vector that can create a digital twin / copy / replica of the AI ​​module.

[0017] Attackers typically generate random queries of varying sizes and shapes as input specifications and begin querying the model with these arbitrary queries. This query produces input-output pairs for the random queries and generates a secondary dataset inferred from a pre-trained model. The attacker then takes this I / O pair and uses the secondary dataset to train a new model from scratch. This is a black-box model attack vector where no prior knowledge of the original model is required. As prior information about the model becomes available and increases, attackers evolve towards more intelligent attacks. Attackers select relevant datasets at their disposal to extract the model more efficiently. This is an attack vector based on domain-intelligent models. Using these methods, model-stealing attacks can be demonstrated across different models and datasets.

[0018] It must be understood that this disclosure specifically discloses methods for training submodules in an AI system and methods for preventing the capture of AI modules in an AI system. While these methods describe only a series of steps to achieve these objectives, they are implemented in an AI system, which can be a combination of hardware and software, or a combination thereof.

[0019] Figure 1 An AI system (10) is depicted. The AI ​​system (10) includes an input interface (11), an interceptor module (18), an AI module (12), a submodule (14), an interceptor notification module (20), an information gain module (16), and at least an output interface (22). The input interface (11) receives input data from at least one user. The input interface (11) is a hardware interface in which the user can input his / her query to the AI ​​module (12).

[0020] The interceptor module (18) is configured to intercept the user upon information gain. The information gain is calculated based on the input attack query exceeding a predefined threshold. The interceptor module (18) is also configured to modify the first output generated by the AI ​​module (12). This is done only when the input is identified as an attack vector.

[0021] The AI ​​module (12) processes the input data and generates first output data corresponding to the input. The AI ​​module (12) executes a first model (M) based on the input to generate a first output. This model can be any of a group consisting of artificial neural networks, convolutional neural networks, recurrent neural networks, etc. The first model includes a first set of network parameters and hyperparameters.

[0022] Neural networks are generated from biological neural networks or brain cells, namely neurons. Network parameters include, but are not limited to, layers, filters, etc. For simplicity, in computer science, a network of neurons is represented as a set of layers. These layers are classified into three categories: input, hidden, and output. Each network has a single input layer and a single output layer. Different layers perform different kinds of transformations / operations on their inputs. Data flows through the network starting from the input layer and moves through the hidden layers until it reaches the output layer. The layer located between the input and output layers is called a hidden layer. However, the number of hidden layers varies depending on the requirements or complexity of the operation to be performed. Filters are primarily used in convolutional neural networks (CNNs). Filters are used to slice through the data using convolutions and map them one by one, learning different parts of the input data. In the case of images, filters slice through the image and map it to learn its different parts. Again, the number of filters in a CNN varies depending on the requirements or complexity of the operation to be performed. Hyperparameters are parameters whose values ​​are used to control the learning process. Hyperparameters can be given / selected when learning network parameters during the training phase. Hyperparameters are typically characterized by the learning rate, learning mode, and batch size. In principle, they have a limited impact on the performance of the model, but they can affect the speed and quality of the learning process.

[0023] Submodule (14) is configured to identify attack vectors from the received input data. Figure 2 A submodule (14) in the AI ​​system (10) is depicted. The submodule (14) includes a first model, at least a second model, and a comparator (143). The second model includes a second set of network parameters and hyperparameters. For example, if the first model has “m1” layers and “m2” filters, corresponding to the first set of hyperparameters (e.g., a learning rate of “a”), then the second model will have “n1” layers and “n2” filters, corresponding to the second set of hyperparameters (e.g., a learning rate of “b”). Those skilled in the art will understand that for different forms of data, “n” models (with “n” sets of network parameters and hyperparameters) will be required. The value of “n” is dynamic, i.e., the number of models executed by the submodule will change. This depends on the current and historical values ​​of the information gain calculated by the information gain module. The comparator (143) receives and compares the outputs received when various models are executed with the same input.

[0024] The interceptor notification module (20) sends a notification to the owner of the AI ​​system (10) when an attack vector is detected. The notification can be sent in any audio / video / text format.

[0025] The information gain module (16) is configured to calculate the information gain and send the information gain value to the interceptor module (18). The information gain is calculated using an information gain method. In one embodiment, if the extracted information gain exceeds a predefined threshold, the AI ​​system (10) is configured to lock the user out of the system. If the cumulative information gain extracted by multiple users exceeds the predefined threshold, the locking system is activated.

[0026] The output interface (22) sends output to the at least one user. When the submodule (14) does not identify an attack vector from the received input, the output sent by the output interface (22) includes first output data. When an attack vector is detected from the input, the output sent by the output interface (22) includes modified output received from the interceptor module (18).

[0027] It must be understood that each building block of the AI ​​system (10) can be implemented in different architectural frameworks depending on the application. In one embodiment of the architectural framework, all building blocks of the AI ​​system (10) are implemented in hardware, i.e., each building block can be hard-coded into a microprocessor chip. This is particularly possible when the building blocks are physically distributed over a network (where each building block resides on a separate computer system across the network). In another embodiment of the architectural framework of the AI ​​system (10), it is implemented as a combination of hardware and software, i.e., some building blocks are hard-coded into a microprocessor chip, while others are implemented in software that can reside in the microprocessor chip or in the cloud.

[0028] Figure 3 The method steps (200) for training a submodule (14) in the AI ​​system (10) are shown. The AI ​​system (10) includes the above-described steps. Figure 1 and Figure 2 The components described herein. Submodule (14) is trained using the dataset used to train AI module (12). Submodule (14) is trained using the dataset used to train AI module (12). Submodule (14) executes a first model (M) and a second model, and includes a comparator for comparing the outputs of at least two models. The first model (M), as explained in the preceding paragraphs, is executed by AI module (12) and includes a first set of network parameters and hyperparameters. The second model includes a second set of network parameters and hyperparameters.

[0029] In step 201, the first model (M) and at least the second model receive the original dataset as input and perform an operation using the input. The at least two models contain different numbers of classes or different classes labeled. The total class value is different when the attack vector passes through all these models. If the class values ​​are different, the data pointer is considered the attack vector. In step 202, the behavior of the submodule (14) is recorded.

[0030] In performing method step (200), in embodiments of the invention, the at least first model and the at least second model use different network initialization methods. Network initialization methods essentially initialize the model weights with small random numbers. Initializing neural networks is a crucial part of deep learning. The method of initializing neural networks determines whether they can converge well and quickly. In this invention, the weights are initialized in ways that differ between the mean and variance of the first model (M) and the at least second model. For example, in embodiments, the first model (M) can be initialized using zero initialization (network weights are initialized with zero), and the at least second model can be initialized using random initialization (network weights are initialized with random numbers other than zero).

[0031] Figure 4 Method steps (300) for preventing the capture of AI module (12) in AI system (10) are shown. AI system (10) and its components have been described in the preceding paragraphs using... Figure 1 and Figure 2 An explanation has been provided. Those skilled in the art will understand that the submodule (14) trained by method step (200) is now used in real time to prevent the capture of the AI ​​module (12) in the AI ​​system (10).

[0032] In step 301, the input interface (11) receives input data from at least one user. In step 302, the input data is sent to the AI ​​module (12) via the interceptor module (18). In step 303, the AI ​​module (12) calculates first output data by executing a first model (M) based on the input data.

[0033] In step 304, while the submodule (14) processes the input data to identify the attack vector, the identification information of the attack vector is sent to the information gain module (16). Processing the input data also includes two stages. First, the first model (M) and at least a second model within the submodule (14) are executed using the input data. The first model includes a first set of network parameters and hyperparameters. The second model includes a second set of network parameters and hyperparameters. Next, the outputs received during the execution of the at least two models are compared. The attack vector is determined from the input based on this comparison. If the received outputs are identical, it means the input is not an attack vector. However, if the comparator (143) detects a difference in the output, it infers that the input is an attack vector.

[0034] Once the attack vector identification information is sent to the information gain module (16), the information gain is calculated. The information gain is then sent to the interceptor module (18). In this embodiment, if the information gain exceeds a predefined threshold, the user is intercepted, and a notification is sent to the owner of the AI ​​system (10) using the interceptor notification module (20). If the information gain is below the predefined threshold, the interceptor module (18) can modify the first output generated by the AI ​​module (12) to send it to the output interface (22), even though an attack vector has been detected.

[0035] Furthermore, user profiles can be used to determine whether a user is a habitual attacker, a one-off attack, or merely an isolated incident. Based on the user profile, the steps to unlock the system can be determined. If it is a first-time attacker, the user may be temporarily locked out. If the attacker is a habitual attacker, more stringent locking procedures can be recommended.

[0036] It must be understood that the embodiments described in the above detailed description are merely illustrative and not intended to limit the scope of the invention. Any modifications to the methods of training submodule (14) and preventing capture of AI module (12) are contemplated and constitute part of this invention. The scope of the invention is defined only by the claims.

Claims

1. An AI system (10) for preventing capture of the AI ​​module (12), comprising at least: Input interface (11) for receiving input data from at least one user; The interceptor module (18) is configured to intercept at least one user to prevent the capture of the AI ​​module (12). The AI ​​module (12) is used to process the input data and generate first output data corresponding to the input data, and the AI ​​module executes the first model; Submodule (14) includes the first model and at least one second model, wherein the first model and the second model contain the number of different classes or categories of the tags, and the submodule is configured to execute the first model (M) and the at least one second model, compare the outputs received when executing the at least two models, and identify an attack vector based on the comparison; The information gain module (16) is configured to calculate the information gain and send the information gain value to the interceptor module (18). An interceptor notification module (20) is configured to send a notification to the owner of the AI ​​system (10) upon detection of an attack vector. The interceptor notification module (20) is also configured to modify first output data generated by the AI ​​module (12). Output interface (22) is used to send output to the at least one user.

2. The AI ​​system (10) according to claim 1, wherein, When the submodule (14) fails to identify the attack vector from the received input, the output sent by the output interface (22) includes the first output data.

3. The AI ​​system (10) according to claim 1, wherein, The first model includes a first set of network parameters and hyperparameters.

4. The AI ​​system (10) according to claim 1, wherein, The second model includes a second set of network parameters and hyperparameters.

5. A method for training a submodule (14) in an AI system (10) according to any one of claims 1-4, the AI ​​system (10) comprising at least one AI module (12) executing a first model (M), a dataset for training the AI ​​module (12), the submodule (14) executing the first model (M) and at least one second model, the submodule comprising a comparator for comparing the outputs of at least two models, the method comprising the following steps: The dataset is used to execute a first model (M) and at least one second model in the submodule (14), the first model (M) including a first set of network parameters and hyperparameters, and the second model including a second set of network parameters and hyperparameters; Record the behavior of the submodule (14), The datasets mentioned are associated with computer vision, speech recognition, natural language processing, audio recognition, healthcare, autonomous driving, manufacturing, and robotics.

6. The method for training a submodule (14) in the AI ​​system (10) according to claim 5, wherein, The at least first model and the at least one second model use different technical network initialization methods.

7. A method for preventing the capture of an AI module (12) in an AI system (10), the method comprising the steps of: Input data is received from at least one user through the input interface (11); The input data is sent to the AI ​​module (12) through the interceptor module (18); The first output data is calculated by executing the first model (M) based on the input data through the AI ​​module (12); The input data is processed by submodule (14) to identify the attack vector from the input data, and the identification information of the attack vector is sent to the information gain module (16). The processing of the input data further includes: Execute the first model (M) and at least one second model, wherein the first model and the second model contain different classes or the number of classes of the tags; Compare the outputs received when executing the at least two models; The input data is determined as an attack vector based on the comparison.

8. The method for preventing the capture of the AI ​​module (12) in the AI ​​system (10) according to claim 7, wherein, The first model includes a first set of network parameters and hyperparameters.

9. The method for preventing the capture of the AI ​​module (12) in the AI ​​system (10) according to claim 7, wherein, The second model includes a second set of network parameters and hyperparameters.

Images