Threat modeling methods, query methods and related components for artificial intelligence application systems

Abstract

The present invention provides a threat modeling method, a query method and related components for an artificial intelligence application system, which are applied to the field of computer technology. In order to solve the problem that threat modeling of an artificial intelligence application system cannot be implemented in related technologies, the method includes: constructing a data flow diagram based on each element in the artificial intelligence application system and the data flow between the elements; the elements are components in the artificial intelligence application system; for each data flow, determining the relevant elements corresponding to the data flow; determining the threats corresponding to the data flow and the corresponding relevant elements according to each preset threat type; formulating countermeasures corresponding to the threats; generating threat modeling results based on each data flow, the threats corresponding to the data flow and the corresponding relevant elements, and the countermeasures corresponding to the threats; the present invention can implement threat modeling of the artificial intelligence application system during use, which is beneficial to improving the security and reliability of the artificial intelligence application system.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a threat modeling method, threat query method, device, electronic device, and computer-readable storage medium for an artificial intelligence application system. Background Art

[0002] Software systems are often exposed to a variety of threats, which can originate from both external and internal sources and have potentially devastating consequences. Threat modeling is an engineering- and risk-based approach for identifying, assessing, and managing security threats. The goal is to develop and deploy better software and IT (information technology) systems that meet an organization's security and risk objectives.

[0003] As artificial intelligence (AI) system applications gradually become a key component of enterprise digital development, AI applications face more security risks. Therefore, it is necessary to identify and protect the security risks of AI application systems and improve the security and reliability of AI application systems.

[0004] In view of this, how to implement threat modeling of artificial intelligence application systems is a problem that technical personnel in this field need to solve. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a threat modeling method, threat query method, device, electronic device and computer-readable storage medium for an artificial intelligence application system, which can realize threat modeling of the artificial intelligence application system and is conducive to improving the security and reliability of the artificial intelligence application system.

[0006] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0007] An embodiment of the present invention provides a threat modeling method for an artificial intelligence application system, including:

[0008] Construct a data flow diagram based on the elements in the artificial intelligence application system and the data flows between the elements; the elements are components in the artificial intelligence application system;

[0009] For each of the data streams, determining a relevant element corresponding to the data stream;

[0010] Determining threats corresponding to the data stream and corresponding related elements according to various preset threat types;

[0011] Develop countermeasures commensurate with the threats described;

[0012] A threat modeling result is generated according to each of the data flows, the threats corresponding to the data flows and the corresponding related elements, and the countermeasures corresponding to the threats.

[0013] In some embodiments, determining the threat corresponding to the data flow and the corresponding related elements according to each preset threat type includes:

[0014] Obtaining the content type of the data stream;

[0015] Determining the confidentiality requirement of the data stream according to the content type of the data stream; the confidentiality requirement is whether the data information needs to be encrypted;

[0016] Determining, based on the content type of the data stream, an integrity requirement of the data stream; the integrity requirement being whether integrity protection is performed on the data;

[0017] Obtaining element information of relevant elements corresponding to the data stream;

[0018] Determining a target threat type and threats corresponding to the target threat type based on the preset threat types, in combination with confidentiality requirements and integrity requirements of the data flow and element information of the relevant elements;

[0019] The threat types include:

[0020] At least two of the following threats: phishing threat, tampering threat, repudiation threat, information leakage threat, denial of service threat, privilege escalation threat, data security threat, and model security threat.

[0021] In some embodiments, the element information includes a combination of one or more of element type, element security mechanism, data network transmission protocol, data transmission mechanism, element storage type, storage data type, and trust boundary type.

[0022] In some embodiments, formulating a response measure corresponding to the threat includes:

[0023] Performing a risk assessment on the threat to determine a threat score for the threat;

[0024] Determining a threat level corresponding to the threat score based on a pre-established correspondence between the threat score and the threat level;

[0025] According to the threat level, a response measure corresponding to the threat is formulated.

[0026] In some embodiments, performing risk assessment on the threat and determining a threat score of the threat includes:

[0027] Establishing a threat scoring table in advance based on threat directions and threat results corresponding to different threat values ​​under the threat directions;

[0028] determining a threat result of the threat in each threat direction;

[0029] Determining a threat value corresponding to the threat direction according to the threat result;

[0030] The threat values ​​corresponding to the respective threat directions are accumulated to obtain a threat score corresponding to the threat.

[0031] In some embodiments, the threat corresponding to the data security threat includes one or more of a data poisoning attack, an adversarial sample attack, and a privacy leak;

[0032] And / or, the threats corresponding to the model security include one or more of prompt injection attacks, member inference and sensitive data extraction, model extraction, and output content compliance.

[0033] Another embodiment of the present invention provides a threat query method for an artificial intelligence application system, including:

[0034] Get threat query instructions;

[0035] Obtaining current data flow information according to the threat query instruction;

[0036] Determining, according to the current data stream information, a current related element corresponding to the current data stream information;

[0037] Determining corresponding threats and countermeasures from pre-established threat modeling results based on the current data flow information and the corresponding current related elements; wherein the threat modeling results are obtained based on the threat modeling method for the artificial intelligence application system as described above;

[0038] Present the threats and countermeasures.

[0039] Another embodiment of the present invention provides a threat modeling device for an artificial intelligence application system, comprising:

[0040] A construction module, configured to construct a data flow diagram based on the elements in the artificial intelligence application system and the data flows between the elements; the elements are components in the artificial intelligence application system;

[0041] A first determining module, configured to determine, for each of the data streams, a related element corresponding to the data stream;

[0042] A second determining module is configured to determine threats corresponding to the data stream and the corresponding related elements according to various preset threat types;

[0043] a formulation module for formulating countermeasures corresponding to the threats;

[0044] The generating module is configured to generate a threat modeling result based on each of the data flows, the threats corresponding to the data flows and the corresponding related elements, and the countermeasures corresponding to the threats.

[0045] Another embodiment of the present invention provides a threat query device for an artificial intelligence application system, including:

[0046] A first acquisition module is used to obtain a threat query instruction;

[0047] A second acquisition module is used to obtain current data flow information according to the threat query instruction;

[0048] A third determining module, configured to determine a current related element corresponding to the current data stream information according to the current data stream information;

[0049] a fourth determination module, configured to determine, based on the current data flow information and the corresponding current related elements, corresponding threats and countermeasures from pre-established threat modeling results; wherein the threat modeling results are obtained based on the threat modeling method for the artificial intelligence application system as described above;

[0050] The display module is used to display the threats and the countermeasures.

[0051] Another embodiment of the present invention provides an electronic device, including:

[0052] memory for storing computer programs;

[0053] A processor is configured to execute the computer program to implement the steps of the threat modeling method for the artificial intelligence application system as described above, or to execute the computer program to implement the threat query steps of the artificial intelligence application system as described above.

[0054] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements the steps of the threat modeling method of the artificial intelligence application system as described above, or executes the computer program to implement the threat query steps of the artificial intelligence application system as described above.

[0055] It can be seen from the above technical solution that the beneficial effects of the present invention are:

[0056] An embodiment of the present invention provides a threat modeling method for an artificial intelligence application system, which takes components in the artificial intelligence application system as elements, constructs a data flow diagram based on the various elements in the artificial intelligence application system and the data flows between the elements, and then determines the relevant elements corresponding to each data flow. Further, based on various preset threat types, the threats corresponding to the data flow and the corresponding relevant elements are determined, and countermeasures corresponding to the threats are formulated, thereby obtaining threats and countermeasures corresponding to each data flow and the corresponding relevant elements. Further, based on the various data flows, the threats corresponding to the data flows and the corresponding relevant elements, and the countermeasures corresponding to the threats, a threat modeling result can be generated. That is, the present invention can realize threat modeling of the artificial intelligence application system during use, which is conducive to improving the security and reliability of the artificial intelligence application system.

[0057] In addition, the present invention also provides corresponding implementation devices, electronic devices and computer-readable storage media for the threat modeling method of artificial intelligence application systems, further making the method more practical, and the devices, electronic devices and computer-readable storage media have corresponding advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0059] Figure 1 A flowchart of a threat modeling method for an artificial intelligence application system provided by an embodiment of the present invention;

[0060] Figure 2 A data flow diagram of an artificial intelligence application system provided by an embodiment of the present invention;

[0061] Figure 3 A flowchart of a threat query method for an artificial intelligence application system provided by an embodiment of the present invention;

[0062] Figure 4 A schematic diagram of the structure of a threat modeling device for an artificial intelligence application system provided by an embodiment of the present invention;

[0063] Figure 5 A schematic diagram of the structure of a threat query device for an artificial intelligence application system provided by an embodiment of the present invention;

[0064] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0066] The terms "including" and "having," as used in the present description and accompanying drawings, and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements and may include steps or elements that are not listed.

[0067] In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0068] Next, a threat modeling method for an artificial intelligence application system provided by an embodiment of the present invention is introduced in detail. Figure 1 A flowchart of a threat modeling method for an artificial intelligence application system provided by an embodiment of the present invention includes:

[0069] S110: Construct a data flow diagram based on the elements in the artificial intelligence application system and the data flows between the elements; the elements are components in the artificial intelligence application system;

[0070] It should be noted that when conducting threat modeling for an artificial intelligence application system, a data flow diagram of the artificial intelligence application system can be constructed based on the various components of the artificial application system, the data processing process between components, the data storage situation, and the data flow between components. This data flow diagram is also called a data flow diagram. The data flow diagram graphically expresses the logical functions of the system, the logical flow of data within the system, and the logical transformation process from the perspective of data transmission and processing. It is the main expression tool of the structured system analysis method and a graphical method for representing software models.

[0071] like Figure 2 As shown in the figure, for an artificial intelligence application system including components such as artificial intelligence application access interface, artificial intelligence application, artificial intelligence model, external API (Application Programming Interface), knowledge vector library and knowledge base, the following can be constructed based on the data transmission between the components. Figure 2The data flow diagram shown in FIG. 4 is a diagram in which the data of each arrow represents a data flow. For example, the user question is the data flow corresponding to the data sent from the artificial intelligence application access interface to the artificial intelligence application, and the question answer is the data flow corresponding to the data sent from the artificial intelligence application to the artificial intelligence application access interface.

[0072] S120: For each data stream, determining a related element corresponding to the data stream;

[0073] It is understood that for each data flow in the data flow diagram, the relevant elements of the data flow can be determined based on the relevant information of the data flow, that is, the sending end and receiving end of the data flow can be determined. For example, for the data flow of user questions, its relevant elements are the artificial intelligence application access interface and the artificial intelligence application, where the artificial intelligence application access interface is the sending end of the data flow and the artificial intelligence application is the receiving end of the data flow. Based on this method, the relevant elements corresponding to each data flow can be determined.

[0074] S130: Determine threats corresponding to the data stream and corresponding related elements according to various preset threat types;

[0075] It should be noted that in actual applications, multiple threat types can be pre-determined, and threat analysis for each threat type can be performed for each data flow and its corresponding related elements. This will determine the threat type corresponding to the data flow and its corresponding related elements, as well as the threats corresponding to that threat type. For example, if the threat types include at least two of the following: impersonation threat, tampering threat, repudiation threat, information leakage threat, denial of service threat, privilege escalation threat, data security threat, and model security threat, then an impersonation threat analysis will be performed on data flow 1 and its related elements (components A and B) to determine whether an impersonation threat exists. If an impersonation threat exists, the specific impersonation threats will be further determined. Tampering threat analysis will then be performed on data flow 1 and its related elements (components A and B) to determine whether a tampering threat exists. If a tampering threat exists, the specific tampering threats will be further determined. This will allow the identification of the various threats against data flow 1 and its related elements (components A and B).

[0076] S140: Develop countermeasures corresponding to the threats;

[0077] Specifically, after identifying the various threats to the data flow and related elements, countermeasures can be further formulated for each threat. In actual applications, corresponding countermeasures can be manually formulated based on the threat. The system obtains countermeasures corresponding to the threat that are manually input or pre-stored in the database.

[0078] S150: Generate threat modeling results based on each data flow, the threats corresponding to the data flow and the corresponding related elements, and the corresponding countermeasures to the threats.

[0079] It should be noted that the data flow and the corresponding related elements can be regarded as a data group. The above method can be used to obtain the threats corresponding to each data group and the corresponding countermeasures. Furthermore, based on the threats corresponding to each data group and the corresponding countermeasures, the threat modeling results of the artificial intelligence application system can be generated, thereby completing the threat modeling of the artificial intelligence application system.

[0080] The embodiments of the present invention take into account the increased security risks faced by AI applications. For example, these risks include AI algorithm security challenges and model algorithm attacks, modifications, and theft. Furthermore, the machine learning nature of AI can lead to new attacks, and AIGC (Artificial Intelligence Generated Content) poses new risks of data leakage and misuse. During the development of AI application systems, large conversational datasets must be collected for training. This process involves increasingly prominent data security issues such as illegal data acquisition, data leakage, and malicious misuse. Therefore, in the embodiments of the present invention, given the integration of AI application systems with AI models and data, potential threats to AI applications may include at least two of the following: impersonation, tampering, repudiation, information leakage, denial of service, privilege escalation, data security threat, and model security threat. In practical applications, data flows and their corresponding elements can be analyzed based on these eight threat categories: impersonation, tampering, repudiation, information leakage, denial of service, privilege escalation, data security threat, and model security threat, thereby further enhancing security threat analysis for AI application systems.

[0081] It should be noted that data security threats to AI application systems, for trained AI models and newer third-party generative AI systems, can include one or more of the following attack surface threats: data poisoning attacks, adversarial sample attacks, and privacy leaks.

[0082] Data poisoning attacks involve intentionally or maliciously introducing false, malicious, or harmful data into a dataset to manipulate, damage, or deceive the performance and output of an AI model. For example, attackers can insert data with misleading labels or features to distort the model's learning process, causing it to deviate from its representation of real data.

[0083] Adversarial sample attacks have become a common attack method for AI applications. Attackers add subtle, imperceptible perturbations to clean samples to cause them to make incorrect predictions or classifications. This phenomenon is called adversarial sample attacks. For example, adding a small piece of tape to a stop sign could confuse the image recognition model embedded in a self-driving car, which could have serious consequences.

[0084] Regarding privacy breaches, AI systems may need to collect and store large amounts of personal data for training and learning. This data may contain sensitive information, such as personal identity, health status, and financial transaction records. If this data is leaked or accessed by unauthorized persons, personal privacy will be seriously violated. In addition, if the AI ​​system is hacked, the attacker may obtain this data, resulting in greater privacy and security risks.

[0085] Model security threats to AI application systems. For trained AI models and newer third-party generative AI systems, model security threats may include one or more of the following attack surface threats: prompt injection attacks, member inference and sensitive data extraction, model extraction, and output content compliance.

[0086] Prompt injection attacks include jailbreaking, prompt leakage, and token smuggling. In these attacks, attackers manipulate input prompts to trigger unexpected behaviors in AI models, potentially causing the AI ​​system to produce inappropriate responses or leak sensitive information. These attacks are particularly powerful when AI systems are combined with other systems or used in software application chains.

[0087] For membership inference and sensitive data extraction, attackers can exploit membership inference attacks to binary infer whether a specific data point is in the training set, thereby causing privacy issues. Data extraction attacks allow attackers to completely reconstruct sensitive information about the training data from the model's responses. This is common when AI models are trained on private datasets, which may contain sensitive organizational data. Attackers can extract confidential information by creating specific prompts.

[0088] Model extraction is a newly discovered form of malicious attack in which an attacker aims to replicate a proprietary trained machine learning model based on the model's queries and responses. For example, an attacker crafts a series of queries and exploits the model's responses to build a replica of the target AI application. Furthermore, possessing a replica of the model allows attackers to perform adversarial attacks or reverse engineer training data, leading to other threats.

[0089] In terms of output content compliance, while providing high-quality text output, AI applications must ensure that the generated content strictly complies with relevant laws and regulations, including social morals and ethical standards, avoid promoting any improper values, and avoid containing any form of discrimination, hate speech, or other speech that may cause social disharmony.

[0090] In addition, for the specific definitions of counterfeit threats, tampering threats, repudiation threats, information leakage threats, denial of service threats, and privilege escalation threats in the embodiments of the present invention, please refer to Table 1:

[0091] Table 1 Threat types and definitions

[0092]

[0093] In some embodiments, the process of determining the threat corresponding to the data flow and the corresponding related elements according to each preset threat type in S130 may include:

[0094] Get the content type of the data stream;

[0095] Determine the confidentiality requirements of the data stream based on the content type of the data stream; the confidentiality requirement is whether the data information needs to be encrypted;

[0096] Determine the integrity requirements of the data stream based on the content type of the data stream; the integrity requirement is whether to perform integrity protection on the data;

[0097] Obtaining element information of relevant elements corresponding to the data stream;

[0098] According to each preset threat type, combined with the confidentiality requirements and integrity requirements of the data flow and the element information of the relevant elements, the target threat type and each threat corresponding to the target threat type are determined;

[0099] Among them, the various threat types include:

[0100] At least two of the following threats: phishing threat, tampering threat, repudiation threat, information leakage threat, denial of service threat, privilege escalation threat, data security threat, and model security threat.

[0101] It should be noted that in embodiments of the present invention, when performing threat determination for a data stream and its corresponding related elements, the content type of the data stream can be obtained. Specifically, the content type of a data stream between two elements in the artificial intelligence application system that transmit information can be predetermined based on the business scope of the artificial intelligence application system. Thus, the content type of the data stream can be determined based on the data stream and its related elements. Furthermore, based on the content type of the data stream, the confidentiality requirements and integrity requirements of the data stream can be determined. In other words, whether the data stream requires encryption and whether data integrity protection is required can be determined. Furthermore, based on the related elements corresponding to the data stream, element information of the related elements can be obtained. For example, if the related elements corresponding to data stream 1 are element A and element B, element information of element A and element B can be obtained. Then, combining the element information of data stream 1, element A, and element B, threat analysis can be performed for each threat type. This can determine what types of threats exist for data stream 1 and elements A and B (i.e., determine the target threat type), and what specific threats under that threat type will be received (i.e., determine the various threats under the target threat type).

[0102] Among them, the element information in the embodiment of the present invention may include one or more combinations of element type, element security mechanism, data network transmission protocol, data transmission mechanism, element storage type, storage data type and trust boundary type.

[0103] It should be noted that, the element information table shown in Table 2 below may be referred to, and the element information of each element may be determined according to the element information table.

[0104] Table 2 Element information table

[0105]

[0106] Among them, Web stands for World Wide Web, HTTP stands for Hypertext Transfer Protocol, HTTPS stands for Hypertext Transfer Protocol Secure, IPSec stands for Internet Protocol Security, TCP stands for Transmission Control Protocol, UDP stands for User Datagram Protocol, RPC stands for Remote Procedure Call, SQL stands for Structured Query Language, XML stands for Extensible Markup Language, and HTML stands for Hyper Text Markup Language.

[0107] That is, for each element, the element type, element security mechanism, data network transmission protocol, data transmission mechanism, element storage type, storage data type and trust boundary type of the element can be determined according to the element information table.

[0108] Then, based on the element information of the data flow and the corresponding related elements, we analyze the counterfeit threat, tampering threat, repudiation threat, information leakage threat, denial of service threat, privilege escalation threat, data security threat and model security threat respectively to determine the corresponding target threat type and each threat.

[0109] For example, if the trust boundary type for the data flow between the AI ​​access interface and the AI ​​application is the Internet network boundary, then the phishing threat analysis can determine that the phishing threats from the Internet include:

[0110] Threat 1: Attackers impersonate users to log in to AI application services;

[0111] Threat 2: An attacker bypasses the current authentication method and impersonates a user to log in.

[0112] Threat 3: Attackers steal user passwords through a man-in-the-middle.

[0113] Threat 4: Attackers steal legitimate user sessions and impersonate users to log in to AI applications.

[0114] In some embodiments, the process of formulating countermeasures corresponding to the threat in S140 may include:

[0115] Conduct risk assessment on threats and determine threat scores;

[0116] Determining the threat level corresponding to the threat score based on a pre-established correspondence between the threat score and the threat level;

[0117] Based on the threat level, formulate corresponding response measures.

[0118] It should be noted that in the embodiment of the present invention, after determining the corresponding threats for each group of data streams and corresponding related elements, a risk assessment can be further performed on each threat to determine a threat score. Then, based on the threat score, a threat level corresponding to the threat can be matched from a pre-established correspondence between threat scores and threat levels. Then, response measures can be formulated for different threat levels and specific threats.

[0119] It is understood that in the process of formulating corresponding countermeasures based on the threat level, the threat level can be combined with the business scope of the artificial intelligence application system to determine whether the threat is harmful. If it is harmful, mitigation measures corresponding to the threat level can be formulated in advance based on the threat level and the corresponding business scope of the artificial intelligence application system. The threat-threat level-artificial intelligence application system business scope-mitigation measure correspondence can be stored in a mitigation measure database. After the threat level of the threat is determined, the corresponding mitigation measure can be retrieved from the mitigation measure database based on the threat level and the corresponding business scope of the artificial intelligence application system as a countermeasure to the threat.

[0120] Furthermore, the process of performing risk assessment on threats and determining threat scores may include:

[0121] Establish a threat scoring table in advance based on threat direction and threat results corresponding to different threat values ​​under threat direction;

[0122] Determine the threat consequences of the threat in each threat direction;

[0123] According to the threat result, determine the threat value corresponding to the threat direction;

[0124] The threat values ​​corresponding to each threat direction are accumulated to obtain a threat score corresponding to the threat.

[0125] It should be noted that the threat direction may include at least two of harmfulness, reproducibility, difficulty of exploitation, affected objects, difficulty of discovery, and compliance (specifically, legal compliance).

[0126] In practical applications, we can construct threat results corresponding to different threat directions and different threat values, as shown in Table 3 below:

[0127] Table 3 Threat results corresponding to different threat values ​​under different threat directions

[0128]

[0129] It is understandable that for each threat and each threat direction, the threat value of the threat result corresponding to the threat direction can be determined. For example, for the harmfulness of threat 1, the threat result of the threat can be determined through analysis to be "obtaining full verification permissions, performing administrator operations, and illegally uploading files." In this case, the threat value of the harmfulness of threat 1 is 3, which is a high threat value. For example, for the difficulty of discovering threat 1, the threat result of the threat 1 in terms of discovery difficulty can be determined through analysis to be "in a private area, visible to some people, requiring in-depth vulnerability exploration." In this case, the threat value of threat 1 in terms of discovery difficulty is 3, which is a medium threat value. In this way, the threat value of each threat in each threat direction can be determined. For this threat, the threat value corresponding to all threat directions can be added together to obtain the threat score corresponding to the threat. The threat level is then determined based on the corresponding threat score, and the corresponding countermeasures for the threat are further determined.

[0130] For example, for data stream 1 and its corresponding related elements, it is determined that there is a counterfeit threat, and specifically:

[0131] Threat 1: attackers impersonate users to log in to AI application services; Threat 2: attackers bypass current authentication methods and impersonate users to log in; Threat 3: attackers steal user passwords through middlemen; Threat 4: attackers steal normal user sessions and impersonate users to log in to AI applications;

[0132] Then, the threat score of each threat is as follows:

[0133] Threat 1: The threat score for an attacker impersonating a user to log in to an AI application service is:

[0134] High D(3) + R(3) + E(3) + A(3) + D(3) +C(1) = 16;

[0135] Threat 2: The attacker bypasses the current authentication method and impersonates the user to log in. The threat score is:

[0136] High D(3) + R(3) + E(3) + A(3) + D(2) +C(1) = 15;

[0137] Threat 3: The threat score of an attacker stealing user passwords through a man-in-the-middle attack is:

[0138] High D(3) + R(3) + E(2) + A(3) + D(1) +C(1) = 14;

[0139] Threat 4: An attacker steals a legitimate user's session and impersonates the user to log into an AI application. The threat score is:

[0140] High D(3) + R(1) + E(1) + A(3) + D(1) +C(1) = 10.

[0141] For example, it is determined that a data security threat exists for data stream 2 and its corresponding related elements, and specifically:

[0142] Threat 1: Data poisoning attack, Threat 2: Adversarial sample attack, and Threat 3: Privacy leakage. The specific threat scores for each threat are as follows:

[0143] The threat score for Threat 1 Data Poisoning Attack is:

[0144] High D(3) + R(3) + E(3) + A(3) + D(3) + C(1) = 18;

[0145] The threat score of threat 2 adversarial attack is:

[0146] High D(3) + R(3) + E(3) + A(3) + D(2) + C(1) = 17.

[0147] The threat score for threat 3 privacy leakage is:

[0148] High D(3) + R(3) + E(2) + A(3) + D(1) + C(3) = 15.

[0149] For example, it is determined that model security threats still exist for data flow 2 and its corresponding related elements, and specifically:

[0150] Threat 1 indicates an injection attack, Threat 2 indicates member inference and sensitive data extraction, Threat 3 indicates model extraction, and Threat 4 indicates output content compliance. The specific threat scores for each threat are as follows:

[0151] Threat 1 indicates that the threat score for the injection attack is:

[0152] High D(3) + R(3) + E(3) + A(3) + D(3) + C(1) = 16;

[0153] The threat score for Threat 2 Membership Inference and Sensitive Data Extraction is:

[0154] High D(3) + R(3) + E(3) + A(3) + D(2) + C(1) = 15;

[0155] The threat score extracted from the Threat 3 model is:

[0156] High D(3) + R(3) + E(2) + A(3) + D(1) + C(1) = 13;

[0157] The threat score for Threat 4 output content compliance is:

[0158] High D(3) + R(3) + E(3) + A(3) + D(3) + C(3) = 18.

[0159] It's important to note that the threat score for each threat ranges from 6 to 18. For example, for general websites (i.e., websites without special requirements, such as financial websites), a threat score of 14 to 18 corresponds to a high-risk vulnerability, a threat score of 10 to 13 corresponds to a medium-risk vulnerability, and a threat score of 6 to 9 corresponds to a low-risk vulnerability. The threat score of each threat can be used to determine the threat level and, based on the actual situation, determine which risks must be avoided and which can be mitigated.

[0160] In practical applications, the mitigation measures corresponding to different threat types can be referred to Table 4:

[0161] Table 4 Mitigation measures

[0162]

[0163] For example, for data stream 1 and its corresponding related elements, it is determined that there is a counterfeit threat, and specifically:

[0164] Threat 1: attackers impersonate users to log in to AI application services; Threat 2: attackers bypass current authentication methods and impersonate users to log in; Threat 3: attackers steal user passwords through middlemen; Threat 4: attackers steal normal user sessions and impersonate users to log in to AI applications;

[0165] The threat score, threat level, and response measures for each threat are as follows:

[0166] Threat 1: Attackers impersonate users to log in to artificial intelligence application services:

[0167] Threat score: High D(3) + R(3) + E(3) + A(3) + D(3) + C(1) = 16; the corresponding threat level is a high-risk vulnerability;

[0168] Mitigation measures: Add authentication functions, authenticate through username + password, or use other authentication methods such as two-factor authentication, SMS, fingerprint, etc.

[0169] Threat 2: The attacker bypasses the current authentication method and impersonates the user to log in. The threat score is:

[0170] High D(3) + R(3) + E(3) + A(3) + D(2) + C(1) = 15; the corresponding threat level is a high-risk vulnerability;

[0171] Mitigation measures: Add graphic verification codes to prevent brute force cracking, increase password complexity requirements to prevent brute force cracking, return unified prompts for authentication errors, ensure the security of password reset and password retrieval logic, and be careful of bypassing;

[0172] Threat 3: Attackers steal user passwords through man-in-the-middle:

[0173] Threat score: High D(3) + R(3) + E(2) + A(3) + D(1) + C(1) = 13; the corresponding threat level is a high-risk vulnerability;

[0174] Mitigation measures: Use HTTPS for login requests, enable HSTS, ensure the security of SSL certificates, use secure algorithms in the algorithm suite, and encrypt or hash passwords before transmission;

[0175] Threat 4: Attackers steal legitimate user sessions and impersonate users to log into AI applications:

[0176] Threat score: High D(3) + R(1) + E(1) + A(3) + D(1) + C(1) = 10; the corresponding threat level is a high-risk vulnerability;

[0177] Mitigation measures: Session ID length is greater than 24 bits, secure random number generation is used, session is forced to change before and after login, and session expiration time is limited.

[0178] For example, it is determined that a data security threat exists for data stream 2 and its corresponding related elements, and specifically:

[0179] Threat 1: Data poisoning attack, Threat 2: Adversarial sample attack, and Threat 3: Privacy leakage. The specific threat scores, threat levels, and countermeasures for each threat are as follows:

[0180] Threat 1: Data poisoning attack:

[0181] Threat score: High D(3) + R(3) + E(3) + A(3) + D(3) + C(1) = 18; the corresponding threat level is a high-risk vulnerability;

[0182] Countermeasures (also known as mitigation measures): data poisoning detection;

[0183] Threat 2 Adversarial Sample Attack:

[0184] Threat score: High D(3) + R(3) + E(3) + A(3) + D(2) + C(1) = 17; the corresponding threat level is a high-risk vulnerability;

[0185] Countermeasures (also known as mitigation measures): adversarial sample identification;

[0186] Threat 3 Privacy Leakage:

[0187] Threat score: High D(3) + R(3) + E(2) + A(3) + D(1) + C(3) = 15; the corresponding threat level is a high-risk vulnerability;

[0188] Countermeasures (also known as mitigation measures): output content identification and filtering.

[0189] For example, it is determined that model security threats still exist for data flow 2 and its corresponding related elements, and specifically:

[0190] Threat 1 indicates an injection attack, Threat 2 indicates member inference and sensitive data extraction, Threat 3 indicates model extraction, and Threat 4 indicates output content compliance. The specific threat scores and threat level response measures for each threat are as follows:

[0191] Threat 1 Tips Injection Attack:

[0192] Threat score: High D(3) + R(3) + E(3) + A(3) + D(3) + C(1) = 16, the threat level is a high-risk vulnerability;

[0193] Countermeasures (also known as mitigation measures): identification and filtering of data input content;

[0194] Threat 2: Member Inference and Sensitive Data Extraction:

[0195] Threat score: High D(3) + R(3) + E(3) + A(3) + D(2) + C(1) = 15, the threat level is a high-risk vulnerability;

[0196] Countermeasures (also known as mitigation measures): adversarial sample identification;

[0197] Threat 3 Model Extraction:

[0198] Threat score: High D(3) + R(3) + E(2) + A(3) + D(1) + C(1) = 13, the threat level is a high-risk vulnerability;

[0199] Countermeasures (also known as mitigation measures): input parameter verification, rate limiting;

[0200] Threat 4 output content compliance:

[0201] Threat score: High D(3) + R(3) + E(3) + A(3) + D(3) + C(3) = 18, the threat level is a high-risk vulnerability;

[0202] Countermeasures (also known as mitigation measures): output content identification and filtering.

[0203] It can be seen that the embodiment of the present invention takes the components in the artificial intelligence application system as elements, constructs a data flow diagram based on the various elements in the artificial intelligence application system and the data flows between the elements, and then determines the relevant elements corresponding to each data flow, and further determines the threats corresponding to the data flow and the corresponding relevant elements according to the preset threat types, and formulates countermeasures corresponding to the threats, thereby obtaining threats and countermeasures corresponding to each data flow and the corresponding relevant elements. Further, based on the various data flows, the threats corresponding to the data flows and the corresponding relevant elements, and the countermeasures corresponding to the threats, a threat modeling result can be generated; that is, the present invention can realize threat modeling of the artificial intelligence application system during use, which is conducive to improving the security and reliability of the artificial intelligence application system.

[0204] In addition, the present invention can also improve the efficiency of security threat analysis of artificial intelligence applications to a certain extent, reduce the difficulty of implementing threat modeling, greatly improve operability, improve the comprehensiveness of threat analysis, and reduce product safety risks caused by insufficient identification of human-caused security risks.

[0205] Based on the above examples, please refer to Figure 3 Another embodiment of the present invention provides a threat query method for an artificial intelligence application system, the method comprising:

[0206] S210: Obtain threat query instructions;

[0207] S220: Obtain current data flow information according to the threat query instruction;

[0208] S230: Determine a current related element corresponding to the current data stream information according to the current data stream information;

[0209] S240: Determine corresponding threats and countermeasures from pre-established threat modeling results based on the current data flow information and the corresponding current related elements; wherein the threat modeling results are obtained based on the threat modeling method of the artificial intelligence application system as described above;

[0210] S250: Display threats and countermeasures.

[0211] It should be noted that in an embodiment of the present invention, after constructing a corresponding threat model for an artificial intelligence application system, a threat query instruction can be obtained, and the current data flow information can be obtained according to the threat query instruction. The sending component and the receiving component of the data flow information are further determined based on the current data flow information, thereby determining the current relevant elements corresponding to the current data flow, and then determining the corresponding threats and countermeasures from the established threat modeling results based on the current data flow and the current relevant elements.

[0212] In one embodiment, threats and corresponding countermeasures can be displayed in a list format for easy review by staff. Furthermore, updates input by staff can be accepted and relevant elements can be updated based on the updates. The updates input by staff are determined based on the threats and corresponding countermeasures, thereby improving the security and reliability of the AI ​​application system.

[0213] It can be understood that the embodiments of the present invention have the same beneficial effects as the above-mentioned method embodiments, and for the implementation process of the threat modeling method of the artificial intelligence application system involved in the embodiments of the present invention, please refer to the above-mentioned embodiments, and the present invention will not be repeated here.

[0214] The present invention also provides a corresponding device for the threat modeling method of an artificial intelligence application system, further making the method more practical. Among them, the device can be described from the perspective of functional modules and hardware. The threat modeling device for the artificial intelligence application system provided by the present invention is introduced below. The device is used to implement the threat modeling method for the artificial intelligence application system provided by the present invention. In this embodiment, the threat modeling device for the artificial intelligence application system may include or be divided into one or more program modules. The one or more program modules are stored in a storage medium and executed by one or more processors to complete the threat modeling method for the artificial intelligence application system disclosed in the above embodiment. The program module referred to in the present invention refers to a series of computer program instruction segments that can perform specific functions. It is more suitable for describing the execution process of the threat modeling device of the artificial intelligence application system in the storage medium than the program itself. The following description will specifically introduce the functions of each program module of this embodiment. The threat modeling device for the artificial intelligence application system described below and the threat modeling method for the artificial intelligence application system described above can be referenced to each other.

[0215] From the perspective of functional modules, see Figure 4 , Figure 4 This is a structural diagram of a threat modeling device for an artificial intelligence application system provided by the present invention in a specific embodiment. The device may include:

[0216] A construction module 11 is used to construct a data flow diagram based on the elements in the artificial intelligence application system and the data flows between the elements; the elements are components in the artificial intelligence application system;

[0217] A first determining module 12 is configured to determine, for each data stream, a relevant element corresponding to the data stream;

[0218] A second determining module 13 is configured to determine threats corresponding to the data stream and the corresponding related elements according to various preset threat types;

[0219] Development module 14, used to develop response measures corresponding to the threat;

[0220] The generating module 15 is configured to generate a threat modeling result based on each data flow, the threats corresponding to the data flow and the corresponding related elements, and the corresponding countermeasures to the threats.

[0221] In some embodiments, the second determining module 13 includes:

[0222] A first acquiring unit, configured to acquire a content type of a data stream;

[0223] A first determining unit is configured to determine a confidentiality requirement of the data stream according to a content type of the data stream; the confidentiality requirement is whether the data information needs to be encrypted;

[0224] A second determining unit is configured to determine an integrity requirement of the data stream according to a content type of the data stream; the integrity requirement is whether integrity protection is performed on the data;

[0225] A second acquiring unit, configured to acquire element information of relevant elements corresponding to the data stream;

[0226] a third determining unit, configured to determine a target threat type and threats corresponding to the target threat type based on preset threat types, in combination with confidentiality requirements and integrity requirements of the data stream and element information of relevant elements;

[0227] Among them, the various threat types include:

[0228] At least two of the following threats: phishing threat, tampering threat, repudiation threat, information leakage threat, denial of service threat, privilege escalation threat, data security threat, and model security threat.

[0229] In some embodiments, the element information includes a combination of one or more of element type, element security mechanism, data network transmission protocol, data transmission mechanism, element storage type, storage data type, and trust boundary type.

[0230] In some embodiments, the formulation module 14 includes:

[0231] a fourth determination unit, configured to perform a risk assessment on the threat and determine a threat score of the threat;

[0232] a fifth determining unit, configured to determine a threat level corresponding to the threat score based on a pre-established correspondence between the threat score and the threat level;

[0233] The formulation unit is used to formulate response measures corresponding to the threat according to the threat level.

[0234] In some embodiments, the fourth determining unit includes:

[0235] Establish a subunit for pre-establishing a threat scoring table based on threat directions and threat results corresponding to different threat values ​​under threat directions;

[0236] A first determining subunit is used to determine the threat result of the threat in each threat direction;

[0237] a second determining subunit, configured to determine a threat value corresponding to the threat direction according to the threat result;

[0238] The calculation subunit is used to accumulate the threat values ​​corresponding to each threat direction to obtain a threat score corresponding to the threat.

[0239] In some embodiments, threats corresponding to data security include one or more of data poisoning attacks, adversarial sample attacks, and privacy leaks;

[0240] And / or, threats corresponding to model security include one or more of prompt injection attacks, member inference and sensitive data extraction, model extraction, and output content compliance.

[0241] It should be noted that the threat modeling device for the artificial intelligence application system provided in the embodiment of the present invention has the same beneficial effects as the threat modeling method for the artificial intelligence application system provided in the above embodiment, and for the specific introduction of the threat modeling method for the artificial intelligence application system involved in the embodiment of the present invention, please refer to the above embodiment, and the present invention will not go into details here.

[0242] The present invention also provides a corresponding device for the threat query method for an artificial intelligence application system, further enhancing the practicality of the method. The device can be described from the perspective of both functional modules and hardware. The following describes the threat query device for an artificial intelligence application system provided by the present invention. This device is used to implement the threat query method for an artificial intelligence application system provided by the present invention. In this embodiment, the threat query device for an artificial intelligence application system may include or be divided into one or more program modules. These one or more program modules are stored in a storage medium and executed by one or more processors to implement the threat query method for an artificial intelligence application system disclosed in the above embodiment. A program module, as referred to in the present invention, refers to a series of computer program instruction segments capable of performing a specific function. These modules are more suitable for describing the execution process of the threat query device for an artificial intelligence application system in a storage medium than the program itself. The following description will specifically introduce the functions of each program module in this embodiment. The threat query device for an artificial intelligence application system described below and the threat query method for an artificial intelligence application system described above can be referenced in conjunction with each other.

[0243] Please refer to Figure 5 Another embodiment of the present invention provides a threat query device for an artificial intelligence application system, including:

[0244] A first acquisition module 21 is used to acquire a threat query instruction;

[0245] A second acquisition module 22 is used to obtain current data flow information according to the threat query instruction;

[0246] A third determining module 23 is configured to determine a current related element corresponding to the current data stream information according to the current data stream information;

[0247] A fourth determination module 24 is configured to determine corresponding threats and countermeasures from pre-established threat modeling results based on the current data flow information and the corresponding current related elements; wherein the threat modeling results are obtained based on the threat modeling method of the artificial intelligence application system as described above;

[0248] The display module 25 is used to display threats and countermeasures.

[0249] It should be noted that the threat query device of the artificial intelligence application system provided in the embodiment of the present invention has the same beneficial effects as the above embodiment.

[0250] Figure 6 A structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 6 As shown, the electronic device includes: a memory 60 for storing computer programs;

[0251] The processor 61 is configured to implement the steps of the threat modeling method for the artificial intelligence application system of the above embodiment when executing a computer program.

[0252] The electronic device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0253] The processor 61 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 61 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing content required to be displayed on the display screen. In some embodiments, the processor 61 may also include an artificial intelligence (AI) processor for handling computational operations related to machine learning.

[0254] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 60 is at least used to store the following computer program 601, wherein, after the computer program is loaded and executed by the processor 61, it can implement the relevant steps of the threat modeling method of the artificial intelligence application system disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 602 may include Windows, Unix, Linux, etc. The data 603 may include but is not limited to relevant data of the threat modeling results of the artificial intelligence application system.

[0255] In some embodiments, the electronic device may further include a display screen 62 , an input / output interface 63 , a communication interface 64 , a power supply 65 , and a communication bus 66 .

[0256] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown in the figure.

[0257] It is understood that if the threat modeling method for the artificial intelligence application system in the above-mentioned embodiment is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and performs all or part of the steps of the method of each embodiment of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, registers, hard drives, removable disks, CD-ROMs, magnetic disks, or optical disks, and other media that can store program code.

[0258] Based on this, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the threat modeling method for the artificial intelligence application system as described above are implemented.

[0259] Based on this, an embodiment of the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the threat modeling method for the above-mentioned artificial intelligence application system.

[0260] The above describes in detail the threat modeling method, threat query method, device, electronic device, and computer-readable storage medium for an artificial intelligence application system provided by the embodiments of the present invention. The various embodiments are described in a progressive manner throughout this specification, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between the various embodiments can be referenced to each other. The device disclosed in the embodiments corresponds to the method disclosed in the embodiments, so the description is relatively brief. For relevant details, refer to the method description.

[0261] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0262] The above describes in detail the threat modeling method, threat query method, device, electronic device, and computer-readable storage medium for an artificial intelligence application system provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only intended to help understand the method and core concept of the present invention. It should be noted that, for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A threat modeling method for an artificial intelligence application system, characterized in that: include: Construct a data flow diagram based on the elements in the artificial intelligence application system and the data flow between the elements; The elements are components in the artificial intelligence application system; For each of the data streams, determining a relevant element corresponding to the data stream; The relevant elements are the artificial intelligence application access interface and the artificial intelligence application; Determining threats corresponding to the data stream and corresponding related elements according to various preset threat types; Develop countermeasures commensurate with the threats described; Generating a threat modeling result based on each of the data flows, the threats corresponding to the data flows and the corresponding related elements, and the countermeasures corresponding to the threats; wherein: The determining, according to each preset threat type, threats corresponding to the data stream and the corresponding related elements includes: Obtaining the content type of the data stream; Determining the confidentiality requirement of the data stream according to the content type of the data stream; the confidentiality requirement is whether the data information needs to be encrypted; Determining, based on the content type of the data stream, an integrity requirement of the data stream; the integrity requirement being whether integrity protection is performed on the data; Obtaining element information of relevant elements corresponding to the data stream; Determining a target threat type and threats corresponding to the target threat type based on the preset threat types, in combination with confidentiality requirements and integrity requirements of the data flow and element information of the relevant elements; The threat types include: At least two of the following threats: impersonation threat, tampering threat, repudiation threat, information leakage threat, denial of service threat, privilege escalation threat, data security threat, and model security threat; The obtaining of the content type of the data stream includes: The content type of the data flow between two elements with information transmission in the artificial intelligence application system is predetermined according to the business scope of the artificial intelligence application system, and the content type of the data flow is determined according to the data flow and related elements.

2. The threat modeling method for an artificial intelligence application system according to claim 1, characterized in that: The element information includes one or more combinations of element type, element security mechanism, data network transmission protocol, data transmission mechanism, element storage type, storage data type and trust boundary type.

3. The threat modeling method for an artificial intelligence application system according to claim 1, characterized in that: The formulation of response measures corresponding to the threat includes: Performing a risk assessment on the threat to determine a threat score for the threat; Determining a threat level corresponding to the threat score based on a pre-established correspondence between the threat score and the threat level; According to the threat level, a response measure corresponding to the threat is formulated.

4. The threat modeling method for an artificial intelligence application system according to claim 3, characterized in that: The performing risk assessment on the threat and determining the threat score of the threat includes: Establishing a threat scoring table in advance based on threat directions and threat results corresponding to different threat values ​​under the threat directions; determining a threat result of the threat in each threat direction; Determining a threat value corresponding to the threat direction according to the threat result; The threat values ​​corresponding to the respective threat directions are accumulated to obtain a threat score corresponding to the threat.

5. The threat modeling method for an artificial intelligence application system according to any one of claims 1 to 4, characterized in that: The threats corresponding to the data security threats include one or more of data poisoning attacks, adversarial sample attacks, and privacy leaks; And / or, the threats corresponding to the model security include one or more of prompt injection attacks, member inference and sensitive data extraction, model extraction, and output content compliance.

6. A threat query method for an artificial intelligence application system, characterized in that: include: Get threat query instructions; Obtaining current data flow information according to the threat query instruction; Determining, according to the current data stream information, a current related element corresponding to the current data stream information; Determining corresponding threats and countermeasures from pre-established threat modeling results based on the current data flow information and the corresponding current related elements; wherein the threat modeling results are obtained based on the threat modeling method for an artificial intelligence application system according to any one of claims 1 to 5; Present the threats and countermeasures.

7. A threat modeling device for an artificial intelligence application system, characterized in that: include: A construction module is used to construct a data flow diagram based on the various elements in the artificial intelligence application system and the data flows between the elements; The elements are components in the artificial intelligence application system; A first determining module, configured to determine, for each of the data streams, a related element corresponding to the data stream; The relevant elements are the artificial intelligence application access interface and the artificial intelligence application; A second determining module is configured to determine threats corresponding to the data stream and the corresponding related elements according to various preset threat types; a formulation module for formulating countermeasures corresponding to the threats; A generating module is configured to generate a threat modeling result based on each of the data flows, the threats corresponding to the data flows and the corresponding related elements, and the countermeasures corresponding to the threats; wherein: The second determining module includes: A first acquiring unit, configured to acquire a content type of a data stream; A first determining unit is configured to determine a confidentiality requirement of the data stream according to a content type of the data stream; the confidentiality requirement is whether the data information needs to be encrypted; A second determining unit is configured to determine an integrity requirement of the data stream according to a content type of the data stream; the integrity requirement is whether to perform integrity protection on the data; A second acquiring unit, configured to acquire element information of relevant elements corresponding to the data stream; a third determining unit, configured to determine a target threat type and threats corresponding to the target threat type according to preset threat types, in combination with confidentiality requirements and integrity requirements of the data flow and element information of the relevant elements; Among them, the various threat types include: At least two of the following threats: impersonation threat, tampering threat, repudiation threat, information leakage threat, denial of service threat, privilege escalation threat, data security threat, and model security threat; The first acquisition unit is specifically used to predetermine the content type of the data flow between two elements with information transmission in the artificial intelligence application system according to the business scope of the artificial intelligence application system, and determine the content type of the data flow according to the data flow and related elements.

8. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the threat modeling method for an artificial intelligence application system as described in any one of claims 1 to 5, or to execute the computer program to implement the steps of the threat modeling method for an artificial intelligence application system as described in claim 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the threat modeling method for an artificial intelligence application system as described in any one of claims 1 to 5, or executes the computer program to implement the steps of the threat modeling method for an artificial intelligence application system as described in claim 6.

Images