AI program analysis and detection method and system

Abstract

The invention discloses an AI program analysis and detection method and system. The method comprises the following steps: S1, extracting static code security features of a code of a to-be-detected AI program before operation; s2, a time sequence heterogeneous graph is constructed based on a system call event sequence when the AI program to be detected runs in the isolation sandbox environment, entity nodes of the time sequence heterogeneous graph represent system entities, relation edges of the time sequence heterogeneous graph represent the interaction behavior relation of the system entities, and the system entities are in the system call event sequence; processing the time sequence heterogeneous graph by using a graph neural network to extract graph embedding features as dynamic behavior security features representing the overall dynamic behavior mode of the AI program to be detected; s3, analyzing the to-be-detected AI program by using a pre-trained first analysis model to obtain a deep analysis feature of the to-be-detected AI program; and S4, fusing the static code security feature, the dynamic behavior security feature and the deep analysis feature, and inputting the fused features into a risk classification model to obtain a risk assessment result.

Description

Technical Field

[0001] This invention relates to the field of computer security technology, specifically to an AI program analysis and detection method and system. Background Technology

[0002] With the development of artificial intelligence technology, more and more programs are being assisted or directly generated by AI.

[0003] The widespread adoption of AI code generation tools has significantly lowered the barrier to entry for programming, but the accompanying security risks are becoming increasingly prominent. AI-generated programs may pose the following unique threats: models learn from publicly available codebases containing malicious code, resulting in programs that are "inherently malicious"; the logic of AI-generated code may be extremely complex or counterintuitive, making it difficult for traditional human auditing and static analysis tools to understand its true intent; misuse due to missing context: AI may correctly use a dangerous function but ignore its proper security context (such as failure to validate input), leading to potential vulnerabilities; adversarial generation: attackers may use carefully crafted prompts to induce AI to generate malicious code that can bypass traditional security detection mechanisms.

[0004] Users often lack the professional security review capabilities when using these programs, and may be vulnerable to malicious code. Currently, there is a lack of security detection tools specifically designed for AI-generated programs.

[0005] Therefore, this paper proposes a detection method and tool for AI-generated software, which can be used to automatically detect and analyze potential security risks in AI-generated programs. Summary of the Invention

[0006] To address these issues, this invention proposes an AI program analysis and detection method and system.

[0007] According to one aspect of the present invention, an AI program analysis and detection method is proposed, comprising the following steps:

[0008] S1, extract the static code security features of the AI ​​program to be tested before it runs;

[0009] S2, construct a temporal heterogeneous graph based on the system call event sequence of the AI ​​program to be tested running in the isolated sandbox environment, wherein the entity nodes of the temporal heterogeneous graph represent system entities, the relation edges of the temporal heterogeneous graph represent the interaction behavior relationships of the system entities, and use graph neural networks to process the temporal heterogeneous graph to extract graph embedding features as dynamic behavior security features characterizing the overall dynamic behavior pattern of the AI ​​program to be tested;

[0010] S3, use the pre-trained first analysis model to analyze the AI ​​program to be detected, and obtain the deep analysis features of the AI ​​program to be detected;

[0011] S4. The static code security features, dynamic behavior security features, and deep analysis features are integrated and input into the risk classification model to obtain the final risk assessment result.

[0012] Specifically, the construction of the temporal heterogeneous graph based on the system call event sequence of the AI ​​program under test running in the isolated sandbox environment in S2 includes:

[0013] Obtain the sequence of all system call events generated by the AI ​​program under test during its operation. Each system call event includes timestamp information, process identifier that initiated the call, and call parameters. Construct corresponding entity nodes and feature labels based on the call parameters or process identifier. Construct different types of relationship edges and corresponding behavior type labels based on the semantics of the system call events. Construct time sequence information based on the timestamps.

[0014] Specifically, the entity nodes include process nodes, file nodes, network nodes, and memory nodes, and the calling parameters include file address parameters, network address parameters, and memory address parameters; the process nodes are constructed based on the process identifier, and the feature tags of the process nodes include the permission level information of the process nodes;

[0015] The file node is constructed based on the file address parameters, and the feature label of the file node includes the sensitivity level information of the file node;

[0016] The network node is constructed based on the network address parameters, and the feature tags of the network node include the domain name reputation information of the network node;

[0017] The memory node is constructed based on the memory address parameters, and the feature label of the memory node includes the read and write attribute information of the memory node.

[0018] Specifically, the different types of relationship edges include: process lifecycle management relationship edges, file system operation relationship edges, network communication relationship edges, inter-process communication relationship edges, registry operation relationship edges, and permission and privilege operation relationship edges;

[0019] The behavior type labels for the process lifecycle management relationship edges include: derivation and termination;

[0020] The behavior type labels of the file system operation relationship edges include: create, read, write, delete, rename, and execute;

[0021] The behavior type labels for the network communication relationship edges include: connect, send, receive, and listen;

[0022] The behavior type labels for the inter-process communication relationship edges include: pipe communication, shared memory, and signal sending;

[0023] The behavior type labels of the registry operation relationship edges include: registry read, registry write, and registry creation;

[0024] The behavior type labels for the permission and privilege operation relationship edge include: privilege escalation and access token theft.

[0025] Specifically, S1 includes: processing the code of the AI ​​program to be detected using a second analysis model, wherein the second analysis model is a large language model trained on code data to generate static code security features that understand the code semantics of the AI ​​program to be detected.

[0026] Specifically, S3 includes processing fused features using the first analysis model. The fused features are based at least on the static code security features generated in S1 and the dynamic behavior security features generated in S2. The first analysis model is a multimodal pre-trained model, used to output deep analysis features based on the association between the static code security features and the dynamic behavior security features.

[0027] Specifically, S4 includes:

[0028] The static code security features, dynamic behavior security features, and deep analysis features are treated as a set of multimodal sequences;

[0029] A pre-trained Transformer encoder is used to perform cross-modal attention computation on the multimodal sequence to generate a unified feature representation;

[0030] The unified feature representation is input into the risk classification model to obtain the risk classification result output by the risk classification model.

[0031] According to one aspect of the present invention, an AI program analysis and detection system is proposed, comprising the following modules according to any one of the first aspects:

[0032] The static feature extraction module is configured to extract static code security features of the AI ​​program to be detected before it runs.

[0033] The dynamic feature extraction module is configured to construct a temporal heterogeneous graph based on the system call event sequence of the AI ​​program under test running in an isolated sandbox environment. The entity nodes of the temporal heterogeneous graph represent system entities, and the relation edges of the temporal heterogeneous graph represent the interaction behavior relationships of the system entities. The graph neural network is used to process the temporal heterogeneous graph to extract graph embedding features as dynamic behavior security features that characterize the overall dynamic behavior pattern of the AI ​​program under test.

[0034] The deep feature extraction module is configured to analyze the AI ​​program to be detected using a pre-trained first analysis model to obtain the deep analysis features of the AI ​​program to be detected.

[0035] The risk assessment output module is configured to integrate the static code security features, the dynamic behavior security features, and the deep analysis features, and input them into the risk classification model to obtain the final risk assessment result.

[0036] According to one aspect of the present invention, a computer program product is provided having a computer program stored thereon, which, when executed by a processor, performs the method as described in the first aspect.

[0037] According to one aspect of the present invention, an electronic device is provided, comprising:

[0038] One or more processors;

[0039] Storage device for storing one or more programs;

[0040] When the one or more programs are executed by the one or more processors, the one or more processors perform the method as described in the first aspect.

[0041] The advantages of this invention are:

[0042] 1. Identify potential security threats in AI-generated programs;

[0043] 2. Automated detection of malicious code and inappropriate behavior;

[0044] 3. Provide a detailed security analysis report;

[0045] 4. Reduce the risks for users using AI-generated programs. Attached Figure Description

[0046] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many anticipated advantages of the embodiments will be readily recognized as they become better understood through reference to the following detailed description. Elements in the drawings are not necessarily to scale. The same reference numerals refer to corresponding similar parts.

[0047] Figure 1 A flowchart illustrating an AI program analysis and detection method according to the present invention is shown;

[0048] Figure 2 A schematic diagram of the structure of an AI program analysis and detection system according to the present invention is shown;

[0049] Figure 3 A schematic diagram of a computer system architecture suitable for implementing the embodiments of this application is shown. Detailed Implementation

[0050] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0051] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0052] Figure 1 An AI program analysis and detection method is shown, including the following steps:

[0053] S1, extract the static code security features of the AI ​​program to be tested before it runs;

[0054] S2, construct a temporal heterogeneous graph based on the system call event sequence of the AI ​​program to be tested running in the isolated sandbox environment, wherein the entity nodes of the temporal heterogeneous graph represent system entities, the relation edges of the temporal heterogeneous graph represent the interaction behavior relationships of the system entities, and use graph neural networks to process the temporal heterogeneous graph to extract graph embedding features as dynamic behavior security features characterizing the overall dynamic behavior pattern of the AI ​​program to be tested;

[0055] S3, use the pre-trained first analysis model to analyze the AI ​​program to be detected, and obtain the deep analysis features of the AI ​​program to be detected;

[0056] S4. The static code security features, dynamic behavior security features, and deep analysis features are integrated and input into the risk classification model to obtain the final risk assessment result.

[0057] Specifically, the construction of the temporal heterogeneous graph based on the system call event sequence of the AI ​​program under test running in the isolated sandbox environment in S2 includes:

[0058] Obtain the sequence of all system call events generated by the AI ​​program under test during its operation. Each system call event includes timestamp information, process identifier that initiated the call, and call parameters. Construct corresponding entity nodes and feature labels based on the call parameters or process identifier. Construct different types of relationship edges and corresponding behavior type labels based on the semantics of the system call events. Construct time sequence information based on the timestamps.

[0059] The AI ​​program to be tested is run in a controlled, isolated sandbox environment (such as Cuckoo Sandbox, Docker containers with seccomp profiles), and the complete system call event stream from program startup to termination is captured using technologies such as system call hooking or eBPF.

[0060] Specifically, the entity nodes include process nodes, file nodes, network nodes, and memory nodes, and the calling parameters include file address parameters, network address parameters, and memory address parameters; the process nodes are constructed based on the process identifier, and the feature tags of the process nodes include the permission level information of the process nodes;

[0061] The file node is constructed based on the file address parameters, and the feature label of the file node includes the sensitivity level information of the file node;

[0062] The network node is constructed based on the network address parameters, and the feature tags of the network node include the domain name reputation information of the network node;

[0063] The memory node is constructed based on the memory address parameters, and the feature label of the memory node includes the read and write attribute information of the memory node.

[0064] Based on the semantics of the system call type, directed edges are established between the involved entity nodes. The edge type (relationship) and behavior type labels are shown below:

[0065] Specifically, the different types of relationship edges include: process lifecycle management relationship edges, file system operation relationship edges, network communication relationship edges, inter-process communication relationship edges, registry operation relationship edges, and permission and privilege operation relationship edges;

[0066] The behavior type labels for the process lifecycle management relationship edges include: derivation and termination;

[0067] The behavior type labels of the file system operation relationship edges include: create, read, write, delete, rename, and execute;

[0068] The behavior type labels for the network communication relationship edges include: connect, send, receive, and listen;

[0069] The behavior type labels for the inter-process communication relationship edges include: pipe communication, shared memory, and signal sending;

[0070] The behavior type labels of the registry operation relationship edges include: registry read, registry write, and registry creation;

[0071] The behavior type labels for the permission and privilege operation relationship edge include: privilege escalation and access token theft.

[0072] In some embodiments, each edge also carries a weight, which can be calculated from the timestamp information of the system call. For example, the time difference of the event occurrence can be used as the time decay weight of the edge, so that the graph neural network (GNN) can learn the temporal dependencies of behavior.

[0073] In one embodiment, at timestamp 100ms, a fork system call occurs, with the core parameters being the parent process PID = 1234 and the child process PID = 1235. This involves the parent process and the newly spawned child process, and the behavior is described as the parent process spawning the child process to perform image preprocessing tasks. At timestamp 180ms, an open system call occurs, with the parameters being the file path = / data / sensitive_img / 001.jpg and the access mode = read-only. This involves the child process (PID = 1235) spawned in step 1 and the target file, and the behavior is described as the child process initiating a read request for the sensitive image file. Build process node P1: The core identifier is PID=1234 (parent process), and the feature label is permission level=ordinary user (uid=1000); Build process node P2: The core identifier is PID=1235 (child process), and the feature label is permission level=ordinary user (initial uid=1000); Build file node F1: The core identifier is path= / data / sensitive_img / 001.jpg, and the feature label is sensitivity level information=highly sensitive (containing core business image data).

[0074] Based on the semantics of the type of each system call event, directed edges are established between the involved entity nodes, labeled with user-defined relationship edge types and behavior type tags, and bound with event timestamps to reflect temporal sequence. Edge E1 is constructed: starting node P1, ending node P2, relationship edge type is process lifecycle management relationship edge, behavior type tag is derivation, temporal attribute is timestamp 100ms, and semantic description is that parent process P1 creates child process P2, triggering process lifecycle derivation behavior.

[0075] Construct edge E2: starting node P2, ending node F1, the relationship edge type is a file system operation relationship edge, the behavior type label is read, the time sequence attribute is a timestamp of 180ms, and the semantic description is that process P2 initiates a file F1 read request through an open call, and the read operation is completed by a subsequent read call.

[0076] A temporally heterogeneous graph G = (V, E, T) is formed. The node set V contains P1, P2, and F1, covering user-defined entities; the edge set E contains E1 to E2. The temporal attribute T includes 100ms and 180ms to ensure that the graph neural network can accurately capture the temporal dependencies of dynamic behaviors.

[0077] GNNs utilize a multi-layered message-passing mechanism to allow nodes to aggregate information from their neighbors and edges. For example, a file node aggregates information from all process nodes that have visited it, thereby updating its own representation. This process ensures that the final embedding of each node includes its local structure and global context information within the entire behavioral graph. After multiple iterations, graph pooling layers (such as global average pooling and attention pooling) aggregate the final embeddings of all nodes into a single, fixed-dimensional graph-level embedding vector, namely the dynamic behavioral safety feature F. dynamic This vector represents the overall behavioral pattern of the program under test throughout its operation.

[0078] Specifically, S1 includes: processing the code of the AI ​​program to be detected using a second analysis model, wherein the second analysis model is a large language model trained on code data to generate static code security features that understand the code semantics of the AI ​​program to be detected.

[0079] Large language models, such as CodeLlama, CodeBERT, or InternLM, are used as secondary analysis models after being pre-trained on massive amounts of code and natural language text. These models are capable of gaining a deep understanding of the syntax and semantics of programming languages.

[0080] The input code is tokenized and language-specific markers are added. For longer code files, sliding window or function-level splitting can be used.

[0081] Based on the hidden state vector corresponding to the specific labels output by the model, or by performing average pooling / max pooling on the hidden states of all labels, a fixed-dimensional, dense feature vector is generated, which is the static code security feature. This vector encodes information such as the code's control flow, data flow, API usage patterns, potential vulnerability patterns (e.g., SQL injection, command injection), and the risks of referenced external libraries. Finally, a vector with dimension d is output. s static code security feature vector F static .

[0082] Specifically, S3 includes processing fused features using the first analysis model. The fused features are based at least on the static code security features generated in S1 and the dynamic behavior security features generated in S2. The first analysis model is a multimodal pre-trained model, used to output deep analysis features based on the association between the static code security features and the dynamic behavior security features.

[0083] A multimodal pre-trained model is used as the first analysis model. This model typically includes an encoder for processing code features (from S1) and an encoder for processing behavioral features (from S2). Finally, they interact through a cross-modal fusion module (such as Cross-Attention, Co-Attention).

[0084] The model learns by performing attention computation between two modalities. For example, it might learn: "When requests.get(url) appears in the code, if a network connection edge to a malicious domain appears in the runtime dynamic graph, then this is a highly suspicious association."

[0085] The model ultimately outputs a deep analysis feature vector F. deep This vector encodes the consistency or inconsistency between static intent and dynamic behavior, revealing the program's potential malicious purpose.

[0086] Specifically, S4 includes:

[0087] The static code security features, dynamic behavior security features, and deep analysis features are treated as a set of multimodal sequences;

[0088] A pre-trained Transformer encoder is used to perform cross-modal attention computation on the multimodal sequence to generate a unified feature representation;

[0089] The unified feature representation is input into the risk classification model to obtain the risk classification result output by the risk classification model.

[0090] F static Fdynamic and F deep Treating it as a multimodal sequence, a pre-trained Transformer encoder is used to process the sequence. The Transformer's self-attention mechanism can automatically learn the importance weights and complex interactions between the three features. For example, it might find that for a certain type of ransomware, dynamic behavior features (massive file encryption writes) are more discriminative than static features. The vector corresponding to the labels in the Transformer output is taken as the unified feature representation F after fusion. unified The unified feature representation is input into a risk classification model (usually a fully connected neural network with a Softmax function at the end), and the final classification result is output, such as "benign", "malicious - backdoor", "malicious - data breach", "suspicious", etc., and a confidence score can be attached.

[0091] In summary, this invention greatly enriches the expressive power and context of behavioral information by upscaling the linear system call sequence into a temporal heterogeneous graph, enabling the model to "understand" the complex interactions between system entities and significantly improving the ability to detect advanced persistent threats (APTs) and multi-stage attacks.

[0092] Cross-modal analysis is performed using a multimodal pre-trained model, which enables the comparison between what the code "says" and what the behavior "does". This allows the detection of malicious code that disguises itself and evades detection. For example, it can detect "grayscale" programs that appear normal in static analysis but behave abnormally at runtime.

[0093] The entire process is highly automated, reducing the cost of manual reverse analysis. It is particularly suitable for integration into continuous integration / continuous deployment (CI / CD) pipelines to achieve automated security auditing of AI models.

[0094] Graph neural networks and attention mechanisms offer a degree of interpretability. By visualizing attention weights and important nodes / edges in the graph, security analysts can trace which code snippets and runtime behaviors contribute most to classification decisions, aiding in security assessments.

[0095] According to one aspect of the present invention, an AI program analysis and detection system is proposed, comprising the following modules according to any one of the first aspects:

[0096] The static feature extraction module 201 is configured to extract static code security features of the code of the AI ​​program to be detected before it runs.

[0097] The dynamic feature extraction module 202 is configured to construct a temporal heterogeneous graph based on the system call event sequence of the AI ​​program to be detected running in an isolated sandbox environment. The entity nodes of the temporal heterogeneous graph represent system entities, and the relation edges of the temporal heterogeneous graph represent the interaction behavior relationships of the system entities. The graph neural network is used to process the temporal heterogeneous graph to extract graph embedding features as dynamic behavior security features that characterize the overall dynamic behavior pattern of the AI ​​program to be detected.

[0098] The deep feature extraction module 203 is configured to analyze the AI ​​program to be detected using a pre-trained first analysis model to obtain the deep analysis features of the AI ​​program to be detected.

[0099] The risk assessment output module 204 is configured to integrate the static code security features, the dynamic behavior security features, and the deep analysis features, and input them into the risk classification model to obtain the final risk assessment result.

[0100] The following is for reference. Figure 3 It shows a schematic diagram of the structure of a computer system 300 suitable for implementing electronic devices according to embodiments of the present application. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0101] like Figure 3 As shown, the computer system 300 includes a central processing unit (CPU) 301, which performs various appropriate actions and processes based on programs stored in read-only memory (ROM) 302 or programs loaded from storage section 309 into random access memory (RAM) 304. The RAM 304 also stores various programs and data required for the operation of the system 300. The CPU 301, ROM 302, ROM 303, and RAM 304 are interconnected via a bus 305. An input / output (I / O) interface 306 is also connected to the bus 305.

[0102] The following components are connected to I / O interface 306: an input section 307 including a keyboard, mouse, etc.; an output section 308 including a liquid crystal display (LCD) and speakers, etc.; a storage section 309 including a hard disk, etc.; and a communication section 310 including a network interface card such as a LAN card and a modem, etc. The communication section 310 performs communication processing via a network such as the Internet. A drive 311 is also connected to I / O interface 306 as needed. A removable medium 312, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 311 as needed so that computer programs read from it can be installed into storage section 309 as needed.

[0103] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts are implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program is downloaded and installed from a network via communication section 310, and / or installed from removable medium 312. When the computer program is executed by central processing unit (CPU) 301, it performs the functions defined in the methods of this application.

[0104] It should be noted that the computer-readable storage medium of this application is a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium is, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium is any tangible medium containing or storing a program that is used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium includes a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium or any computer-readable storage medium other than a computer-readable storage medium may transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0105] Computer program code for performing the operations of this application is written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code executes entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer is connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or connected to an external computer (e.g., via the Internet using an Internet service provider).

[0106] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram represents a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually execute substantially in parallel, and they may sometimes execute in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, is implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0107] The modules described in the embodiments of this application are implemented in software or hardware.

[0108] In another aspect, this application also provides a computer-readable storage medium included in the electronic device described in the above embodiments; or existing independently and not assembled into the electronic device. The computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: S1, extract static code security features of the code of the AI ​​program to be tested before execution; S2, construct a temporal heterogeneous graph based on the system call event sequence of the AI ​​program to be tested running in an isolated sandbox environment, wherein the entity nodes of the temporal heterogeneous graph represent system entities, and the relational edges of the temporal heterogeneous graph represent the interactive behavior relationships of the system entities, and use a graph neural network to process the temporal heterogeneous graph to extract graph embedding features as dynamic behavior security features characterizing the overall dynamic behavior pattern of the AI ​​program to be tested; S3, analyze the AI ​​program to be tested using a pre-trained first analysis model to obtain deep analysis features of the AI ​​program to be tested; S4, fuse the static code security features, the dynamic behavior security features, and the deep analysis features, input them into a risk classification model, and obtain the final risk assessment result.

[0109] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. An AI program analysis and detection method, characterized in that, Includes the following steps: S1, extract the static code security features of the AI ​​program to be tested before it runs; S2, construct a temporal heterogeneous graph based on the system call event sequence of the AI ​​program to be tested running in the isolated sandbox environment, wherein the entity nodes of the temporal heterogeneous graph represent system entities, the relation edges of the temporal heterogeneous graph represent the interaction behavior relationships of the system entities, and use graph neural networks to process the temporal heterogeneous graph to extract graph embedding features as dynamic behavior security features characterizing the overall dynamic behavior pattern of the AI ​​program to be tested; S3, use the pre-trained first analysis model to analyze the AI ​​program to be detected, and obtain the deep analysis features of the AI ​​program to be detected; S4. The static code security features, dynamic behavior security features, and deep analysis features are integrated and input into the risk classification model to obtain the final risk assessment result.

2. The AI ​​program analysis and detection method according to claim 1, characterized in that, The construction of the temporal heterogeneity graph based on the system call event sequence of the AI ​​program under test running in the isolated sandbox environment in S2 specifically includes: Obtain the sequence of all system call events generated by the AI ​​program under test during its operation. Each system call event includes timestamp information, process identifier that initiated the call, and call parameters. Construct corresponding entity nodes and feature labels based on the call parameters or process identifier. Construct different types of relationship edges and corresponding behavior type labels based on the semantics of the system call events. Construct time sequence information based on the timestamps.

3. The AI ​​program analysis and detection method according to claim 2, characterized in that, The entity nodes include process nodes, file nodes, network nodes, and memory nodes, and the calling parameters include file address parameters, network address parameters, and memory address parameters; The process node is constructed based on the process identifier, and the feature label of the process node includes the permission level information of the process node; The file node is constructed based on the file address parameters, and the feature label of the file node includes the sensitivity level information of the file node; The network node is constructed based on the network address parameters, and the feature tags of the network node include the domain name reputation information of the network node; The memory node is constructed based on the memory address parameters, and the feature label of the memory node includes the read and write attribute information of the memory node.

4. The AI ​​program analysis and detection method according to claim 2, characterized in that, The different types of relationship edges include: process lifecycle management relationship edges, file system operation relationship edges, network communication relationship edges, inter-process communication relationship edges, registry operation relationship edges, and permission and privilege operation relationship edges; The behavior type labels for the process lifecycle management relationship edges include: derivation and termination; The behavior type labels of the file system operation relationship edges include: create, read, write, delete, rename, and execute; The behavior type labels for the network communication relationship edges include: connect, send, receive, and listen; The behavior type labels for the inter-process communication relationship edges include: pipe communication, shared memory, and signal sending; The behavior type labels of the registry operation relationship edges include: registry read, registry write, and registry creation; The behavior type labels for the permission and privilege operation relationship edge include: privilege escalation and access token theft.

5. The AI ​​program analysis and detection method according to claim 1, characterized in that, S1 specifically includes: processing the code of the AI ​​program to be detected using a second analysis model, wherein the second analysis model is a large language model trained on code data to generate static code security features that understand the code semantics of the AI ​​program to be detected.

6. The AI ​​program analysis and detection method according to claim 1, characterized in that, S3 specifically includes processing fused features using the first analysis model. The fused features are based at least on the static code security features generated by S1 and the dynamic behavior security features generated by S2. The first analysis model is a multimodal pre-trained model, used to output deep analysis features based on the association between the static code security features and the dynamic behavior security features.

7. The AI ​​program analysis and detection method according to claim 1, characterized in that, S4 specifically includes: The static code security features, dynamic behavior security features, and deep analysis features are treated as a set of multimodal sequences; A pre-trained Transformer encoder is used to perform cross-modal attention computation on the multimodal sequence to generate a unified feature representation; The unified feature representation is input into the risk classification model to obtain the risk classification result output by the risk classification model.

8. An AI program analysis and detection system, characterized in that, The method according to any one of claims 1 to 7 comprises the following modules: The static feature extraction module is configured to extract static code security features of the AI ​​program to be detected before it runs. The dynamic feature extraction module is configured to construct a temporal heterogeneous graph based on the system call event sequence of the AI ​​program under test running in an isolated sandbox environment. The entity nodes of the temporal heterogeneous graph represent system entities, and the relation edges of the temporal heterogeneous graph represent the interaction behavior relationships of the system entities. The graph neural network is used to process the temporal heterogeneous graph to extract graph embedding features as dynamic behavior security features that characterize the overall dynamic behavior pattern of the AI ​​program under test. The deep feature extraction module is configured to analyze the AI ​​program to be detected using a pre-trained first analysis model to obtain the deep analysis features of the AI ​​program to be detected. The risk assessment output module is configured to integrate the static code security features, the dynamic behavior security features, and the deep analysis features, and input them into the risk classification model to obtain the final risk assessment result.

9. A computer program product, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

10. An electronic device, comprising: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

Images