AI model training code attack identification method and related device
By periodically and automatically detecting and identifying malicious code during AI model training, the high cost and low effectiveness of existing technologies in preventing attacks on AI model training code are solved, achieving efficient and low-cost attack identification and localization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN INTELLIFUSION TECHNOLOGIES CO LTD
- Filing Date
- 2024-12-25
- Publication Date
- 2026-06-26
Smart Images
Figure CN122286752A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence, and in particular to a method and related equipment for identifying attacks on AI model training code. Background Technology
[0002] Computing power is a crucial resource for maintaining the advancement of AI technology. In AI model training, especially large-scale model training, there are instances where training code contains malicious logic, causing training to fail as expected and resulting in a significant waste of computing time. These can be categorized into two types: internal and external. (1) Internal types generally refer to code intentionally added by model trainers (or collaborating members within the same team) that causes training failure; (2) External types generally refer to cases where model trainers do not have malicious intent but introduce attack code by using open-source code libraries to accelerate development. The ways in which attack code is introduced are further divided into two types: one is that model trainers install a version of an open-source code library that appears to function normally but contains malicious code due to a spelling error; the other is that the author of the open-source code library gains the user's trust and then includes malicious code in subsequent upgrade versions.
[0003] Currently, the main method for preventing attacks on AI model training code is manual review of the code logic. If any doubts are found, the code submitter is promptly contacted for clarification before the code is allowed to enter the training process (i.e., the "Code Review" process). From the perspective of defending against attacks on training code, the drawbacks of this process include: (1) Code reviewers are generally senior development engineers in the company, and code review itself consumes a lot of time and energy. In other words, the company itself has already invested a considerable amount of money to defend against the above attacks; (2) Due to the level of code reviewers or time and energy issues, or due to excessive trust in code submitters, hidden attack code may be overlooked because it is not given enough attention; (3) Code attacks on open source library code are considered to be low-probability events, so they are mostly ignored, i.e., no effective defense is formed. Therefore, the existing methods for preventing attacks on AI model training code have the problems of high cost and low effectiveness. Summary of the Invention
[0004] This invention provides a method for identifying attacks on AI model training code, aiming to address the problems of high cost and low effectiveness in existing methods for preventing attacks on AI model training code. By automatically detecting the presence of malicious code in the AI model training code through its execution cycle, and then locating or issuing warnings based on the type of malicious code when it is found, this method reduces the manpower costs users need to invest in defending against attacks on AI model training code while improving the effectiveness of attack code identification.
[0005] In a first aspect, embodiments of the present invention provide a method for identifying attacks on AI model training code, the method comprising the following steps:
[0006] During the AI model training process, the presence of malicious code in the AI model training code is detected based on the execution cycle of the AI model training code.
[0007] If the offensive code exists, then the offensive code is determined to be either a readable type or an unreadable type;
[0008] If the attack code is of a readable type, then the attack code is located;
[0009] If the attack code is of an unreadable type, a warning will be issued for the attack code.
[0010] Optionally, detecting whether the AI model training code contains malicious code based on the execution periodicity of the AI model training code includes:
[0011] Determine the execution cycle of the AI model training code, and the average time of each training iteration in each execution cycle;
[0012] Within each execution cycle, a stack probing sequence is acquired at a preset frequency, the stack probing sequence including the call levels of functions at different levels;
[0013] The iteration period in the execution cycle is calculated based on the power spectral density of the stack probing sequence;
[0014] Based on the average time and the iteration cycle, it is determined whether the AI model training code contains malicious code.
[0015] Optionally, calculating the iteration period in the execution cycle based on the power spectral density of the stack probing sequence includes:
[0016] Calculate the autocorrelation sequence of the stack probing sequence;
[0017] The autocorrelation sequence is transformed by discrete Fourier transform to obtain the digital frequency with the highest power.
[0018] The iteration cycle in the execution cycle is determined based on the digital frequency with the highest power.
[0019] Optionally, determining whether the AI model training code contains malicious code based on the average time and the iteration period includes:
[0020] Calculate the absolute value of the first difference between the average time and the iteration period;
[0021] Calculate the first ratio between the absolute value of the first difference and the average time. If the first ratio is less than a preset first threshold, it is determined that the probability of the AI model training code containing malicious code within the execution cycle is less than the probability threshold; otherwise, it is determined that the AI model training code contains malicious code.
[0022] Optionally, determining whether the AI model training code contains malicious code based on the average time and the iteration period includes:
[0023] If the current execution cycle is the first execution cycle, then the reference cycle is taken as the iteration cycle;
[0024] If the current execution cycle is not the first execution cycle, then based on the iteration cycle, the average time, and the reference cycle corresponding to the historical execution cycles, it is determined whether there is malicious code in the AI model training code;
[0025] If the AI model training code does not contain any malicious code, then the reference period of the current execution period is updated based on the iteration period and the reference period corresponding to the historical execution period.
[0026] Optionally, if the attack code is of a readable type, then locating the attack code includes:
[0027] If the attack code is of a readable type, then the behavior type of the attack code is determined;
[0028] If the behavior type of the attack code is a data operation type, then when a new connection to the Internet or a file deletion operation is detected, the execution frame stack of the training code is tracked to obtain the code location when the new connection to the Internet or file deletion operation is detected.
[0029] If the behavior type of the attack code is computationally expensive, then based on the code of the main function, a function link relationship containing functions at different levels is established. The functions at each level in the function link relationship are reviewed through a large language model. Based on the functions that fail the review, the code location of the attack code is determined.
[0030] Optionally, if the attack code is of an unreadable type, then issuing a warning for the attack code includes:
[0031] If the attack code is of an unreadable type, obtain the dynamic link library loaded during the AI model training process;
[0032] Determine whether the loaded dynamic link library is a pre-prepared dynamic link library;
[0033] If the loaded dynamic link library is not a pre-prepared dynamic link library, a warning will be issued for the loaded dynamic link library.
[0034] Secondly, embodiments of the present invention also provide an AI model training code attack identification device, the AI model training code attack identification device comprising:
[0035] The detection module is used to detect whether there is malicious code in the AI model training code during the AI model training process, based on the execution cycle of the AI model training code.
[0036] The first processing module is used to determine whether the attack code is a readable type or an unreadable type if the attack code exists.
[0037] The second processing module is used to locate the attack code if the attack code is of a readable type.
[0038] The third processing module is used to issue a warning for the attack code if the attack code is of an unreadable type.
[0039] Thirdly, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the AI model training code attack identification method provided in embodiments of the present invention.
[0040] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps in the AI model training code attack identification method provided in the embodiments of the present invention.
[0041] In this embodiment of the invention, during the AI model training process, the presence of malicious code in the AI model training code is detected based on the execution cycle of the AI model training code. If malicious code is found, it is determined whether the malicious code is of a readable or unreadable type. If the malicious code is of a readable type, it is located; if the malicious code is of an unreadable type, an alert is issued. This invention automates the detection of malicious code in AI model training code by utilizing the execution cycle of the AI model training code. When malicious code is found, it is located or an alert is issued based on the type of malicious code. This reduces the human resource cost for users to defend against attacks on AI model training code and improves the effectiveness of identifying malicious code. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0043] Figure 1 This is a flowchart of an AI model training code attack identification method provided by an embodiment of the present invention;
[0044] Figure 2 This is a schematic diagram of the structure of an AI model training code attack identification device provided in an embodiment of the present invention;
[0045] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0047] like Figure 1 As shown, Figure 1 This is a flowchart of an AI model training code attack identification method provided by an embodiment of the present invention. The AI model training code attack identification method includes the following steps:
[0048] 101. During the AI model training process, detect whether there is malicious code in the AI model training code based on the execution cycle of the AI model training code.
[0049] In this embodiment of the invention, the above-described AI model training code attack identification method can be applied to a model training platform. This platform can be built on a server or a distributed server and includes a training data interface (for user uploads), a model data interface (for user uploads), AI model training code, a database containing a list of dynamic link libraries for commonly used deep learning software libraries, and an AI model training code attack identification application. The training data interface can be used to acquire training data, which can be images, text, or speech, or other data suitable for training AI models. The model data interface can be used to acquire the model to be trained. The AI model training code can be used to execute AI model training. The AI model training code attack identification application can be used to implement each step of the AI model training code attack identification method.
[0050] The aforementioned attack code is often executed probabilistically or conditionally (e.g., triggered only in the mid-to-late stages of training) to avoid premature detection due to frequent execution. Considering the obvious periodicity of the execution process of normal AI model training code, and that the aforementioned attack methods increase runtime and disrupt this periodicity, the existence of such code can be inferred from the periodicity indicators of training.
[0051] The theoretical periodicity of the AI model training code during execution can be predetermined. Then, during the AI model training process, the actual periodicity is detected. The theoretical periodicity is compared with the actual periodicity. If the actual periodicity is the same as or close to the theoretical periodicity, it can be considered that the probability of the existence of attack code is low, or that the AI model training code does not contain attack code. If the actual periodicity differs greatly from the theoretical periodicity, it can be considered that the probability of the existence of attack code is high, or that the AI model training code contains attack code.
[0052] 102. If malicious code exists, determine whether the malicious code is a readable or unreadable type.
[0053] In this embodiment of the invention, upon determining that malicious code exists in the AI model training code, it can be further determined whether the malicious code is of a readable or unreadable type. The readable or unreadable type can be determined based on the code language environment of the AI model training code. Some code language environments have high readableness and can be considered readable, while others have low readableness and can be considered unreadable. Therefore, the malicious code can be determined as readable or unreadable based on the code language environment of the AI model training code. If the code language environment of the AI model training code is readable, the corresponding malicious code is readable; if the code language environment of the AI model training code is unreadable, the corresponding malicious code is unreadable. Examples of readable code language environments include Python code, Easy Language code, etc., while examples of unreadable code language environments include C language code, C++ language code, etc.
[0054] 103. If the attack code is of a readable type, then locate the attack code.
[0055] In this embodiment of the invention, since the attack code is a readable type and has strong interpretability, it can be located by monitoring the network and IO operations, or by function calls.
[0056] By locating the malicious code, its position can be found in the AI model training code. The malicious code can then be generated into a report for users to view. These users can be AI model trainers or CodeReviewers.
[0057] 104. If the malicious code is of an unreadable type, then issue a warning for the malicious code.
[0058] In this embodiment of the invention, since the malicious code is an unreadable type and has poor interpretability, it can be directly warned and a warning report can be generated for the AI model trainers or code reviewers to view. This allows the AI model trainers or code reviewers to know that there is malicious code in the AI model training code, and then to manually review the AI model training code.
[0059] In this embodiment of the invention, during the AI model training process, the presence of malicious code in the AI model training code is detected based on the execution cycle of the AI model training code. If malicious code is found, it is determined whether the malicious code is of a readable or unreadable type. If the malicious code is of a readable type, it is located; if the malicious code is of an unreadable type, an alert is issued. This invention automates the detection of malicious code in AI model training code by utilizing the execution cycle of the AI model training code. When malicious code is found, it is located or an alert is issued based on the type of malicious code. This reduces the human resource cost for users to defend against attacks on AI model training code and improves the effectiveness of identifying malicious code.
[0060] It is understood that in the specific implementation of this application, data such as code data, database data, and training data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required. Furthermore, the collection, use, and processing of related data, as well as the training, deployment, and invocation of algorithm models, must comply with the relevant laws, regulations, and standards of the relevant countries and regions.
[0061] Optionally, in the step of detecting whether there is malicious code in the AI model training code based on the execution periodicity of the AI model training code, the execution period of the AI model training code and the average time of each training iteration in each execution period can be determined; within each execution period, a stack probing sequence is obtained at a preset frequency, the stack probing sequence including the call layers of functions at different levels; the iteration period in the execution period is calculated based on the power spectral density of the stack probing sequence; and the presence of malicious code in the AI model training code is determined based on the average time and the iteration period.
[0062] In this embodiment of the invention, the attack code is often executed using a probabilistic or conditionally triggered method to avoid premature detection due to frequent execution. The execution process of normal AI model training code exhibits a clear periodicity, while the execution of attack code disrupts this periodicity. Therefore, this embodiment of the invention can infer the existence of attack code by monitoring the periodicity indicators of the training code.
[0063] Specifically, within a cycle T (e.g., 3 minutes), according to a preset frequency (f) s A function is executed at 1 kHz to obtain a stack probing sequence. The stack probing sequence records the stack level of the executed code frames relative to the main function at different times. This function dynamically reflects the periodic changes in code execution.
[0064] In one embodiment, considering that deep functions execute for a shorter time and have a larger statistical variance, it is necessary to reduce the influence of deep functions on the numerical value to avoid misjudgment. Specifically, when calculating the function value, the function value is weighted according to the stack level of the currently executed code frame; the deeper the stack, the smaller the weighting coefficient.
[0065] The above function can be represented by the following formula:
[0066]
[0067] Where stack_depth(n) represents the stack depth of the code frame being executed at time n relative to the main function (for example, stack_depth(n) = 0 when executing the main function itself, stack_depth(n) = 1 when executing the code of a sub-function directly called by the main function, stack_depth(n) = 2 when executing the code of a grandchild function called by the main function, and so on). C is a constant in (0, 1), which can generally be taken as...
[0068] Record the number of training iterations completed within the cycle and calculate the average time for each training iteration. Then, perform power spectral density analysis on the stack probing sequence to obtain the maximum power spectral density, and estimate the iteration cycle from the power spectral density.
[0069] By comparing the estimated iteration period with the average time, the probability of the attack code being executed within that execution period can be determined. If the estimated iteration period differs significantly from the average time, it indicates a higher probability that the attack code will be executed within that period, confirming the presence of malicious code in the AI model training code. Conversely, if the estimated iteration period differs slightly from the average time, it indicates a lower probability that the attack code will be executed within that period, confirming the absence of malicious code in the AI model training code.
[0070] Optionally, in the step of calculating the iteration period in the execution cycle based on the power spectral density of the stack probe sequence, the autocorrelation sequence of the stack probe sequence can be calculated; the autocorrelation sequence can be transformed by discrete Fourier transform to obtain the digital frequency with the highest power; and the iteration period in the execution cycle can be determined based on the digital frequency with the highest power.
[0071] In this embodiment of the invention, the autocorrelation sequence is a measure of the similarity between a signal and itself at different time delays. For stack probing sequences, calculating the autocorrelation sequence can further reveal periodic patterns in the sequence.
[0072] Calculate the power spectral density of the F(n) sequence, take the maximum value, and estimate the iteration period from the power spectrum. For the sequence F(n) collected above, n=0,1,2,...N-1, first calculate its autocorrelation sequence:
[0073]
[0074] Perform a Discrete Fourier Transform on the above autocorrelation sequence:
[0075]
[0076] Take S FF (e jw The maximum value is used to obtain the digital frequency with the highest power, Max(S). FF (e jw )).
[0077] The conversion yields:
[0078]
[0079] Where N is the number of data points in the acquisition sequence, f s The sampling rate used when acquiring the sequence. m is a fixed parameter introduced from the calculation of the autocorrelation sequence, and fixed parameters introduced from the discrete Fourier transform, such as ω, j, t, e, etc.
[0080] Optionally, in the step of determining whether there is malicious code in the AI model training code based on the average time and the iteration cycle, the absolute value of the first difference between the average time and the iteration cycle can be calculated; the first ratio between the absolute value of the first difference and the average time can be calculated; if the first ratio is less than a preset first threshold, it is determined that the probability of the AI model training code containing malicious code within the execution cycle is less than the probability threshold; otherwise, it is determined that the AI model training code contains malicious code.
[0081] In this embodiment of the invention, the first ratio between the absolute value of the first difference and the average time is used to measure the degree of difference between the average time and the iteration cycle. The larger the first ratio, the greater the difference between the average time and the iteration cycle, and the more likely it is that the AI model training code contains malicious code. The smaller the first ratio, the smaller the difference between the average time and the iteration cycle, and the lower the probability that the AI model training code contains malicious code within the execution cycle, or it can be directly determined that the AI model training code does not contain malicious code.
[0082] Specifically, the calculation of the first ratio mentioned above is as follows:
[0083]
[0084] If satisfied (ε is the first threshold, a constant, which can be 0.05), indicating that the probability of the attack code being executed within the monitoring period is low; otherwise, the attack may have already occurred, and measures should be taken (such as step 103 or 104 in the embodiments of the present invention).
[0085] Optionally, in the step of determining whether the AI model training code contains malicious code based on the average time and the iteration cycle, if the current execution cycle is the first execution cycle, then the reference cycle can be taken as the iteration cycle; if the current execution cycle is not the first execution cycle, then the AI model training code contains malicious code based on the iteration cycle, the average time, and the reference cycle corresponding to the historical execution cycles; if the AI model training code does not contain malicious code, then the reference cycle of the current execution cycle is updated based on the iteration cycle and the reference cycle corresponding to the historical execution cycles.
[0086] In this embodiment of the invention, if it is the first execution cycle, since there is no historical data available for reference, the current iteration cycle is directly used as the reference cycle. The first execution cycle can be compared with the above-mentioned average time and iteration cycle to determine whether there is malicious code in the AI model training code.
[0087] For execution cycles other than the first, the current iteration cycle, average time, and the reference cycle corresponding to historical execution cycles can be considered comprehensively. The presence of malicious code can be determined by comparing the difference between the current iteration cycle and the historical reference cycle, and the ratio of this difference to the average time.
[0088] Specifically, the absolute value of the second difference between the reference period corresponding to the historical execution period and the current iteration period can be calculated, the product between the average time and the reference period corresponding to the historical execution period can be calculated, and the second ratio between the absolute value of the second difference and the product can be calculated. If the second ratio is less than the preset second threshold, it indicates that the monitoring period is consistent with the historical period and there is no attack or the attack has not been triggered. Otherwise, it is determined that there is malicious code in the AI model training code.
[0089] If the current execution cycle is determined to be free of malicious code, then the reference cycle of the current execution cycle is updated based on the current iteration cycle and the historical reference cycle.
[0090] If it is the first monitoring period, take the reference period. For other monitoring periods, if (δ is the second threshold, a constant, which can be 0.05). This indicates that the monitoring period is consistent with the historical period, and there is no attack or the attack has not been triggered. τ is then updated using the following method: (ρ is generally taken as 0.1);
[0091] This step ensures the timeliness and accuracy of the reference period, enabling it to better reflect the normal behavior patterns of the code.
[0092] Implementation details
[0093] Processing of the first execution cycle:
[0094] Determine the iteration period for the first execution cycle (as described in Section 3) and use it as the initial reference period.
[0095] Record the iteration cycle and average time to provide a data basis for subsequent comparisons.
[0096] Handling of non-first execution cycles:
[0097] In each non-first execution cycle, the difference between the current iteration cycle and the reference cycle of the previous execution cycle is first calculated.
[0098] Calculate the ratio of this difference to the average time of the current execution cycle (i.e., the first ratio in the fourth text).
[0099] This ratio is compared to a preset first threshold. If the ratio is less than the first threshold, it indicates that the difference between the current iteration cycle and the historical reference cycle is within an acceptable range, and therefore it is highly likely that there is no malicious code in the current execution cycle; otherwise, it is considered that there is a high probability that there is malicious code.
[0100] Reference period update:
[0101] If the current execution cycle is determined to be free of malicious code, then the reference cycle of the current execution cycle is updated using a weighted average or moving average method based on the current iteration cycle and the reference cycle of the previous execution cycle.
[0102] The purpose of the update is to make the reference cycle more closely reflect the current code behavior pattern and improve the accuracy of subsequent judgments.
[0103] The judgment method based on average time and iteration cycle, combined with historical execution cycle information, can comprehensively consider current and historical data, thereby improving the accuracy and reliability of detection.
[0104] Optionally, in the step of locating the attack code if it is a readable type, the behavior type of the attack code can be determined if it is a readable type; if the behavior type of the attack code is a data operation type, then when a new connection to the Internet or a file deletion operation is detected, the execution frame stack of the training code is tracked to obtain the code location when the new connection to the Internet or the file deletion operation occurs; if the behavior type of the attack code is a computationally expensive type, then based on the code of the main function, a function link relationship containing functions at different levels is established, and the functions at each level in the function link relationship are reviewed through a large language model. Based on the functions that fail the review, the code location of the attack code is determined.
[0105] In this embodiment of the invention, the training code for the AI model is often written in Python, which is highly readable. Therefore, it is possible to locate the specific location of the attack code and identify the responsible party.
[0106] For attacks that leak sensitive information or delete data on the internet, the method for locating them is as follows:
[0107] To monitor the network and I / O operations of the operating system (such as Linux, the strace tool can be used for this purpose).
[0108] When a new connection to the Internet or a file deletion operation is detected, the execution frame stack of the training code is immediately traced (i.e., which line of code the code is running at; Python code can be located using pystack) to obtain the specific code location of the corresponding operation.
[0109] The method for locating attacks that intentionally execute useless code that consumes a large amount of CPU / GPU computing time is as follows:
[0110] Based on the code of the main function, a link relationship is established: "external input → sub-function 1 → sub-function 1 output → function 2 input → function 2 → ... → external output". Since there may be multiple inputs and outputs, and the sub-functions may not be in a simple serial call relationship, the above link relationship eventually forms a directed acyclic graph G.
[0111] Traverse all nodes in G. If a node with an out-degree of 0 is found (excluding "Output"), it indicates the presence of useless code or side-effect code (such as log printing). To further identify whether the code at this location is attack code, execute steps (3) to (5); if no such suspicious situation is found, skip steps (3) to (5) and execute step (6).
[0112] Extract the complete code of the main function. If the suspected useless code is a call to a readable sub-function, extract the complete code of that sub-function as well.
[0113] The extracted code was then queried with the LLM (Large Language Model) to preliminarily determine whether it was useless code that consumed a lot of CPU / GPU computing time.
[0114] Extract conclusions from the LLM's response and implement the strategy:
[0115] If the answer is yes, the relevant code and the LLM answer will be sent to a specialist for focused review;
[0116] If the answer is no, the automatic review of the main function will pass;
[0117] If the answer requires analysis of the implementation of certain grandchild functions, and if the code of the specified grandchild functions is also readable, then extract the code of the grandchild functions for LLM to supplement the analysis and regenerate the answer. Repeat this step until LLM can give a clear answer (for example, LLM also requires the implementation of the sub-functions of the grandchild functions to confirm the conclusion); if the specified grandchild functions are not readable, then remind a specialist to assess the risks of the grandchild functions.
[0118] Using a breadth-first search, the main function calls sub-function 1, sub-function 2, etc., and sub-function 1 calls grandchildren function 1, grandchildren function 2, etc., in turn, until all readable code is checked.
[0119] Optionally, in the step of issuing a warning for attack code if the attack code is of an undecipherable type, if the attack code is of an undecipherable type, the dynamic link library loaded during the execution of AI model training can be obtained; it can be determined whether the loaded dynamic link library is a pre-prepared dynamic link library; if the loaded dynamic link library is not a pre-prepared dynamic link library, a warning can be issued for the loaded dynamic link library.
[0120] In this embodiment of the invention, code written in C++ or similar formats needs to be compiled into binary files to run, resulting in very low readability. Therefore, this type of attack code is more stealthy and requires early warning. The method is as follows:
[0121] For dynamic link libraries (.so files or .dll files) of commonly used deep learning software libraries, version information is obtained from the file name and other means, the MD5 value of the library file is calculated, and the data is stored in the database as a key-value pair relationship of "file name-version number-MD5 value".
[0122] When training an AI model, trace the paths of the loaded dynamic link libraries (for example, on a Linux system, you can enable the lld tool to trace libraries by setting the environment variable "LD_TRACE_LOADED_OBJECTS=1");
[0123] For each tracked dynamic link library, extract its filename, version number, and MD5 value, and compare them with the database. If the filename or version number is missing, or the MD5 value is inconsistent, it indicates that the dynamic link library is an unapproved new version or a tampered version, and a warning should be issued.
[0124] like Figure 2 As shown, this embodiment of the invention provides an AI model training code attack identification device, which includes:
[0125] The detection module 201 is used to detect whether there is malicious code in the AI model training code during the AI model training process, based on the execution periodicity of the AI model training code.
[0126] The first processing module 202 is used to determine whether the attack code is a readable type or an unreadable type if the attack code exists.
[0127] The second processing module 203 is used to locate the attack code if the attack code is of a readable type;
[0128] The third processing module 204 is used to issue a warning for the attack code if the attack code is of an unreadable type.
[0129] Optionally, the first processing module 202 is further configured to determine the execution cycle of the AI model training code and the average time of each training iteration in each execution cycle; within each execution cycle, acquire a stack probing sequence at a preset frequency, the stack probing sequence including the call layers of functions at different levels; calculate the iteration cycle in the execution cycle based on the power spectral density of the stack probing sequence; and determine whether there is malicious code in the AI model training code based on the average time and the iteration cycle.
[0130] Optionally, the first processing module 202 is further configured to calculate the autocorrelation sequence of the stack probing sequence; perform a discrete Fourier transform on the autocorrelation sequence to obtain the digital frequency with the highest power; and determine the iteration cycle in the execution cycle based on the digital frequency with the highest power.
[0131] Optionally, the first processing module 202 is further configured to calculate the absolute value of a first difference between the average time and the iteration period; calculate a first ratio between the absolute value of the first difference and the average time; if the first ratio is less than a preset first threshold, then determine that the probability of the AI model training code containing malicious code within the iteration period is less than the probability threshold; otherwise, determine that the AI model training code contains malicious code.
[0132] Optionally, the first processing module 202 is further configured to: if the current execution cycle is the first execution cycle, then take the reference cycle as the iteration cycle; if the current execution cycle is not the first execution cycle, then determine whether the AI model training code contains malicious code based on the iteration cycle, the average time, and the reference cycle corresponding to the historical execution cycle; if the AI model training code does not contain malicious code, then update the reference cycle of the current execution cycle based on the iteration cycle and the reference cycle corresponding to the historical execution cycle.
[0133] Optionally, the second processing module 203 is further configured to: if the attack code is of a readable type, determine the behavior type of the attack code; if the behavior type of the attack code is a data operation type, then when a new connection to the Internet or a file deletion operation is detected, track the execution frame stack of the training code to obtain the code location when the new connection to the Internet or file deletion operation occurs; if the behavior type of the attack code is of a computational cost type, then based on the code of the main function, establish a function link relationship containing functions at different levels, examine each level of functions in the function link relationship through a large language model, and determine the code location of the attack code based on the functions that fail the examination.
[0134] Optionally, the third processing module 204 is further configured to: if the attack code is of an unreadable type, obtain the dynamic link library loaded during the execution of AI model training; determine whether the loaded dynamic link library is a pre-prepared dynamic link library; and if the loaded dynamic link library is not a pre-prepared dynamic link library, issue a warning for the loaded dynamic link library.
[0135] like Figure 3 As shown, this embodiment of the invention also provides an electronic device, including a processor, which can execute any of the above-mentioned AI model training code attack identification methods.
[0136] Specifically, it includes processor 301 and memory 302, as well as a computer program stored in memory 302 and capable of running on processor 301 that executes the AI model training code attack identification method, wherein:
[0137] The processor 301 executes the following steps: The AI model training code stored in the memory 302 attacks the recognition method in the calculator program.
[0138] During the AI model training process, the presence of malicious code in the AI model training code is detected based on the execution cycle of the AI model training code.
[0139] If the offensive code exists, then the offensive code is determined to be either a readable type or an unreadable type;
[0140] If the attack code is of a readable type, then the attack code is located;
[0141] If the attack code is of an unreadable type, a warning will be issued for the attack code.
[0142] Optionally, the processor 301's execution of the method to detect whether the AI model training code contains malicious code based on the execution periodicity of the AI model training code includes:
[0143] Determine the execution cycle of the AI model training code, and the average time of each training iteration in each execution cycle;
[0144] Within each execution cycle, a stack probing sequence is acquired at a preset frequency, the stack probing sequence including the call levels of functions at different levels;
[0145] The iteration period in the execution cycle is calculated based on the power spectral density of the stack probing sequence;
[0146] Based on the average time and the iteration cycle, it is determined whether the AI model training code contains malicious code.
[0147] Optionally, the processor 301's calculation of the iteration cycle based on the power spectral density of the stack probing sequence includes:
[0148] Calculate the autocorrelation sequence of the stack probing sequence;
[0149] The autocorrelation sequence is transformed by discrete Fourier transform to obtain the digital frequency with the highest power.
[0150] The iteration cycle in the execution cycle is determined based on the digital frequency with the highest power.
[0151] Optionally, the process executed by processor 301 to determine whether the AI model training code contains malicious code based on the average time and the iteration period includes:
[0152] Calculate the absolute value of the first difference between the average time and the iteration period;
[0153] Calculate the first ratio between the absolute value of the first difference and the average time. If the first ratio is less than a preset first threshold, then determine that the probability of the AI model training code containing malicious code within the iteration period is less than the probability threshold; otherwise, determine that the AI model training code contains malicious code.
[0154] Optionally, the process executed by processor 301 to determine whether the AI model training code contains malicious code based on the average time and the iteration period includes:
[0155] If the current execution cycle is the first execution cycle, then the reference cycle is taken as the iteration cycle;
[0156] If the current execution cycle is not the first execution cycle, then based on the iteration cycle, the average time, and the reference cycle corresponding to the historical execution cycles, it is determined whether there is malicious code in the AI model training code;
[0157] If the AI model training code does not contain any malicious code, then the reference period of the current execution period is updated based on the iteration period and the reference period corresponding to the historical execution period.
[0158] Optionally, the step of processor 301 locating the attack code if it is of a readable type includes:
[0159] If the attack code is of a readable type, then the behavior type of the attack code is determined;
[0160] If the behavior type of the attack code is a data operation type, then when a new connection to the Internet or a file deletion operation is detected, the execution frame stack of the training code is tracked to obtain the code location when the new connection to the Internet or file deletion operation is detected.
[0161] If the behavior type of the attack code is computationally expensive, then based on the code of the main function, a function link relationship containing functions at different levels is established. The functions at each level in the function link relationship are reviewed through a large language model. Based on the functions that fail the review, the code location of the attack code is determined.
[0162] Optionally, if the malicious code executed by processor 301 is of an unreadable type, a warning is issued regarding the malicious code, including:
[0163] If the attack code is of an unreadable type, obtain the dynamic link library loaded during the AI model training process;
[0164] Determine whether the loaded dynamic link library is a pre-prepared dynamic link library;
[0165] If the loaded dynamic link library is not a pre-prepared dynamic link library, a warning will be issued for the loaded dynamic link library.
[0166] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the AI model training code attack identification method provided in this invention and achieves the same technical effect. To avoid repetition, it will not be described again here.
[0167] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0168] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.
Claims
1. A method for identifying attacks on AI model training code, characterized in that, The method includes the following steps: During the AI model training process, the presence of malicious code in the AI model training code is detected based on the execution cycle of the AI model training code. If the offensive code exists, then the offensive code is determined to be either a readable type or an unreadable type; If the attack code is of a readable type, then the attack code is located; If the attack code is of an unreadable type, a warning will be issued for the attack code.
2. The AI model training code attack identification method as described in claim 1, characterized in that, The step of detecting whether there is malicious code in the AI model training code based on the execution periodicity of the AI model training code includes: Determine the execution cycle of the AI model training code, and the average time of each training iteration in each execution cycle; Within each execution cycle, a stack probing sequence is acquired at a preset frequency, the stack probing sequence including the call levels of functions at different levels; The iteration period in the execution cycle is calculated based on the power spectral density of the stack probing sequence; Based on the average time and the iteration cycle, it is determined whether the AI model training code contains malicious code.
3. The AI model training code attack identification method as described in claim 2, characterized in that, The calculation of the iteration period in the execution cycle based on the power spectral density of the stack probing sequence includes: Calculate the autocorrelation sequence of the stack probing sequence; The autocorrelation sequence is transformed by discrete Fourier transform to obtain the digital frequency with the highest power. The iteration cycle in the execution cycle is determined based on the digital frequency with the highest power.
4. The AI model training code attack identification method as described in claim 2, characterized in that, The step of determining whether the AI model training code contains malicious code based on the average time and the iteration period includes: Calculate the absolute value of the first difference between the average time and the iteration period; Calculate the first ratio between the absolute value of the first difference and the average time. If the first ratio is less than a preset first threshold, it is determined that the probability of the AI model training code containing malicious code within the execution cycle is less than the probability threshold; otherwise, it is determined that the AI model training code contains malicious code.
5. The AI model training code attack identification method as described in claim 2, characterized in that, The step of determining whether the AI model training code contains malicious code based on the average time and the iteration period includes: If the current execution cycle is the first execution cycle, then the reference cycle is taken as the iteration cycle; If the current execution cycle is not the first execution cycle, then based on the iteration cycle, the average time, and the reference cycle corresponding to the historical execution cycles, it is determined whether there is malicious code in the AI model training code; If the AI model training code does not contain any malicious code, then the reference period of the current execution period is updated based on the iteration period and the reference period corresponding to the historical execution period.
6. The AI model training code attack identification method as described in claims 1 to 5, characterized in that, If the malicious code is of a readable type, then locating the malicious code includes: If the attack code is of a readable type, then the behavior type of the attack code is determined; If the behavior type of the attack code is a data operation type, then when a new connection to the Internet or a file deletion operation is detected, the execution frame stack of the training code is tracked to obtain the code location when the new connection to the Internet or file deletion operation is detected. If the behavior type of the attack code is computationally expensive, then based on the code of the main function, a function link relationship containing functions at different levels is established. The functions at each level in the function link relationship are reviewed through a large language model. Based on the functions that fail the review, the code location of the attack code is determined.
7. The AI model training code attack identification method as described in any one of claims 1 to 5, characterized in that, If the malicious code is of an unreadable type, a warning is issued for the malicious code, including: If the attack code is of an unreadable type, obtain the dynamic link library loaded during the AI model training process; Determine whether the loaded dynamic link library is a pre-prepared dynamic link library; If the loaded dynamic link library is not a pre-prepared dynamic link library, a warning will be issued for the loaded dynamic link library.
8. A device for identifying attacks on AI model training code, characterized in that, The AI model training code attack detection device includes: The detection module is used to detect whether there is malicious code in the AI model training code during the AI model training process, based on the execution cycle of the AI model training code. The first processing module is used to determine whether the attack code is a readable type or an unreadable type if the attack code exists. The second processing module is used to locate the attack code if the attack code is of a readable type. The third processing module is used to issue a warning for the attack code if the attack code is of an unreadable type.
9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps in the AI model training code attack identification method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps in the AI model training code attack identification method as described in any one of claims 1 to 7.