Security detection method, device and equipment
By converting the nodes to be tested in the source code file into text markers and performing semantic analysis, combined with the pre-trained security detection model, the problem that existing vulnerability detection tools are difficult to cope with diversified vulnerabilities, and achieving higher vulnerability detection accuracy Sex and flexibility.
Patent Information
- Application Number
- CN202410966626.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-08
AI Technical Summary
Existing automated vulnerability detection tools are difficult to deal with diversified vulnerabilities, resulting in low accuracy in vulnerability detection and the inability to effectively identify risk vulnerabilities with varying forms.
By converting the nodes to be tested in the source code file into text markers, and using the semantic analysis model for semantic analysis, combined with the pre-trained security detection model, the detection results of whether there is a security risk are output, which improves the accuracy and flexibility of vulnerability detection sex.
It improves the accuracy and flexibility of vulnerability detection, can better identify diversified risk vulnerabilities, and enhances the ability of source code security detection.
Smart Images

Figure CN118916882A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of network security technology, and in particular to a security detection method, device and equipment. Background Art
[0002] With the rapid development of Internet technology, software has become an indispensable tool in all walks of life. Whether in office, daily life or other industrial scenarios, the operation of program software is indispensable. Correspondingly, software security issues have followed one after another. Some attackers find vulnerabilities in program software and attack the program software through the vulnerabilities to illegally obtain personal information, commercial confidential information, etc. Therefore, before the program software is put online, the vulnerabilities in the source code need to be carefully checked to avoid leaving opportunities for attackers.
[0003] Although the current automated vulnerability detection tools have improved the efficiency of vulnerability detection to a certain extent, they can only detect fixed-form vulnerabilities. With the diversification of vulnerability types and forms, the current static vulnerability detection tools are difficult to cope with the diversification of vulnerabilities, resulting in low accuracy of vulnerability detection. Summary of the invention
[0004] The present application provides a security detection method, device and equipment for improving the accuracy of source code security detection.
[0005] In a first aspect, an embodiment of the present application provides a security detection method, comprising: determining a node to be tested according to a source code file to be tested and a first condition, and generating a text tag for the node to be tested, wherein the first condition is used to screen nodes in the source code file that are at risk of being attacked, and each tag in the text tag corresponds to a type of node information of the node to be tested; converting the text tag into an integer sequence according to a correspondence between preset tags and integer identifiers; performing semantic analysis on the integer sequence through a semantic analysis model to obtain a first vector, wherein the first vector is used to indicate semantic information of the node to be tested in the source code file; performing security detection on the first vector through a pre-trained security detection model, and outputting a security detection result indicating whether the node to be tested has a security risk.
[0006] In an embodiment of the present application, the node information of the node to be tested is converted into the form of an integer sequence, and the integer sequence is output to the semantic analysis model. The node information of the node to be tested is semantically analyzed through the semantic analysis model, so that the program semantics of the node to be tested in the source code file can be better understood. Even for risk vulnerabilities with variable forms, the security detection model can detect whether there is a security risk in the node to be tested based on the semantic analysis results, that is, the accuracy of security detection is improved. In addition, compared with the method of detecting based on fixed vulnerability feature matching in the prior art, the embodiment of the present application discovers risk vulnerabilities from the semantic level, which is more conducive to identifying more diverse risk vulnerabilities and also improves the flexibility of security detection.
[0007] In a possible implementation, the semantic analysis model includes a language model and a word embedding model; performing semantic analysis on the integer sequence through the semantic analysis model to obtain a first vector includes: inputting the integer sequence into the word embedding model, performing lexical semantic analysis on the integer sequence through the word embedding model to obtain a second vector, wherein the second vector is used to indicate node semantic information of the node to be tested; inputting the second vector into the language model, performing contextual semantic analysis on the second vector through the language model to obtain the first vector.
[0008] In this embodiment, the word embedding model and the language model are used to perform lexical semantic analysis and contextual semantic analysis on the node information of the node to be tested, respectively, which can better understand the node semantic information of the node to be tested itself and the contextual semantic information of the node to be tested in the source code file, which is conducive to improving the accuracy of security detection.
[0009] In a possible implementation, the word embedding model is obtained by retraining a pre-trained word embedding model using a sample data set, and the language model is obtained by retraining a pre-trained language model using the sample data set, and the sample data set includes various types of risk features.
[0010] In this embodiment, the word embedding model and the language model are retrained using sample data sets of various types of risk features, so that the retrained word embedding model and the language model can better understand and represent the semantics of the node to be tested in the scenario described in this application, thereby facilitating improving the accuracy of security detection.
[0011] In a possible implementation, the security detection is performed on the first vector through a pre-trained security detection model, and the security detection result is output, including: based on the convolution layer of the security detection model, extracting the risk feature information in the first vector; based on the pooling layer of the security detection model, reducing the dimension of the risk feature information, and outputting a third vector; based on the bidirectional long short-term memory network layer in the security detection model, extracting the time series feature information in the third vector, and outputting a fourth vector; based on the dense layer in the fully connected layer in the security detection model, reducing the feature dimension of the fourth vector, and outputting a fifth vector, and combining the classification function in the fully connected layer, operating the fifth vector to determine the risk value of the node to be tested; based on the fully connected layer, judging whether the risk value is greater than a preset risk threshold, and if so, outputting the security detection result that the node to be tested has a security risk, and if not, outputting the security detection result that the node to be tested does not have a security risk.
[0012] In a possible implementation, determining the node to be tested based on the source code file to be tested and the first condition includes: parsing the source code file into a first abstract syntax tree, and traversing to obtain irrelevant nodes in the first abstract syntax tree, where the irrelevant nodes are risk-free nodes predefined in the source code file; deleting irrelevant nodes in the first abstract syntax tree, and reconstructing the first abstract syntax tree to obtain a second abstract syntax tree; serializing the second abstract syntax tree, and determining the node to be tested that meets the first condition from the serialized second abstract syntax tree.
[0013] In a possible implementation, generating the text mark of the node to be tested includes: extracting key node information of the node to be tested, wherein the key node information includes node type, node attribute, node value, and tree structure information of the node; converting the key node information into a preset format to generate the text mark.
[0014] In a second aspect, an embodiment of the present application provides a security detection device, comprising: a determination module, used to determine a node to be tested according to a source code file to be tested and a first condition, and generate a text tag of the node to be tested, wherein the first condition is used to screen nodes in the source code file that are at risk of being attacked, and each tag in the text tag corresponds to a type of node information of the node to be tested; a conversion module, used to convert the text tag into an integer sequence according to a correspondence between a preset tag and an integer identifier; a semantic analysis module, used to perform semantic analysis on the integer sequence through a semantic analysis model to obtain a first vector, wherein the first vector is used to indicate the semantic information of the node to be tested in the source code file; a detection module, used to perform security detection on the first vector through a pre-trained security detection model, and output a security detection result indicating whether the node to be tested has a security risk.
[0015] In a possible implementation, the semantic analysis model includes a language model and a word embedding model; the semantic analysis module is specifically used to: input the integer sequence into the word embedding model, perform lexical semantic analysis on the integer sequence through the word embedding model, and obtain a second vector, wherein the second vector is used to indicate node semantic information of the node to be tested; input the second vector into the language model, and perform contextual semantic analysis on the second vector through the language model to obtain the first vector.
[0016] In a possible implementation, the word embedding model is obtained by retraining a pre-trained word embedding model using a sample data set, and the language model is obtained by retraining a pre-trained language model using the sample data set, and the sample data set includes various types of risk features.
[0017] In one possible implementation, the detection module is specifically used to: extract risk feature information from the first vector based on the convolutional layer of the security detection model; reduce the dimension of the risk feature information based on the pooling layer of the security detection model, and output a third vector; extract time series feature information from the third vector based on the bidirectional long short-term memory network layer in the security detection model, and output a fourth vector; reduce the feature dimension of the fourth vector based on the dense layer in the fully connected layer in the security detection model, and output a fifth vector, and perform operations on the fifth vector in combination with the classification function in the fully connected layer to determine the risk value of the node to be tested; based on the fully connected layer, determine whether the risk value is greater than a preset risk threshold, and if so, output a security detection result indicating that the node to be tested has a security risk, and if not, output a security detection result indicating that the node to be tested does not have a security risk.
[0018] In a possible implementation, the determination module is specifically used to: parse the source code file into a first abstract syntax tree, and traverse to obtain irrelevant nodes in the first abstract syntax tree, where the irrelevant nodes are risk-free nodes predefined in the source code file; delete irrelevant nodes in the first abstract syntax tree, and reconstruct the first abstract syntax tree to obtain a second abstract syntax tree; serialize the second abstract syntax tree, and determine the nodes to be tested that meet the first condition from the serialized second abstract syntax tree.
[0019] In a possible implementation, the determination module is specifically used to: extract key node information of the node to be tested, the key node information including node type, node attribute, node value, and node tree structure information; convert the key node information into a preset format to generate the text tag.
[0020] In a third aspect, an embodiment of the present application provides a security detection device, comprising: at least one processor, and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the method described in the first aspect and any possible implementation manner by executing the instructions stored in the memory.
[0021] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed on a computer, the computer executes the method described in the first aspect and any possible implementation manner.
[0022] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed on a computer, enables the method described in the first aspect and any possible implementation manner to be implemented.
[0023] The beneficial effects of the second to fifth aspects can be referred to the contents described in the first aspect above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 A schematic diagram of an application scenario of a security detection method provided in an embodiment of the present application;
[0025] Figure 2 A schematic diagram of a safety detection method provided in an embodiment of the present application;
[0026] Figure 3 A schematic diagram of the structure of a security detection model provided in an embodiment of the present application;
[0027] Figure 4A schematic diagram of the structure of a safety detection device provided in an embodiment of the present application;
[0028] Figure 5 A schematic diagram of the structure of a safety detection device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0029] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.
[0030] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.
[0031] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description. It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.
[0032] In order to better understand the technical solution provided by the embodiments of the present application, Figure 1 The application scenarios to which the embodiments of the present application are applicable are briefly described.
[0033] Please refer to Figure 1 , is a schematic diagram of an application scenario of a security detection method provided in an embodiment of the present application. Figure 1 As shown, the scenario diagram includes a device 110 and a safety detection apparatus 120 , and wired or wireless communication can be performed between the device 110 and the safety detection apparatus 120 .
[0034] Among them, the device 110 refers to a device with functions such as code writing, compiling or code storage, code running, etc., and is specifically a terminal device or server, and the terminal device is, for example, a personal computer (PC), a tablet computer, a laptop computer, a PDA, and a mobile internet device (MID). The security detection device 120 refers to a device or module with specific data processing functions and security detection functions. When the security detection device 120 is a module, it can be a functional module on the device 110 or in other devices, and the embodiments of the present application are not limited to this.
[0035] Exemplarily, the security detection device 120 can obtain the source code file to be detected from the device 110, and determine the node to be tested in the source code file that has the risk of being attacked according to the source code file and the first condition, and generate a text tag of the node to be tested, based on the correspondence between the preset tag and the integer identifier, convert the text tag into an integer sequence, perform semantic analysis on the integer sequence through a semantic analysis model, obtain a first vector, perform security detection on the first vector through a security detection model, and obtain a security detection result of whether the node to be tested has a security risk. Among them, the specific implementation method of the security detection device 120 determining whether the node to be tested has a security risk will be described in detail below.
[0036] Please refer to Figure 2 , is a flow chart of a safety detection method provided in an embodiment of the present application. The following is a flow chart of a safety detection method performed by a safety detection device. Figure 2 The safety detection device is, for example, Figure 1 The safety detection device 120 is shown.
[0037] S201, determining the node to be tested according to the source code file to be tested and the first condition, and generating text tags for the node to be tested, wherein the first condition is used to screen nodes in the source code file that are at risk of being attacked, and each tag in the text tags corresponds to a type of node information of the node to be tested.
[0038] The source code file to be tested may refer to a code file with security detection requirements, and the security detection requirements may refer to requirements such as vulnerability detection. The source code file can be obtained from other devices, such as Figure 1The device 110 shown; alternatively, the source code file may be input into the security detection device by a user, which is not limited in this embodiment of the present application.
[0039] In the source code file, important nodes have a great influence on the realization of the program functions of the source code file. Therefore, important nodes are usually the nodes that attackers prefer to attack, that is, important nodes in the source code file are at greater risk of being attacked. Based on this, the embodiment of the present application determines the more important nodes in the source code file as nodes to be tested to ensure the security of the source code file. In the specific implementation, the security detection device can determine the nodes to be tested according to the first condition. The first condition may indicate the characteristics of nodes or important nodes in the source code file that are at risk of being attacked. The characteristics may involve aspects such as semantic relevance, security impact, code logic, data flow, dependencies, declarations and definitions, code specifications, maintainability, specific domain knowledge, and contextual information. The above characteristics are illustrated below.
[0040] 1. Semantic relevance: whether the node carries information that has a significant impact on the program semantics. For example, control flow statements (if, while, do, etc.) and function definitions have a direct impact on the execution flow of the program.
[0041] 2. Security impact: whether the node may be related to security vulnerabilities. For example, input processing, exception handling, memory management, etc. may contain security vulnerabilities.
[0042] 3. Code logic: whether the node represents the key logic part of the code, such as loops, conditional judgments, function calls, etc.
[0043] 4. Data flow: Whether the node involves the definition, transfer or modification of data, especially those variables and parameters that may affect the program state.
[0044] 5. Dependencies: Whether the node has other code parts that it depends on, for example, function nodes called by other functions or variables referenced in multiple places.
[0045] 6. Declarations and Definitions: Declaration and definition nodes for variables, types, functions, etc., as they provide context for the rest of the program.
[0046] 7. Coding standards: Conformance to specific coding standards or style guides. Some standards may require special attention to specific coding patterns or practices.
[0047] 8. Maintainability: Code readability and maintainability. Clearly defined functions and good modularization can improve code maintainability.
[0048] 9. Domain-specific knowledge: In a specific application domain, some specific AST nodes may be more important than other nodes because they are closely related to the domain logic.
[0049] 10. Contextual information: The context of the node in the code. For example, the use of a variable under certain conditions may have an important impact on the program behavior.
[0050] Exemplarily, important nodes in a source code file include: control flow nodes (if, while, and do), function call nodes (function names), declaration nodes (such as declaring methods, parameter lists, and variables), and data types / return types.
[0051] The specific method in which the security detection device determines the node to be tested from the source code file is described in detail below.
[0052] The security detection device may parse the source code file into a first abstract syntax tree (AST). Exemplarily, the source code file may be parsed by a syntax analyzer to generate the first abstract syntax tree. The syntax analyzer may be, for example, ANTLR (a No other tool for language recognition).
[0053] The security detection device traverses the first abstract syntax tree and determines irrelevant nodes in the first abstract syntax tree. The irrelevant nodes are risk-free nodes predefined in the source code file. It should be understood that the risk-free nodes here are nodes in the source code file with extremely low risk of being attacked compared to the nodes to be tested. Irrelevant nodes can be judged based on the following characteristics:
[0054] 1. Context independence: Some nodes may not provide useful information about security vulnerabilities in a specific context. For example, in the context of vulnerability detection, syntactic elements such as comments or spaces usually do not contain information about potential vulnerabilities.
[0055] 2. Functionality irrelevance: Nodes that are not related to the main logic or functionality of the program, such as imported libraries or module declarations, may not be relevant for vulnerability detection.
[0056] 3. Security irrelevance: nodes that do not directly affect the security features of the program, for example, some specific metadata or document strings.
[0057] 4. Redundancy: In some cases, there may be redundant nodes in the AST, such as repeated variable declarations or redundant code blocks.
[0058] 5. Complexity irrelevance: Certain complex structures may not be important for vulnerability detection, such as deeply nested code blocks, if they do not contain potential security vulnerabilities.
[0059] 6. Specific language features: Some programming language-specific syntactic structures may not be related to security vulnerabilities, for example, macro definitions or template metaprogramming.
[0060] Exemplarily, the security detection device may identify irrelevant nodes in the first abstract syntax tree through a static analysis tool, and the static analysis tool may be configured with the features of the irrelevant nodes described above. When an irrelevant node is identified, the irrelevant node may be marked, and the mark is used to indicate that the irrelevant node can be deleted. Then, after the security detection device completes the traversal of the first abstract syntax tree, all irrelevant nodes in the first abstract syntax tree may be deleted. Exemplarily, the security detection device may use an abstract syntax tree traversal algorithm to traverse the first abstract syntax tree, and the security detection device may further modify the traversal algorithm so that during the traversal process, irrelevant nodes in the first abstract syntax tree may be deleted synchronously. Among them, the traversal algorithm is, for example, a depth-first search algorithm or a breadth-first search algorithm.
[0061] After the security detection device deletes irrelevant nodes in the first abstract syntax tree, the first abstract syntax tree can be reconstructed to obtain a second abstract syntax tree. Reconstructing the first abstract syntax tree can ensure the integrity and consistency of the structure of the abstract syntax tree, thereby avoiding the impact on the security detection result.
[0062] In order to facilitate subsequent processing and analysis, the security detection device can serialize the second abstract syntax tree to obtain a serialized second abstract syntax tree. Serialization can be understood as converting the second abstract syntax tree from a tree structure to a linear format. The following uses the depth-first search algorithm as an example to illustrate the way in which the security detection device serializes the second abstract syntax tree.
[0063] The security detection device may start from the root node of the second abstract syntax tree, traverse each node along the depth of the tree until a leaf node is reached, and then start backtracking. During the traversal process, the depth (or height) of each node and the parent node information of each node are recorded. After the traversal is completed, the security detection device may save the nodes in the second abstract syntax tree in a linear data structure according to their access order in the second abstract syntax tree, and the linear data structure is, for example, a list or an array.
[0064] In order to be able to understand the original tree structure and maintain the original tree structure information after serialization, the security detection device can add additional tree structure information in the linear format, the tree structure information such as node index, height, depth, parent node reference, etc.
[0065] In practical applications, for abstract syntax trees with complex nested structures, the depth-first search algorithm can be preferred. The depth-first search algorithm can ensure that all nodes in the abstract syntax tree are visited and can be processed in a specific order (such as pre-order, in-order, or post-order).
[0066] After obtaining the serialized second abstract syntax tree, the security detection device can filter out the nodes to be tested that meet the first condition from the serialized second abstract syntax tree based on the first condition. The specific content of the first condition can be referred to the content described above, and will not be repeated here. The number of nodes to be tested can be one or more, and the embodiment of the present application does not limit this.
[0067] Further, the safety detection device can generate a text tag of the node to be tested. Alternatively, it can be understood that the node to be tested is converted into a formalized format that is easy for machine processing, and this format can be in the form of a text tag (token). Specifically, the safety detection device can extract the key node information of the node to be tested, convert the key node information into a preset format, and generate a text tag. Among them, the key node information may include node type, node attribute, node value, node tree structure information, etc. Exemplarily, the key node information is illustrated below.
[0068] 1. Node type: The type of each node, such as function declaration, control flow statement (if, while), variable declaration, etc.
[0069] 2. Node attributes: Node attributes, such as the return type of a function (return_type), parameter list (parameter_list), data type of a variable (such as int), etc.
[0070] 3. Node value: the specific value or name of the node, such as function name (function_name), variable name, etc.
[0071] 4. Structural information: The structural information of the node in the second abstract syntax tree, which may include relationships such as parent node, child node, and sibling node.
[0072] 5. Metadata: such as the location information of the node in the source code.
[0073] In the embodiment of the present application, the specific form of the text tag can be a structured tag sequence. For example, for a function declaration, the tag sequence can be expressed as: ["return_type", "void", "function_name", "myFunction", "parameter_list", "(int arg1, int arg2)", "statements", "..."]; for an if statement, the tag sequence can be expressed as: ["if", "condition", "statement_block"].
[0074] S202: Convert the text mark into an integer sequence according to a preset correspondence between the mark and the integer identifier.
[0075] The correspondence between the preset marks and the integer identifiers may be pre-configured in the safety detection device, and the safety detection device may be pre-configured with the correspondence between all marks and integer identifiers.
[0076] The security detection device can then identify each tag in the text tag, determine the unique integer identifier corresponding to each tag, and generate an integer sequence for the text tag, which can also be represented as a Token ID.
[0077] Exemplarily, the process of converting the text token into an integer sequence by the security detection device can be implemented by a tokenizer configured in the security detection device, wherein an identification table is maintained in the tokenizer, and the identification table includes all tokens and their corresponding unique integer identifications.
[0078] In a possible implementation, the safety detection device may also add an identifier at the start position and the end position of the integer sequence of the node to be tested to mark the start and end of the integer sequence. The identifier may be in the form of a character, for example, the start identifier may be set to [st], and the end identifier may be expressed as [end].
[0079] In a possible implementation, when there are multiple nodes to be tested, the sequence lengths of the integer sequences of the multiple nodes to be tested can be made the same by padding or truncation techniques. Specifically, when there are multiple nodes to be tested, a standard sequence length of the nodes to be tested can be set, and the standard sequence length can be set based on actual needs, which can be a model requirement for specific use. If the safety detection device determines that the length of the integer sequence of the node to be tested is less than the standard sequence length, it will extend the length of the integer sequence to the standard sequence length by adding a "padding" mark. If the length of the integer sequence exceeds the standard sequence length, the integer sequence will be truncated, retaining only the beginning or most important part of the sequence, so that the length of the integer sequence reaches the standard sequence length.
[0080] The sequence lengths of multiple integer sequences are set to the same length, on the one hand, to meet the input requirements of the model; on the other hand, it can ensure that all integer sequences have the same dimension before being input into the model, so that the model can perform batch processing and parallel calculations on multiple integer sequences to improve the calculation efficiency of the model; furthermore, the same sequence lengths of all integer sequences can enable the model to perform the same operations on all integer sequences, thereby ensuring the consistency of the execution methods of all integer sequences and ensuring the accuracy of the results output by the model.
[0081] It should be understood that the conversion of the text mark into an integer sequence here is only to convert the text mark into a format that is easy for machine processing. Each integer identifier in the integer sequence corresponds to a mark in the text mark, and the text mark indicates the key node information in the node to be tested. Therefore, the integer sequence can also indicate the key node information in the node to be tested.
[0082] S203, performing semantic analysis on the integer sequence through a semantic analysis model to obtain a first vector, where the first vector is used to indicate semantic information of the node to be tested in the source code file.
[0083] The security detection device performs semantic analysis on the integer sequence through a semantic analysis model, and can obtain a first vector indicating the semantic information of the node to be tested in the source code file, which is equivalent to obtaining the program semantics of the node to be tested in the source code file. Therefore, it can subsequently identify whether the node to be tested has a security risk based on the semantic information.
[0084] In one possible implementation, the semantic analysis model may include a language model and a word embedding model, wherein the language model may be a pre-trained natural language processing model, and the language model may be a pre-trained language model based on Bidirectional Encoder Representations from Transformers (BERT). The pre-training process of BERT may be through Extract-Transform-Load (ETL), and therefore, the language model may also be referred to as the EtlBERT language model.
[0085] In this implementation, the security detection device may first input the integer sequence into the word embedding model, perform lexical semantic analysis on the integer sequence through the word embedding model, and obtain a second vector, which is used to indicate the node semantic information of the node to be tested. The node semantic information may include semantic information such as node type, node attribute, node value, node name, etc.
[0086] Furthermore, the security detection device inputs the second vector into the language model, performs context semantic analysis on the second vector through the language model, and obtains the first vector. That is, the first vector substantially includes the node semantic information of the node to be tested and the context semantic information of the node to be tested in the source code file.
[0087] In a possible implementation, in order to improve the accuracy of the semantic analysis results of the word embedding model and the language model on the node to be tested, the pre-trained word embedding model and the pre-trained language model can be retrained through a sample data set, and the sample data set includes various types of risk features, and specifically can be a source code file set including various types of risk features. During the retraining process, the sample data set is input into the pre-trained word embedding model and the pre-trained language model, and the parameters of the pre-trained word embedding model and the pre-trained language model are fine-tuned based on the output results thereof, so that the re-trained word embedding model and the re-trained language model can have higher accuracy in the semantic analysis of the node to be tested.
[0088] S204, performing a security check on the first vector using a pre-trained security check model, and outputting a security check result indicating whether there is a security risk in the node to be checked.
[0089] The security detection model can be constructed by a convolutional neural network (CNN) layer, a bidirectional long short-term memory network (LSTM) layer, and a fully connected layer. The CNN layer is constructed by a convolutional layer and a pooling layer. The convolutional layer is used to automatically learn risk features, such as the features of software vulnerabilities. The output calculation of the CNN layer can be as follows:
[0090] p t =tanh(v t *w t +bias t )
[0091] where p t is the output value of the convolutional layer, tanh is the activation function, v t is the input sequence, w t is the weight of the convolution kernel, bias t is the bias value. The maximum pooling is selected by the pooling layer. The output vector after convolution pooling is as follows:
[0092] X t =F maxpooling (p t )
[0093] Among them, X t is the output of the CNN layer, Xt It is the input of the bidirectional LSTM layer.
[0094] The bidirectional LSTM layer is composed of two LSTM networks. In the bidirectional LSTM, there are three main gates: input gate, forget gate, and output gate. The input gate manages the flow of information to the memory cells, the forget gate manages the flow of information out of the memory cells, and the output gate manages the flow of information out of the LSTM and to the output. The input data will be passed through the sigmoid activation function.
[0095] Among them, the output calculation formula of the input gate is as follows:
[0096] Input t =σ(U i *s t-1 +Weight i *X t +bias i )
[0097] Among them, Input t represents the input gate, σ is the sigmoid function, U i and Weight i is the weight matrix, bias i is bias.
[0098] The calculation formula of the forget gate is as follows:
[0099] Foget t =σ(U f *s t-1 +Weight f *X t +bias f )
[0100] Among them, Foget t represents the forget gate, σ is the sigmoid function, U f and Weight f is the weight matrix, bias f is bias.
[0101] The output calculation formula of the output gate is as follows:
[0102] Ouput t =σ(U o *s t-1 +Weight o *X t +bias o )
[0103] Among them, Ouput trepresents the output gate, σ is the sigmoid function, U o and Weight o is the weight matrix, bias o is bias.
[0104] The basic structure of the LSTM unit includes logic, activation function tanh, and the new input vector I t , the output s of the previous time step t-1 , current output s t , old cell state Ce t-1 and the new cell state Ce t .
[0105] The fully connected layer includes a dense layer and a classification function. The dense layer is a fully connected neural network layer used to reduce the vector dimension. The classification function is, for example, a Sigmoid function.
[0106] Each network layer in the security detection model can be pre-trained based on a sample data set, which refers to a set of source code files including various types of risk features. The training process is consistent with the training process of other existing models, and the embodiments of this application will not be described in detail.
[0107] Combine the following Figure 3 The structural diagram of a security detection model shown in FIG. 1 is a detailed description of a specific method of performing a security detection on a first vector through a security detection model and outputting a security detection result indicating whether a node to be detected has a security risk. Figure 3 The security detection model shown includes a convolutional neural network layer, a bidirectional long short-term memory network layer, and a fully connected layer, wherein the convolutional neural network layer includes a convolution layer and a pooling layer, and the fully connected layer includes a dense layer and a classification function.
[0108] The safety detection device inputs the first vector into the safety detection model, and the convolution layer in the safety detection model extracts the risk feature information in the first vector based on the convolution operation, and outputs the risk feature information, which refers to the feature information required for safety detection. Since the dimension of the risk feature information output by the convolution layer is relatively high, the risk feature information can be used as the input of the pooling layer, and the important risk features in the risk feature information are retained based on the pooling layer to reduce the dimension of the risk feature information, and output a third vector.
[0109] Since the source code entry is actually text data, the convolution layer in the embodiment of the present application is a one-dimensional convolution layer, which is suitable for processing sequence data. In addition, in the specific implementation, the number and length of filters in the convolution layer can be set according to actual needs. The number of filters determines the number of risk feature types that the CNN layer can capture, and the length of the filter affects the ability of the CNN layer to identify local information in the first vector. Exemplarily, in the embodiment of the present application, the number of filters can be set to 15 and the length of the filter can be set to 20.
[0110] The third vector is used as the input of the bidirectional LSTM layer, and the temporal feature information in the third vector is extracted based on the bidirectional LSTM layer to output the fourth vector. Based on the ability of bidirectional LSTM to detect long-term dependencies, the bidirectional LSTM can simultaneously consider the contextual information before and after the current moment, thereby more effectively capturing the key risk features in the third vector.
[0111] The fourth vector is used as the input of the fully connected layer. Based on the dense layer in the fully connected layer, it can reduce the feature dimension of the risk feature information from the bidirectional LSTM, that is, the feature dimension of the fourth vector, and output the fifth vector. And combined with the classification function, the fifth vector is operated to determine the risk value of the node to be tested. Specifically, after the dense layer integrates the features of the fourth vector, the integrated risk feature information is reduced in dimension, and the low-dimensional fifth vector is output. Then, the fifth vector is operated using the weighted sum calculation method to obtain a data value. Combined with the classification function in the fully connected layer, the data value is converted into a probability value, which is the risk value of the node to be tested. The risk value can be used to indicate the probability that the node to be tested has a security risk. Finally, it can be determined based on the fully connected layer whether the risk value is greater than the preset risk threshold. If so, the security detection result that the node to be tested has a security risk is output. If not, the security detection result that the node to be tested does not have a security risk is output. Among them, the risk threshold can be in a pre-configured security detection model, and the risk threshold can be set according to actual needs. The embodiment of the present application does not limit this. The risk value is usually between 0 and 1.
[0112] For example, taking vulnerability detection as an example, if the risk value output by the classification function is 0.3, the risk threshold is 0.5, and 0.3<0.5, then it is determined that the source code corresponding to the node to be tested is not vulnerable to attack. If the risk value output by the classification function is 0.7, then 0.7>0.5, then it is determined that the source code corresponding to the node to be tested is vulnerable to attack.
[0113] In the fully connected layer, the vector dimension of the fourth vector is reduced through a dense layer, which is conducive to extracting important risk feature information in the fourth vector and also helps to reduce computational complexity.
[0114] In practical applications, the semantic analysis model and the security detection model can be integrated into a hybrid model to implement the above functions, or can be implemented separately, which is not limited in the embodiments of the present application.
[0115] Based on the same inventive concept, an embodiment of the present application provides a safety detection device, which is used to implement any of the above safety detection methods, for example Figure 2 The safety detection method shown, and the device can also realize the functions of the safety detection device mentioned above.
[0116] Please refer to Figure 4 , is a schematic diagram of the structure of a safety detection device provided in an embodiment of the present application. Figure 4 As shown, the security detection device 400 includes a determination module 401 , a conversion module 402 , a semantic analysis module 403 and a detection module 404 .
[0117] Exemplarily, a determination module 401 is used to determine the node to be tested based on the source code file to be tested and a first condition, and to generate a text tag for the node to be tested, wherein the first condition is used to screen nodes in the source code file that are at risk of being attacked, and each tag in the text tag corresponds to a type of node information of the node to be tested; a conversion module 402 is used to convert the text tag into an integer sequence based on a correspondence between a preset tag and an integer identifier; a semantic analysis module 403 is used to perform semantic analysis on the integer sequence through a semantic analysis model to obtain a first vector, and the first vector is used to indicate the semantic information of the node to be tested in the source code file; a detection module 404 is used to perform security detection on the first vector through a pre-trained security detection model, and output a security detection result indicating whether the node to be tested has a security risk.
[0118] In one possible implementation, the semantic analysis model includes a language model and a word embedding model; the semantic analysis module 403 is specifically used to: input the integer sequence into the word embedding model, perform lexical semantic analysis on the integer sequence through the word embedding model, and obtain a second vector, where the second vector is used to indicate the node semantic information of the node to be tested; input the second vector into the language model, perform contextual semantic analysis on the second vector through the language model, and obtain a first vector.
[0119] In a possible implementation, the word embedding model is obtained by retraining a pre-trained word embedding model using a sample data set, and the language model is obtained by retraining a pre-trained language model using a sample data set, wherein the sample data set includes various types of risk features.
[0120] In one possible implementation, the detection module 404 is specifically used to: extract risk feature information in the first vector based on the convolution layer of the security detection model; reduce the dimension of the risk feature information based on the pooling layer of the security detection model, and output a third vector; extract time series feature information in the third vector based on the bidirectional long short-term memory network layer in the security detection model, and output a fourth vector; reduce the feature dimension of the fourth vector based on the dense layer in the fully connected layer in the security detection model, and output a fifth vector, and perform operations on the fifth vector in combination with the classification function in the fully connected layer to determine the risk value of the node to be tested; based on the fully connected layer, determine whether the risk value is greater than a preset risk threshold, and if so, output a security detection result indicating that the node to be tested has a security risk, and if not, output a security detection result indicating that the node to be tested does not have a security risk.
[0121] In a possible implementation, the determination module 401 is specifically used to: parse the source code file into a first abstract syntax tree, and traverse to obtain irrelevant nodes in the first abstract syntax tree, where the irrelevant nodes are risk-free nodes predefined in the source code file; delete the irrelevant nodes in the first abstract syntax tree, and reconstruct the first abstract syntax tree to obtain a second abstract syntax tree; serialize the second abstract syntax tree, and determine the nodes to be tested that meet the first condition from the serialized second abstract syntax tree.
[0122] In a possible implementation, the determination module 401 is specifically used to: extract key node information of the node to be tested, the key node information including node type, node attribute, node value, and tree structure information of the node; convert the key node information into a preset format to generate a text tag.
[0123] Based on the same inventive concept, an embodiment of the present application provides a safety detection device, which is used to implement any of the above safety detection methods, for example Figure 2 The safety detection method shown, and the device can also realize the functions of the safety detection device mentioned above.
[0124] Please refer to Figure 5 , is a schematic diagram of the structure of a safety detection device provided in an embodiment of the present application. Figure 5 As shown, the safety detection device 500 includes at least one processor 501 and a memory 502 communicatively connected to the at least one processor 501 .
[0125] The processor 501 may be a general purpose processor or a dedicated processor. The processor 501 may include, for example, a baseband processor or a central processing unit. The baseband processor may be used to process communication protocols and communication data. The central processing unit may be used to control the security detection device 500, execute software programs and / or process data. Different processors may be independent devices or may be arranged in one or more processing circuits, for example, integrated in one or more dedicated integrated circuits.
[0126] In one embodiment, the memory stores instructions that can be executed by at least one processor. The at least one processor implements the functions of the security detection device by executing the instructions stored in the memory, and accordingly, can also implement the steps performed by the security detection device mentioned above.
[0127] In this embodiment, the security detection device 500 can also implement the functions of the previous security detection device 400, and at least one processor 501 in the security detection device 500 can also implement the functions of the previous text determination module 401, the conversion module 402, the semantic analysis module 403 and the detection module 404.
[0128] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed on a computer, the computer executes any of the above-mentioned security detection methods, for example, Figure 2 The safety detection method shown.
[0129] Based on the same inventive concept, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed on a computer, enables any of the above security detection methods, for example, Figure 2 The safety detection method shown is implemented.
[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0132] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0134] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A safety detection method, characterized in that: include: Determine the node to be tested according to the source code file to be tested and the first condition, and generate a text tag of the node to be tested, wherein the first condition is used to screen nodes in the source code file that are at risk of being attacked, and each tag in the text tag corresponds to a type of node information of the node to be tested; According to the correspondence between the preset tags and the integer identifiers, the text tags are converted into integer sequences; Performing semantic analysis on the integer sequence through a semantic analysis model to obtain a first vector, where the first vector is used to indicate semantic information of the node to be tested in the source code file; The first vector is subjected to security detection by using a pre-trained security detection model, and a security detection result indicating whether the node to be detected has a security risk is output.
2. The method according to claim 1, characterized in that The semantic analysis model includes a language model and a word embedding model; performing semantic analysis on the integer sequence through the semantic analysis model to obtain a first vector includes: Inputting the integer sequence into the word embedding model, performing lexical semantic analysis on the integer sequence through the word embedding model, and obtaining a second vector, wherein the second vector is used to indicate node semantic information of the node to be tested; The second vector is input into the language model, and contextual semantic analysis is performed on the second vector through the language model to obtain the first vector.
3. The method according to claim 2, characterized in that The word embedding model is obtained by retraining a pre-trained word embedding model using a sample data set, and the language model is obtained by retraining a pre-trained language model using the sample data set, wherein the sample data set includes various types of risk features.
4. The method according to claim 1, characterized in that: The performing security detection on the first vector by using a pre-trained security detection model and outputting a security detection result includes: Extracting risk feature information in the first vector based on the convolutional layer of the security detection model; Based on the pooling layer of the security detection model, reduce the dimension of the risk feature information and output a third vector; Based on the bidirectional long short-term memory network layer in the security detection model, extract the time series feature information in the third vector and output a fourth vector; Based on the dense layer in the fully connected layer in the security detection model, reduce the feature dimension of the fourth vector, output a fifth vector, and combine the classification function in the fully connected layer to operate the fifth vector to determine the risk value of the node to be tested; Based on the fully connected layer, determine whether the risk value is greater than a preset risk threshold. If so, output a security detection result indicating that the node to be tested has a security risk. If not, output a security detection result indicating that the node to be tested does not have a security risk.
5. The method according to any one of claims 1 to 4, characterized in that: The step of determining the node to be tested according to the source code file to be tested and the first condition includes: Parsing the source code file into a first abstract syntax tree, and traversing to obtain irrelevant nodes in the first abstract syntax tree, wherein the irrelevant nodes are risk-free nodes predefined in the source code file; Deleting irrelevant nodes in the first abstract syntax tree, and reconstructing the first abstract syntax tree to obtain a second abstract syntax tree; The second abstract syntax tree is serialized, and a node to be tested that satisfies the first condition is determined from the serialized second abstract syntax tree.
6. The method according to any one of claims 1 to 4, characterized in that: The step of generating a text label of the node to be tested comprises: Extract key node information of the node to be tested, wherein the key node information includes node type, node attribute, node value, and tree structure information of the node; The key node information is converted into a preset format to generate the text mark.
7. A safety detection device, characterized in that: include: A determination module, used to determine the node to be tested according to the source code file to be tested and a first condition, and generate a text tag of the node to be tested, wherein the first condition is used to screen nodes in the source code file that are at risk of being attacked, and each tag in the text tag corresponds to a type of node information of the node to be tested; A conversion module, used for converting the text mark into an integer sequence according to a preset correspondence between the mark and the integer identifier; A semantic analysis module, used for performing semantic analysis on the integer sequence through a semantic analysis model to obtain a first vector, wherein the first vector is used for indicating semantic information of the node to be tested in the source code file; The detection module is used to perform a security detection on the first vector through a pre-trained security detection model, and output a security detection result indicating whether the node to be tested has a security risk.
8. A safety detection device, characterized in that: include: at least one processor, and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the method according to any one of claims 1 to 6 by executing the instructions stored in the memory.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.
10. A computer program product, characterized in that The invention comprises computer instructions, which, when executed on a computer, enable the method according to any one of claims 1 to 6 to be implemented.
Citation Information
Cited By
Programming review method and system based on multi-modal data
CN120804726A
