Java deserialization vulnerability detection system and method based on reinforcement learning
Through a detection system based on reinforcement learning, the serialization-aware code attribute graph and fuzzy decision-making model are used to solve the accuracy and scalability problems of JAVA deserialization vulnerability detection in the existing technology, and achieve efficient and accurate vulnerability detection.
Patent Information
- Application Number
- CN202111629096.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-28
- Publication Date
- 2025-01-21
- Estimated Expiration
- 2041-12-28
AI Technical Summary
In the detection of JAVA deserialization vulnerabilities, the existing technology has problems such as incomplete dynamic characteristics considerations, incomplete call chain construction and large dynamic analysis overhead, which leads to difficulty in ensuring accuracy and poor scalability.
A detection system based on reinforcement learning is adopted to achieve efficient detection of JAVA deserialization vulnerabilities through vulnerability data collection, serialization-aware code attribute graph construction, fuzzy policy learning and fuzzy decision-making models.
It improves the accuracy and scalability of detection, can more accurately identify potential vulnerability call chains, and verifies the exploitability of the call chain through fuzz testing, reducing the cost of manual verification.
Smart Images
Figure CN114462043B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of software security, and in particular relates to a Java deserialization vulnerability detection system and method based on reinforcement learning. Background Art
[0002] As a mechanism for dynamically capturing program states, JAVA deserialization often leads to a series of malicious vulnerabilities such as remote code execution, expression injection, and system resource access, and is listed as one of the OWASP TOP 10 most serious vulnerabilities in 2018. JAVA deserialization vulnerabilities are widely present in many large open source projects, threatening software security. How to accurately and efficiently detect potential JAVA deserialization vulnerability call chains has become a challenging task. Existing JAVA deserialization vulnerability mining work mostly uses static detection methods to model JAVA programs, analyze potential dangerous call sites in programs and the controllability of variables between calling methods, and mine potential vulnerability call chains through graph traversal algorithms and other methods. However, due to the dynamic characteristics of the JAVA language, such as reflection, polymorphism, dynamic class loading, etc., the call graphs constructed by most static analysis methods are mostly unreliable, resulting in many false positives and false negatives in the analysis of the call chain itself. In addition, most manual inspection methods are currently used to verify the mined call chains, which has a large manual overhead.
[0003] At present, some works have used a hybrid analysis method combining dynamic and static analysis to detect JAVA deserialization vulnerabilities. For example, the document "Du Xiaoyu, Ye He, Wen Weiping. Java deserialization vulnerability call chain mining method based on bytecode search [J]. Information Network Security, 2020, 20(07): 19-29." uses taint analysis and symbolic execution technology to generate a call tree from the deserialization vulnerability entry point to the dangerous function, and uses a depth-first search algorithm to search the call tree and generate related call chains. Some work has also begun to use fuzz testing methods to automatically verify the accuracy of call chain mining, such as the paper "Shawn Rasheed, and Jens Dietrich. 2020. A Hybrid Analysis to Detect Java Serialisation Vulnerabilities. In 35th IEEE / ACM International Conference on Automated Software Engineering (ASE'20). IEEE, Melbourne, Australia, 2020.", which uses static analysis to detect the call chain connecting the deserialization call site and the dangerous call, and at the same time finds the data flow from the deserialized object to the dangerous call site through the heap access path, and constructs the actual input through fuzz testing to verify the availability of the call chain. However, the above methods still have problems such as incomplete consideration of JAVA dynamic characteristics, incomplete call chain construction, and high dynamic analysis overhead, which makes it difficult to guarantee the accuracy of the method and poor scalability. Summary of the invention
[0004] Purpose of the invention: In view of the problems existing in the above-mentioned prior art, the present invention provides a Java deserialization vulnerability detection system and method based on reinforcement learning, which has the characteristics of wider application fields and higher accuracy.
[0005] Technical solution: To achieve the above-mentioned invention object, the present invention specifically adopts the following technical solution:
[0006] The invention provides a Java deserialization vulnerability detection system based on reinforcement learning, comprising: a vulnerability data collection module, used for collecting vulnerability data to obtain a vulnerability data set; a vulnerability call chain mining module, used for mining potential vulnerability call chains in the vulnerability data set by constructing a serialization-aware code property graph SCPG, and obtaining a vulnerability call chain set; a fuzzy strategy learning module, used for generating fuzzy seeds from the vulnerability call chain set, using reinforcement learning-guided fuzzy testing to perform fuzzy testing on the scheduling process of the fuzzy seeds and constructing a fuzzy decision model according to the test results; a vulnerability detection module, used for performing static analysis on a JAVA file to be detected and constructing a serialization-aware code property graph SCPG to mine potential vulnerability call chains in the file, and obtaining a file vulnerability call chain set, and using the fuzzy decision model to detect the file vulnerability call chain set, and outputting a deserialization vulnerability call chain with an exploitation risk.
[0007] Furthermore, in the vulnerability data collection module,
[0008] Collect vulnerability-related information from JAVA benchmark vulnerability libraries and open source vulnerability libraries with deserialization vulnerabilities. The vulnerability-related information includes: vulnerable project code, vulnerability call chain, and vulnerability exploitation POC; use the JAVA static analysis framework Soot to compile the vulnerable project code and convert it into a general code intermediate representation IR.
[0009] Furthermore, in the vulnerability call chain mining module,
[0010] According to the code intermediate representation IR, a serialization-aware code property graph SCPG is constructed through pointer analysis. The property graph includes: class relationship graph, method alias graph and method call graph; dangerous call sites are extracted according to the vulnerability call chain, and a dangerous call library is constructed. It is checked whether there are dangerous call sites in the dangerous call library in the constructed serialization-aware code property graph SCPG; class methods associated with dangerous call sites are analyzed according to data flow to determine whether the parameters of the class methods are controllable. If the parameters of the class methods are irrelevant to the return value, it indicates that the return result is uncontrollable and the vulnerability call chain cannot be constructed; if the parameters of the class methods are related to the return value, it indicates that the return result is controllable, and the class methods are included in the set of exploitable transit points; with the magic method in the deserialization process or the serializable JDK class that implements the magic method and automatically calls other methods as the entry point and the deserialized dangerous call site as the end point, all vulnerability call chains consisting of "entry point->exploitable transit point->dangerous call site" are searched in the serialization-aware code property graph SCPG using a depth-first algorithm, and a set of vulnerability call chains is obtained.
[0011] Furthermore, in the fuzzy strategy learning module,
[0012] Randomly select an entry point from the set of vulnerability call chains, and generate an injection object based on the class attributes of the entry point. The injection object is a fuzzy seed. Scan the attributes of all class methods that have a call relationship with the entry point as a candidate attribute set to guide the generation and mutation of fuzzy seeds in fuzzy testing; use an incremental clustering algorithm to cluster the fuzzy seeds in the candidate fuzzy seed pool and construct a multi-level seed tree. Each node in the i-th layer of the tree represents a cluster represented by a coverage metric, and the child node in the i+1th layer represents a sub-cluster. The leaf nodes of the seed tree are associated with the fuzzy seeds in the candidate fuzzy seed pool.
[0013] For a fuzzy seed s used for fuzz testing in round t, calculate its coverage metric C l The fuzzy reward SeedReward(s,l,t):
[0014]
[0015] Where fcov[s,l,t] represents the coverage metric C on the multi-level seed tree l The feature coverage of l is the number of layers of the seed tree. is the scarcity of program feature F, num_hits[F] represents the number of hits of feature F;
[0016] Propagate the fuzzy reward to the seed cluster scheduled by the upper layer to calculate the reward of the current seed cluster:
[0017]
[0018] In the formula, a l Represents the seed cluster formed by all n nodes in the lth layer of the seed tree;
[0019] Evaluate the fuzzy performance of node a FuzzPerf(a):
[0020] FuzzPerf(a)=Q(a,t)+U(a)
[0021] Where Q(a,t) is the average reward of node a after t rounds of fuzzy testing, which is calculated by the following formula:
[0022]
[0023] Where N(a,t) represents the number of times node a is selected from the end of the tth round of testing to the cumulative number of times node a is selected, t′ is the last round in which node a is selected, w is the discount factor, and p is the cumulative operation variable;
[0024] U(a) is the estimated radius factor for the number of times node a is selected, and is calculated using the following formula:
[0025]
[0026] Where a′ is the parent node of node a, Y[a] represents the number of fuzzy seeds in the cluster of node a, N[a] represents the cumulative number of times node a is selected, and C is a predefined parameter used to evaluate whether this round of fuzz testing attempts to select fuzzy seeds that have never been used from the fuzzy seed pool or high-quality fuzzy seeds that have achieved higher coverage or call chain depth in previous fuzz tests;
[0027] Aggregate the scarcity of features covered by the fuzzy seeds to calculate the rarity of the fuzzy seeds (a l ):
[0028]
[0029] Where P is the JAVA program to be tested, rareness[F] is the scarcity of program feature F, and C l is the coverage metric;
[0030] By obtaining the rarity Rareness (a l ) and fuzzy performance FuzzPerf(a) calculate the priority score Score(a) of the fuzzy seeds in the fuzzy seed pool, and select the fuzzy seed with the highest priority score for this round of fuzz testing:
[0031] Score(a)=Rareness(a)×FuzzPerf(a)
[0032] Perform fuzz testing on the vulnerable project code to determine whether the high-priority fuzzy seed input selected in this round of fuzz testing can reach the dangerous call site. If it can be reached, terminate the fuzzy test and build a fuzzy decision model. If it cannot be reached, further evaluate the quality of the fuzzy seed input in this round of fuzz testing. If this round of fuzz testing improves the branch coverage or triggers more available transit points in the call chain traversal, add the fuzzy seed to the fuzzy seed pool, otherwise, discard the fuzzy seed.
[0033] Furthermore, in the vulnerability detection module,
[0034] The JAVA file to be tested is compiled using the Soot static analysis framework, and a serialization-aware code property graph SCPG is constructed. According to the serialization-aware code property graph SCPG and the existing dangerous call library, it is checked whether the JAVA file to be tested has a class method with controllable variables and associated with the dangerous call site. If so, the associated method is included in the set of exploitable transit points. If not, vulnerability call chain mining is no longer performed. In the serialization-aware code property graph SCPG, a depth-first algorithm is used to search for all vulnerability call chains consisting of "entry point->exploitable transit point->dangerous call site" to obtain a set of file vulnerability call chains. Using a fuzzy decision model, starting from the entry point, an injection object is regenerated, and the injection object is a new fuzzy seed. Then, the file vulnerability call chain set is fuzz tested to output a deserialized vulnerability call chain with exploitation risk.
[0035] In addition, the present invention provides a Java deserialization vulnerability detection method based on reinforcement learning, including: step 1, collecting vulnerability data to obtain a vulnerability data set; step 2, mining potential vulnerability call chains in the vulnerability data set by constructing a serialization-aware code property graph SCPG to obtain a vulnerability call chain set; step 3, generating fuzzy seeds from the vulnerability call chain set, using reinforcement learning-guided fuzzy testing to perform fuzzy testing on the scheduling process of the fuzzy seeds and constructing a fuzzy decision model according to the test results; step 4, performing static analysis on the JAVA file to be detected and constructing a serialization-aware code property graph SCPG to mine the potential vulnerability call chains in the file to obtain a file vulnerability call chain set, and using the fuzzy decision model to detect the file vulnerability call chain set, and outputting a deserialization vulnerability call chain with an exploitation risk.
[0036] Furthermore, in step 1,
[0037] Collect vulnerability-related information from JAVA benchmark vulnerability libraries and open source vulnerability libraries with deserialization vulnerabilities. The vulnerability-related information includes: vulnerable project code, vulnerability call chain, and vulnerability exploitation POC; use the JAVA static analysis framework Soot to compile the vulnerable project code and convert it into a general code intermediate representation IR.
[0038] Furthermore, in step 2,
[0039] According to the code intermediate representation IR, a serialization-aware code property graph SCPG is constructed through pointer analysis. The property graph includes: class relationship graph, method alias graph and method call graph; dangerous call sites are extracted according to the vulnerability call chain, and a dangerous call library is constructed. It is checked whether there are dangerous call sites in the dangerous call library in the constructed serialization-aware code property graph SCPG; class methods associated with dangerous call sites are analyzed according to data flow to determine whether the parameters of the class methods are controllable. If the parameters of the class methods are irrelevant to the return value, it indicates that the return result is uncontrollable and the vulnerability call chain cannot be constructed; if the parameters of the class methods are related to the return value, it indicates that the return result is controllable, and the class methods are included in the set of exploitable transit points; with the magic method in the deserialization process or the serializable JDK class that implements the magic method and automatically calls other methods as the entry point and the deserialized dangerous call site as the end point, all vulnerability call chains consisting of "entry point->exploitable transit point->dangerous call site" are searched in the serialization-aware code property graph SCPG using a depth-first algorithm, and a set of vulnerability call chains is obtained.
[0040] Furthermore, in step 3,
[0041] Randomly select an entry point from the set of vulnerability call chains, and generate an injection object based on the class attributes of the entry point. The injection object is a fuzzy seed. Scan the attributes of all class methods that have a call relationship with the entry point as a candidate attribute set to guide the generation and mutation of fuzzy seeds in fuzzy testing; use an incremental clustering algorithm to cluster the fuzzy seeds in the candidate fuzzy seed pool and construct a multi-level seed tree. Each node in the i-th layer of the tree represents a cluster represented by a coverage metric, and the child node in the i+1th layer represents a sub-cluster. The leaf nodes of the seed tree are associated with the fuzzy seeds in the candidate fuzzy seed pool.
[0042] For a fuzzy seed s used for fuzz testing in round t, calculate its coverage metric C l The fuzzy reward SeedReward(s,l,t):
[0043]
[0044] Where fcov[s,l,t] represents the coverage metric C on the multi-level seed tree l The feature coverage of l is the number of layers of the seed tree. is the scarcity of program feature F, num_hits[F] represents the number of hits of feature F;
[0045] Propagate the fuzzy reward to the seed cluster scheduled by the upper layer to calculate the reward of the current seed cluster:
[0046]
[0047] In the formula, a l Represents the seed cluster formed by all n nodes in the lth layer of the seed tree;
[0048] 1 ,…,a n ,a n+1 > represents the node sequence in the seed tree selected in round t, a n+1 is the seed node of fuzzy seed s; evaluate the fuzzy performance of node a FuzzPerf(a):
[0049] FuzzPerf(a)=Q(a,t)+U(a)
[0050] Where Q(a,t) is the average reward of node a after t rounds of fuzzy testing, which is calculated by the following formula:
[0051]
[0052] Where N(a,t) represents the number of times node a is selected from the end of the tth round of testing to the cumulative number of times node a is selected, t′ is the last round in which node a is selected, w is the discount factor, and p is the cumulative operation variable;
[0053] U(a) is the estimated radius factor for the number of times node a is selected, and is calculated using the following formula:
[0054]
[0055] Where a′ is the parent node of node a, Y[a] represents the number of fuzzy seeds in the cluster of node a, N[a] represents the cumulative number of times node a is selected, and C is a predefined parameter used to evaluate whether this round of fuzz testing attempts to select fuzzy seeds that have never been used from the fuzzy seed pool or high-quality fuzzy seeds that have achieved higher coverage or call chain depth in previous fuzz tests;
[0056] Aggregate the scarcity of features covered by the fuzzy seeds to calculate the rarity of the fuzzy seeds (a l ):
[0057]
[0058] Where P is the JAVA program to be tested, rareness[F] is the scarcity of program feature F, and C l is the coverage metric;
[0059] By obtaining the rarity Rareness (a l ) and fuzzy performance FuzzPerf(a) calculate the priority score Score(a) of the fuzzy seeds in the fuzzy seed pool, and select the fuzzy seed with the highest priority score for this round of fuzz testing:
[0060] Score(a)=Rareness(a)×FuzzPerf(a)
[0061] Perform fuzz testing on the vulnerable project code to determine whether the high-priority fuzzy seed input selected in this round of fuzz testing can reach the dangerous call site. If it can be reached, terminate the fuzzy test and build a fuzzy decision model. If it cannot be reached, further evaluate the quality of the fuzzy seed input in this round of fuzz testing. If this round of fuzz testing improves the branch coverage or triggers more available transit points in the call chain traversal, add the fuzzy seed to the fuzzy seed pool, otherwise, discard the fuzzy seed.
[0062] Furthermore, in step 4,
[0063] The JAVA file to be tested is compiled using the Soot static analysis framework, and a serialization-aware code property graph SCPG is constructed. According to the serialization-aware code property graph SCPG and the existing dangerous call library, it is checked whether the JAVA file to be tested has a class method with controllable variables and associated with the dangerous call site. If so, the associated method is included in the set of exploitable transit points. If not, vulnerability call chain mining is no longer performed. In the serialization-aware code property graph SCPG, a depth-first algorithm is used to search for all vulnerability call chains consisting of "entry point->exploitable transit point->dangerous call site" to obtain a set of file vulnerability call chains. Using a fuzzy decision model, starting from the entry point, an injection object is regenerated, and the injection object is a new fuzzy seed. Then, the file vulnerability call chain set is fuzz tested to output a deserialized vulnerability call chain with exploitation risk.
[0064] Beneficial effects:
[0065] (1) Model the JAVA deserialization vulnerability call chain from a graph perspective. By constructing a serialization-aware code property graph SCPG (including class relationship graph, method alias graph, and method call graph), we can fully explore the class properties and call relationships of deserialization class methods that are closely related to deserialization vulnerabilities.
[0066] (2) A reinforcement learning-based fuzz testing method is proposed to verify potential vulnerability call chains. Compared with directly using fuzz testing to verify the true reachability of the call chain, the present invention guides the seed generation and mutation of each fuzz test by training a decision model on the vulnerability items of the known call chain, thereby optimizing the fuzz testing process.
[0067] (3) Different from the traditional automatic JAVA deserialization vulnerability mining method, this method can automatically generate injection objects through fuzz testing, which makes it easier for developers to understand and reproduce the vulnerabilities. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] Figure 1 It is a flow chart of the Java deserialization vulnerability detection method based on reinforcement learning of the present invention.
[0069] Figure 2 It is the serialization-aware code property graph SCPG generated by the XStream deserialization chain CVE-2021-21346 in the present invention.
[0070] Figure 3 It is a framework flow chart of the fuzzy test used in the present invention. DETAILED DESCRIPTION
[0071] The specific implementation of the present invention is described below with reference to the accompanying drawings.
[0072] Embodiment 1
[0073] like Figure 1 As shown, this embodiment discloses a Java deserialization vulnerability detection system based on reinforcement learning, including: a vulnerability data collection module, used to collect vulnerability data to obtain a vulnerability data set; a vulnerability call chain mining module, used to mine potential vulnerability call chains in the vulnerability data set by constructing a serialization-aware code property graph SCPG to obtain a vulnerability call chain set; a fuzzy strategy learning module, used to generate fuzzy seeds from the vulnerability call chain set, use reinforcement learning-guided fuzzy testing to perform fuzzy testing on the scheduling process of the fuzzy seeds and build a fuzzy decision model according to the test results; a vulnerability detection module, used to perform static analysis on the JAVA file to be detected and construct a serialization-aware code property graph SCPG to mine the potential vulnerability call chain in the file, obtain a file vulnerability call chain set, and use the fuzzy decision model to detect the file vulnerability call chain set, and output a deserialization vulnerability call chain with an exploit risk.
[0074] Furthermore, in the vulnerability data collection module,
[0075] Collect vulnerability-related information from JAVA benchmark vulnerability libraries and open source vulnerability libraries with deserialization vulnerabilities. The vulnerability-related information includes: vulnerable project code, vulnerability call chain, and vulnerability exploitation POC; use the JAVA static analysis framework Soot to compile the vulnerable project code and convert it into a general code intermediate representation IR.
[0076] Furthermore, in the vulnerability call chain mining module,
[0077] According to the code intermediate representation IR, the serialization-aware code property graph SCPG is constructed through pointer analysis, such as Figure 2 As shown in FIG. 1 , the property graph includes: a class relationship graph, a method alias graph, and a method call graph; extract dangerous call sites (such as Runtime.exec(), Method.invoke(), etc., which can be used to execute malicious code) according to the vulnerability call chain, and build a dangerous call library, and check whether there are dangerous call sites in the dangerous call library in the constructed serialization-aware code property graph SCPG; analyze the class methods associated with the dangerous call sites according to the data flow, and judge whether the parameters of the class methods are controllable. If the parameters of the class methods are not related to the return value, it means that the return result is uncontrollable and the vulnerability call chain cannot be constructed; if the parameters of the class methods are related to the return value, then the table The return result is controllable, and the method is included in the set of exploitable transit points. The magic method in the deserialization process (such as Object.readObject, Object.finalize, etc.) or the serializable JDK class that implements the magic method and automatically calls other methods (such as Object.hashCode, Object.equals, etc.) is used as the entry point, and the deserialization dangerous call site is used as the end point. The depth-first algorithm is used to search for all vulnerable call chains consisting of "entry point->exploitable transit point->dangerous call site" in the serialization-aware code property graph SCPG, and a set of vulnerable call chains is obtained.
[0078] Further, such as Figure 3 As shown, in the fuzzy strategy learning module,
[0079] Randomly select an entry point from the set of vulnerability call chains, and generate an injection object based on the class attributes of the entry point. The injection object is a fuzzy seed. Scan the attributes of all class methods that have a call relationship with the entry point as a candidate attribute set to guide the generation and mutation of fuzzy seeds in fuzzy testing; use an incremental clustering algorithm to cluster the fuzzy seeds in the candidate fuzzy seed pool and construct a multi-level seed tree. Each node in the i-th layer of the tree represents a cluster represented by a coverage metric, and the child node in the i+1th layer represents a sub-cluster. The leaf nodes of the seed tree are associated with the fuzzy seeds in the candidate fuzzy seed pool.
[0080] For a fuzzy seed s used for fuzz testing in round t, calculate its coverage metric C l The fuzzy reward SeedReward(s,l,t):
[0081]
[0082] Where fcov[s,l,t] represents the coverage metric C on the multi-level seed tree lThe feature coverage of l is the number of layers of the seed tree. is the scarcity of program feature F, num_hits[F] represents the number of hits of feature F;
[0083] Propagate the fuzzy reward to the seed cluster scheduled by the upper layer to calculate the reward of the current seed cluster:
[0084]
[0085] In the formula, a l Represents the seed cluster formed by all n nodes in the lth layer of the seed tree;
[0086] 1 ,…,a n ,a n+1 > represents the node sequence in the seed tree selected in round t, a n+1 is the seed node of fuzzy seed s; evaluate the fuzzy performance of node a FuzzPerf(a):
[0087] FuzzPerf(a)=Q(a,t)+U(a)
[0088] Where Q(a,t) is the average reward of node a after t rounds of fuzzy testing, which is calculated by the following formula:
[0089]
[0090] Where N(a,t) represents the number of times node a is selected from the end of the tth round of testing to the cumulative number of times node a is selected, t′ is the last round in which node a is selected, w is the discount factor, and p is the cumulative operation variable;
[0091] U(a) is the estimated radius factor for the number of times node a is selected, and is calculated using the following formula:
[0092]
[0093] Where a′ is the parent node of node a, Y[a] represents the number of fuzzy seeds in the cluster of node a, N[a] represents the cumulative number of times node a is selected, and C is a predefined parameter used to evaluate whether this round of fuzz testing attempts to select fuzzy seeds that have never been used from the fuzzy seed pool or high-quality fuzzy seeds that have achieved higher coverage or call chain depth in previous fuzz tests;
[0094] Aggregate the scarcity of features covered by the fuzzy seeds to calculate the rarity of the fuzzy seeds (a l ):
[0095]
[0096] Where P is the JAVA program to be tested, rareness[F] is the scarcity of program feature F, and C l is the coverage metric;
[0097] By obtaining the rarity Rareness (a l ) and fuzzy performance FuzzPerf(a) calculate the priority score Score(a) of the fuzzy seeds in the fuzzy seed pool, and select the fuzzy seed with the highest priority score for this round of fuzz testing:
[0098] Score(a)=Rareness(a)×FuzzPerf(a)
[0099] Perform fuzz testing on the vulnerable project code to determine whether the high-priority fuzzy seed input selected in this round of fuzz testing can reach the dangerous call site. If it can be reached, terminate the fuzzy test and build a fuzzy decision model. If it cannot be reached, further evaluate the quality of the fuzzy seed input in this round of fuzz testing. If this round of fuzz testing improves the branch coverage or triggers more available transit points in the call chain traversal, add the fuzzy seed to the fuzzy seed pool, otherwise, discard the fuzzy seed.
[0100] Furthermore, in the vulnerability detection module,
[0101] Use the Soot static analysis framework to compile the JAVA file to be tested and build a serialization-aware code property graph SCPG, such as Figure 2 As shown; according to the serialization-aware code property graph SCPG and the existing dangerous call library, check whether there is a class method with controllable variables and associated with the dangerous call site in the JAVA file to be detected. If so, the associated method is included in the set of available transfer points; if not, no vulnerability call chain mining is performed; in the serialization-aware code property graph SCPG, a depth-first algorithm is used to search for all vulnerability call chains consisting of "entry point->available transfer point->dangerous call site" to obtain a file vulnerability call chain set; using a fuzzy decision model, starting from the entry point, regenerate the injection object, which is a new fuzzy seed, and then perform fuzzy testing on the file vulnerability call chain set to output a deserialized vulnerability call chain with an exploit risk.
[0102] Embodiment 2
[0103] like Figure 1As shown, this embodiment provides a Java deserialization vulnerability detection method based on reinforcement learning, including: step 1, collecting vulnerability data to obtain a vulnerability data set; step 2, mining potential vulnerability call chains in the vulnerability data set by constructing a serialization-aware code property graph SCPG to obtain a vulnerability call chain set; step 3, generating fuzzy seeds from the vulnerability call chain set, using reinforcement learning-guided fuzzy testing to perform fuzzy testing on the scheduling process of the fuzzy seeds and constructing a fuzzy decision model according to the test results; step 4, performing static analysis on the JAVA file to be detected and constructing a serialization-aware code property graph SCPG to mine potential vulnerability call chains in the file, obtain a file vulnerability call chain set, and use the fuzzy decision model to detect the file vulnerability call chain set, and output a deserialization vulnerability call chain with an exploit risk.
[0104] Furthermore, in step 1,
[0105] Collect vulnerability-related information from JAVA benchmark vulnerability libraries and open source vulnerability libraries with deserialization vulnerabilities. The vulnerability-related information includes: vulnerable project code, vulnerability call chain, and vulnerability exploitation POC; use the JAVA static analysis framework Soot to compile the vulnerable project code and convert it into a general code intermediate representation IR.
[0106] Furthermore, in step 2,
[0107] According to the code intermediate representation IR, the serialization-aware code property graph SCPG is constructed through pointer analysis, such as Figure 2As shown in FIG. 1 , the property graph includes: a class relationship graph, a method alias graph, and a method call graph; extract dangerous call sites (such as Runtime.exec(), Method.invoke(), etc., which can be used to execute malicious code) according to the vulnerability call chain, and build a dangerous call library, and check whether there are dangerous call sites in the dangerous call library in the constructed serialization-aware code property graph SCPG; analyze the class methods associated with the dangerous call sites according to the data flow, and judge whether the parameters of the class methods are controllable. If the parameters of the class methods are not related to the return value, it means that the return result is uncontrollable and the vulnerability call chain cannot be constructed; if the parameters of the class methods are related to the return value, then the table The return result is controllable, and the method is included in the set of exploitable transit points. The magic method in the deserialization process (such as Object.readObject, Object.finalize, etc.) or the serializable JDK class that implements the magic method and automatically calls other methods (such as Object.hashCode, Object.equals, etc.) is used as the entry point, and the deserialization dangerous call site is used as the end point. The depth-first algorithm is used to search for all vulnerable call chains consisting of "entry point->exploitable transit point->dangerous call site" in the serialization-aware code property graph SCPG, and a set of vulnerable call chains is obtained.
[0108] Further, such as Figure 3 As shown, in step 3,
[0109] Randomly select an entry point from the set of vulnerability call chains, and generate an injection object based on the class attributes of the entry point. The injection object is a fuzzy seed. Scan the attributes of all class methods that have a call relationship with the entry point as a candidate attribute set to guide the generation and mutation of fuzzy seeds in fuzzy testing; use an incremental clustering algorithm to cluster the fuzzy seeds in the candidate fuzzy seed pool and construct a multi-level seed tree. Each node in the i-th layer of the tree represents a cluster represented by a coverage metric, and the child node in the i+1th layer represents a sub-cluster. The leaf nodes of the seed tree are associated with the fuzzy seeds in the candidate fuzzy seed pool.
[0110] For a fuzzy seed s used for fuzz testing in round t, calculate its coverage metric C l The fuzzy reward SeedReward(s,l,t):
[0111]
[0112] Where fcov[s,l,t] represents the coverage metric C on the multi-level seed tree l The feature coverage of l is the number of layers of the seed tree. is the scarcity of program feature F, num_hits[F] represents the number of hits of feature F;
[0113] Propagate the fuzzy reward to the seed cluster scheduled by the upper layer to calculate the reward of the current seed cluster:
[0114]
[0115] In the formula, a l Represents the seed cluster formed by all n nodes in the lth layer of the seed tree;
[0116] 1 ,…,a n ,a n+1 > represents the node sequence in the seed tree selected in round t, a n+1 is the seed node of fuzzy seed s; evaluate the fuzzy performance of node a FuzzPerf(a):
[0117] FuzzPerf(a)=Q(a,t)+U(a)
[0118] Where Q(a,t) is the average reward of node a after t rounds of fuzzy testing, which is calculated by the following formula:
[0119]
[0120] Where N(a,t) represents the number of times node a is selected from the end of the tth round of testing to the cumulative number of times node a is selected, t′ is the last round in which node a is selected, w is the discount factor, and p is the cumulative operation variable;
[0121] U(a) is the estimated radius factor for the number of times node a is selected, and is calculated using the following formula:
[0122]
[0123] Where a′ is the parent node of node a, Y[a] represents the number of fuzzy seeds in the cluster of node a, N[a] represents the cumulative number of times node a is selected, and C is a predefined parameter used to evaluate whether this round of fuzz testing attempts to select fuzzy seeds that have never been used from the fuzzy seed pool or high-quality fuzzy seeds that have achieved higher coverage or call chain depth in previous fuzz tests;
[0124] Aggregate the scarcity of features covered by the fuzzy seeds to calculate the rarity of the fuzzy seeds (a l ):
[0125]
[0126] Where P is the JAVA program to be tested, rareness[F] is the scarcity of program feature F, and C l is the coverage metric;
[0127] By obtaining the rarity Rareness (a l ) and fuzzy performance FuzzPerf(a) calculate the priority score Score(a) of the fuzzy seeds in the fuzzy seed pool, and select the fuzzy seed with the highest priority score for this round of fuzz testing:
[0128] Score(a)=Rareness(a)×FuzzPerf(a)
[0129] Perform fuzz testing on the vulnerable project code to determine whether the high-priority fuzzy seed input selected in this round of fuzz testing can reach the dangerous call site. If it can be reached, terminate the fuzzy test and build a fuzzy decision model. If it cannot be reached, further evaluate the quality of the fuzzy seed input in this round of fuzz testing. If this round of fuzz testing improves the branch coverage or triggers more available transit points in the call chain traversal, add the fuzzy seed to the fuzzy seed pool, otherwise, discard the fuzzy seed.
[0130] Furthermore, in step 4,
[0131] Use the Soot static analysis framework to compile the JAVA file to be tested and build a serialization-aware code property graph SCPG, such as Figure 2 As shown; according to the serialization-aware code property graph SCPG and the existing dangerous call library, check whether there is a class method with controllable variables and associated with the dangerous call site in the JAVA file to be detected. If so, the associated method is included in the set of available transfer points; if not, no vulnerability call chain mining is performed; in the serialization-aware code property graph SCPG, a depth-first algorithm is used to search for all vulnerability call chains consisting of "entry point->available transfer point->dangerous call site" to obtain a file vulnerability call chain set; using a fuzzy decision model, starting from the entry point, regenerate the injection object, which is a new fuzzy seed, and then perform fuzzy testing on the file vulnerability call chain set to output a deserialized vulnerability call chain with an exploit risk.
[0132] There are many methods and approaches to implement the technical solution of the present invention, and the above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.
Claims
1. A Java deserialization vulnerability detection system based on reinforcement learning, characterized in that: include: A vulnerability data collection module is used to collect vulnerability data to obtain a vulnerability data set; A vulnerability call chain mining module is used to mine potential vulnerability call chains in the vulnerability dataset by constructing a serialization-aware code property graph SCPG to obtain a vulnerability call chain set; A fuzzy strategy learning module, used to generate fuzzy seeds from the set of vulnerability call chains, perform fuzzy testing on the scheduling process of the fuzzy seeds using reinforcement learning-guided fuzzy testing, and build a fuzzy decision model based on the test results; The vulnerability detection module is used to perform static analysis on the JAVA file to be detected and construct a serialization-aware code property graph SCPG to mine the potential vulnerability call chain in the file, obtain a file vulnerability call chain set, and use the fuzzy decision model to detect the file vulnerability call chain set, and output a deserialized vulnerability call chain with an exploit risk.
2. The Java deserialization vulnerability detection system based on reinforcement learning according to claim 1 is characterized in that: In the vulnerability data collection module, Collect vulnerability-related information from JAVA benchmark vulnerability libraries and open source vulnerability libraries that contain deserialization vulnerabilities. The vulnerability-related information includes: vulnerability project code, vulnerability call chain, and vulnerability exploitation POC; Use the JAVA static analysis framework Soot to compile the vulnerable project code and convert it into a general code intermediate representation IR.
3. The Java deserialization vulnerability detection system based on reinforcement learning according to claim 2 is characterized in that: In the vulnerability call chain mining module, According to the code intermediate representation IR, a serialization-aware code property graph SCPG is constructed through pointer analysis. The property graph includes: class relationship graph, method alias graph and method call graph. Extract dangerous call sites according to the vulnerability call chain, build a dangerous call library, and check whether there are dangerous call sites in the dangerous call library in the built serialization-aware code property graph SCPG; According to the data flow analysis, class methods associated with dangerous call sites are analyzed to determine whether the parameters of the class methods are controllable. If the parameters of the class methods are irrelevant to the return value, the return result is uncontrollable and the vulnerability call chain cannot be constructed. If the parameters of the class methods are related to the return value, the return result is controllable and the class methods are included in the set of exploitable transfer points. Taking the magic method in the deserialization process or the serializable JDK class that implements the magic method and automatically calls other methods as the entry point and the deserialized dangerous call site as the end point, the depth-first algorithm is used to search all vulnerability call chains consisting of "entry point->exploitable transit point->dangerous call site" in the serialization-aware code property graph SCPG, and the vulnerability call chain set is obtained.
4. The Java deserialization vulnerability detection system based on reinforcement learning according to claim 3 is characterized in that: In the fuzzy strategy learning module, Randomly select an entry point from the set of vulnerability call chains, and generate an injection object based on the class attributes of the entry point. The injection object is the fuzzy seed. Scan the attributes of all class methods that have a call relationship with the entry point as a candidate attribute set to guide the generation and mutation of fuzzy seeds in fuzz testing. An incremental clustering algorithm is used to cluster the fuzzy seeds in the candidate fuzzy seed pool and construct a multi-level seed tree. Each node in the i-th layer of the tree represents a cluster represented by a coverage metric, and the child nodes in the i+1th layer represent subclusters. The leaf nodes of the seed tree are associated with the fuzzy seeds in the candidate fuzzy seed pool. For a fuzzy seed s used for fuzz testing in round t, calculate its coverage metric C l The fuzzy reward SeedReward(s,l,t): Where fcov[s,l,t] represents the coverage metric C on the multi-level seed tree l The feature coverage of l is the number of layers of the seed tree. is the scarcity of program feature F, num_hits[F] represents the number of hits of feature F; Propagate the fuzzy reward to the seed cluster scheduled by the upper layer to calculate the reward of the current seed cluster: In the formula, a l Represents the seed cluster formed by all n nodes in the lth layer of the seed tree; Evaluate the fuzzy performance of node a FuzzPerf(a): FuzzPerf(a)=Q(a,t)+U(a) Where Q(a,t) is the average reward of node a after t rounds of fuzzy testing, which is calculated by the following formula: Where N(a,t) represents the number of times node a is selected from the end of the tth round of testing to the cumulative number of times node a is selected, t′ is the last round in which node a is selected, w is the discount factor, and p is the cumulative operation variable; U(a) is the estimated radius factor for the number of times node a is selected, and is calculated using the following formula: Where a′ is the parent node of node a, Y[a] represents the number of fuzzy seeds in the cluster of node a, N[a] represents the cumulative number of times node a is selected, and C is a predefined parameter used to evaluate whether this round of fuzz testing attempts to select fuzzy seeds that have never been used from the fuzzy seed pool or high-quality fuzzy seeds that have achieved higher coverage or call chain depth in previous fuzz tests; Aggregate the scarcity of features covered by the fuzzy seeds to calculate the rarity of the fuzzy seeds (a l ): Where P is the JAVA program to be tested, rareness[F] is the scarcity of program feature F, and C l is the coverage metric; By obtaining the rarity Rareness (a l ) and fuzzy performance FuzzPerf(a) calculate the priority score Score(a) of the fuzzy seeds in the fuzzy seed pool, and select the fuzzy seed with the highest priority score for this round of fuzz testing: Score(a)=Rareness(a)×FuzzPerf(a) Perform fuzz testing on the vulnerable project code to determine whether the high-priority fuzzy seed input selected in this round of fuzz testing can reach the dangerous call site. If it can be reached, terminate the fuzzy test and build a fuzzy decision model. If it cannot be reached, further evaluate the quality of the fuzzy seed input in this round of fuzz testing. If this round of fuzz testing improves the branch coverage or triggers more available transit points in the call chain traversal, add the fuzzy seed to the fuzzy seed pool, otherwise, discard the fuzzy seed.
5. The Java deserialization vulnerability detection system based on reinforcement learning according to claim 4 is characterized in that: In the vulnerability detection module, Use the Soot static analysis framework to compile the JAVA file to be tested and build a serialization-aware code property graph SCPG; According to the serialization-aware code property graph SCPG and the existing dangerous call library, check whether the JAVA file to be tested has a class method with controllable variables and associated with the dangerous call site. If so, include the associated method in the set of available transfer points; If it does not exist, no vulnerability call chain mining will be performed; In the serialization-aware code property graph SCPG, a depth-first algorithm is used to search for all vulnerability call chains consisting of "entry point->exploitable transit point->dangerous call site" to obtain a set of file vulnerability call chains; Using the fuzzy decision model, starting from the entry point, the injection object is regenerated, and the injection object is a new fuzzy seed. Then, the file vulnerability call chain set is fuzz tested to output the deserialized vulnerability call chain that has the risk of exploitation.
6. A Java deserialization vulnerability detection method based on reinforcement learning, comprising: Step 1, collect vulnerability data to obtain a vulnerability data set; Step 2, mining potential vulnerability call chains in the vulnerability dataset by constructing a serialization-aware code property graph SCPG to obtain a set of vulnerability call chains; Step 3, generating a fuzzy seed from the set of vulnerability call chains, performing a fuzzy test on the scheduling process of the fuzzy seed using a reinforcement learning-guided fuzzy test, and building a fuzzy decision model according to the test results; Step 4, statically analyze the JAVA file to be detected and construct a serialization-aware code property graph SCPG to mine the potential vulnerability call chains in the file, obtain a file vulnerability call chain set, and use the fuzzy decision model to detect the file vulnerability call chain set, and output a deserialization vulnerability call chain with an exploit risk.
7. The Java deserialization vulnerability detection method based on reinforcement learning according to claim 6 is characterized in that: In step 1, Collect vulnerability-related information from JAVA benchmark vulnerability libraries and open source vulnerability libraries that contain deserialization vulnerabilities. The vulnerability-related information includes: vulnerability project code, vulnerability call chain, and vulnerability exploitation POC; Use the JAVA static analysis framework Soot to compile the vulnerable project code and convert it into a general code intermediate representation IR.
8. The Java deserialization vulnerability detection method based on reinforcement learning according to claim 7 is characterized in that: In step 2, According to the code intermediate representation IR, a serialization-aware code property graph SCPG is constructed through pointer analysis. The property graph includes: class relationship graph, method alias graph and method call graph. Extract dangerous call sites according to the vulnerability call chain, build a dangerous call library, and check whether there are dangerous call sites in the dangerous call library in the built serialization-aware code property graph SCPG; According to the data flow analysis, class methods associated with dangerous call sites are analyzed to determine whether the parameters of the class methods are controllable. If the parameters of the class methods are irrelevant to the return value, the return result is uncontrollable and the vulnerability call chain cannot be constructed. If the parameters of the class methods are related to the return value, the return result is controllable and the class methods are included in the set of exploitable transfer points. Taking the magic method in the deserialization process or the serializable JDK class that implements the magic method and automatically calls other methods as the entry point and the deserialized dangerous call site as the end point, the depth-first algorithm is used to search all vulnerability call chains consisting of "entry point->exploitable transit point->dangerous call site" in the serialization-aware code property graph SCPG, and the vulnerability call chain set is obtained.
9. The Java deserialization vulnerability detection method based on reinforcement learning according to claim 8 is characterized in that: In step 3, Randomly select an entry point from the set of vulnerability call chains, and generate an injection object based on the class attributes of the entry point. The injection object is the fuzzy seed. Scan the attributes of all class methods that have a call relationship with the entry point as a candidate attribute set to guide the generation and mutation of fuzzy seeds in fuzz testing. An incremental clustering algorithm is used to cluster the fuzzy seeds in the candidate fuzzy seed pool and construct a multi-level seed tree. Each node in the i-th layer of the tree represents a cluster represented by a coverage metric, and the child nodes in the i+1th layer represent subclusters. The leaf nodes of the seed tree are associated with the fuzzy seeds in the candidate fuzzy seed pool. For a fuzzy seed s used for fuzz testing in round t, calculate its coverage metric C l The fuzzy reward SeedReward(s,l,t): Where fcov[s,l,t] represents the coverage metric C on the multi-level seed tree l The feature coverage of l is the number of layers of the seed tree. is the scarcity of program feature F, num_hits[F] represents the number of hits of feature F; Propagate the fuzzy reward to the seed cluster scheduled by the upper layer to calculate the reward of the current seed cluster: In the formula, a l Represents the seed cluster formed by all n nodes in the lth layer of the seed tree; Evaluate the fuzzy performance of node a FuzzPerf(a): FuzzPerf(a)=Q(a,t)+U(a) Where Q(a,t) is the average reward of node a after t rounds of fuzzy testing, which is calculated by the following formula: Where N(a,t) represents the number of times node a is selected from the end of the tth round of testing to the cumulative number of times node a is selected, t′ is the last round in which node a is selected, w is the discount factor, and p is the cumulative operation variable; U(a) is the estimated radius factor for the number of times node a is selected, and is calculated using the following formula: Where a′ is the parent node of node a, Y[a] represents the number of fuzzy seeds in the cluster of node a, N[a] represents the cumulative number of times node a is selected, and C is a predefined parameter used to evaluate whether this round of fuzz testing attempts to select fuzzy seeds that have never been used from the fuzzy seed pool or high-quality fuzzy seeds that have achieved higher coverage or call chain depth in previous fuzz tests; Aggregate the scarcity of features covered by the fuzzy seeds to calculate the rarity of the fuzzy seeds (a l ): Where P is the JAVA program to be tested, rareness[F] is the scarcity of program feature F, and C l is the coverage metric; By obtaining the rarity Rareness (a l ) and fuzzy performance FuzzPerf(a) calculate the priority score Score(a) of the fuzzy seeds in the fuzzy seed pool, and select the fuzzy seed with the highest priority score for this round of fuzz testing: Score(a)=Rareness(a)×FuzzPerf(a) Perform fuzz testing on the vulnerable project code to determine whether the high-priority fuzzy seed input selected in this round of fuzz testing can reach the dangerous call site. If it can be reached, terminate the fuzzy test and build a fuzzy decision model. If it cannot be reached, further evaluate the quality of the fuzzy seed input in this round of fuzz testing. If this round of fuzz testing improves the branch coverage or triggers more available transit points in the call chain traversal, add the fuzzy seed to the fuzzy seed pool, otherwise, discard the fuzzy seed.
10. The Java deserialization vulnerability detection method based on reinforcement learning according to claim 9 is characterized in that: In step 4, Use the Soot static analysis framework to compile the JAVA file to be tested and build a serialization-aware code property graph SCPG; According to the serialization-aware code property graph SCPG and the existing dangerous call library, check whether the JAVA file to be tested has a class method with controllable variables and associated with the dangerous call site. If so, include the associated method in the set of available transfer points; If it does not exist, no vulnerability call chain mining will be performed; In the serialization-aware code property graph SCPG, a depth-first algorithm is used to search for all vulnerability call chains consisting of "entry point->exploitable transit point->dangerous call site" to obtain a set of file vulnerability call chains; Using the fuzzy decision model, starting from the entry point, the injection object is regenerated, and the injection object is a new fuzzy seed. Then, the file vulnerability call chain set is fuzz tested to output the deserialized vulnerability call chain that has the risk of exploitation.
Citation Information
Patent Citations
Deserialization vulnerability detection method and device
CN111931184A
Steganographic embedding of hidden payload
US20170169737A1