Big data security risk assessment method and system

Abstract

The invention provides a big data security risk assessment method and system, and relates to the technical field of risk assessment, and the method comprises the steps: formulating a test case according to a security rule, obtaining a test result, and collecting operation data if the test result accords with expectation; otherwise, an alarm is triggered. Then feature extraction and weighted average processing are conducted on the operation data to obtain first data, whether the data exceed a preset threshold value or not is judged, if yes, the first collection step is executed again, and if not, an alarm is triggered, and potential risks in big data can be reduced.

Description

Technical Field

[0001] The present application relates to the technical field of risk assessment, and in particular to a big data security risk assessment method and system. Background Art

[0002] With the rapid advancement of information technology, big data has gradually penetrated into all areas of social economy and has become a key force in promoting innovation and development in all walks of life. From finance, medical care to education and transportation, the application of big data is everywhere, and the application of big data can provide strong data support for corporate decision-making. However, the widespread application of big data is also accompanied by a series of severe security challenges.

[0003] Big data systems store a huge amount of sensitive information, including but not limited to personal privacy, business secrets, etc. Once this information is illegally attacked or leaked, it will not only cause serious infringement of personal privacy, but may also cause significant economic losses to the company. Therefore, big data security has become a major issue that cannot be ignored.

[0004] Traditional security risk assessment methods are powerless when faced with complex threats in big data environments. These methods often rely on manual experience and static rules. However, big data is huge in scale and in diverse formats. Traditional methods are prone to miss potential security risks when processing these data, thereby increasing the risks of these big data. Summary of the invention

[0005] In order to reduce potential risks in big data, the present application provides a big data security risk assessment method and system.

[0006] In the first aspect, the present application provides a big data security risk assessment method, which adopts the following technical solution: A big data security risk assessment method includes the following steps: Data collection: including the first collection and the second collection; First collection: formulate multiple test cases according to the security rules of the assessment target and obtain the test results; Second collection: collect the running data during the test; First judgment: judge whether the test result meets expectations. If yes, execute the second acquisition step; if no, execute the alarm step; First assessment: includes first processing and second judgment; First processing: extracting features from the operation data to obtain operation features, performing weighted average processing on feature values ​​of the operation features, and recording the processing results as first data; Second judgment: judging whether the first data is greater than a preset threshold, if so, executing the first acquisition step; if not, executing the alarm step; Alarm: Send an alarm signal.

[0007] This application formulates multiple test cases according to the security rules of the evaluation target, and obtains the test results of each test case. Then, it is judged whether the test results meet expectations. If the test results meet expectations, it means that the evaluation target performs well in the current test scenario. After that, the operation data during the test process will be collected, and the operation data will be further analyzed and processed to reveal problems or abnormal situations that the test case failed to capture. If the test results do not meet expectations, an alarm signal is issued. This application can directly reflect whether the evaluation target meets the preset safety standards or performance requirements. Subsequently, feature extraction is performed on the operation data to obtain the operation features, and the feature values ​​of the operation features are weighted averaged, and the processing results are recorded as the first data. Subsequently, by comparing the first data with the preset threshold, it can be judged whether the evaluation target is in a safe state. If the first data is greater than the preset threshold, it means that there is no abnormality or performance degradation in the evaluation target. After the preset time interval, the first acquisition step is re-executed to further observe and verify the security of the evaluation target. If the first data is not greater than the preset threshold, it means that there are potential risks or vulnerabilities in the evaluation target, and an alarm signal needs to be issued to prompt relevant personnel to take measures to solve the problem. This application tests the assessment objectives and, on the premise that the test results meet expectations, further analyzes the operating data during the test to determine whether there are potential risks or vulnerabilities, thereby improving the security of big data.

[0008] Optionally, the method further comprises: First modeling: Integrate the running status of the evaluation target during the test into a state set, integrate the actions taken when the evaluation target changes its running status into an action set, and establish a Markov decision process model; Second modeling: Taking the running status of the evaluation target during the test as the node and the dependency relationship between the running statuses as the directed edge, a Bayesian network model is established; First reasoning: Input each running state of the evaluation target into the Bayesian network model in sequence, and output the next running state of the evaluation target and the corresponding probability; First setting: using the next running state and the corresponding probability in the first inference step to update the state transition probability of the Markov decision process model to obtain a new Markov decision process model; Second reasoning: Input the current operating state of the evaluation target into the new Markov decision process model and output the operating state with the greatest risk at the next moment.

[0009] By constructing a Markov decision process model, this application can formally represent the state transition and decision-making process of the evaluation target during the test process, which helps to understand the behavior pattern of the evaluation target in different states and the state changes that may occur after taking different actions. Subsequently, the Bayesian network model represents the state transition of the evaluation target during the test process through nodes (representing the operating state) and directed edges (representing the dependencies between the operating states). Subsequently, each operating state of the evaluation target is input into the Bayesian network model, and the next operating state of the evaluation target and its corresponding probability can be output. Subsequently, the next operating state and its corresponding probability are used to update the state transition probability of the Markov decision process model, so that the Markov decision process model also takes into account the past operating state of the evaluation target when performing related actions. Subsequently, the current operating state of the evaluation target is input into the new Markov decision process model, and the new Markov decision process model outputs the operating state when the risk is the highest according to the current operating state of the evaluation target. The present application adopts the Bayesian network model to make up for the defect that the Markov decision process model cannot take into account the past state of the evaluation target when making predictions, that is: the conditional probability output by the Bayesian network model replaces the state transition probability in the Markov decision process model, so that the output of the Markov decision process model can consider the current and past operating states at the same time, so that the Markov decision process model can more comprehensively consider the historical operating states when making decisions, thereby improving the accuracy and robustness of the prediction results.

[0010] Optionally, the method further comprises: Second calculation: using statistical analysis algorithms, calculate the probability of risk occurring at the next moment based on all the probabilities in the first reasoning step; The third judgment: judging whether the probability of the risk occurring at the next moment is greater than the preset threshold, if so, executing the early warning step; if not, executing the first collection step; Early warning: Send out early warning signals.

[0011] This application calculates the overall probability of risk occurring in the evaluation target during the test process through each operating state and its corresponding probability obtained in the first reasoning step, so as to evaluate the risk level of the evaluation target in the current operating state. If the probability of risk occurrence exceeds the preset threshold, it means that the evaluation target has a high risk in the current operating state and an immediate alarm is required; if the risk probability does not exceed the threshold, the first acquisition step can be continued to further test the evaluation target. This application calculates the probability of risk in the next operating state based on the state transition probability of the current operating state, and then decides whether to issue a warning signal in advance, so that operation and maintenance personnel can take measures in advance to reduce safety risks.

[0012] Optionally, the method further comprises: First acquisition: obtain the action taken by the evaluation target when the probability of risk occurrence is greater than the preset threshold, record it as the target action, and locate the risk program in the test case according to the target action; Second acquisition: acquiring the execution log of the risk program, locating the risk position according to the execution log, and executing the early warning step, wherein the early warning step also includes outputting the risk position.

[0013] This application obtains the actions taken by the assessment target in the current running state when the probability of risk occurrence is greater than a preset threshold. By parsing these actions, it is possible to locate the specific program segment in the test case program that may cause risks, that is, the risk program. Subsequently, the execution log of the program is obtained. By analyzing the execution log in detail, the specific location where the risk occurs can be obtained, such as a function, a line of code, or a data processing link. The execution log contains rich runtime information, such as variable values, function call sequence, exception information, etc. This information helps to quickly locate the risk location. Through log analysis, the time for risk investigation can be significantly shortened. Subsequently, the risk location is output in the early warning step, which helps the operation and maintenance personnel to quickly understand the risk situation and take effective countermeasures to prevent the problem from further expanding, thereby further improving the security of the data.

[0014] Optionally, the method further comprises: Find vulnerabilities: Use static analysis tools to disassemble the assessment target and obtain dangerous instructions of the assessment target; Third acquisition: obtain the calling function of the dangerous instruction and determine the calling point address according to the calling function; First construction: construct a target stack for storing the call point address, and set the initial state of the target stack to be empty; The first operation: running the risk program, updating the target stack, obtaining the call point address in the updated target stack, and executing the early warning step, wherein the early warning step further includes: outputting the call point address in the updated target stack.

[0015] This application uses static analysis tools to disassemble the evaluation target to identify dangerous instructions that may cause risks. Dangerous instructions are associated with known vulnerability patterns, such as buffer overflows, formatted string vulnerabilities, etc. Through static analysis, potential code defects can be discovered without actually running the program. Subsequently, after identifying the dangerous instructions, the functions and call point addresses called by these dangerous instructions are obtained. The calling functions and call point addresses help understand the triggering methods and conditions of dangerous instructions. By obtaining the calling functions and call point addresses, the context environment in which the dangerous instructions are executed can be constructed. Subsequently, a target stack for storing the call point addresses is constructed, and its initial state is set to empty. The target stack is used to record the call point addresses of dangerous instructions encountered during the running of the risk program. Subsequently, the risk program is run and the target stack is updated according to the execution of the program. During the program running process, whenever a call point of a dangerous instruction is encountered, the corresponding call point address is pushed into the target stack. By actually running the risk program, the call situation of the dangerous instruction can be dynamically captured. Subsequently, after the program runs, the call point address in the target stack is obtained, and the early warning step is performed. This application can more comprehensively identify and assess risks by combining static analysis and dynamic operation, thereby improving the accuracy of early warnings.

[0016] Optionally, the method further comprises: Set execution mode: including second build, fourth acquisition, fourth judgment and change status; Second build: build the code execution engine; Fourth acquisition: obtain the program start line of the risk program, recorded as the first code line; obtain the program end line of the risk program, recorded as the second code line; Fourth judgment: judge whether the test case runs to the first code line. If so, stop running, save the current memory and registers, and execute the step of changing the state; if not, re-execute the fourth judgment step after the current code line is completed; Change status: synchronize the current execution status of the test case to the code execution engine, convert the memory or register when running to the first code line into a symbolic expression, record it as the first symbolic expression, load the first symbolic expression into the code execution engine, set the risk program between the first code line and the second code line to run in the code execution engine, and execute the first running step.

[0017] The present application constructs a code execution engine, which is used to execute the code of the risk program. Subsequently, the starting line (first code line) and the ending line (second code line) of the risk program are obtained. These two lines of code represent the starting and ending positions of the risk program. By obtaining these key code lines, the risk can be more accurately located and analyzed. Subsequently, during the execution of the test case, it is constantly determined whether it has run to the first code line. Once it runs to the first code line, it stops running immediately, and saves the current memory and register state to accurately capture the execution state of the test case before the key code line, which provides a basis for the subsequent state synchronization and the operation of the risk program in the code execution engine. Subsequently, the current execution state of the test case is synchronized to the code execution engine, and the value of the memory or register at the first code line is converted into a symbolic expression, and then loaded into the code execution engine to continue to store the running data of the risk program. At the same time, the risk program between the first code line and the second code line is set to run in the code execution engine. Through state synchronization and the loading of the first symbolic expression, the risk program can be run in a symbolic manner in the code execution engine to reduce the data dependency between the risk program and the program code before the first code line, while improving the flexibility and accuracy of the test.

[0018] Optionally, the method further comprises: Symbol conversion: convert the memory or register when running to the second code line into a symbolic expression, which is recorded as the second symbolic expression; Feature extraction: extract the features of the second symbolic expression, recorded as the first feature; Fifth judgment: judging whether the first feature is the same as any vulnerability feature in the preset vulnerability feature set, if so, executing the early warning step; if not, executing the step of adding features; Adding features: adding the first feature to the vulnerability feature set, and executing the early warning step, wherein the early warning step also includes outputting the first feature.

[0019] The present application converts the memory or register at the second code line into a symbolic expression, thereby converting the state of the memory or register into a symbolic form, which is convenient for feature extraction and comparison. The introduction of symbolic expressions makes the description of the memory or register state more abstract and universal, which is conducive to cross-platform and cross-architecture analysis and early warning. Subsequently, features are extracted from the second symbolic expression and recorded as the first feature. Feature extraction is a key step in identifying whether a risk program has a vulnerability. By extracting key information to characterize the state of the memory or register after the risk program runs, through feature extraction, the complex memory or register state can be simplified into a feature vector that is easy to compare and analyze, thereby improving the accuracy and efficiency of early warning. Subsequently, it is determined whether the extracted first feature is the same as any one of the vulnerability features in the preset vulnerability feature set. Through the preset vulnerability feature set, known vulnerabilities in the risk program can be quickly and accurately identified, reducing the risk of false positives and false negatives. If the first feature is different from any of the features in the preset vulnerability feature set, it is added to the vulnerability feature set. By continuously updating the vulnerability feature set, the present application can adapt to the ever-changing threat environment and improve the ability to identify and warn new vulnerabilities.

[0020] Optionally, the method further comprises: Format conversion: Use a compiler to convert the risk program into a binary format, record the converted risk program as a new risk program, and execute the vulnerability search step.

[0021] This application uses a compiler to convert the risk program from source code or intermediate representation to binary format to convert the risk program from high-level language to machine language. The new risk program is a converted binary executable file. The binary format is closer to the actual execution state of the program, so the analysis results based on binary are more accurate.

[0022] In the second aspect, the present application provides a big data security risk assessment system, which adopts the following technical solutions: A big data security risk assessment system, comprising: a processor and a memory, The memory stores program code; The processor executes the steps of the method when calling the program code in the memory.

[0023] In summary, the present application includes at least one of the following beneficial technical effects: 1. This application has formulated multiple test cases based on the security rules of the evaluation target, and obtained the test results of each test case. Then, it determines whether the test results meet expectations. If the test results meet expectations, it means that the evaluation target performs well in the current test scenario. After that, the operation data during the test process will be collected, and the operation data will be further analyzed and processed to reveal problems or abnormal situations that the test cases failed to capture. If the test results do not meet expectations, an alarm signal will be issued. This application can directly reflect whether the evaluation target meets the preset security standards or performance requirements.

[0024] 2. This application performs feature extraction on the operation data, obtains the operation characteristics, performs weighted averaging on the characteristic values ​​of the operation characteristics, and records the processing results as the first data. Subsequently, by comparing the first data with the preset threshold, it can be determined whether the evaluation target is in a safe state. If the first data is greater than the preset threshold, it means that there is no abnormality or performance degradation in the evaluation target. At this time, the first collection step is re-executed after a preset time interval to further observe and verify the safety of the evaluation target. If the first data is not greater than the preset threshold, it means that there are potential risks or loopholes in the evaluation target, and an alarm signal needs to be issued to prompt relevant personnel to take measures to solve the problem.

[0025] 3. This application tests the assessment objectives and, on the premise that the test results meet expectations, further analyzes the operating data during the test to determine whether there are potential risks or vulnerabilities, thereby improving the security of big data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of Example 1 of the present application; Figure 2 is a flow chart of Example 2 of the present application; Figure 3 is a flow chart of Example 3 of the present application; Figure 4 This is a flow chart of Example 4 of the present application. DETAILED DESCRIPTION

[0027] The following combination Figures 1 to 4 This application is described in further detail.

[0028] Example 1: This example discloses a method for assessing the security risks of big data. Figure 1, the method includes: S1 data collection, S2 first judgment, S3 first evaluation and S4 alarm. First, formulate test cases and obtain test results according to safety rules. If the test results meet expectations, collect the operation data during the test; otherwise, issue an alarm signal. Then, perform feature extraction and weighted average processing on the operation data to obtain first data, and compare it with a preset threshold. If the first data is greater than the threshold, re-execute the first collection step after a certain interval; otherwise, issue an alarm signal. The process of this embodiment is as follows: S1 data collection, including S11 first collection and S12 second collection.

[0029] S11 first collects and designs multiple test cases based on the security rules of the evaluation target. These test cases are designed to fully cover various possible states and interaction scenarios of the evaluation target to accurately evaluate its security.

[0030] Once all the test cases are developed, they are executed sequentially in a controlled environment and the test results of each test case are carefully recorded.

[0031] S12 collects the running data during the test, including key performance indicators such as execution log, memory usage, CPU load, and network traffic.

[0032] S2 is the first judgment, which determines whether the test results meet expectations. If so, it means that the evaluation target performs normally under the current test case. Then S12 is executed for the second collection to determine whether there are potential vulnerabilities or risks. If not, it means that the evaluation target has security risks or abnormal behaviors, and S4 alarm needs to be executed to feedback these security risks or abnormal behaviors to the operation and maintenance personnel.

[0033] The test results meet the expected standards if the test results are consistent with the expected results listed in the test case, or the data in the test results are accurate and consistent with the expected data.

[0034] S3 first evaluation, including S31 first processing and S32 second judgment.

[0035] S31: First processing, extracting operation features from the operation data by feature extraction technology, and then performing weighted average processing on the feature values ​​of these operation features to obtain an index that comprehensively reflects the operation status of the evaluation target, namely, the first data.

[0036] S32 is the second judgment, judging whether the first data is greater than a preset threshold. If so, it indicates that there are no potential risks or loopholes in the assessment target. After a preset time interval, S11 is executed for the first collection to continuously monitor the assessment target. If not, it means that there are risks or loopholes in the assessment target, and S4 alarm needs to be executed to remind the operation and maintenance personnel that there are risks or loopholes in the assessment target.

[0037] S4 alarm, sends out an alarm signal.

[0038] The following is an explanation of this solution using a specific case.

[0039] S11 First Collection: Based on the security rules of the online payment system, multiple test cases were designed. For example, to test whether the online payment system can correctly identify and prevent illegal logins, and how the online payment system responds when the transaction amount exceeds the user's account balance.

[0040] In a controlled environment, test the system in accordance with the designed test cases. For example, simulate illegal login and record the response of the online payment system; try to make excessive transactions and observe how the online payment system responds.

[0041] After each test case is run, record the corresponding test results.

[0042] S12 Second collection: In the process of running the test case, the operation data of the online payment system is collected, and the operation data includes key performance indicators such as execution log, memory usage, CPU load, and network traffic.

[0043] S2 First judgment: Compare the collected test results with the expected results. For example, if the online payment system successfully blocks illegal logins or correctly handles excessive transaction requests, the test results are considered to be in line with expectations. On the contrary, if the online payment system fails to respond to these test cases correctly, it is considered that there are security risks or abnormal behavior.

[0044] In this case, assuming that the test results are in line with expectations, it means that the online payment system performs normally under the current test case. Therefore, the second collection of S12 can be performed to continue collecting operation data to determine whether there are potential vulnerabilities or risks.

[0045] S3 First Assessment S31 first processing: extract key operation features from the collected operation data, such as CPU load peak 85%, memory usage average 60%, network delay time 50ms, etc.

[0046] The characteristic values ​​of the above-mentioned operation characteristics are processed by weighted averaging to obtain the first data.

[0047] 0.2×85%+0.3×60%+0.5×0.05=0.375.

[0048] 0.2 is the weight of the CPU load peak; 0.3 is the weight of the average memory usage; 0.5 is the weight of the network delay time; and 0.375 is the first data.

[0049] S32 Second judgment: Compare the first data with a preset threshold.

[0050] If the first data is greater than a preset threshold, it is considered that there is no potential risk or loophole in the online payment system, and after a preset time interval, S11 first collection is executed; otherwise, it is considered that there is a risk or loophole, and S4 alarm is executed.

[0051] S4 Alarm: Sends an alarm signal to prompt operation and maintenance personnel to deal with potential security risks in a timely manner.

[0052] This embodiment can test the assessment target using test cases formulated according to the security rules of the assessment target and obtain test results. When the test results meet expectations, further analysis is performed using the operating data to determine whether there are risks or vulnerabilities in the assessment target, thereby improving the security of big data.

[0053] Embodiment 2: In the Markov decision process model, the state transition probability is only related to the current state and does not consider the past state. However, the evaluation target is strongly correlated in each link of the test process. Therefore, this embodiment replaces the state transition probability in the Markov decision process model with the conditional probability in the Bayesian network model, so that the Markov decision process model can take the past operating state into account when predicting the operating state at the next moment. Figure 2 The difference between this embodiment and embodiment 1 is that the method further includes: S51 is the first modeling, which integrates the running status of the evaluation target during the test into a state set, integrates the actions taken when the evaluation target changes the running status into an action set, and establishes a Markov decision process model.

[0054] The state set of the Markov decision process model includes all the possible operating states that the evaluation target may experience during the test. For example, the possible operating states of an online payment system include "normal operation", "operation failure", "attack", "system crash", etc.

[0055] The action set of the Markov decision process model includes all actions that the evaluation target can take, and the evaluation target can change the current operating state by taking actions. For example, for an online payment system, possible actions include "calling the database port", "entering the payment password", "checking the balance", etc.

[0056] The state transition probability of the Markov decision process model represents the probability of evaluating the target transitioning from the current running state to the next running state after taking a certain action.

[0057] The reward function of the Markov decision process model refers to: defining the reward function according to the goal of the security test, such as giving positive rewards to the operating state that successfully defends against attacks and negative rewards to the operating state that is attacked, in order to quantify the degree of quality of the target reaching a certain operating state after taking a certain action.

[0058] For example, the reward function The calculation model can be: ; Among them, s represents the current running state of the evaluation target. a represents the action taken by the agent. s′ represents the running state of the evaluation target after executing the action. is the reward value, which can be set as needed. The value of is 1.

[0059] In other embodiments, the reward function The calculation model can also be: ; The reward for evaluating the target to transfer from the current running state s to the running state s′ is , The value of can be positive, negative or zero. When the value of is positive, it indicates that a positive reward is given; when When the value of is negative, it indicates that a negative reward is given; when When the value of is zero, it means no reward is given.

[0060] S52 is a second modeling, in which the running state of the evaluation target during the test is used as a node, and the dependency relationship between each running state is used as a directed edge to establish a Bayesian network model.

[0061] The nodes of the Bayesian network model correspond to the possible states of the evaluation target, such as "normal operation", "under attack", etc.

[0062] The directed edges of the Bayesian network model represent the dependency between the running states, that is, the occurrence of one running state may affect the probability of another running state. For example, the "under attack" state will increase the probability of the "system crash" state.

[0063] S53 is the first reasoning, which inputs each operating state of the evaluation target into the Bayesian network model respectively, and uses the Bayesian theorem and the known transition probability (i.e., dependency) to calculate the next operating state of the evaluation target and its corresponding probability.

[0064] S54 is the first setting, using the next operating state and its corresponding probability in the first reasoning of S53 to update the state transition probability of the Markov decision process model to obtain a new Markov decision process model.

[0065] S55 is the second reasoning, which inputs the current operating state of the evaluation target into a new Markov decision process model. The Markov decision process model can find the optimal action set given the current operating state, and output the operating state with the highest risk at the next moment after executing the action set.

[0066] S56 is the second calculation, which uses a statistical analysis algorithm to calculate the probability of the risk occurring at the next moment based on all the probabilities in the first reasoning in S53.

[0067] S57 is the third judgment, which determines whether the probability of a risk occurring at the next moment is greater than a preset threshold. If so, it means that the assessment target will face a higher security risk at the next moment, and it is necessary to execute S58 warning; if not, it means that the security situation at the next moment is acceptable, and then execute S11 first collection.

[0068] S58 warning, issuing a warning signal.

[0069] In the traditional Markov decision process model, the calculation method of the state transition probability depends only on the current operating state and the actions taken, while ignoring the impact that the past operating state may have on the current and future operating states. However, the operating states of many software systems often have a certain historical dependency, that is, the current operating state may be affected by a series of previous operating states. In order to capture this historical dependency, this embodiment combines the Bayesian network model with the Markov decision process model, and replaces the state transition probability in the Markov model with the conditional probability in the Bayesian network to capture the historical dependency of the evaluation target operating state, so that the past operating state can be more comprehensively considered when predicting the evaluation target to change the operating state, thereby improving the prediction accuracy.

[0070] The Bayesian network model represents the dependency between variables through nodes and directed edges. In this embodiment, each possible operating state of the evaluation target is taken as a node of the Bayesian network, and the dependency between the operating states is represented as directed edges. In this way, the conditional probability in the Bayesian network model can be used to quantify the probability of one operating state transferring to another operating state given its predecessor state. By applying the conditional probability in the Bayesian network model to the Markov decision process model, the replacement of the traditional state transition probability is achieved, so that the new Markov decision process model can take into account the current state and action while also taking the past operating state into the prediction range, thereby improving the accuracy of the prediction.

[0071] Example 3: Reference Figure 3 The difference between this embodiment and embodiment 2 is that the method further includes: S61 first obtains, when the probability of risk occurrence is greater than a preset threshold, the action taken by the evaluation target is obtained, recorded as the target action. Then, the risk program in the program of the test case is located according to these target actions.

[0072] S62 second acquisition, acquiring the execution log of the risky program, the execution log contains detailed information when the program is running, such as variable values, function calls, error information, etc. By analyzing the execution log, abnormal behavior or potential problems in the program can be identified, thereby locating the risk location.

[0073] S63 format conversion, using a compiler to convert the risk program into a binary format, and recording the converted risk program as a new risk program.

[0074] A compiler is a program that converts source code written in a high-level programming language (such as C, C++, Java, etc.) into machine language (binary format).

[0075] The compiler first reads the source code and breaks it down into a series of tokens, which are the basic building blocks of the source code. Then, the compiler organizes the tokens into a parse tree according to the grammatical rules of the language to represent the structure of the source code. Then, the compiler checks the semantic correctness of the parse tree, such as whether the variables have been declared, whether the types match, etc. The compiler converts the parse tree into an intermediate representation (IR). Then, the compiler optimizes the intermediate code to improve the execution efficiency of the program or reduce the code size. Finally, the compiler converts the optimized intermediate code into binary code.

[0076] The converted binary code is a machine instruction that can be directly executed by a computer. The machine instructions are organized into executable files or library files, which contain all necessary information of the program, such as code segments, data segments, relocation information, etc.

[0077] S64 looks for vulnerabilities and uses static analysis tools to disassemble the evaluation target. Through disassembly, the internal structure of the evaluation target binary file can be analyzed to identify dangerous instructions that may cause security vulnerabilities.

[0078] S65 The third acquisition: in the program code of the evaluation target, the instruction is called through a certain function. By identifying the calling function of the dangerous instruction, the specific position of the dangerous instruction in the program (ie, the calling point address) can be further located.

[0079] S66: First construction, constructing a target stack for storing the call point address, and setting the initial state of the target stack to be empty.

[0080] The target stack is a data structure used to record the call point addresses corresponding to dangerous instructions during program execution. By building the target stack, it is possible to determine which instructions were called during the test using the test case.

[0081] S67 is the first operation, which runs the risk program. During the program operation, whenever a call function corresponding to a dangerous instruction is encountered, the corresponding call point address is pushed into the target stack. In this way, as the risk program is executed, the target stack is continuously updated, and the call point address in the updated target stack is obtained, and S58 warning is executed. The S58 warning also includes: outputting the risk position and the call point address in the target stack.

[0082] In other embodiments, whether a dangerous instruction has been called can be determined by judging whether the call point address is stored in the updated target stack. If so, it indicates that the dangerous instruction has not been called during the execution of the risk program, and then the first acquisition step is executed to continue the test; if not, it means that the dangerous instruction has been called during the execution of the risk program, and the early warning step needs to be executed.

[0083] This embodiment locates risky programs by evaluating target actions, uses execution logs and static analysis tools to determine risk locations and dangerous instructions, builds and updates the target stack to track the call point address, and finally decides whether to re-collect data or trigger an early warning mechanism based on the stack status.

[0084] Example 4: Reference Figure 4 The difference between this embodiment and embodiment 3 is that the method further includes: S71 sets the execution mode: including S711 second construction, S712 fourth acquisition, S713 fourth judgment and S714 change status.

[0085] S711 The second construction is to build a code execution engine, which is a specially designed software component for executing risky programs in binary format. It can execute code in an isolated environment to reduce the damage to the actual system or data. The code execution engine has functions such as memory management, register simulation, and instruction execution, which enables it to simulate the operation of the program without relying on actual hardware.

[0086] S712 fourth acquisition: obtain the program start line of the risk program source code, recorded as the first code line; obtain the program end line of the risk program source code, recorded as the second code line.

[0087] S713 is the fourth judgment, which determines whether the test case has run to the first code line. If so, stop running, save the current memory and registers, and execute S714 to change the state; if not, after the current code line is completed, re-execute S713, the fourth judgment.

[0088] S714 changes the state. In order to run the risk program in the code execution engine, it is necessary to convert the memory or register state at the first code line into a symbolic expression (i.e., the first symbolic expression). The first symbolic expression represents the state of the test case when it is executed to the starting point of the risk program.

[0089] Subsequently, the first symbolic expression is loaded into the code execution engine, and the risk program (i.e., the code between the first code line and the second code line) is set to run in the code execution engine, and then the behavior of the risk program can be analyzed in an isolated and secure environment without affecting the actual system or data. Then, S67 first run is executed, and after running S67 first run, it also includes: S72 symbol conversion converts the memory or register when running to the second code line into a symbolic expression, which is recorded as the second symbolic expression.

[0090] S73 feature extraction, using feature extraction technology to extract the features of the second symbolic expression, recorded as the first feature.

[0091] S74 is the fifth judgment, judging whether the first feature is the same as any vulnerability feature in the preset vulnerability feature set. If so, it means that a vulnerability has been found, and then S58 is executed for warning; if not, it means that the vulnerability corresponding to the first feature is not recorded in the vulnerability feature set, and then S75 is executed to add features.

[0092] S75 adds a feature, adds the first feature to the vulnerability feature set, and executes S58 warning, and the S58 warning also includes outputting the first feature.

[0093] In other embodiments, after executing S714 to change the state, the following steps are also included: Collect paths. When the program is running, dynamically collect the execution path of the program according to the state of variables and changes in branch conditions. By analyzing the execution flow of the program, record the variable values ​​and branch selections at each branch point, and generate path constraints. Then, use the preset SMT (Satisfiability Modulo Theories) solver to solve the path constraints to obtain the solution set of all possible execution paths.

[0094] The sixth judgment is to judge whether the solution set is empty. If so, it means that under the given variables and branch conditions, an execution path that satisfies all constraints cannot be found, and then the output step is executed; if not, the first run of S67 is executed.

[0095] Output, signal that the output path is unreachable.

[0096] This embodiment generates path constraints based on the variables and branch conditions when the risk program is running, and uses a preset SMT (Satisfiability Modulo Theories) solver to solve the path constraints to obtain a solution set. This embodiment can generate path constraints through a formal method and use the SMT solver to solve these constraints. It can accurately analyze the execution path of the risk program and discover potential unreachable paths or path conditions. Then, determine whether the solution set is empty. If the solution set is empty, it means that according to the current variables and branch conditions, it is impossible to find an execution path that satisfies all path constraints, that is, the path is unreachable, so that the unreachable path can be quickly identified to reduce the waste of resources in subsequent execution.

[0097] The following is further explained with specific cases.

[0098] Assume there is a risky program risky_program, which is known to contain a potential security vulnerability.

[0099] 1. Build a code execution engine CodeExecutionEngine, which has functions such as memory management, register simulation and instruction execution. It can execute binary code in an isolated environment.

[0100] 2. By analyzing the source code of the test case TestCase where risky_program is located, it is determined that the starting line of the program (the first code line) is line 100 and the ending line (the second code line) is line 200.

[0101] 3. During the execution of the test case TestCase, monitor whether the first code line (i.e., line 100) has been reached. When the test case runs to line 100, stop executing and save the current memory and register status.

[0102] 4. Convert the saved memory and register states into a symbolic expression FirstSymbolExpression, which represents the state of the test case when it executes to the starting point of the risk program.

[0103] Load the symbolic expression FirstSymbolExpression into the code execution engine, and set the code from line 100 to line 200 to run in the engine.

[0104] 5. When running the risk program in the code execution engine, the execution path of the program is dynamically collected according to the state of the variables and the changes in the branch conditions. By analyzing the execution flow of the program, the variable values ​​and branch selections at each branch point are recorded to generate path constraints. Then, these path constraints are solved using the SMT solver to obtain the solution set of all possible execution paths.

[0105] 6. Check whether the solution set returned by the SMT solver is empty. If the solution set is empty, it means that under the given variables and branch conditions, it is impossible to find an execution path that satisfies all constraints, and then outputs a signal that the path is unreachable. Otherwise, the first run step will continue.

[0106] 7. When the risky program risky_program executes to the second code line (assuming it is line 200) in the isolated environment, the memory and register states at this time are automatically captured. Subsequently, these states are converted into symbolic expressions using symbolic execution technology, recorded as SecondSymbolExpression, which represents the state of the program when it executes to the second code line.

[0107] 8. Extract key features from SecondSymbolExpression, which can reflect the behavior or status of the risk program at a specific execution point, and record them as FirstFeature.

[0108] 9. Determine whether FirstFeature matches any feature in the preset vulnerability feature set.

[0109] The vulnerability feature set is a predefined database containing known vulnerability features. If FirstFeature matches a feature in the feature set, it is considered that a corresponding vulnerability is found in risky_program, and S58 warning is executed to notify the operation and maintenance personnel to conduct further analysis and response.

[0110] If FirstFeature does not match any feature in the vulnerability feature set, it means that an unknown vulnerability is discovered, or the vulnerability corresponding to the feature has not been recorded in the vulnerability feature set, and S75 is executed to add a feature.

[0111] 10. Add FirstFeature to the vulnerability feature set to expand the coverage of the feature set and improve the accuracy of future vulnerability detection.

[0112] This embodiment firstly builds a code execution engine, obtains the code scope of risky programs, monitors the execution progress of test cases, and synchronizes and executes risky programs, so as to safely analyze and repair potential security risks. Then, through the steps of collecting paths, symbol conversion, feature extraction, and feature matching, it can realize dynamic analysis and vulnerability detection of risky programs.

[0113] Embodiment 5: This embodiment discloses a big data security risk assessment system, including a processor and a memory, The memory stores program code; The processor executes the steps of the method when calling the program code in the memory.

[0114] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A big data security risk assessment method, characterized in that: include: Data collection: including the first collection and the second collection; First collection: formulate multiple test cases according to the security rules of the assessment target and obtain the test results; Second collection: collect the running data during the test; First judgment: judge whether the test result meets expectations. If so, execute the second acquisition step; If not, execute the warning steps; First assessment: includes first processing and second judgment; First processing: extracting features from the operation data to obtain operation features, performing weighted average processing on feature values ​​of the operation features, and recording the processing results as first data; Second judgment: judging whether the first data is greater than a preset threshold, and if so, executing the first acquisition step; If not, execute the warning steps; Alarm: Send an alarm signal.

2. The big data security risk assessment method according to claim 1 is characterized in that: The method further comprises: First modeling: Integrate the running status of the evaluation target during the test into a state set, integrate the actions taken when the evaluation target changes its running status into an action set, and establish a Markov decision process model; Second modeling: Taking the running status of the evaluation target during the test as the node and the dependency relationship between the running statuses as the directed edge, a Bayesian network model is established; First reasoning: Input each running state of the evaluation target into the Bayesian network model in sequence, and output the next running state of the evaluation target and the corresponding probability; First setting: using the next running state and the corresponding probability in the first inference step to update the state transition probability of the Markov decision process model to obtain a new Markov decision process model; Second reasoning: Input the current operating state of the evaluation target into the new Markov decision process model and output the operating state with the greatest risk at the next moment.

3. The big data security risk assessment method according to claim 2 is characterized in that: The method further comprises: Second calculation: using statistical analysis algorithms, calculate the probability of risk occurring at the next moment based on all the probabilities in the first reasoning step; The third judgment: judging whether the probability of the risk occurring at the next moment is greater than the preset threshold, if so, executing the early warning step; if not, executing the first collection step; Early warning: Send out early warning signals.

4. The big data security risk assessment method according to claim 3 is characterized in that: The method further comprises: First acquisition: obtain the action taken by the evaluation target when the probability of risk occurrence is greater than the preset threshold, record it as the target action, and locate the risk program in the test case according to the target action; Second acquisition: acquiring the execution log of the risk program, locating the risk position according to the execution log, and executing the early warning step, wherein the early warning step also includes outputting the risk position.

5. The big data security risk assessment method according to claim 4 is characterized in that: The method further comprises: Find vulnerabilities: Use static analysis tools to disassemble the assessment target and obtain dangerous instructions of the assessment target; Third acquisition: obtain the calling function of the dangerous instruction and determine the calling point address according to the calling function; First construction: construct a target stack for storing the call point address, and set the initial state of the target stack to be empty; The first operation is to run the risk program, update the target stack, obtain the call point address in the updated target stack, and execute the early warning step, wherein the early warning step further includes: outputting the call point address in the updated target stack.

6. The big data security risk assessment method according to claim 5, characterized in that: The method further comprises: Set execution mode: including second build, fourth acquisition, fourth judgment and change status; Second build: build the code execution engine; Fourth acquisition: obtain the program start line of the risk program, recorded as the first code line; obtain the program end line of the risk program, recorded as the second code line; Fourth judgment: judge whether the test case runs to the first code line. If so, stop running, save the current memory and registers, and execute the step of changing the state; if not, re-execute the fourth judgment step after the current code line is completed; Change status: synchronize the current execution status of the test case to the code execution engine, convert the memory or register when running to the first code line into a symbolic expression, record it as the first symbolic expression, load the first symbolic expression into the code execution engine, set the risk program between the first code line and the second code line to run in the code execution engine, and execute the first running step.

7. The big data security risk assessment method according to claim 6, characterized in that: The method further comprises: Symbol conversion: convert the memory or register when running to the second code line into a symbolic expression, which is recorded as the second symbolic expression; Feature extraction: extract the features of the second symbolic expression, recorded as the first feature; Fifth judgment: judging whether the first feature is the same as any vulnerability feature in the preset vulnerability feature set, if so, executing the early warning step; if not, executing the step of adding features; Adding features: adding the first feature to the vulnerability feature set, and executing the early warning step, wherein the early warning step also includes outputting the first feature.

8. The big data security risk assessment method according to any one of claims 5 to 7, characterized in that: The method further comprises: Format conversion: Use a compiler to convert the risk program into a binary format, record the converted risk program as a new risk program, and execute the vulnerability search step.

9. A big data security risk assessment system, characterized in that: include: processor and memory, The memory stores program code; When the processor calls the program code in the memory, the steps of the method according to any one of claims 1 to 8 are executed.

Images