Abnormal modeling behavior detection method based on goal-driven and data mining

Abstract

The application discloses a target-driven and data-mining-based abnormal modeling behavior detection method, relates to target-oriented requirement modeling analysis and data mining technology, and comprises the following steps: firstly, a target-oriented requirement modeling analysis technology is used to model normal modeling behaviors of a modeler; then, data mining technology is used to mine and identify real modeling operation data, to obtain normal modeling behavior patterns from the data, and to verify the normal modeling behaviors defined in the model of the previous step; finally, the screened normal modeling behavior patterns are used to detect abnormal modeling behaviors in the modeling operation data. The application is suitable for collecting modeling process data of the modeler on a modeling platform, modeling the modeling behaviors of the modeler to obtain a normal modeling behavior target model, then combining real modeling process data to extract normal modeling behavior patterns to form a normal modeling behavior pattern set, and finally detecting an abnormal modeling behavior process.

Description

TECHNICAL FIELD

[0001] The present application relates to goal-oriented requirement modeling analysis and data mining techniques. First, the goal-oriented requirement modeling analysis technique is used to model the normal modeling behavior of modelers. Then, the data mining technique is used to mine and identify the real modeling operation data, to obtain the normal modeling behavior pattern from the data, and to verify the normal modeling behavior defined in the model of the previous step. Finally, the normal modeling behavior pattern after screening is used to detect abnormal modeling behavior in the modeling operation data. BACKGROUND

[0002] In the fields of science, technology, engineering, mathematics, etc., it is very important to learn the modeling techniques of model-driven engineering. Therefore, it is very important to improve the quality and efficiency of modeling teaching for modeling education and practice. The existing methods mainly focus on evaluating the results of modeling rather than mining the knowledge hidden behind the modeling process. Therefore, it is very important to explore the modeling process data and mine the modeling problems hidden behind the modeling behavior for improving the quality of modeling teaching.

[0003] In the past two decades, goal-oriented requirement engineering has received extensive attention. In this field, goals are used as a useful conceptualization to elicit modeling and analyzing requirements, capturing alternatives and conflicts. Further explanation, goals are elicited and conceptualized according to some form of model. Goal-oriented requirement engineering analysis contains two commonly used analysis methods. The first is to elicit new goals by asking "why" questions. Compared with the determination of high-level goals, basic goals are easy to determine from the initial material. And high-level goals are elicited by asking "why" questions about the current goals and answering them. The second is to elicit new goals by asking "how to do" questions. For a goal model diagram, a large number of goals need to be refined until any situation in the future software and environment can be represented by sub-goals. Therefore, the highest level of goals becomes requirements in the previous step, and becomes the assumption premise in the process of goal refinement. More specific goals can be determined by asking "how to do" questions. This patent will use goal models and combine domain knowledge to model the modeling behavior of modelers.

[0004] Data mining is a task of mining information from database to make decision or explore. One of the most popular tasks under data mining is sequential pattern mining. Sequential pattern mining is a core task of exploring interesting, unknown, potential and useful patterns in database. Sequential pattern mining focuses on the order of events or elements to mine interesting patterns. The confidence of the pattern can be determined by various indicators, such as frequency, length, etc. The present patent focuses on the frequency of modeling behavior and other features in the log information, and explores and mines the normal modeling behavior pattern from the real modeling process data. SUMMARY

[0005] The present patent proposes a set of target-driven and data-mining-based abnormal modeling behavior detection methods, which are suitable for collecting modeling process data of modelers on a modeling platform, modeling the modeling behavior of the modelers to obtain a normal modeling behavior target model, then combining the real modeling process data to extract normal modeling behavior patterns to form a normal modeling behavior pattern set, and finally detecting the abnormal modeling behavior process.

[0006] The present patent comprises the following steps:

[0007] Step 1, based on the domain knowledge of data flow graph, from the top-down perspective, systematically build a target model of normal modeling behavior of data flow graph for modelers, and design a target expression to represent the order relationship of normal modeling behavior.

[0008] Step 2, focus on the real modeling process information in the log information, pre-process the modeling process information, then focus on the features of the continuous occurrence frequency of internal modeling operations of modeling behavior and the occurrence frequency of modeling behavior patterns, use the method of data mining to mine a frequent modeling behavior pattern set from the real modeling process data, then identify a normal modeling behavior pattern set from the frequent modeling behavior pattern set, and verify and adjust the target model in step 1.

[0009] Step 3, use the normal modeling behavior pattern set obtained in step 2 to capture normal modeling behavior, and the part not captured in the modeling process data is abnormal modeling behavior.

[0010] Further, the process of step 1 of building a target model of normal modeling behavior of data flow graph for modelers based on the domain knowledge of data flow graph, and designing a target expression to represent the order relationship of normal modeling behavior, specifically comprises:

[0011] Step 1.1, set the top target of the target model. Because the normal modeling behavior of data flow graph is studied, the top target is set as "the necessary modeling behavior sequence of modelers building data flow graph".

[0012] Step 1.2, determine two kinds of modeling operations that need to be logged. The first kind is the operation that directly acts on the data flow graph elements. The second kind is the modeling operation that does not directly act on the data flow graph elements, but has an impact on the modeling process.

[0013] Step 1.3, determine the modeling operation that can be added to the target model of normal modeling behavior among the modeling operations that are logged. The modeling operation that is logged is divided into three categories of strong correlation, correlation and weak correlation according to the degree of relevance to the top goal of the target model. The strong correlation modeling operation is the operation that directly acts on the data flow graph elements and frequently appears in the modeling process according to the top goal. The related modeling operation is the operation remaining after the strong correlation operation is removed from the operation that directly acts on the data flow graph elements. The weak correlation operation is the operation that does not directly act on the data flow graph elements. We add the strong correlation modeling operation to the target model.

[0014] Step 1.4, build the target model of normal modeling behavior. The strong correlation modeling operation added to the target model is modeled from the perspectives of action and goal. From the perspective of action, the strong correlation modeling operation can be abstracted as addition and modification at a high level of the target model. From the perspective of goal, the actions of processing, data storage and external entity in the data flow graph are the same, so these three elements can be abstracted as nodes. Then the nodes and the actions of data flow in the data flow graph are similar at a high level, so the nodes and the data flow can be abstracted as elements.

[0015] Step 1.5, represent the goal expression of the order relationship between goals. Define the order relationship between goals, and set the goal expression for the goals in the target model except the leaf nodes (the strong correlation modeling operation added to the target model) by using the defined order relationship.

[0016] Further, the step 2 pre-processes the modeling process information, then uses the method of data mining to mine the set of frequent modeling behavior patterns from the real modeling process data, and then identifies the set of normal modeling behavior patterns from the set of frequent modeling behavior patterns and verifies and adjusts the target model in step 1, which specifically includes:

[0017] Step 2.1, pre-process the real modeling process data according to the characteristics of the modeling process constructed by the modeler. Set the time interval between two modeling operations ≤ 300s. If it exceeds 300s, the second modeling operation is taken as the start of a new modeling sequence. At the same time, the refresh canvas operation also needs to be taken as the start of a new modeling sequence due to its special significance. In order to ensure the completeness of the modeling behavior represented by the modeling sequence, the length of the modeling sequence is set to ≥ 10.

[0018] Step 2.2, mining frequent modeling behavior patterns from modeling process data. In the modeling sequence set obtained after step 2.1, if the same modeling operation appears continuously for 2 times or more, then the sub-modeling operation string of the continuous appearance of this operation is expressed by this operation and a plus sign.

[0019] Step 2.3, in the modeling sequence set obtained after step 2.2, all sub-modeling operation strings with length ≥ 2 are mined, and the frequency of the appearance of these sub-modeling operation strings in different operation sequences is recorded, the sub-modeling operation strings with frequency ≥ 2 are added to the set to constitute the frequent modeling behavior pattern set.

[0020] Step 2.4, in the frequent modeling behavior pattern set obtained after step 2.3, first, the normal modeling behavior patterns are identified by using a semi-automatic method. First, if the frequent modeling behavior patterns appear in all modeling sequences with a frequency more than half (frequency ≥ total number of modeling sequences / 2), they are screened out, and artificial inspection is performed by using domain knowledge to judge whether they are normal modeling behavior patterns. If yes, they are added to the normal modeling behavior pattern set, and if not, they are skipped. Second, the frequent modeling behavior patterns with total number of modeling sequences / 2>frequency≥total number of modeling sequences / 3 are found, and artificial inspection is performed to judge whether they are normal modeling behavior patterns. Third, the frequent modeling behavior patterns with total number of modeling sequences / 3>frequency≥total number of modeling sequences / 10 are found, and artificial inspection is performed to judge whether they are normal modeling behavior patterns. According to the obtained normal modeling behavior pattern set, the target model obtained in step 1 is modified, and a new target expression is designed for the adjusted target model.

[0021] Further, the normal modeling behavior pattern set obtained in step 2 is used in step 3 to capture normal modeling behavior, and the part not captured in the modeling process data is abnormal modeling behavior, which specifically includes:

[0022] Step 3.1, the normal modeling behavior pattern set obtained in step 2 is designed into a regular form.

[0023] Step 3.2, the normal modeling behavior is captured from the modeling process data by using the regular form of the normal modeling behavior pattern.

[0024] Step 3.3, the part of the modeling process data except the captured normal modeling behavior is the abnormal modeling behavior.

[0025] The beneficial effects of the present application are:

[0026] First, the modeling platform for collecting modeling process data in the present application has been put into use in software engineering courses in colleges and universities.

[0027] Secondly, the application proposes an innovative method, which defines and mines normal behavior of modelers from the modeling process of modelers, and then detects abnormal behavior of modelers. From abnormal modeling behavior, many knowledge about modeling process of modelers can be obtained. For example, availability of modeling tools can be explored, and aspects of modeling knowledge of students can be obtained. However, most of the existing methods start from modeling results (modeling homework, etc.), and improve the effectiveness of modeling teaching by evaluating modeling results (scoring modeling homework, etc.). BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 Fig. a: target model local graph - adding processing; Fig. b: target model local graph - modifying data flow, which are two local graphs of the target model graph of normal modeling behavior of the application for constructing data flow graph.

[0029] Figure 2 The algorithm for merging same continuous operation is the algorithm of the application.

[0030] Figure 3 The algorithm for mining frequent pattern is the algorithm of the application.

[0031] Figure 4 The algorithm for detecting abnormal modeling behavior is the algorithm of the application. DETAILED DESCRIPTION

[0032] In the following, a brief overview of the application is given in order to provide a basic understanding of some aspects of the application. It should be understood that this overview is not an exhaustive overview of the application. It is not intended to identify key or important parts of the application nor is it intended to limit the scope of the application. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later.

[0033] Step 1, based on domain knowledge of data flow graph, the target model of normal modeling behavior of modelers is constructed from the top-down perspective, and the target expression is designed to represent the order relationship of normal modeling behavior, and the specific operation is as follows:

[0034] The application takes the necessary modeling behavior sequence of modelers for constructing data flow graph as the top-level target construction target model. The application determines 27 kinds of modeling operations that need to be recorded when constructing a model in a modeling platform, which are all related to the construction of the target model. The operations can be divided into three categories according to the degree of relevance to the target model: strong correlation, correlation, and weak correlation. The application only adds the strongly related operations to the target model. The application combines domain knowledge to model.

[0035] For modeling operation, it is composed of action + object. Action can be abstracted as add and modify in high level. Object is composed of process, data store, external entity, data flow. The first three objects can be abstracted as node, node and data flow can be abstracted as element. First, from high level action add, refine. Add element can be refined as add node or add data flow. Then add node can be refined as add process or add data store or add external entity. Next, from high level action modify, refine. High level target modify element can be refined as modify node or modify data flow. Because modify process, data store and external entity action is modify size, move, edit. So, modify node can be refined as modify node size or move node or edit node. Modify node size can be refined as modify process or data store or external entity size. Move node or edit node and modify node size have the same refine process. Thus, the above steps can build normal modeling behavior target model. Part of target model is shown in Fig. 1a and Fig. 1b. Figure 1

[0036] Next, in order to express the relationship between targets, the present application designs target expression, the specific content is shown in Table 1. Add process and modify data flow two sub-target model graph in target model, respectively shown in Fig. 1a and Fig. 1b. Use add process (such as G7 in Fig. 1a) to explain its target expression definition. Add process must contain add process operation or paste process operation (marked as Q, I), and can contain change process size or move process or edit process (marked as B, E, L). Add process expression is shown in G7 in Fig. 1a. Its meaning is that operation Q or I appears at least once (marked as (Q|I)+), and operation B or E or L appears zero or more times (marked as (B|E|L)*). Use modify data flow (such as G6 in Fig. 1b) to explain its target expression definition. Modify data flow must include edit data flow or move data flow edit area or move data flow head and tail around connection element (marked as D, J, Y). Its target expression meaning is that D or J or Y appears at least once (marked as (D|J|Y)+). Figure 1 Figure 1 Figure 1 Figure 1 Figure 1

[0037] Table 1 target expression syntax

[0038]

[0039] ​​​​​​Step 2, focusing on the real modeling process information in the log information, pre-processing the modeling process information, then focusing on the number of internal modeling operations of modeling behavior and the number of modeling behavior patterns, using data mining methods to mine the frequent modeling behavior pattern set from the real modeling process data, then identifying the normal modeling behavior pattern set from the frequent modeling behavior pattern set, and verifying and adjusting the target model in step 1, the specific operation is as follows:

[0040] Preprocessing the real modeling process data. In order to obtain the modeling operation sequence with continuous operation time, define two operation time intervals ≤ 300s. If it exceeds 300s, the second operation is regarded as the start of a new sequence. Operations with refresh meaning also need to be regarded as the start of a new sequence. After the above operation, in order to ensure that the sequence is meaningful, define the sequence length ≥ 10. If it is less than 10, this sequence will be removed.

[0041] Then, the frequent behavior pattern needs to be mined from the sequence. The present application designs two algorithms to achieve this process. The same continuous modeling operation has no effect on extracting the modeling operation intention, so we design an algorithm to remove the same continuous operation, as shown in Figure 2 In order to obtain frequent behavior patterns, the present application designs a frequent pattern mining algorithm according to the condition that the frequency of behavior patterns ≥ 2 and the length ≥ 2, as shown in Figure 3

[0042] Then, the normal modeling behavior pattern set needs to be identified from the frequent modeling behavior pattern set, and the target model in step 1 needs to be verified and adjusted. In general, after arranging the frequent modeling behavior pattern set in descending order according to the number of patterns appearing in the modeling sequence, manual inspection is performed according to the interval. First, find the interval where the frequent modeling behavior pattern appears more than half of all modeling sequences (frequency ≥ total number of modeling sequences / 2), and use domain knowledge to manually check whether they are normal modeling behavior patterns. If so, add them to the normal modeling behavior pattern set, if not, skip. Second, find the frequent modeling behavior pattern with total number of modeling sequences / 2 > frequency ≥ total number of modeling sequences / 3, and manually check whether they are normal modeling behavior patterns. Third, find the frequent modeling behavior pattern with total number of modeling sequences / 3 > frequency ≥ total number of modeling sequences / 10, and manually check whether they are normal modeling behavior patterns. According to the normal modeling behavior pattern set obtained by the above three steps, modify the target model obtained in step 1, and design a new target expression for the adjusted target model.

[0043] Step 3, using the normal modeling behavior pattern set obtained in step 2 to capture normal modeling behavior, and the part of the modeling process data that is not captured is abnormal modeling behavior, the specific operation is as follows:​

[0044] First, the normal modeling behavior patterns set obtained in step 2 is regularized.

[0045] Then, in real modeling process data, the normal modeling behavior patterns in regular form are used to capture the normal modeling behavior. The part not captured in the modeling process data is the detected abnormal modeling behavior. The algorithm for detecting abnormal modeling behavior is shown in Figure 4

[0046] The above merely describes the preferred embodiments of the present application and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.​

Claims

1. An anomaly modeling behavior detection method based on target-driven and data mining, characterized in that, Includes the following steps: Step 1: Based on the domain knowledge of data flow graphs, systematically construct target models for the normal modeling behavior of data flow graphs from a top-down perspective, and design target expressions to represent the sequential relationships of normal modeling behavior; Step 2 focuses on the actual modeling process information in the log information. The modeling process information is preprocessed, and then the characteristics of the number of consecutive occurrences of modeling operations and the number of occurrences of modeling behavior patterns are observed. Data mining methods are used to mine a set of frequent modeling behavior patterns from the actual modeling process data. Then, a set of normal modeling behavior patterns is identified from the set of frequent modeling behavior patterns, and the target model in Step 1 is verified and adjusted. Step 3: Using the set of normal modeling behavior patterns obtained in Step 2, capture normal modeling behavior, while the part of the modeling process data that is not captured is abnormal modeling behavior. Step 1, based on domain knowledge of the data flow graph, involves constructing a target model for the normal modeling behavior of the modeler in building the data flow graph, and designing a target expression to represent the sequential relationship of the normal modeling behavior. Specifically, this includes: Step 1.1, set the top-level objective of the target model; because the study focuses on the normal modeling behavior domain of data flow graphs, the top-level objective is set as the necessary modeling behavior sequence for the modeler to construct the data flow graph; Step 1.2: Identify two types of modeling operations that need to be logged; the first type is operations that directly affect data flow graph elements; the second type is modeling operations that do not directly affect data flow graph elements but have an impact on the modeling process. Step 1.3: Identify the modeling operations recorded in the logs that will be added to the target model as normal modeling behavior. The recorded modeling operations are categorized into three types based on their relevance to the top-level objective of the target model: strongly related, related, and weakly related. Strongly related modeling operations are those that directly affect data flow graph elements and frequently appear in the modeling process based on the top-level objective. Related modeling operations are those remaining after removing strongly related operations from those that directly affect data flow graph elements. Weakly related operations are those that do not directly affect data flow graph elements. The strongly related modeling operations are then added to the target model. Step 1.4: Construct the target model for normal modeling behavior; add strongly related modeling operations to the target model and model them from both action and target perspectives; from the action perspective, strongly related modeling operations are abstracted as addition and modification at the high level of the target model; from the target perspective, the actions of the three elements of processing, data storage, and external entities in the data flow diagram are the same, so these three elements are abstracted as nodes; then, the actions of nodes and data flows in the data flow diagram are similar at the high level, so nodes and data flows are abstracted as elements. Step 1.5: Define the target expression representing the sequential relationship between targets; define the sequential relationship between targets, and use the defined sequential relationship to set the target expression for targets in the target model excluding leaf nodes.

2. The anomaly modeling behavior detection method based on target-driven and data mining according to claim 1, characterized in that, Step 2 involves preprocessing the modeling process information, then using data mining methods to extract a set of frequent modeling behavior patterns from the real modeling process data. Next, a set of normal modeling behavior patterns is identified from this set, and the target model from Step 1 is verified and adjusted. Specifically, this includes: Step 2.1: Based on the characteristics of the modeler's model building process, preprocess the data of the actual modeling process; set the time interval between two modeling operations to ≤300s; if it exceeds 300s, the second modeling operation is taken as the start of a new modeling sequence; at the same time, the canvas refresh operation, due to its special significance, also needs to be taken as the start of a new modeling sequence; in order to ensure the integrity of the modeling behavior represented by the modeling sequence, the length of the modeling sequence is set to ≥10. Step 2.2: Mine frequent modeling behavior patterns from the modeling process data; In the modeling sequence set obtained after processing in Step 2.1, if the same modeling operation appears twice or more consecutively, the sub-modeling operation strings that appear consecutively with this operation are represented by this operation and a plus sign; Step 2.3: In the modeling sequence set obtained after processing in Step 2.2, all sub-modeling operation strings with a length ≥ 2 are mined, and the frequency of these sub-modeling operation strings in different operation sequences is recorded. Sub-modeling operation strings with a frequency ≥ 2 are added to the set to form a set of frequent modeling behavior patterns. Step 2.4: In the set of frequent modeling behavior patterns obtained after processing in Step 2.3, normal modeling behavior patterns are first identified using a semi-automatic method. First, if a frequent modeling behavior pattern appears more than half the time in all modeling sequences (i.e., frequency ≥ total number of modeling sequences / 2), it is filtered out and manually checked using domain knowledge to determine whether it is a normal modeling behavior pattern. If it is, it is added to the set of normal modeling behavior patterns; otherwise, it is skipped. Second, frequent modeling behavior patterns where total number of modeling sequences / 2 > frequency ≥ total number of modeling sequences / 3 are found and manually checked to determine whether they are normal modeling behavior patterns. Third, frequent modeling behavior patterns where total number of modeling sequences / 3 > frequency ≥ total number of modeling sequences / 10 are found and manually checked to determine whether they are normal modeling behavior patterns. Based on the obtained set of normal modeling behavior patterns, the target model obtained in Step 1 is modified, and a new target expression is designed for the adjusted target model.

3. The anomaly modeling behavior detection method based on target-driven and data mining according to claim 1, characterized in that, In step 3, the normal modeling behavior pattern set obtained in step 2 is used to capture normal modeling behavior. The portion of the modeling process data not captured represents abnormal modeling behavior, specifically including: Step 3.1: Design the set of normal modeling behavior patterns obtained in Step 2 into a regular form; Step 3.2: Use regular expressions to capture normal modeling behavior patterns from the modeling process data; Step 3.3: In the modeling process data, the portion other than the captured normal modeling behavior is considered abnormal modeling behavior.

Images