A platform for software engineering capability assessment based on behavior and learning data

Abstract

The present invention proposes a platform for software engineering capability assessment based on behavior and learning data, including: a data collection module, a database, a data analysis module, a capability evaluation module and a data visualization module; the data collection module is used to collect the user's software engineering learning behavior data and store it in the database as raw data; the data analysis module processes the raw data into a unified protocol format data specified by the capability evaluation module, and stores it in the database; the capability evaluation module imports the unified protocol format data into a pre-built capability evaluation model to obtain the capability evaluation result, and stores it in the database; the data visualization module reads the corresponding data from the database according to user needs for visual display. The present invention can analyze the learning progress characteristics of students based on learning behavior data, so that students can understand their own learning characteristics, and teachers can understand the learning progress and characteristics of different students, and then conduct characteristic lesson preparation and teaching.

Description

Technical Field

[0001] The present invention relates to the field of software engineering and educational technology, and in particular to a platform for evaluating software engineering capabilities based on behavior and learning data. Background Art

[0002] Online education is a combination of Internet technology and traditional education. Teaching relies on the Internet, which fundamentally breaks the limitations of space and time, improves teaching efficiency to a certain extent, and makes learning more humane and life-oriented. Current course websites focus on course learning and course management, and by simulating each link of offline teaching, every online user can learn in the same way.

[0003] However, in software engineering education, traditional MOOCs can no longer meet the requirements. In software engineering education, there are many knowledge points and students need to practice to master them. The existing platforms do not have the ability to analyze student data in detail, nor can they evaluate students' practical activities, so as to adjust teachers' teaching plans according to students' learning progress. Summary of the invention

[0004] Purpose of the invention: To make up for the deficiencies of the prior art, the present invention proposes a platform for software engineering ability assessment based on behavior and learning data. Based on the learning behavior data generated by students in the process of learning software engineering courses, the learning progress characteristics of students can be analyzed, so that students can understand their own learning characteristics, and teachers can understand the learning progress and characteristics of different students, and then conduct characteristic lesson preparation and teaching.

[0005] Technical solution: To achieve the above technical effects, the present invention proposes a platform for software engineering capability assessment based on behavior and learning data, including: a data collection module, a database, a data analysis module, a capability evaluation module and a data visualization module;

[0006] The data collection module is used to collect the user's software engineering learning behavior data and store it in the database as raw data;

[0007] The data analysis module processes the original data into the unified format data specified by the capability evaluation module and stores it in the database;

[0008] The capability evaluation module imports the data in the unified protocol format into the pre-built capability evaluation model, obtains the capability evaluation results, and stores them in the database;

[0009] The data visualization module reads the corresponding data from the database for visualization according to user needs.

[0010] Several optional methods are also provided below, but they are not intended to be additional limitations on the above-mentioned overall solution, but are merely further supplements or preferences. Under the premise that there are no technical or logical contradictions, each optional method can be combined with the above-mentioned overall solution separately, and multiple optional methods can also be combined.

[0011] Optionally, the software engineering learning behavior data includes: learning event information, question training data, examinations, document scoring data, code submission data, and identity information.

[0012] Optionally, the visualization includes: visualization of raw data, visualization of data in a unified protocol format, and visualization of capability assessment results.

[0013] Optionally, the data analysis module uses a 5W1H analysis method to analyze the original data to obtain unified protocol format data, in which the attributes of the event are retained, and the attributes include: person, time, object, operation, and description.

[0014] Optionally, the capability evaluation model is constructed by:

[0015] Set up an evaluation index system, including setting up first-level indicators and subdividing each first-level indicator into multiple dimensions, and designing a scoring method for each subdivided indicator; the first-level indicators include: demand, design, construction, testing, operation and maintenance, and human-computer interaction, and the dimensions include: foundation, technology, process, tool, and quality;

[0016] The fuzzy hierarchical analysis method is used to calculate the indicator weight relationship; then the established indicator system is used as the evaluation factor set, the calculated indicator weight relationship is used as the weight set, the comment set is set, and a fuzzy comprehensive evaluation model is constructed. The fuzzy evaluation results of user data can be obtained through the fuzzy comprehensive evaluation model.

[0017] Optionally, a single-dimensional clustering rating model is also introduced into the capability evaluation model, that is, based on the evaluation index system, the K-means++ algorithm is used to calculate the grade of each single-dimensional index in the overall evaluation result.

[0018] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0019] The present invention proposes a software engineering teaching / learning auxiliary platform, which, by constructing a unique capability evaluation model, can quickly analyze the collected data on the premise of fully collecting student learning behavior data to obtain the students' learning progress characteristics, so that the students themselves can understand their own learning characteristics and thus adjust their learning progress; it can allow teachers to understand the learning progress and characteristics of different students so as to carry out characteristic lesson preparation and teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A diagram of the system structure involved in the embodiment. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. However, it should be understood that the present invention can be implemented in various forms, and some exemplary and non-limiting embodiments shown in the accompanying drawings and described below are not intended to limit the present invention to the specific embodiments described.

[0022] It should be understood that, where technically feasible, the technical features listed above for different embodiments can be combined with each other to form other embodiments within the scope of the present invention. In addition, the specific examples and embodiments described in the present invention are non-limiting, and the structures, steps, and sequences described above can be modified accordingly without departing from the scope of protection of the present invention.

[0023] Example:

[0024] Figure 1 An exemplary system structure diagram of the present invention is shown, including a data collection module, a database, a data analysis module, a capability evaluation module and a data visualization module. The database is MySQL. The data collection module is used to collect the user's software engineering learning behavior data and store it in the database as raw data; the data analysis module processes the raw data into the unified protocol format data specified by the capability evaluation module, and stores it in the database; the capability evaluation module imports the unified protocol format data into the pre-built capability evaluation model to obtain the capability evaluation results, and stores them in the database; the data visualization module reads the corresponding data from the database according to user needs for visual display. Since SEEC (Software Engineering Education Cloud) Software Engineering Education Cloud has provided software engineering courses and development teaching practice platforms and a set of related subsystems, in this embodiment, the SEEC platform is cited as the data collection object.

[0025] Several key technologies involved in this embodiment are: determination of capability evaluation indicators (involving FAHP fuzzy hierarchical analysis method, iceberg model, 5W1H analysis method), K-means++ clustering algorithm, continuous integration tool Jenkins, and code quality analysis tool Sonarqube.

[0026] 1. FAHP (Fuzzy Analytic Hierarchy Process)

[0027] The Analytic Hierarchy Process (AHP) was proposed by famous operations researcher TL Satty and others in the 1970s. The AHP is a multi-objective decision analysis method that combines qualitative and quantitative analysis methods. The AHP decomposes the target into multiple hierarchical relationships such as criteria, indicators, and plans, and conducts qualitative and quantitative analysis to obtain decision results. The specific steps of AHP are: (1) Establishing a hierarchical indicator system; (2) Obtaining a judgment matrix; (3) Performing a single hierarchical sorting; (4) Performing a total hierarchical sorting. The judgment matrix is ​​obtained by comparing the importance of indicators two by two. The hierarchical sorting process is to calculate the weight relationship of indicators. The fuzzy AHP is based on the AHP and improves the shortcomings of the traditional AHP that it is difficult to ensure the consistency of thinking when there are too many indicators. The fuzzy AHP improves the way the AHP obtains the judgment matrix. The AHP uses exact numbers to measure the relationship between elements, while the fuzzy AHP uses fuzzy numbers to measure the relationship between elements. Therefore, the fuzzy AHP uses a fuzzy consistent matrix as the judgment matrix.

[0028] The relevant definition of the fuzzy consistent matrix is ​​as follows:

[0029] Let the matrix R = (r ij ) n×n .

[0030] Definition (1): Satisfy 0≤r ij ≤1, (i, j = 1, 2..., n), R is the fuzzy matrix.

[0031] Definition (2): satisfy r ij +r ji =1, (i, j=1, 2, ..., n), R is the fuzzy complementary matrix.

[0032] Definition (3): The fuzzy matrix R satisfies r ij =r ik -r jk +0.5, R is the fuzzy consistent matrix.

[0033] Theorem: r i is the row sum of the fuzzy complementary matrix R, through The matrix after mathematical transformation is a fuzzy consistent matrix.

[0034] After obtaining the index weight ranking, the fuzzy comprehensive evaluation method can be used to evaluate the final results. The fuzzy comprehensive evaluation method requires the selection of appropriate membership functions and operators, the calculation of the obtained fuzzy evaluation matrix and matrix, and finally the fuzzy evaluation result is obtained to make a decision on the specified plan.

[0035] 2. Iceberg Model

[0036] In the evaluation of software engineering capabilities, in addition to determining the evaluation indicators, we also need to establish a capability evaluation model. Capabilities belong to the category of psychology, usually referring to the skills and art of completing certain activities, including the specific ways to complete certain activities and the psychological characteristics required to successfully complete certain activities. The "Iceberg Model" proposed by the famous American psychologist McClelland is one of the more famous capability quality models. Nowadays, the way companies recruit talents is not entirely to examine professional capabilities, but also to conduct many professional quality tests. It can be seen that the criteria for judging capabilities are not only reflected in knowledge and skills. Therefore, when establishing a software engineering capability evaluation model, the "Iceberg Model" is also used to refer to the collection of user behavior information. The Iceberg Model compares the comprehensive capabilities of the testee to an iceberg. The part above the iceberg is the external performance that is easy to examine in real life, such as basic knowledge, professional skills, etc. These parts are relatively easier to change and develop through training. The part below the iceberg often represents the inherent characteristics of people, which are not easy to measure and are not easy to change through external influences, such as social roles, self-cognition, traits and motivation.

[0037] 3. 5W1H analysis method (data analysis module)

[0038] In order to examine the characteristics under the iceberg of students, the system needs to collect user behavior information in each subsystem. In order to unify the information content, the 5W1H analysis method was referred to during the collection, and various types of information were defined as behavioral event information, and the attribute standards of the information were defined. The "5W1H" analysis method is to raise questions and think about the selected projects, processes or operations from six aspects: cause (WHY), object (WHAT), place (WHERE), time (WHEN), person (WHO), and method (HOW). This simple and direct way of questioning and thinking can deepen and scientificize the content of thinking. In the face of event information collection in the system, this article adaptively modified 5W1H, removed the cause (WHY), and retained the five attributes of the event: person, time, object, operation, and description.

[0039] 4. K-means++ clustering algorithm

[0040] When analyzing students' single-dimensional abilities, the simple and effective K-means clustering algorithm can be used for grade evaluation, so that students can understand the position of their single-dimensional abilities in the overall situation. The advantage of the K-means algorithm is its simplicity and high efficiency. Its basic idea is: select the initial cluster center; calculate the distance of the remaining points from each cluster center and divide them into corresponding categories; recalculate the center of each category and divide each point again until the cluster center no longer changes.

[0041] Although K-means is simple and efficient, it also has problems such as the selection of K value and the rationality of the initial center selection. The K-means++ algorithm improves the way the initial cluster center is selected in the K-means algorithm, and improves the final error of the classification result. The basic idea of ​​selecting the initial cluster center of K-means++ is that the distance between the initial centers is as far as possible. The steps for selecting the initial cluster center are:

[0042] (1) Randomly select a sample as the first cluster center;

[0043] (2) Calculate the shortest distance between each sample and the existing cluster center, and calculate the probability of each sample being selected as the next cluster center. Use the roulette wheel method to select the next cluster center;

[0044] (3) Repeat steps (2) until all K cluster centers are selected.

[0045] 5. Jenkins

[0046] Jenkins is an open source continuous integration (CI) tool that provides a user-friendly interface. It originated from Hudson and is mainly used for continuous and automatic building / testing of software projects and monitoring the operation of external tasks. Jenkins is written in Java and can run in popular servlet containers such as Tomcat or independently. It is usually used in conjunction with version control tools (SCM) and build tools. Commonly used version control tools include SVN and GIT, and build tools include Maven, Ant, and Gradle. In addition to using continuous integration delivery, you can also add Sonarqube quality inspection steps to Jenkins' Pipeline. You can then query the inspection results and perform secondary analysis through the web API provided by Sonarqube.

[0047] 6. Sonarqube

[0048] As software products in the IT industry are constantly updated, customers have higher and higher requirements for software products. Therefore, how to deliver high-quality software code, analyze code quality in a timely and continuous manner, and provide reasonable solutions has become a problem that must be solved at present. SonarQube is an open source platform for managing the quality of source code. SonarQube is not only a quality data reporting tool, but also a code quality management platform. It supports code quality management and detection in more than 20 programming languages, such as Java, C#, C / C++, PL / SQL, Cobol, JavaScript, Groovy, etc. In current enterprise development, more and more people use SonarQube to check the quality of code, helping developers to find code defects in time, improve code quality, and increase development speed. SonarQube supports plug-ins, and can also be placed on the server in B / S mode to generate analysis reports. SonarQube provides a web API for querying code analysis results, which can be used for secondary use.

[0049] 7. Construction of capability assessment model and capability assessment

[0050] According to the idea of ​​AHP, and referring to the research on software engineering professional curriculum system in the field of software engineering education, the software engineering capability measurement standards are divided into six aspects: requirements, design, construction, testing, operation and maintenance, and human-computer interaction. The second-level learning measurement standards are subdivided for each indicator dimension: foundation, technology, process, tool, and quality, forming the following software engineering capability evaluation index table.

[0051] Base technology process tool quality need design structure test Operations Human-computer interaction

[0052] For each secondary indicator under each primary indicator dimension, a corresponding scoring method is designed, as shown in the following table.

[0053]

[0054]

[0055] Next, we calculate the indicator weights. The fuzzy analytic hierarchy process is used to calculate the indicator weights. First, we need to establish the priority relationship matrix of the indicators. The platform uses a pairwise importance judgment method and a quantitative scale of 0.1 to 0.9 as shown in the table below to build a priority relationship matrix between indicators.

[0056]

[0057] The priority relationship matrix, namely the aforementioned fuzzy complementary matrix, is constructed. Since the secondary indexes use the same attribute measurement relative to the primary indexes, two priority relationship matrices are generated in total.

[0058] First-level indicator priority relationship matrix

[0059]

[0060]

[0061] Secondary indicator priority relationship matrix

[0062] <![CDATA[A i1 ]]> <![CDATA[A i2 ]]> <![CDATA[A i3 ]]> <![CDATA[A i4 ]]> <![CDATA[A i5 ]]> <![CDATA[p i ]]> <![CDATA[A i1 ]]> 0.5 0.7 0.8 0.8 0.8 3.6 <![CDATA[A i2 ]]> 0.3 0.5 0.7 0.6 0.7 2.8 <![CDATA[A i3 ]]> 0.2 0.3 0.5 0.4 0.5 1.9 <![CDATA[A i4 ]]> 0.2 0.4 0.6 0.5 0.6 2.3 <![CDATA[A i5 ]]> 0.2 0.3 0.5 0.4 0.5 1.9

[0063] The fuzzy consistency matrix is ​​calculated according to the above formula and used for the subsequent calculation of indicator weights. The fuzzy consistency matrix is ​​as follows.

[0064] First-level indicator fuzzy consistency matrix

[0065]

[0066] Secondary index fuzzy consistency matrix

[0067]

[0068]

[0069] The weight is calculated using the following weight calculation formula:

[0070]

[0071] represents the weight of the i-th secondary indicator under the k-th primary indicator, and the parameter a satisfies The condition is, n represents the hierarchy of the fuzzy consistent judgment matrix. The resolution of the weight ranking calculation results can be improved. In this way, the fuzzy consistent judgment matrix can be converted into the weight matrix of the indicator.

[0072] After calculating the weights of each indicator, according to the fuzzy comprehensive evaluation method, it is also necessary to construct a fuzzy evaluation matrix. The process of fuzzy comprehensive evaluation requires the clarification of the evaluation factor set, comment set and indicator weight set.

[0073] The evaluation factor set has been described above, including the first-level indicator set {requirements, design, construction, testing, operation and maintenance, human-computer interaction}, and each item in the first-level indicator set corresponds to a second-level indicator set {foundation, technology, process, tool, quality}.

[0074] The comment set mainly describes the results of software engineering capability assessment. This platform divides software engineering capability into five levels, namely A, B, C, D, and E. Considering the score range of the indicator and the convenience of subsequent calculation, the numerical intervals of the five levels are determined as [1, 0.8], (0.8, 0.6], (0.6, 0.4], (0.4, 0.2], (0.2, 0].

[0075] The weight set can be obtained by the above method, and the first-level indicator weight set can be defined as The secondary indicator weight set can be defined as

[0076] The construction of the fuzzy evaluation matrix requires measuring the influence of each indicator in the evaluation factor set on each level in the review set, which is also called the membership degree of the factor element to the review set element. We can construct the fuzzy evaluation matrix U = {u ij} nxm , where n is the number of indicator elements in the factor set, and m is the number of evaluation levels in the comment set. The definition of the membership function is as follows:

[0077]

[0078]

[0079]

[0080]

[0081]

[0082] Among them, x i is the value of the i-th evaluation index of the evaluation object.

[0083] After obtaining the fuzzy relationship between the factor set and the comment set (i.e., the fuzzy evaluation matrix U), we need to select a suitable fuzzy synthesis operator to synthesize the previously obtained indicator weight vector with the fuzzy relationship matrix to obtain the fuzzy comprehensive evaluation vector of the evaluation object. The fuzzy operator of weighted summation finally obtains the fuzzy comprehensive evaluation model B=wU=(b1, b2, ..., b n ). When calculating the final evaluation result Q, it is necessary to i After normalization, the calculation formula of Q is obtained as follows:

[0084]

[0085] Where n is the number of elements in the comment set, and Q is the final evaluation result of the student's software engineering ability.

[0086] Experimental Results

[0087] Since the modules of the data analysis subsystem support each other, the user-oriented use cases are mainly focused on information modification, task creation, and viewing of learning analysis views. Therefore, different methods are used when testing different modules.

[0088] In this experiment, both pull and push methods were used in the data collection process. In the test of the method of pulling data through OpenFeign, Junit test was mainly used to check whether the acquisition results met expectations. In the test of the method of collecting data reported by other subsystems through OpenFeign (that is, data pushed by other subsystems), the Postman tool was mainly used to call the system interface to check whether the data was successfully reported and entered into the database for persistence. As shown in Table 1, the method of the data collection module passed all tests and can collect data normally.

[0089] Table 1 External data collection test case table

[0090]

[0091]

[0092] In addition to the data generated in other systems, there is also a part of data generated from this system. One is the learning task event data, and the other is the student identity data. As shown in Table 2, the student task section function and the information section function work normally.

[0093] Table 2 Internal data collection test case table

[0094]

[0095]

[0096] The results generated in the data analysis module cannot be fully visible. Part of them are used for system visualization, while part of them are used for the establishment of the evaluation model and capability evaluation in the capability evaluation module. When testing the analysis module, capability evaluation module, and visualization module, the main verification is the overall function, that is, the capability view display and capability evaluation report display. As shown in Table 3, the data analysis module, capability evaluation module, and data visualization module passed all tests.

[0097] Table 3 Data interaction test case table

[0098]

[0099] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for evaluating software engineering capabilities based on behavior and learning data, characterized in that: include: Data collection module, database, data analysis module, capability assessment module and data visualization module; The database used is MySQL; The data collection module is used to collect the user's software engineering learning behavior data and store it in the database as raw data; the data analysis module uses the 5W1H analysis method to process the raw data into the unified protocol format data specified by the capability assessment module and store it in the database; the capability assessment module imports the unified protocol format data into the pre-built capability assessment model to obtain the capability assessment results and store them in the database; The data visualization module reads the corresponding data from the database for visualization according to user needs; the Software Engineering Education Cloud SEEC platform is used as the data collection object; The method for constructing the capability assessment model is as follows: setting an assessment indicator system, including setting a first-level indicator and subdividing each first-level indicator into multiple dimensions, and designing a scoring method for each subdivided indicator; The fuzzy analytic hierarchy process is used to calculate the indicator weight relationship; then the established indicator system is used as the evaluation factor set, the calculated indicator weight relationship is used as the weight set, the comment set is set, and the fuzzy comprehensive evaluation model is constructed. The fuzzy evaluation results of user data are obtained through the fuzzy comprehensive evaluation model; The capability assessment model also introduces a single-dimensional clustering rating model, that is, based on the assessment indicator system, the K-means++ algorithm is used to calculate the level of each single-dimensional indicator in the overall assessment result; The weight is calculated using the following weight calculation formula: represents the weight of the i-th secondary indicator under the k-th primary indicator, and the parameter a satisfies The condition, n represents the hierarchy of the fuzzy consistent judgment matrix; when Improve the resolution of weight ranking calculation results; convert the fuzzy consistent judgment matrix into the weight matrix of indicators; After calculating the weights of each indicator, according to the fuzzy comprehensive evaluation method, a fuzzy evaluation matrix needs to be constructed; the fuzzy comprehensive evaluation process requires the clarification of the evaluation factor set, comment set and indicator weight set; The evaluation factor set includes the first-level indicator set: requirements, design, construction, testing, operation and maintenance, and human-computer interaction; Each item in the primary indicator set corresponds to a secondary indicator set: foundation, technology, process, tool, quality; The comment set is a description of the results of software engineering capability assessment. The software engineering capability is divided into five levels, namely A, B, C, D, and E. Considering the score range of the indicator and the convenience of subsequent calculation, the numerical intervals of the five levels are determined as [1, 0.8], (0.8, 0.6], (0.6, 0.4], (0.4, 0.2], (0.2, 0]; The weight set is obtained through the above method, and the first-level indicator weight set is defined as The secondary indicator weight set is defined as The construction of the fuzzy evaluation matrix requires measuring the impact of each indicator in the evaluation factor set on each level in the comment set. The elements of these factors belong to the elements of the comment set. Constructing the fuzzy evaluation matrix U = {u ij } nxm , where n is the number of indicator elements in the evaluation factor set, and m is the number of evaluation levels in the comment set; the definition of the membership function is as follows: Among them, x i is the value of the i-th evaluation indicator of the evaluation object; After obtaining the fuzzy relationship between the factor set and the comment set, i.e., the fuzzy evaluation matrix U, it is necessary to select a suitable fuzzy synthesis operator to synthesize the previously obtained indicator weight vector with the fuzzy relationship matrix, thereby obtaining the fuzzy comprehensive evaluation vector of the evaluation object; The fuzzy operator of weighted summation finally obtains the fuzzy comprehensive evaluation model B = wU = (b1, b2, ..., b n ); When calculating the final evaluation result Q, it is necessary to i After normalization, the calculation formula of Q is obtained as follows: Among them, n is the number of elements in the comment set; Q is the final software engineering ability evaluation result of the students.

2. The method for software engineering capability assessment according to claim 1, characterized in that: The visualization display includes: visualization display of original data, visualization display of unified protocol format data, and visualization display of capability assessment results.

Images