Big data full-process practical training and assessment method and system based on multi-resource collaboration

Abstract

The invention relates to the technical field of education, and discloses a big data full-process practical training and assessment method and system based on multi-resource collaboration, and the method comprises the steps: capturing an operation sequence executed by a student for completing a practical training task in real time in a big data practical training environment of various heterogeneous resources, synchronously collecting resource calling data triggered by each operation step in the operation sequence; based on the operation sequence and the resource calling data, constructing a student dynamic behavior graph capable of reflecting the problem solving strategy and the process behavior of the student; performing multi-dimensional quantitative comparison on the student dynamic behavior map and an expert reference map pre-constructed for the same practical training task to generate a comparison analysis result; and according to a comparative analysis result, performing attribution analysis on the operation sequence of the trainee, and finally generating a quantitative score and a diagnosis report of the comprehensive practical ability of the trainee. The method overcomes the limitation that the traditional assessment method only depends on the final result to score, and enables the assessment to be more objective, comprehensive and deep.

Description

Technical Field

[0001] This application relates to the field of educational technology, specifically to a big data-based full-process training and assessment method and system based on multi-resource collaboration. Background Technology

[0002] With the booming development of the big data industry, the demand for talent with big data engineering practice capabilities is increasing. To meet this demand, various big data training platforms have emerged. Existing platforms generally focus on providing students with simulated big data environments and training in functional operations, such as performing single tasks like data cleaning, extraction, transformation, loading, and model training. After students complete the tasks, the platform or instructor typically provides a grade by checking the correctness of the final output data, reports, or models.

[0003] However, while this results-oriented assessment method can determine whether a student answered correctly, it struggles to measure the quality of their work. Specifically, existing technologies are insufficient in evaluating a student's in-depth problem-solving strategies, the efficiency of their operational steps, and the rationality of their utilization of computational and storage resources. For example, one student might obtain the correct result by chance through a series of redundant and inefficient operations (such as full table scans and unreasonable data skew handling), while another student achieves the same goal through concise and efficient algorithms and data structure design. Under the current assessment system, the two might receive similar scores, which clearly fails to accurately reflect the significant differences in their engineering practice abilities.

[0004] In summary, while existing technologies can record some operation logs or monitor isolated resource indicators, they generally lack a mechanism to effectively correlate and accurately attribute fragmented operational behaviors with underlying resource consumption. Furthermore, they cannot conduct in-depth, quantitative, and structured evaluations of the trainees' overall problem-solving process from the perspectives of strategy and effectiveness, resulting in assessment results that are often one-sided and lack effective diagnostic and guidance value. Summary of the Invention

[0005] The purpose of this application is to provide a big data full-process training and assessment method based on multi-resource collaboration, including: in a big data training environment with multiple heterogeneous resources, capturing in real time the operation sequence executed by the trainee to complete the training task, and simultaneously collecting resource call data triggered by each operation step in the operation sequence; based on the operation sequence and the resource call data, constructing a trainee dynamic behavior graph that reflects the trainee's problem-solving strategy and process behavior, wherein the operation steps are mapped to nodes in the trainee dynamic behavior graph, the logical dependencies or temporal relationships between the operation steps are mapped to edges in the trainee dynamic behavior graph, and the resource call data is associated with the nodes that execute the corresponding operation steps; performing a multi-dimensional quantitative comparison between the trainee dynamic behavior graph and a pre-constructed expert benchmark graph for the same training task, generating a comparative analysis result representing the differences between the two in terms of problem-solving path, operation efficiency, and resource effectiveness; based on the comparative analysis result, performing attribution analysis on the trainee's operation sequence, and finally generating a quantitative score and diagnostic report on the trainee's comprehensive practical ability.

[0006] The above technical solution structures the trainees' invisible thinking strategies and operational processes into measurable behavioral maps. By comparing these maps with those of experts, it achieves penetrating analysis from surface-level operational behaviors to deep problem-solving strategies, and from static results to dynamic processes. This overcomes the limitations of traditional assessment methods that rely solely on final results for scoring, making the evaluation more objective, comprehensive, and in-depth.

[0007] Optionally, the real-time capture of the operation sequence performed by the trainee to complete the training task includes: intercepting and structurally recording various interactive instructions, program calls and state change events of the trainee during the entire training task execution period through an event listening agent pre-set in the training environment, so as to form an operation event flow with a temporal relationship, wherein the event listening agent can cover the operating system layer, application software layer and service interface layer.

[0008] By using the above technical solution and pre-setting multi-level event listening agents in the training environment, it is possible to intercept various operation event streams of trainees in a non-intrusive manner in a comprehensive and real-time manner, ensuring that the data foundation for subsequent graph construction is complete and accurate, and without interfering with the trainees' normal operating experience.

[0009] Optionally, in the student dynamic behavior graph, the attributes of the nodes include operation type, execution parameters or key code snippets, and the attributes of the edges include data flow relationship or control flow transmission relationship between nodes.

[0010] Optionally, associating the resource call data with the node executing the corresponding operation step includes binding the key performance indicators generated by each operation step during its execution lifecycle as extended attributes of the node corresponding to that operation step, wherein the key performance indicators include computing resource consumption, peak storage resource usage, or data network transmission overhead.

[0011] By defining rich attributes for the nodes and edges of the student dynamic behavior graph, such as operation type, code snippets, and data flow relationships, and binding key performance indicators such as central processing unit, memory, and input / output as node attributes, the system successfully correlates and attributes scattered operational behaviors with underlying resource consumption. This enables multi-dimensional modeling of the cost and benefits of each operational step for the student, providing a foundation for subsequent performance analysis.

[0012] Optionally, the expert benchmark graph is derived from a benchmark graph library containing a variety of typical problem-solving strategies. The benchmark graph library includes an optimal path graph representing industry best practices, a standard path graph representing standard problem-solving methods, or a typical error graph representing common error patterns.

[0013] By introducing an expert benchmark library containing various typical problem-solving strategies such as optimal paths, standard paths, and typical error paths, the above technical solutions provide objective, scientific, and multi-level reference standards for evaluation. Evaluation is no longer a subjective judgment, but a quantitative comparison between trainees' actual behavior and industry best practices, making the evaluation results more credible.

[0014] Optionally, the multi-dimensional quantitative comparison includes at least structural similarity analysis, that is, using graph theory algorithms to calculate the difference in topological structure between the student's dynamic behavior graph and the expert's benchmark graph, in order to evaluate the degree of conformity between the student's overall problem-solving approach and the expert's approach.

[0015] Optionally, the multi-dimensional quantitative comparison also includes path efficiency analysis, which involves comparing the differences between the student's dynamic behavior graph and the expert benchmark graph in terms of critical path length, total number of nodes, or number of parallel branches to assess whether the student's operation sequence has redundancy, detours, or suboptimal choices.

[0016] Optionally, the multi-dimensional quantitative comparison also includes node performance analysis, that is, comparing the corresponding or logically similar nodes in the student dynamic behavior graph and the expert benchmark graph one by one with their associated resource consumption performance indicators, in order to identify and locate the specific high-consumption operation links that lead to system performance bottlenecks.

[0017] By using the above technical solutions, and through comparative analysis from three core dimensions—structural similarity (the degree of conformity of problem-solving approaches), path efficiency (the redundancy of operational processes), and node effectiveness (the resource consumption of specific operations)—the gap between trainees and experts can be comprehensively and three-dimensionally quantified, and the key operational links that lead to inefficiency or errors can be accurately located.

[0018] Optionally, the step of performing attribution analysis on the student's operation sequence based on the comparative analysis results, and finally generating a quantitative score and diagnostic report on the student's comprehensive practical ability, includes: automatically identifying inefficient or erroneous links in the student's operation sequence that deviate significantly from the expert benchmark map based on the comparative analysis results; performing attribution analysis on the effectiveness of the student's operation steps, the superiority or inferiority of the problem-solving strategy, and the rationality of resource consumption; and generating a personalized diagnostic report containing specific optimization suggestions and ability deficiency analysis by combining the corresponding correct or efficient links in the expert benchmark map.

[0019] The above technical solutions can automatically identify weak links in the trainee's process and, combined with the correct paradigms in the expert graph, generate personalized diagnostic reports that include specific optimization suggestions and analysis of skill gaps. The assessment is no longer a cold score, but becomes an intelligent coach that drives trainees' skill improvement, making the assessment results truly have accurate diagnostic value and practical guiding significance.

[0020] The second objective of this application is to provide a big data full-process training and assessment system based on multi-resource collaboration, comprising: a data capture module, used to capture in real time the operation sequence executed by the trainee to complete the training task in a big data training environment with multiple heterogeneous resources, and synchronously collect resource call data triggered by each operation step in the operation sequence; a graph construction module, connected to the data capture module, used to construct a dynamic behavior graph of the trainee that reflects the trainee's problem-solving strategy and process behavior based on the operation sequence and the resource call data; a comparative analysis module, connected to the graph construction module, used to perform multi-dimensional quantitative comparison between the dynamic behavior graph of the trainee and a pre-constructed expert benchmark graph for the same training task, and generate comparative analysis results representing the differences between the two in terms of problem-solving path, operation efficiency and resource effectiveness; and an evaluation and scoring module, connected to the comparative analysis module, used to perform attribution analysis on the trainee's operation sequence based on the comparative analysis results, and finally generate a quantitative score and diagnostic report on the trainee's comprehensive practical ability. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the big data full-process training and assessment method based on multi-resource collaboration proposed in this application.

[0022] Figure 2This is a flowchart of the big data full-process training and assessment system based on multi-resource collaboration, as described in this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit the scope of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0024] This application uses the analysis of DAU (Daily Active User) and PV (Page Views) of a large website using Apache Spark (a fast and general-purpose computing engine designed for large-scale data processing) as an example to elaborate in detail the complete implementation process of a big data full-process training and assessment method and system based on multi-resource collaboration proposed in this application.

[0025] Specifically, trainees are required to process a terabyte-level raw website access log file in Apache Commons log format using the Spark framework in a pre-configured big data training environment. The task objective is to write and execute a Spark application to accurately calculate the number of unique visitor IPs for a specified day as a proxy metric for DAU and the total number of page requests as a proxy metric for PV, and output the final results in key-value pair text format to a specified HDFS (Hadoop Distributed File System) directory; for example, the output should be in the format of "DAU, 1580293" and "PV, 89345021".

[0026] This training environment is configured with multiple virtual server nodes, forming a cluster that coordinates computing and storage resources. The training environment comes pre-installed with a Linux operating system, Hadoop suite, Spark runtime environment, and commonly used development tools, such as IntelliJ IDEA and other IDEs (Integrated Development Environments) and command-line terminals.

[0027] like Figure 1 As shown in the figure, the big data full-process training and assessment method based on multi-resource collaboration in this application includes the following steps.

[0028] S01: In a big data training environment with multiple heterogeneous resources, the operation sequence executed by the trainee to complete the training task is captured in real time, and the resource call data triggered by each operation step in the operation sequence is collected synchronously. The multiple heterogeneous resources include computing, storage and network.

[0029] Specifically, when trainees begin their practical training tasks, the system's backend initiates a comprehensive monitoring and data capture mechanism. This mechanism records every command the trainee enters in the terminal and every code editing, saving, and compiling operation in the integrated development environment in a non-intrusive manner, forming a complete operation sequence with precise timestamps. Simultaneously, the system collects underlying resource call data triggered during the execution of each critical operation step. For example, when a trainee submits and executes a Spark job, the system records the cumulative CPU (Central Processing Unit) time consumed by the job process, peak memory usage, disk I / O / I bytes, and network data transfer volume.

[0030] To achieve comprehensive, real-time, and non-intrusive capture of student operations, this application pre-configures multi-layered event listening agents in the training environment. These agents intercept and structurally record various interactive commands, program calls, and state change events of the student throughout the entire training task execution, forming a time-series flow of operational events. These event listening agents are designed as lightweight background services or plugins that can cover the operating system layer, application software layer, and service interface layer. They are transparent to the student, ensuring the non-intrusiveness of the capture process.

[0031] Specifically, in the application software layer agent, a customized plugin is pre-installed in the integrated development environment such as IntelliJ IDEA used by the trainees. This plugin can listen to and capture various user behavior events within the IDE, such as which source code file the trainee opened, when the code was modified, when the file was saved, and when the compilation or packaging operation was triggered. By listening to these events, the system can obtain a complete record of the trainee's activities during the coding phase and code snapshots at various points in time.

[0032] In the operating system layer proxy, a hook function for command execution is injected by modifying the shell configuration file in the Linux command-line terminal environment used by the student. This hook function is triggered before and after each user command is actually executed. Before command execution, the hook function captures the complete command string and its parameters, the current working directory, and the execution timestamp. After command execution, it captures the command's exit code, a summary or full text of standard output and standard error, and the command's end timestamp. The exit code is a numerical value indicating whether the execution was successful or failed, typically 0 for success. In this way, every interaction the student makes in the command line, whether it's file operations, environment checks, or task submissions, is accurately recorded.

[0033] In the service interface layer proxy, for critical commands that start background distributed jobs, such as commands to submit Spark applications, the service interface layer proxy's monitoring is further deepened. When the service interface layer proxy captures a student executing such a command, it parses the command parameters to obtain the unique identifier of the submitted application. Subsequently, it periodically interacts with the application interface of the cluster's resource manager to track the job's running status in real time, such as whether it is running, successful, failed, or terminated. At the same time, it can also use the monitoring tools provided by the operating system to monitor the driver and executor processes started by the job, collecting performance counters such as CPU utilization, memory usage, and disk read / write bytes at a set frequency (e.g., once per second). When the job ends, the resource consumption data throughout its entire lifecycle is summarized to obtain key performance indicators such as cumulative CPU seconds, peak memory, and total input / output traffic.

[0034] Understandably, all captured events, whether from the IDE, command line, or resource monitoring, are formatted into a unified structured data record containing event type, timestamp, and detailed load, and sent in real time to the background graph building module in sequence.

[0035] S02: Based on operation sequences and resource call data, construct a dynamic behavior map of the student that reflects the student's problem-solving strategies and process behavior.

[0036] Specifically, the system performs structured processing on the discrete operation event streams and resource data captured in the previous step, constructing a dynamic behavior graph of students that can comprehensively reflect their problem-solving strategies and process behaviors. In this dynamic behavior graph, each specific operation step of the student is mapped to a node in the graph, such as uploading log files to HDFS, writing data cleaning logic, and executing Spark jobs. The logical dependencies or temporal relationships between these operation steps are mapped to edges connecting the nodes. Furthermore, the resource call data collected in the previous step is used as an attribute and bound to the node that performs the corresponding operation step.

[0037] Understandably, the graph construction module is responsible for transforming temporal operation sequences into graphical representations that deeply reveal the learner's underlying logic and strategies. Suppose the system captures the following simplified operation sequence: The learner first creates a directory in HDFS to store input data. Next, they upload local log files to this HDFS directory. Then, they write and save a Spark program in the IDE, whose logic uses the `groupByKey` operator to count PV and DAU respectively. After writing the program, they submit the Spark application via the command line, specifying the input and output paths. The application executes successfully after several minutes. During this period, the system monitors that the Spark job's peak memory usage is 128GB and the cumulative CPU time is 3600 seconds. Finally, the learner uses commands to view the list of output directories and prints the contents of one of the result files to verify its correctness.

[0038] The graph construction module processes this operation sequence as follows: each meaningful operation step is converted into a node in the student's dynamic behavior graph. For example, command execution steps such as creating an HDFS directory, uploading files, submitting Spark applications, and viewing output will generate operation nodes; the code files written and saved by the student can themselves serve as code asset nodes; and file or directory paths involved in the operation, such as input log files and output result directories, will be identified and generate data nodes.

[0039] In the learner dynamic behavior graph, each node contains rich attributes to accurately describe its characteristics. For example, the attributes of a node include operation type, execution parameters, or key code snippets.

[0040] Specifically, the attributes of an operation node should at least include the operation type, the complete command string, the start and end timestamps of the operation, the exit code, and the working directory at the time of execution. The attributes of a code asset node should at least include the programming language, the complete source code text or key logic fragments, and a timestamp associated with the file save event. The programming language includes Scala, etc., and the timestamp associated with the file save event is used for version control. The attributes of a data node should at least include the full path of the file or directory, the data type, and the file size.

[0041] Understandably, for each Spark application submission node, the system binds the resource consumption data collected during its execution as an extended attribute of that node. This extended attribute is a composite attribute, containing multiple key performance indicators, such as computational resource consumption, peak storage resource usage, total disk read bytes, and data network transmission overhead. In this way, the resource overhead caused by the specific program written by the student is precisely and indivisibly bound to the node executing that program.

[0042] In the student dynamic behavior graph, edges are used to represent the relationships between nodes, mainly data flow relationships and control flow transmission relationships. When the output of one operation becomes the input of another operation, the system creates a data flow edge; for example, the file upload operation node writes data to the input data node, while the Spark application submission operation node reads data from that data node. Therefore, a clear data link is formed between the file upload node and the application submission node through the data nodes; the edge attributes can be defined to describe the type of data flow, such as read or write.

[0043] By default, all operation nodes are connected in timestamp order to form a main control flow, representing the sequence of student operations. In addition, operation nodes that submit Spark applications establish a control flow edge representing the execution relationship with the Spark program file code asset nodes they execute.

[0044] The resulting student dynamic behavior map is no longer a simple log, but a strategic map that allows us to examine from a global perspective how students organize data, write code, and execute commands to solve problems. This student dynamic behavior map clearly shows each step of the student's operation and how these operations are interconnected and how data is transferred. The resource consumption of each key operation is also clearly marked.

[0045] S03: Compare the trainee dynamic behavior map with the expert benchmark map pre-built for the same training task in multiple dimensions to generate comparative analysis results representing the differences between the two in terms of problem-solving path, operational efficiency and resource effectiveness.

[0046] Specifically, the system pre-configures an expert benchmark graph library for the same training task (e.g., DAU / PV analysis). This library stores various representative solution graphs built by senior big data engineers, such as optimal path graphs representing industry best practices, standardized path graphs representing standard problem-solving methods, and typical error graphs representing common error patterns. The system rigorously compares the real-time generated student dynamic behavior graphs with these expert benchmark graphs from multiple dimensions, including the structure of the problem-solving path, the efficiency of the operational steps, and the resource efficiency of specific operations, and generates detailed comparative analysis results.

[0047] The objectivity and scientific nature of the evaluation are derived from comparison with high-quality references. To this end, this application has constructed an expert benchmark graph library. For example, for the DAU / PV analysis task, the expert benchmark graph library pre-stores at least the following three types of graphs pre-constructed by senior big data engineers or domain experts: optimal path graph, standard path graph, or typical error graph representing common error patterns.

[0048] Specifically, the optimal path graph represents the industry best practices for solving this problem, with efficient algorithms and resource-saving features. For example, before performing any expensive shuffle operations (such as aggregation), use filter operators to remove as many invalid log lines as possible. For PV and DAU calculations, use operators such as reduceByKey or aggregateByKey, because they perform local aggregation on the map side, which greatly reduces the amount of data that needs to be shuffled in the network.

[0049] Canonical path graphs represent textbook or functionally correct standard solutions, but may not be optimal in terms of performance. They use logically clear but slightly less efficient operators, such as groupByKey. This approach first gathers all data with the same key into the same reduce task and then performs the aggregation. This can generate huge network shuffle overhead and memory pressure on large-scale data.

[0050] Typical error graphs represent common error patterns that beginners often make, which can lead to serious performance problems or errors. For example, in a Spark program, directly calling the .collect() method on a terabyte-scale elastic distributed dataset loaded from HDFS in an attempt to pull all the data into the driver's memory for processing will immediately cause the driver to overflow memory, which is a serious taboo in big data processing.

[0051] These expert benchmark graphs also follow the aforementioned graph structure and attribute definitions, including operation nodes, data nodes, and bound resource efficiency indicators. After receiving the trainee dynamic behavior graph, the comparative analysis module will automatically compare it with the graphs in the expert benchmark graph library in three dimensions: structural similarity analysis, path efficiency analysis, and node efficiency analysis.

[0052] Specifically, structural similarity analysis is used to assess the degree to which a student's overall problem-solving approach aligns with that of an expert. It can employ graph edit distance or its variants to calculate the minimum cost required to convert the student's dynamic behavior graph into an expert benchmark graph. This cost consists of a series of predefined editing operations, such as node replacement or addition / deletion. Specifically, replacing the `groupByKey` operation node in the student's dynamic behavior graph with the `reduceByKey` node in the expert benchmark graph incurs a higher cost. Redundant operations or missing necessary steps in the student's dynamic behavior graph also incur corresponding costs. The final calculated graph edit distance is a numerical value; the smaller the value, the closer the student's problem-solving approach and step organization are to the expert's, and the higher the structural similarity score.

[0053] Path efficiency analysis is used to evaluate whether there are redundancies, detours, or suboptimal choices in the trainee's operation sequence. It focuses on analyzing the critical data processing path from the initial data node to the final data node. By comparing the total number of nodes or the path length on this path between the trainee's dynamic behavior graph and the expert benchmark graph, the redundancy of the operation can be intuitively reflected. For example, the trainee's path may have several more nodes than the expert's optimal path due to multiple unnecessary intermediate data landings. In addition, the system will also analyze whether the topology of the trainee's dynamic behavior graph reflects the parallelism of task decomposition.

[0054] Node performance analysis is used to accurately pinpoint the specific high-consumption operations that cause system performance bottlenecks. This is achieved by finding nodes in the student dynamic behavior graph and the expert benchmark graph that correspond to the functions, such as the node that submits the Spark application in both graphs. Then, the resource consumption performance indicators bound to these nodes are compared one by one. For example, the comparative analysis found that the peak memory of the student's Spark job node was 128GB, while the peak memory of the node corresponding to the expert's optimal path was only 15GB. This huge difference provides quantitative data evidence for subsequent attribution analysis.

[0055] S04: Based on the comparative analysis results, conduct attribution analysis on the trainee's operation sequence, and finally generate a quantitative score and diagnostic report on the trainee's comprehensive practical ability.

[0056] Specifically, based on the comparative analysis results above, the system automatically identifies inefficient or erroneous steps in the student's operational sequence that deviate significantly from the expert benchmark graph. It then performs attribution analysis on the effectiveness of the student's operational steps, the quality of their problem-solving strategies, and the rationality of their resource consumption. For example, if it finds that a certain computational node in the student's dynamic behavior graph consumes ten times more memory than the corresponding node in the expert benchmark graph, the system can pinpoint that this is due to the student using inappropriate Spark operators. Finally, the system combines the corresponding correct or efficient steps in the expert benchmark graph with all comparative dimensions to generate a personalized diagnostic report that includes a comprehensive quantitative score, multi-dimensional capability gap analysis, and specific optimization suggestions for the inefficient or erroneous steps. This allows for an in-depth assessment and effective guidance of the student's comprehensive practical abilities.

[0057] The assessment and scoring module transforms the complex comparative analysis results into feedback that directly guides trainees. This module is based on a rule-based and machine learning model-driven inference engine that receives discrepancy data from the comparative analysis module and performs logical inferences. For example, the engine can pre-set rules such as: if the trainee's dynamic behavior graph structure is highly similar to the standard path graph, and the memory consumption of the Spark job node is much greater than that of the corresponding node in the optimal path graph, and the trainee's code contains the groupByKey operator, then the system can automatically infer that the cause is an inappropriate aggregation algorithm selection, leading to excessive memory pressure.

[0058] Understandably, the system can score trainees from multiple dimensions and calculate a quantitative score of their comprehensive practical ability. The scoring dimensions may include: strategy compliance based on structural similarity analysis, process efficiency based on path efficiency analysis, CPU performance based on CPU consumption, memory performance based on memory consumption, etc. The final quantitative score is a weighted average of these dimension scores.

[0059] The personalized diagnostic report is the final product delivered to the trainee. It is usually in the form of a webpage with pictures and text or a PDF document. The report content can be quite detailed, and may include, for example, overall evaluation and comprehensive score, capability radar chart, key problem identification and analysis, and complete behavioral path playback.

[0060] Specifically, the overall evaluation and comprehensive score are used to showcase the trainee's overall performance; the ability radar chart graphically compares the trainee's scores across multiple dimensions with those of experts, allowing trainees to clearly see their strengths and weaknesses; the key issue identification and analysis presents a list of the main issues identified by attribution analysis, with each item including a problem description, problem location, data support, and specific optimization suggestions; for example, the system might suggest that trainees "modify the code segment using the groupByKey operator to use the reduceByKey operator, which allows for local aggregation before data shuffling, significantly reducing memory and network overhead"; and the complete behavior path replay provides a visual interface, allowing trainees or teachers to review the entire dynamic behavior graph generation process.

[0061] Through personalized diagnostic reports, assessments are no longer just simple scores, but are transformed into a powerful learning tool that drives learners to reflect on their problem-solving strategies, understand the essence of excellent engineering practices, and thus achieve a spiral increase in their abilities.

[0062] like Figure 2 As shown, to implement the above method, this application also discloses a big data full-process training and assessment system based on multi-resource collaboration, including: a data capture module, a front-end part deployed in the student training environment, including the aforementioned IDE plugin, Shell hook function, and resource monitoring script, used to capture in real time the operation sequence executed by the student to complete the training task in a big data training environment with multiple heterogeneous resources, and synchronously collect the resource call data triggered by each operation step in the operation sequence; a graph construction module, connected to the data capture module, used to dynamically create or update nodes and edges in the graph database based on the operation sequence and resource call data, and construct and maintain a student dynamic graph that reflects the student's problem-solving strategy and process behavior. The behavior graph is responsible for attaching resource efficiency data as attributes to the corresponding operation nodes. The comparative analysis module, connected to the graph construction module, is used to conduct multi-dimensional quantitative comparisons between the student's dynamic behavior graph and the expert benchmark graph pre-built for the same training task, generating comparative analysis results that represent the differences between the two in terms of problem-solving paths, operational efficiency, and resource efficiency. The evaluation and scoring module, connected to the comparative analysis module, is used to perform attribution analysis on the student's operation sequence based on the comparative analysis results, applying the built-in evaluation rule base and scoring model, and finally generating a quantitative score and diagnostic report of the student's comprehensive practical ability. Finally, it dynamically generates a personalized diagnostic report with graphics and text for the front-end interface to be displayed to the student or teacher.

[0063] Through the detailed embodiments described above, this application demonstrates how to transform the complex thought processes and operational behaviors of trainees, which were previously invisible in big data training, into measurable, analyzable, and evaluable structured data models. By quantitatively comparing these models with expert practice, it achieves a profound, comprehensive, and diagnostically valuable assessment of trainees' engineering practice capabilities. This surpasses traditional results-oriented assessment methods and provides an innovative technical path for cultivating high-quality big data talent.

[0064] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0065] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0066] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0067] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0068] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0069] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A big data end-to-end training and assessment method based on multi-resource collaboration, characterized in that, include: In a big data training environment with diverse heterogeneous resources, the operation sequence executed by trainees to complete training tasks is captured in real time, and the resource call data triggered by each operation step in the operation sequence is collected synchronously. Based on the operation sequence and the resource call data, a dynamic behavior graph of the student that reflects the student's problem-solving strategy and process behavior is constructed. The operation steps are mapped to nodes in the dynamic behavior graph, the logical dependencies or temporal relationships between the operation steps are mapped to edges in the dynamic behavior graph, and the resource call data is associated with the nodes that execute the corresponding operation steps. The student dynamic behavior map is compared with the expert benchmark map pre-constructed for the same training task in multiple dimensions to generate comparative analysis results that represent the differences between the two in terms of problem-solving path, operational efficiency and resource effectiveness. Based on the comparative analysis results, attribution analysis is performed on the student's operation sequence, and a quantitative score and diagnostic report on the student's comprehensive practical ability is finally generated.

2. The method according to claim 1, characterized in that, The real-time capture of the operation sequence performed by the trainee to complete the training task includes: intercepting and structurally recording various interactive instructions, program calls and state change events of the trainee during the entire training task execution period through an event listening agent pre-set in the training environment, so as to form an operation event flow with temporal relationship, wherein the event listening agent can cover the operating system layer, application software layer and service interface layer.

3. The method according to claim 1, characterized in that, In the student dynamic behavior graph, the attributes of the nodes include operation type, execution parameters or key code snippets, and the attributes of the edges include data flow relationship or control flow transmission relationship between nodes.

4. The method according to claim 3, characterized in that, Associating the resource call data with the node executing the corresponding operation step includes: binding the key performance indicators generated by each operation step during its execution lifecycle as extended attributes of the node corresponding to that operation step, wherein the key performance indicators include computing resource consumption, peak storage resource usage, or data network transmission overhead.

5. The method according to claim 1, characterized in that, The expert benchmark graphs are derived from a benchmark graph library containing a variety of typical problem-solving strategies. The benchmark graph library includes optimal path graphs representing industry best practices, standard path graphs representing standard problem-solving methods, or typical error graphs representing common error patterns.

6. The method according to claim 1, characterized in that, The multi-dimensional quantitative comparison includes at least structural similarity analysis, that is, using graph theory algorithms to calculate the difference in topological structure between the student's dynamic behavior graph and the expert's benchmark graph, in order to evaluate the degree of conformity between the student's overall problem-solving approach and the expert's approach.

7. The method according to claim 6, characterized in that, The multi-dimensional quantitative comparison also includes path efficiency analysis, which involves comparing the differences between the student dynamic behavior graph and the expert benchmark graph in terms of critical path length, total number of nodes, or number of parallel branches to assess whether the student's operation sequence has redundancy, detours, or suboptimal choices.

8. The method according to claim 7, characterized in that, The multi-dimensional quantitative comparison also includes node performance analysis, which involves comparing the resource consumption performance indicators associated with the functionally corresponding or logically similar nodes in the student dynamic behavior graph and the expert benchmark graph one by one, in order to identify and locate the specific high-consumption operation links that lead to system performance bottlenecks.

9. The method according to claim 1, characterized in that, The process of performing attribution analysis on the student's operation sequence based on the comparative analysis results, and finally generating a quantitative score and diagnostic report on the student's comprehensive practical ability, includes: automatically identifying inefficient or erroneous links in the student's operation sequence that deviate significantly from the expert benchmark map based on the comparative analysis results; performing attribution analysis on the effectiveness of the student's operation steps, the superiority or inferiority of the problem-solving strategy, and the rationality of resource consumption; and generating a personalized diagnostic report containing specific optimization suggestions and ability gap analysis by combining the corresponding correct or efficient links in the expert benchmark map.

10. A big data end-to-end training and assessment system based on multi-resource collaboration, characterized in that, include: The data capture module is used to capture the sequence of operations performed by trainees to complete training tasks in real time in a big data training environment with multiple heterogeneous resources, and to synchronously collect the resource call data triggered by each operation step in the sequence of operations. The graph construction module, connected to the data capture module, is used to construct a dynamic behavior graph of the student that reflects the student's problem-solving strategies and process behavior based on the operation sequence and the resource call data. The comparative analysis module, connected to the graph construction module, is used to perform multi-dimensional quantitative comparison between the trainee dynamic behavior graph and the expert benchmark graph pre-constructed for the same training task, and generate comparative analysis results that characterize the differences between the two in terms of problem-solving path, operational efficiency and resource effectiveness. The assessment and scoring module, connected to the comparative analysis module, is used to perform attribution analysis on the student's operation sequence based on the comparative analysis results, and finally generate a quantitative score and diagnostic report on the student's comprehensive practical ability.

Images