Big data-based vocational education teaching evaluation method

Abstract

This invention relates to the field of big data and vocational education evaluation technology, and discloses a big data-based method for vocational education teaching evaluation. The method includes constructing a teaching evaluation data lake to aggregate and analyze raw teaching behavior data streams from all nodes of the vocational education process; fusing and mapping the analyzed structured and unstructured evaluation elements to generate a unified, time-series behavioral representation sequence; inputting this sequence into a dynamic competency modeling network to output a dynamic competency profile that evolves with the learning process; matching the dynamic competency profile with a pre-set vocational competency map to generate a quantified, stage-specific competency fit; and driving a teaching evaluation decision engine based on this fit to generate preliminary evaluation trigger commands. This method achieves deep integration and unified computational representation of multi-source heterogeneous teaching data, and through dynamic modeling and real-time map matching, promotes a paradigm shift in teaching evaluation from static result judgment to dynamic process measurement.

Description

Technical Field

[0001] This invention relates to the field of big data and vocational education evaluation technology, specifically to a method for evaluating vocational education teaching based on big data. Background Technology

[0002] Current vocational education assessments largely rely on outcome-based, single-dimensional data. Common assessment systems primarily collect and process structured data such as exam scores, attendance records, and task completion status. For unstructured data generated during the teaching process, such as classroom interaction audio, practical training videos, project report texts, and group discussion records, existing technologies typically only allow for independent storage or simple keyword retrieval, failing to effectively correlate and deeply integrate them with structured learning behaviors. This results in fragmented assessment information, making it difficult to comprehensively and accurately depict the continuous state of students during complex skills training and vocational competence development.

[0003] Existing evaluation models are mostly static, typically based on fixed evaluation index systems and weights, measuring academic achievements at a specific point in time. This model lacks the ability to track and model the dynamic development trajectory of students' abilities, resulting in lagging and fixed evaluation results. Meanwhile, vocational education outputs need to align with the actual competency requirements of industry positions, but existing standards or maps are often static and compartmentalized, unable to perform real-time, refined matching and fit calculations with a multi-dimensional student competency model that evolves with the learning process. This leads to delayed teaching feedback, making it difficult to accurately drive personalized teaching interventions and resource recommendations. Summary of the Invention

[0004] The purpose of this invention is to provide a vocational education teaching evaluation method based on big data, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, this invention provides a vocational education teaching evaluation method based on big data, the method comprising:

[0006] A teaching evaluation data lake is constructed in the vocational education management system, which receives raw teaching behavior data streams from nodes throughout the vocational education process;

[0007] In the teaching evaluation data lake, the original teaching behavior data stream is analyzed in layers to extract structured evaluation elements and unstructured evaluation elements;

[0008] The structured evaluation elements and unstructured evaluation elements are fused and mapped to generate a unified behavioral representation sequence for vocational education evaluation objects;

[0009] The unified behavioral representation sequence is input into the dynamic ability modeling network of the vocational education evaluation object, and the dynamic ability profile of the vocational education evaluation object is output.

[0010] The dynamic ability profile is matched with the preset vocational ability map to generate the stage-specific ability fit of the vocational education evaluation object.

[0011] The teaching evaluation decision engine starts the evaluation process based on the stage-specific ability fit, and the teaching evaluation decision engine generates a preliminary evaluation trigger command based on the stage-specific ability fit.

[0012] Preferably, the construction of a teaching evaluation data lake in the vocational education management system, wherein the teaching evaluation data lake receives raw teaching behavior data streams from nodes throughout the vocational education process, includes:

[0013] Identify and mark theoretical teaching nodes, practical training nodes, project assessment nodes, and teacher-student interaction nodes in the entire vocational education process as data source nodes;

[0014] Deploy lightweight data probes on each data source node to continuously capture teaching process records and operation behavior logs containing time-series information generated by the data source node;

[0015] Define the data access specifications for the teaching evaluation data lake, and perform format cleaning on the teaching process records and operation behavior logs according to the data access specifications to form a raw teaching behavior data unit stream with a unified timestamp;

[0016] All raw teaching behavior data units are asynchronously written into the distributed storage layer of the teaching evaluation data lake to form raw teaching behavior data streams.

[0017] Preferably, the step of performing hierarchical analysis on the original teaching behavior data stream in the teaching evaluation data lake to extract structured evaluation elements and unstructured evaluation elements includes:

[0018] The parsing service of the teaching evaluation data lake is invoked to perform layered processing on the original teaching behavior data stream;

[0019] In the first parsing layer, data units containing numerical indicators and discrete labels are identified and separated, and these data units are grouped into a set of structured evaluation elements.

[0020] In the second parsing layer, semantic understanding and feature extraction are performed on the remaining text records, audio segments and image frame data to extract feature vectors that reflect teaching attitude, collaborative tendency and innovative thinking. The feature vectors are then merged into an unstructured evaluation element set.

[0021] Preferably, the step of fusing and mapping the structured evaluation elements with the unstructured evaluation elements to generate a unified behavioral representation sequence for vocational education evaluation objects includes:

[0022] An independent behavioral mapping space is established for the evaluation objects of vocational education, and each element in the set of structured evaluation elements is mapped to a coordinate point in the behavioral mapping space;

[0023] Each feature vector in the unstructured evaluation element set is transformed into a trajectory segment in the behavior mapping space, and the endpoints of the trajectory segment are determined by the initial and final states of the feature vector.

[0024] In the behavior mapping space, coordinate points formed by structured evaluation elements and trajectory segments formed by unstructured evaluation elements are connected in chronological order to generate a continuous behavior trajectory, which is then encoded as a unified behavior representation sequence for vocational education evaluation objects.

[0025] Preferably, the step of inputting the unified behavioral representation sequence into the dynamic competency modeling network of the vocational education evaluation object and outputting a dynamic competency profile of the vocational education evaluation object includes:

[0026] A dynamic capability modeling network is constructed, which includes a temporal feature extraction layer and a capability state inference layer.

[0027] The temporal feature extraction layer performs sliding window analysis on the unified behavioral representation sequence of the input vocational education evaluation object to extract temporal feature segments that reflect the trend of ability change. The ability state inference layer receives the temporal feature segments and, with reference to the historical ability evolution path, infers the comprehensive ability state vector of the vocational education evaluation object at the current moment. The comprehensive ability state vector constitutes the core of the dynamic ability profile.

[0028] Preferably, the step of matching the dynamic competency profile with a preset vocational competency map to generate a stage-specific competency fit for the vocational education evaluation subject includes:

[0029] A vocational competency graph corresponding to the major of the vocational education evaluation object is loaded from the vocational education knowledge base. The vocational competency graph consists of multiple interconnected competency nodes and competency edges. The comprehensive competency state vector in the dynamic competency profile is projected onto the multidimensional competency space defined by the vocational competency graph. The projection intensity of the comprehensive competency state vector on each competency node of the vocational competency graph is calculated. Several competency nodes with projection intensity exceeding a threshold are selected as current matching nodes. Based on the number, weight and topological importance of the current matching nodes in the vocational competency graph, a quantitative stage competency fit degree is generated through aggregation calculation.

[0030] Preferably, the teaching evaluation decision engine initiates the evaluation process based on the stage-based competency fit, and the teaching evaluation decision engine generates a preliminary evaluation trigger instruction based on the stage-based competency fit, including:

[0031] The teaching evaluation decision engine is set with a key threshold. When the received stage ability fit is lower than the key threshold, the in-depth evaluation process is triggered.

[0032] The teaching evaluation decision engine generates a preliminary evaluation trigger instruction that includes the trigger time, target object identifier, and required evaluation dimensions, and sends the preliminary evaluation trigger instruction to the evaluation task scheduling center.

[0033] Preferably, the construction of the teaching evaluation decision engine includes:

[0034] The core rule base for configuring the decision engine stores evaluation triggering conditions and decision logic based on vocational education evaluation standards.

[0035] A status monitoring module is deployed for the decision engine. The status monitoring module listens in real time to the output events of the dynamic capability profile from the dynamic capability modeling network and the calculation results of the phased capability fit.

[0036] An instruction generator is integrated into the decision engine. The instruction generator matches the input state information according to the decision logic in the core rule base and generates corresponding evaluation control instructions.

[0037] A communication interface for the decision engine is established, which is responsible for receiving query requests from external systems and forwarding evaluation trigger instructions generated by the decision engine.

[0038] Preferably, the step of projecting the comprehensive ability state vector from the dynamic ability profile onto the multi-dimensional ability space defined by the professional ability map, and calculating the projection intensity of the comprehensive ability state vector at each ability node of the professional ability map, includes:

[0039] Analyze the vector representation of each ability node in the professional ability graph and construct a set of basis vectors for a multidimensional ability space;

[0040] The integrated capability state vector is orthogonally decomposed according to the basis vector set to obtain the magnitude of the components of the integrated capability state vector in each basis vector direction;

[0041] The magnitude of the component in the direction of the basis vector corresponding to each capability node is normalized to obtain the projection intensity value on the capability node.

[0042] Traverse all ability nodes in the professional ability graph, repeatedly perform the projection calculation process, and generate a projection distribution map containing the projection intensity values ​​of each ability node.

[0043] Preferably, the teaching evaluation decision engine is equipped with a key threshold. When the received stage-based ability fit is lower than the key threshold, the in-depth evaluation process is triggered, including:

[0044] The range of key threshold values ​​is determined based on the statistical distribution of historical evaluation data, and specific key threshold values ​​are set in conjunction with the teaching objectives of the current evaluation cycle.

[0045] The teaching evaluation decision engine continuously monitors the input phased ability fit data stream and compares the phased ability fit of each evaluation object with key thresholds in real time.

[0046] When it is detected that the phased capability fit of an evaluation object is continuously lower than the key threshold for a preset period of time, the decision to start the in-depth evaluation process is triggered.

[0047] Once the in-depth evaluation process is initiated, the teaching evaluation decision engine automatically invokes the expanded evaluation indicator library and enhanced analysis algorithms to conduct a multi-dimensional in-depth evaluation of the target evaluation object.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] By performing hierarchical analysis on the raw teaching behavior data generated at every stage of the vocational education process, extracting structured and unstructured elements, and implementing fusion mapping, a unified behavioral representation sequence for the evaluation subjects is generated. This achieves standardized alignment and unified encoding of structured data such as exam scores and operation records with unstructured data such as classroom audio, practical videos, and text reports. Heterogeneous data is transformed into a vector sequence with temporal correlation, overcoming the limitations of isolated and dimensionally fragmented multi-source data in traditional evaluation, and forming a continuous, complete, and directly usable digital archive that characterizes learning behavior.

[0050] A unified behavioral representation sequence is input into a dynamic capability modeling network. This network processes continuous input through a time-series model, capturing the changing trends and relationships of capability dimensions, and outputting a dynamic capability profile that continuously evolves with the learning process. This profile is then matched and similarity-calculated in real time with a pre-defined occupational capability graph that includes skill nodes, levels, and weights, generating a quantified, stage-specific capability fit. This technical approach transforms evaluation from static scoring based on fixed indicators and weights to continuous modeling of capability growth trajectories and dynamic measurement of their conformity with occupational standards, achieving a fundamental shift in evaluation models from focusing on cross-sectional results to tracking the development process. Attached Figure Description

[0051] Figure 1 This is a schematic diagram illustrating the working principle of the big data-based vocational education teaching evaluation method described in this invention.

[0052] Figure 2 A flowchart for constructing a teaching evaluation data lake and receiving raw teaching behavior data streams;

[0053] Figure 3 A flowchart for generating a unified behavioral representation sequence;

[0054] Figure 4 Grouped bar chart for the effectiveness of in-depth evaluation of vocational education;

[0055] Figure 5 A line graph comparing the performance of the status monitoring module in the vocational education teaching evaluation decision engine. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Please see Figure 1This invention provides a big data-based method for evaluating vocational education teaching. The method includes: constructing a teaching evaluation data lake to aggregate and process multi-source data; the teaching evaluation data lake receives raw teaching behavior data streams from nodes throughout the entire vocational education process, covering real-time information from multiple stages such as theoretical teaching and practical training; performing hierarchical analysis on the raw teaching behavior data streams within the data lake, identifying and separating structured and unstructured evaluation elements; structured evaluation elements include numerical indicators and discrete labels, while unstructured evaluation elements extract feature vectors from text, speech, and image data; fusing and mapping the structured and unstructured evaluation elements to establish a behavior mapping space to generate a unified behavior representation sequence for vocational education evaluation objects, encoding the continuous behavioral trajectory of the evaluation object in the time dimension; inputting the unified behavior representation sequence into a dynamic capability modeling network for vocational education evaluation objects, which includes a temporal feature extraction layer and a capability state inference layer; and outputting a dynamic capability profile of the vocational education evaluation objects, which represents the current capability state of the evaluation object in the form of a comprehensive capability state vector. The dynamic competency profile is matched with a pre-defined vocational competency map, which is loaded from a vocational education knowledge base and defines a multi-dimensional competency space. The matching process generates a phased competency fit degree for the vocational education evaluation subject, which quantifies the degree of matching between the evaluation subject's competency and occupational requirements. Based on the phased competency fit degree, the teaching evaluation decision engine initiates the evaluation process. The teaching evaluation decision engine generates an initial evaluation trigger command based on the phased competency fit degree, which is used to trigger subsequent in-depth evaluations or scheduling tasks.

[0058] In one embodiment of the present invention, see [reference] Figure 2The system identifies and labels theoretical teaching nodes, practical training nodes, project assessment nodes, and teacher-student interaction nodes throughout the vocational education process as data source nodes. The labeling of these data source nodes is defined based on the course chapter settings and practical training activity plans within the teaching management system. Lightweight data probes deployed on each data source node utilize software proxies to continuously capture teaching process records and operational behavior logs generated by the data source node, including time-series information. Teaching process records include, for example, chapter completion timestamps and in-class quiz scores recorded on online learning platforms. Operational behavior logs include, for example, sequence of operation steps and equipment parameter adjustment records recorded on virtual simulation training platforms. A data access specification for the teaching evaluation data lake is defined. This specification explicitly requires all incoming data units to include a uniformly formatted identifier field, timestamp field, and data payload field. The teaching process records and operational behavior logs are format-cleaned according to the data access specification. This cleaning process corrects abnormal timestamps, completes missing identifier fields, and converts the data payload to standard JSON format, forming a raw teaching behavior data unit stream with a uniform timestamp. This raw teaching behavior data unit stream is buffered and transmitted in the form of a message queue. All raw teaching behavior data units are asynchronously written into the distributed storage layer of the teaching evaluation data lake. The distributed storage layer is built based on Hadoop HDFS or object storage system to form raw teaching behavior data streams. The raw teaching behavior data streams are partitioned and stored according to date and data source type for subsequent batch and real-time analysis.

[0059] In some embodiments, the parsing service of the teaching evaluation data lake is invoked. This parsing service, triggered as a resident task on the data lake's computing engine, performs layered processing on the raw teaching behavior data stream. The layered processing logic is based on the structure and semantic complexity of the data units. In the first parsing layer, data units containing numerical indicators and discrete labels are identified and separated. The identification process is based on predefined data pattern matching rules, merging the data units into a structured evaluation element set. This set is stored in a fact table of a relational database for rapid aggregation and querying. In the second parsing layer, semantic understanding and feature extraction are performed on the remaining text records, audio clips, and image frames. The semantic understanding module performs sentiment polarity analysis and keyword extraction on the text discussions in the teacher-student forum. The feature extraction module performs action recognition and attention estimation on clips from practical training operation videos, extracting feature vectors reflecting teaching attitudes, collaborative tendencies, and innovative thinking. The dimension and meaning of these feature vectors are defined by a pre-trained artificial intelligence model. These feature vectors are merged into an unstructured evaluation element set, which is stored in a specially designed vector database in the form of a high-dimensional vector index to support similarity retrieval.

[0060] It is understandable that the data access specification format cleaning process includes a data quality verification step. This step checks the completeness, consistency, and accuracy of the data according to rules. For data units that do not conform to the specification, an error report is generated, triggering a data tracing process. In an example scenario related to CNC machining, raw data units from theoretical teaching nodes might include students' progress percentages and chapter quiz scores in the online "CNC Programming" course. Raw data units from practical training nodes might include operation sequence logs of students writing G-code in simulation software and simulation accuracy scores of the final machined parts. The data cleaning process unifies these two types of data from different sources and with different formats into data units with the same timestamp granularity and field structure.

[0061] Optionally, the raw teaching behavior data unit stream undergoes a data compression and encoding stage before being written to the distributed storage layer. This stage employs a columnar compression algorithm to reduce storage space usage and improve subsequent retrieval efficiency. After merging, the structured evaluation element set can undergo preliminary statistical analysis. This analysis calculates the mean, variance, and trend of each evaluation object across various numerical indicators. These statistical results are also stored as snapshot information in the teaching evaluation data lake.

[0062] The extraction process of unstructured evaluation elements can be achieved by calculating the feature vector dimension value reflecting the tendency to cooperate in text records using the following formula:

[0063]

[0064] in: Indicates within the time window The collaborative tendency feature value extracted from within, This indicates the total number of discussion threads participated in by the evaluated object within this time window. This refers to the text of the query or question posted by the person being evaluated. This indicates the first action taken by other participants in the same thread on the query or question text. One reply text, This represents a text embedding function that maps text into a fixed-dimensional semantic vector. This represents the calculation of the cosine similarity function between two semantic vectors. Feature extraction from speech segments focuses on analyzing changes in intonation and speech rate to aid in assessing teaching attitude. Image frame data processing, tailored to practical training scenarios, uses a pose estimation model to identify whether students' operational postures comply with safety regulations, thus serving as a component of practical behavior characteristics. These unstructured evaluation elements extracted from multimodal data ultimately form the raw materials for subsequent fusion mapping, together with the set of structured evaluation elements.

[0065] It is understandable that the processing of the first and second parsing layers can be executed in parallel pipelines. This parallel pipeline allows the remaining data in the original teaching behavior data stream to immediately enter the unstructured feature extraction process after the initial structured separation, thereby reducing overall processing latency. The lightweight data probe is designed to ensure low resource consumption and will not interfere with the normal operation of the original teaching system. Lightweight data probes are typically implemented in the form of log collection agents or database change data capture technologies.

[0066] In one embodiment of the present invention, see [reference] Figure 3 In its implementation, an independent behavioral mapping space is established for vocational education evaluation subjects. This behavioral mapping space is a multi-dimensional Euclidean space predefined by the teaching evaluation system. Each orthogonal dimension of the behavioral mapping space corresponds to a quantified core behavior or ability indicator, such as "theoretical knowledge mastery," "practical operation proficiency," "teamwork frequency," and "number of innovative attempts." Each element in the structured evaluation element set is mapped to a coordinate point in the behavioral mapping space. The mapping process follows a set of fixed transformation rules. For example, the numerical indicator "chapter test score" is linearly scaled to the "theoretical knowledge mastery" dimension, and the discrete label "practical training project pass status" is converted into a specific discrete value in the "practical operation proficiency" dimension. This determines the precise position of each structured evaluation element in the behavioral mapping space. Each feature vector in the unstructured evaluation element set is transformed into a trajectory segment in the behavior mapping space. The endpoints of the trajectory segment are determined by the initial and final states of the feature vectors. The initial and final states are obtained by clustering the feature vector sequence within a time window to identify its centroid or by identifying state abrupt change points through a temporal segmentation algorithm. For example, from the semantic feature vector sequence of a dialogue, two state points, "beginning of questioning" and "consensus reached," are identified, and a line segment connecting these two points is drawn in the "communication depth" and "consensus building" dimensions of the behavior mapping space. In the behavior mapping space, the coordinate points formed by structured evaluation elements and the trajectory segments formed by unstructured evaluation elements are connected in chronological order. The connection operation uses a cubic spline interpolation algorithm to ensure the smoothness and continuity of the trajectory, generating a continuous behavior trajectory. The behavior trajectory is encoded into a unified behavior representation sequence for vocational education evaluation objects. The encoding process samples the behavior trajectory at fixed time intervals and arranges the multidimensional coordinate values ​​of each sampled point in sequence to form a unified behavior representation sequence.

[0067] In some embodiments, a dynamic competency modeling network is constructed, which uses a gated recurrent unit network as its basic architecture. The temporal feature extraction layer performs sliding window analysis on the unified behavioral representation sequence of the input vocational education evaluation object. The sliding window size is set according to the teaching evaluation cycle, for example, a week or a teaching module as the window length, extracting temporal feature segments reflecting the trend of competency changes. These temporal feature segments capture the short-term fluctuations and long-term evolution patterns of the evaluation object in multidimensional behavioral indicators. The competency state inference layer receives the temporal feature segments and, referring to the historical competency evolution path (trained from the unified behavioral representation sequence of the evaluation object's past performance and passed as a hidden state), infers the comprehensive competency state vector possessed by the vocational education evaluation object at the current moment. This comprehensive competency state vector constitutes the core of the dynamic competency profile, which is ultimately output as a multidimensional vector, where each dimension represents an interpretable competency trait score.

[0068] It is understandable that the representation of a trajectory segment in the behavior mapping space includes not only endpoint information but also attributes describing the segment's shape. In an example scenario of evaluating automotive repair students, the structured evaluation element "engine disassembly and assembly assessment score" is mapped to a high-value coordinate point on the "precision operation ability" dimension of the behavior mapping space. The unstructured evaluation element "semantic evolution features in fault diagnosis discussion" is transformed into an ascending trajectory segment on the "logical reasoning ability" dimension, moving from a "phenomenon listing" state to a "root cause inference" state. These points and segments are connected according to their timestamps, forming the student's behavioral trajectory during a one-month practical training project. The encoding result of the unified behavioral representation sequence is a sequence of the form... The sequence is given by t, where t represents a time point and x, y, and z represent coordinate values ​​in different dimensions of the behavior mapping space.

[0069] To determine the endpoints of a trajectory segment, the state centroid can be calculated from a time-series feature vector using the following formula to define the endpoints:

[0070]

[0071] in: This represents the calculated endpoint vector. The vector representing the number of vectors in this time segment. This indicates the first segment in the fragment. 1 eigenvector Indicates the first The weights of each feature vector. The vector can be assigned based on its temporal position or importance. The dynamic capability modeling network is trained using diachronic uniform behavioral representation sequences labeled with capability levels as samples. Through supervised learning, the network learns to predict the comprehensive capability state vector from the behavioral sequences.

[0072] In some embodiments, the unified behavioral representation sequence is standardized before being input into the dynamic capability modeling network. Standardization scales the values ​​of each dimension to the same range to eliminate the influence of different units of measurement. The temporal feature extraction layer may contain multiple parallel convolutional neural network branches, each capturing local dependency patterns at different time scales. Their outputs are then fused with the backbone of a gated recurrent unit network. The final output of the capability state inference layer is a comprehensive capability state vector, which can be further used to generate dynamic capability profile reports in visual form, such as radar charts or capability development curves.

[0073] It is understandable that the dimensional definition of the behavior mapping space is not static. The dimensional definition can be dynamically configured and adjusted according to the training objectives of different majors to ensure that the mapping process is closely aligned with specific professional competency requirements. Algorithms that connect coordinate points to trajectory segments to generate continuous trajectories need to handle cases of non-uniform timestamp distribution. Handling non-uniform timestamp distribution typically employs a weighted interpolation method based on actual time intervals. During inference, the hidden states of the dynamic competency modeling network are continuously updated. This continuous updating of the hidden states allows the calculation of the comprehensive competency state vector to reflect the latest competency changes of the evaluated object, forming an online learning and evaluation mechanism.

[0074] In one embodiment of the present invention, in a specific implementation, a vocational competency graph corresponding to the major of the vocational education evaluation object is loaded from a vocational education knowledge base. The vocational education knowledge base is a graph database that stores the competency standards and relationships of various industry positions. The vocational competency graph consists of multiple interconnected competency nodes and competency edges. Competency nodes represent specific, observable skills or knowledge domains, and competency edges represent the prerequisite, concurrent, or progressive relationships between nodes. The comprehensive competency state vector in the dynamic competency profile is projected onto the multidimensional competency space defined by the vocational competency graph. The dimensional axis of the multidimensional competency space is consistent with the direction of the spatial basis vector spanned by the vectorized representation of the competency nodes in the vocational competency graph. The projection intensity of the comprehensive ability state vector onto each ability node of the vocational ability graph is calculated. The calculation process is completed through vector dot product and modulus operation. Specifically, in order to calculate the projection intensity of the comprehensive ability state vector onto each ability node of the vocational ability graph, the graph structure data of the vocational ability graph corresponding to the major of the evaluation object needs to be loaded from the graph database of the vocational education knowledge base. The graph embedding service is called to convert each ability node in the graph into a high-dimensional vector representation. Principal component analysis and Schmidt orthogonalization are performed on all ability node vectors to obtain a set of pairwise orthogonal unit basis vectors. These unit basis vectors together span the multidimensional ability space. The comprehensive capability state vector output by the capability state inference layer in the dynamic capability modeling network is subjected to a vector dot product operation with the unit basis vector representing each capability node. The resulting scalar is the projection length of the comprehensive capability state vector in the direction of that basis vector. This projection length is then divided by the magnitude of the unit basis vector. Since the magnitude of the unit basis vector is 1, this ratio is numerically equal to the projection length itself. This calculation result is directly used as the projection intensity value of the comprehensive capability state vector on that capability node. By performing the above dot product and magnitude operation on all capability nodes in the occupational capability graph, a projection distribution map containing the projection intensity values ​​of each capability node can be generated. Several capability nodes with projection intensities exceeding a threshold are selected as the current matching nodes. The threshold is dynamically set according to the strictness of the job capability requirements. Based on the number, weight, and topological importance of the current matching nodes in the occupational capability graph, a quantitative stage capability fit degree is generated through aggregation calculation. The aggregation calculation integrates the intensity value of the matching nodes, the preset weight coefficient, and their centrality index in the graph.

[0075] In some embodiments, the vector representation of each ability node in the professional ability graph is parsed. This vector representation is obtained by training the ability nodes and their associated edges using graph embedding techniques, constructing a set of basis vectors for a multi-dimensional ability space. This set of basis vectors is obtained by performing Schmitt orthogonalization on the vector representations of each ability node. The comprehensive ability state vector is orthogonally decomposed according to the basis vector set. This orthogonal decomposition represents the comprehensive ability state vector as a linear combination of the directions of each basis vector, yielding the component magnitude of the comprehensive ability state vector in each basis vector direction. The component magnitudes in the basis vector directions corresponding to each ability node are normalized, mapping the component magnitudes to the interval between 0 and 1, resulting in a projection intensity value for the ability node. This projection intensity value directly reflects the degree of fit between the comprehensive ability state vector and the dimension represented by that ability node. The projection calculation process is repeated for all ability nodes in the professional ability graph, generating a projection distribution map containing the projection intensity values ​​of each ability node. This projection distribution map visually displays the matching hotspots and weak points between the evaluated object's abilities and professional requirements.

[0076] It's understandable that the loading process of the professional competency map involves precise matching based on the professional identifier of the evaluation object. For example, for the "Software Testing Engineer" profession, the loaded professional competency map includes competency nodes such as "Writing Test Cases," "Executing Automated Tests," "Defect Management and Analysis," and "Test Environment Configuration," along with their interrelationships. The comprehensive competency status vector is a condensed representation that integrates multi-dimensional information such as theoretical scores, practical operation logs, and semantic features of project reports. The projection intensity can be calculated using the following formula:

[0077]

[0078] in: The state vector representing the overall capability is in the th... Projected intensity values ​​at each capability node This represents the overall capability state vector to be projected. The first in the occupational ability map The basis vectors corresponding to each capability node. This represents the dot product operation of vectors. Represents basis vectors The modulus length. In a specific example scenario, the projection intensity value of a student's comprehensive ability state vector onto the "Write Test Cases" ability node. The projection intensity value is 0.85 on the "Test Environment Configuration" capability node. If the threshold is set to 0.45, the "Write Test Cases" node will be selected as the current matching node, while the "Test Environment Configuration" node will not be selected.

[0079] Optionally, the weights of the currently matched nodes can be pre-assigned based on industry expert surveys or historical employment data analysis. Topological importance can be measured using graph algorithm metrics such as degree centrality, eigenvector centrality, or betweenness centrality of capability nodes in the graph. The aggregated calculation of the stage-specific capability fit can be represented as the sum of the weighted product of the projection intensity value of each matched node, the preset weight, and the topological importance score. The projection distribution map not only includes numerical values ​​but can also be overlaid on the original career capability graph structure in the form of a heatmap, forming a visual analysis view.

[0080] In one embodiment of the present invention, the teaching evaluation decision engine sets a key threshold. The range of the key threshold value is determined based on the statistical distribution of historical evaluation data. The statistical distribution of historical evaluation data analyzes the mean, standard deviation, and quantiles of the stage-by-stage ability fit of all students in multiple past teaching cycles. A specific key threshold value is set in conjunction with the teaching objectives of the current evaluation cycle, such as requiring students to reach an industry entry-level skill level after a specific practical training module. When the received stage-by-stage ability fit is lower than the key threshold, a deep evaluation process is triggered. The deep evaluation process covers more granular skill indicators and analytical dimensions compared to conventional evaluation. The teaching evaluation decision engine generates a preliminary evaluation trigger instruction containing the trigger time, target object identifier, and required evaluation dimensions, and sends the preliminary evaluation trigger instruction to the evaluation task scheduling center. The evaluation task scheduling center allocates special assessment resources and initiates corresponding diagnostic assessment tasks based on the instruction content.

[0081] In some embodiments, the teaching evaluation decision engine continuously monitors the input phased ability fit data stream, which uses the real-time results of each evaluation object calculated by the front-end processor as the input source. The phased ability fit of each evaluation object is compared with a key threshold in real time. This real-time comparison is performed in the stream processing engine and generates a Boolean trigger flag. When it is detected that the phased ability fit of an evaluation object is continuously below the key threshold for a preset duration, the deep evaluation process is initiated. The preset duration is used to filter out false triggers caused by instantaneous data fluctuations. After the deep evaluation process is initiated, the teaching evaluation decision engine automatically calls an extended evaluation indicator library and enhanced analysis algorithms to conduct a multi-dimensional in-depth assessment of the target evaluation object. The extended evaluation indicator library contains diagnostic questions and behavioral observations targeting specific weaknesses, and the enhanced analysis algorithm, for example, uses an attention-based neural network to perform attribution analysis on the historical behavioral sequences of the evaluation object.

[0082] It is understandable that the setting of key thresholds is not static. Setting specific key threshold values ​​based on the teaching objectives of the current evaluation cycle allows for intervention and adjustment by teaching administrators. In an example scenario for the "Network Security Operations and Maintenance" major, historical evaluation data shows that students who successfully passed the "Network Attack and Defense Practice" module all had a stage-specific competency fit score higher than 0.75. Therefore, the key threshold value set for this module in the current cycle is 0.75. The teaching evaluation decision engine continuously monitors the stage-specific competency fit score data streams of three students A, B, and C. Refer to Table 1 for a comparison of the observed data with the threshold across three consecutive evaluation time windows.

[0083] Table 1: Comparison of Stage-Specific Competency Alignment and Key Thresholds

[0084]

[0085] Based on the comparison in Table 1, Student B's stage-specific competency fit remained below the critical threshold of 0.75 for two consecutive time windows (1 and 2), meeting the preset duration condition. Therefore, the decision to initiate the in-depth evaluation process was made for Student B. After triggering, the initial evaluation trigger instruction generated by the teaching evaluation decision engine identified the target object as "Student B," and the required evaluation dimensions may specifically specify extended dimensions such as "accuracy of vulnerability scanning" and "standardization of emergency response procedures."

[0086] The dynamic adjustment of the critical threshold can be based on the following formula:

[0087]

[0088] in: This represents the specific key threshold value set for the current evaluation period. This represents the historical average of the capability fit at different stages, calculated based on the statistical distribution of historical evaluation data. Indicates historical standard deviation, It is a coefficient that controls the stringency of the threshold. It is an adjustment amount based on the teaching objectives set in the current evaluation cycle. The teaching administrator will make minor adjustments based on the focus and difficulty of this round of teaching. The preset duration needs to balance sensitivity and stability. The preset duration can be determined based on the frequency of evaluation data updates. For example, if the data is updated daily, the preset duration may be set to 3 days, and if it is updated weekly, it may be set to 2 weeks.

[0089] In some embodiments, the judgment logic for values ​​consistently below a key threshold can incorporate smoothing techniques such as sliding window averaging or exponentially weighted moving averages to eliminate the impact of single-point outliers. The extended evaluation index library invoked after the in-depth evaluation process is initiated is directly associated with the low-projection-intensity capability nodes previously identified in the evaluation object, ensuring the targeted nature of the in-depth assessment. During runtime, the augmented analysis algorithm not only analyzes the target object's own data but may also incorporate peer group data as a baseline for differential attribution. It can be understood that the teaching evaluation decision engine sending the initial evaluation trigger instruction to the evaluation task scheduling center is an asynchronous event-driven process. This asynchronous event-driven process ensures that the main monitoring logic is not blocked by delays in subsequent evaluation tasks. Upon receiving the instruction, the evaluation task scheduling center coordinates various resources, including specialized assessment systems, expert evaluation modules, or additional sensor data acquisition, to execute the in-depth evaluation.

[0090] See Figure 4 This is a grouped bar chart illustrating the effectiveness of in-depth vocational education evaluation. It displays comparative data across five skill dimensions, including "accuracy of vulnerability scanning," under "regular evaluation scores," "scores after in-depth evaluation," and "industry entry standards," verifying the skill improvement effect of the in-depth evaluation stage. Used to validate the effectiveness of the in-depth evaluation process, it serves as a core basis for the "precise remediation" strategy in the vocational education teaching evaluation system, ensuring students' skills meet industry entry requirements. This type of chart is a key tool in vocational education evaluation, demonstrating the value of the "data-driven, precision teaching" model and visually showcasing the actual impact of in-depth evaluation on vocational skills improvement.

[0091] In one embodiment of the present invention, a core rule base for the decision engine is configured. This core rule base stores evaluation triggering conditions and decision logic based on vocational education evaluation standards. Evaluation triggering conditions include, but are not limited to, thresholds for staged capability fit, thresholds for capability development trend slope, thresholds for the projection intensity of specific capability nodes, and their combinations. Decision logic is stored and organized in the form of "if condition, then action" rules or decision tables. A status monitoring module is deployed for the decision engine. This module monitors in real-time the output events of the dynamic capability profile from the dynamic capability modeling network and the calculation results of the staged capability fit. The monitoring mechanism is implemented by subscribing to specific topics in the message middleware or monitoring database change logs. An instruction generator is integrated into the decision engine. The instruction generator matches the input status information according to the decision logic in the core rule base. The matching process performs pattern matching between the currently received dynamic capability profile and staged capability fit and the conditional parts of all rules in the core rule base, generating corresponding evaluation control instructions. These evaluation control instructions include types such as immediately initiating in-depth evaluation, arranging subsequent specialized assessments, sending early warning notifications, or requiring no action. Establish a communication interface for the decision engine. The communication interface is responsible for receiving query requests from external systems and forwarding evaluation trigger instructions generated by the decision engine. The communication interface is usually implemented in the form of an application programming interface gateway or a message queue producer to ensure that the decision engine can exchange data bidirectionally with external systems such as the evaluation task scheduling center and the teaching management platform.

[0092] In some embodiments, the rule matching process in the core rule base can be quantified as the calculation of a rule trigger score, which is used to determine priority when multiple rule conditions are met simultaneously. When the condition part of a rule is satisfied by state information, the instruction generator generates a structured instruction object. In an example scenario, the dynamic capability modeling network outputs a dynamic capability profile of a student in the "Electrical Control" course module. The state monitoring module listens to this event and obtains a score of 0.6 in the "Circuit Diagram Reading" dimension of the profile, while the stage capability fit calculation result is 0.68. There are two related rules in the core rule base: the condition of rule R1 is "stage capability fit < 0.7", and the action is "generate preliminary evaluation trigger instruction"; the condition of rule R2 is "circuit diagram reading capability score < 0.65 and stage capability fit < 0.7", and the action is "generate high-priority evaluation trigger instruction and notify the teacher". The state monitoring module sends the state information "circuit diagram reading capability score = 0.6" and "stage capability fit = 0.68" to the instruction generator. The instruction generator performs a matching calculation on these two rules, calculating a matching score for each rule. Rule R1's condition is fully satisfied, and both sub-conditions of rule R2 are also satisfied simultaneously. To determine which rule is ultimately triggered, the instruction generator uses the following formula to calculate the final trigger score for each rule that meets the conditions:

[0093]

[0094] in: Representation rules The final trigger score, This indicates the total number of conditions for this rule. Indicates the first The weights of each condition are preset within the rule. It is an indicator function, when the first One condition The value is 1 if the condition is met, and 0 otherwise. Assume rule R1 has only one condition, and its weight... It is 1.0, and its Rule R2 has two conditions, each with a weight of 1.0. Based on the calculation results, rule R2 has a higher final trigger score, so the instruction generator ultimately selects to trigger rule R2 and generates a high-priority evaluation trigger instruction.

[0095] It is understandable that the core rule base can be added, deleted, modified, and queried by the academic administrator through the management interface to adapt to changes in evaluation strategies for different courses or stages. The design of the status monitoring module needs to ensure high throughput and low latency. The status monitoring module typically adopts an event-driven architecture to handle concurrent incoming status events. The evaluation control instructions generated by the instruction generator are encapsulated in a standard message format, including fields such as instruction type, target object, triggering rule ID, timestamp, and instruction payload. Optionally, the decision engine's communication interface not only processes output but also receives query requests from external systems, such as querying the latest status of a certain evaluation object or the current rule set version of the decision engine. The implementation of the communication interface needs to consider security authentication and access control to ensure that only authorized system components can submit status information or obtain instructions.

[0096] In some embodiments, the decision logic of the core rule base can be implemented using a rule engine based on the Rete algorithm to achieve efficient pattern matching. After generating an instruction, the instruction generator can record the context information of the decision, including the triggered rule, the input state snapshot, and the calculated final trigger score, in the decision log for auditing and rule optimization analysis. It can be understood that when deploying a state monitoring module for the decision engine, the state monitoring module needs to maintain a state snapshot of all currently monitored evaluation objects. The state snapshot is updated when a new event is received to ensure that rule matching is based on the latest complete context. The evaluation control instructions generated by the instruction generator are pushed to a dedicated instruction message queue through a communication interface. The evaluation task scheduling center, as a consumer, retrieves and executes instructions from this queue.

[0097] See Figure 5 This is a line graph comparing the performance of the status monitoring module in a vocational education teaching evaluation decision engine. It's a double-line graph showing the trends of "baseline latency" and "optimized latency" as event throughput changes, corresponding to the performance evaluation of the status monitoring phase. When event throughput increases, baseline latency shows a rapid upward trend, while optimized latency increases more gradually, demonstrating the performance advantages of the event-driven architecture. This graph is used to verify the performance optimization effect of the status monitoring module and is a core basis for the decision engine's high throughput and low latency operation, ensuring the system can stably monitor the status even when a large amount of teaching behavior data flows in. In decision engine architecture design, this type of chart is a key tool for demonstrating the value of optimization strategies such as "event-driven" approaches, intuitively showing the role of module performance in ensuring system stability.

[0098] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0099] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A vocational education teaching evaluation method based on big data, characterized in that, The method includes: A teaching evaluation data lake is constructed in the vocational education management system, which receives raw teaching behavior data streams from nodes throughout the vocational education process; In the teaching evaluation data lake, the original teaching behavior data stream is analyzed in layers to extract structured evaluation elements and unstructured evaluation elements; The structured evaluation elements and unstructured evaluation elements are fused and mapped to generate a unified behavioral representation sequence for vocational education evaluation objects; The unified behavioral representation sequence is input into the dynamic ability modeling network of the vocational education evaluation object, and the dynamic ability profile of the vocational education evaluation object is output. The dynamic ability profile is matched with a preset vocational ability map to generate the stage-specific ability fit of the vocational education evaluation object. The teaching evaluation decision engine starts the evaluation process based on the stage-specific ability fit, and the teaching evaluation decision engine generates a preliminary evaluation trigger command based on the stage-specific ability fit.

2. The vocational education teaching evaluation method based on big data as described in claim 1, characterized in that, The construction of a teaching evaluation data lake within the vocational education management system, wherein the teaching evaluation data lake receives raw teaching behavior data streams from nodes throughout the entire vocational education process, including: Identify and mark theoretical teaching nodes, practical training nodes, project assessment nodes, and teacher-student interaction nodes in the entire vocational education process as data source nodes; Deploy lightweight data probes on each data source node to continuously capture teaching process records and operation behavior logs containing time-series information generated by the data source node; Define the data access specifications for the teaching evaluation data lake, and perform format cleaning on the teaching process records and operation behavior logs according to the data access specifications to form a raw teaching behavior data unit stream with a unified timestamp; All raw teaching behavior data units are asynchronously written into the distributed storage layer of the teaching evaluation data lake to form raw teaching behavior data streams.

3. The vocational education teaching evaluation method based on big data as described in claim 2, characterized in that, The process involves hierarchical analysis of the original teaching behavior data stream within the teaching evaluation data lake to extract structured and unstructured evaluation elements, including: The parsing service of the teaching evaluation data lake is invoked to perform layered processing on the original teaching behavior data stream; In the first parsing layer, data units containing numerical indicators and discrete labels are identified and separated, and these data units are grouped into a set of structured evaluation elements. In the second parsing layer, semantic understanding and feature extraction are performed on the remaining text records, audio segments and image frame data to extract feature vectors that reflect teaching attitude, collaborative tendency and innovative thinking. The feature vectors are then merged into an unstructured evaluation element set.

4. The vocational education teaching evaluation method based on big data as described in claim 3, characterized in that, The process of fusing and mapping the structured evaluation elements with the unstructured evaluation elements to generate a unified behavioral representation sequence for vocational education evaluation objects includes: An independent behavioral mapping space is established for the evaluation objects of vocational education, and each element in the set of structured evaluation elements is mapped to a coordinate point in the behavioral mapping space; Each feature vector in the unstructured evaluation element set is transformed into a trajectory segment in the behavior mapping space, and the endpoints of the trajectory segment are determined by the initial and final states of the feature vector. In the behavior mapping space, coordinate points formed by structured evaluation elements and trajectory segments formed by unstructured evaluation elements are connected in chronological order to generate a continuous behavior trajectory, which is then encoded as a unified behavior representation sequence for vocational education evaluation objects.

5. The vocational education teaching evaluation method based on big data as described in claim 1, characterized in that, The step of inputting the unified behavioral representation sequence into the dynamic competency modeling network of the vocational education evaluation object and outputting the dynamic competency profile of the vocational education evaluation object includes: A dynamic capability modeling network is constructed, which includes a temporal feature extraction layer and a capability state inference layer. The temporal feature extraction layer performs sliding window analysis on the unified behavioral representation sequence of the input vocational education evaluation object to extract temporal feature segments that reflect the trend of ability change. The ability state inference layer receives the temporal feature segments and, with reference to the historical ability evolution path, infers the comprehensive ability state vector of the vocational education evaluation object at the current moment. The comprehensive ability state vector constitutes the core of the dynamic ability profile.

6. The vocational education teaching evaluation method based on big data as described in claim 5, characterized in that, The step of matching the dynamic competency profile with a preset vocational competency map to generate a stage-specific competency fit for the vocational education evaluation subject includes: A vocational competency graph corresponding to the major of the vocational education evaluation object is loaded from the vocational education knowledge base. The vocational competency graph consists of multiple interconnected competency nodes and competency edges. The comprehensive competency state vector in the dynamic competency profile is projected onto the multidimensional competency space defined by the vocational competency graph. The projection intensity of the comprehensive competency state vector on each competency node of the vocational competency graph is calculated. Several competency nodes with projection intensity exceeding a threshold are selected as current matching nodes. Based on the number, weight and topological importance of the current matching nodes in the vocational competency graph, a quantitative stage competency fit degree is generated through aggregation calculation.

7. The vocational education teaching evaluation method based on big data as described in claim 6, characterized in that, The teaching evaluation decision engine, driven by the stage-based competency fit, initiates the evaluation process. The teaching evaluation decision engine generates a preliminary evaluation trigger instruction based on the stage-based competency fit, including: The teaching evaluation decision engine is set with a key threshold. When the received stage ability fit is lower than the key threshold, the in-depth evaluation process is triggered. The teaching evaluation decision engine generates a preliminary evaluation trigger instruction that includes the trigger time, target object identifier, and required evaluation dimensions, and sends the preliminary evaluation trigger instruction to the evaluation task scheduling center.

8. The vocational education teaching evaluation method based on big data as described in claim 1, characterized in that, The construction of the teaching evaluation decision engine includes: The core rule base for configuring the decision engine stores evaluation triggering conditions and decision logic based on vocational education evaluation standards. A status monitoring module is deployed for the decision engine. The status monitoring module listens in real time to the dynamic capability profile output events from the dynamic capability modeling network and the calculation results of the phased capability fit. An instruction generator is integrated into the decision engine. The instruction generator matches the input state information according to the decision logic in the core rule base and generates corresponding evaluation control instructions. A communication interface for the decision engine is established, which is responsible for receiving query requests from external systems and forwarding evaluation trigger instructions generated by the decision engine.

9. The vocational education teaching evaluation method based on big data as described in claim 6, characterized in that, The step of projecting the comprehensive ability state vector from the dynamic ability profile onto the multidimensional ability space defined by the professional ability map, and calculating the projection intensity of the comprehensive ability state vector on each ability node of the professional ability map, includes: Analyze the vector representation of each ability node in the professional ability graph and construct a set of basis vectors for a multidimensional ability space; The integrated capability state vector is orthogonally decomposed according to the basis vector set to obtain the magnitude of the components of the integrated capability state vector in each basis vector direction; The magnitude of the component in the direction of the basis vector corresponding to each capability node is normalized to obtain the projection intensity value on the capability node. Traverse all ability nodes in the professional ability graph, repeatedly perform the projection calculation process, and generate a projection distribution map containing the projection intensity values ​​of each ability node.

10. The vocational education teaching evaluation method based on big data as described in claim 7, characterized in that, The teaching evaluation decision engine is equipped with a key threshold. When the received stage-based competency fit is lower than the key threshold, the in-depth evaluation process is triggered, including: The range of key threshold values ​​is determined based on the statistical distribution of historical evaluation data, and specific key threshold values ​​are set in conjunction with the teaching objectives of the current evaluation cycle. The teaching evaluation decision engine continuously monitors the input phased ability fit data stream and compares the phased ability fit of each evaluation object with key thresholds in real time. When it is detected that the phased capability fit of an evaluation object is continuously lower than the key threshold for a preset period of time, the decision to start the in-depth evaluation process is triggered. Once the in-depth evaluation process is initiated, the teaching evaluation decision engine automatically invokes the expanded evaluation indicator library and enhanced analysis algorithms to conduct a multi-dimensional in-depth evaluation of the target evaluation object.

Images