A multi-modal vocational education system evaluation system and evaluation method

Abstract

This invention discloses a multimodal vocational education system evaluation system and method, belonging to the field of vocational education evaluation technology. The system includes the following components: a multimodal data acquisition module, a multimodal big data processing module, a multidimensional evaluation system construction module, and an evaluation result output and guidance module. By collecting multiple types of data and utilizing the multimodal big data processing module for in-depth preprocessing, feature extraction, and data fusion, this invention achieves a comprehensive evaluation of the vocational education system. The multidimensional evaluation system construction module combines basic evaluation dimensions, vocational ability evaluation dimensions, resource matching evaluation dimensions, and industry-education integration evaluation dimensions, ensuring the comprehensiveness and scientific nature of the evaluation. This end-to-end technical implementation makes the evaluation results more objective and accurate, providing strong data support for the innovative development of vocational education.

Description

Technical Field

[0001] This invention relates to the field of vocational education evaluation technology, specifically to a multimodal vocational education system evaluation system and evaluation method. Background Technology

[0002] With the rapid development of vocational education, how to scientifically and comprehensively evaluate the quality and effectiveness of the vocational education system has become a key issue in improving the level of vocational education and promoting innovative development in education.

[0003] Traditional vocational education evaluation systems suffer from several shortcomings: First, data sources are limited, often relying solely on student grades, teacher evaluations, or questionnaires, making it difficult to fully reflect the complexity and diversity of the vocational education process. Second, evaluation methods are highly subjective, lacking scientific and objective evaluation indicators and quantitative models, leading to susceptibility to human factors and limitations in accuracy and fairness. Furthermore, traditional evaluation systems lack the technical expertise in data preprocessing, feature extraction, and data fusion, hindering the effective processing and analysis of multimodal data and thus limiting the depth and breadth of evaluation. Finally, traditional evaluation results are presented in a simplistic manner, lacking intuitive, visual presentations and in-depth analysis, making it difficult to quickly translate into practical guidance and promote the continuous improvement of vocational education quality.

[0004] In view of the shortcomings of the traditional vocational education evaluation system, the present invention proposes a multimodal vocational education system evaluation system and evaluation method, which is of particular importance. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a multimodal vocational education system evaluation system and method. This system can collect multiple types of data and utilize a multimodal big data processing module for in-depth preprocessing, feature extraction, and data fusion, achieving a comprehensive evaluation of the vocational education system. Simultaneously, it constructs a multi-dimensional evaluation index system including basic evaluation dimensions, vocational ability evaluation dimensions, resource matching evaluation dimensions, and industry-education integration evaluation dimensions, ensuring the comprehensiveness and scientific nature of the evaluation. Finally, through visualization tools and intelligent analysis models, the evaluation results are intuitively displayed and transformed into practical guidance, effectively promoting the continuous improvement of vocational education quality.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On the one hand, a multimodal vocational education system evaluation system, the system comprising the following components: a multimodal data acquisition module, a multimodal big data processing module, a multidimensional evaluation system construction module, and an evaluation result output and guidance module;

[0007] The multimodal data acquisition module collects various types of data in vocational education scenarios, including at least three types of data: text data, image data, audio data, video data, and sensor data.

[0008] The multimodal big data processing module is communicatively connected to the multimodal data acquisition module and has a built-in multimodal big data model to preprocess, extract features, and fuse data from the acquired multi-type data.

[0009] The multi-dimensional evaluation system construction module communicates with the multi-modal big data processing module to construct a multi-dimensional evaluation index system that includes basic evaluation dimensions, professional ability evaluation dimensions, resource matching evaluation dimensions, and industry-education integration evaluation dimensions, and calculates the quantitative values ​​of each evaluation index based on the integrated multi-modal data.

[0010] The evaluation result output and guidance module communicates with the multi-dimensional evaluation system construction module, and is used to comprehensively analyze the quantitative values ​​of evaluation indicators, generate a visual evaluation report, and output reference guidance for the innovative development of vocational education based on the evaluation results.

[0011] Furthermore, the text data acquisition unit of the multimodal data acquisition module employs industrial-grade OCR optical character recognition technology combined with a domain-pre-trained NLP model to achieve the structured conversion of unstructured text. The specific implementation steps are as follows: First, paper lesson plans, training reports, and enterprise feedback text materials are acquired using a 4K high-definition scanner and converted into editable text via OCR recognition. Then, a BERT model with a finely tuned vocabulary in the vocational education field is used to perform word segmentation, entity recognition, and semantic annotation on the text, removing invalid characters, duplicate content, and format noise. Finally, the processed text data is stored in a distributed text database, and a SHA-256 checksum hash value is generated to ensure the integrity of data transmission and storage. For integrity, the image data acquisition unit uses a global shutter industrial camera with an infrared fill light module to acquire images of equipment operation status and student operation at a frequency of 30 frames per second in the training scenario. The image noise is removed by an adaptive median filtering algorithm, and the images are uniformly scaled to 224×224 pixels using bilinear interpolation to adapt to the input requirements of the subsequent feature extraction model. The audio data acquisition unit uses a directional array microphone to collect teacher-student interaction voices and equipment operation audio in classroom and training scenarios. The audio is denoised by a wavelet transform algorithm, the sampling rate is uniformly set to 16kHz, and MFCC features are extracted for preliminary feature compression to ensure the validity and consistency of the audio data.

[0012] Furthermore, the multimodal feature extraction sub-model of the multimodal big data processing module adopts improved feature extraction strategies for different types of data: for text data, an improved BERT model incorporating professional terminology from the vocational education field is used. By introducing industry standards and training manual corpora during the pre-training stage, the model's ability to understand professional semantics is enhanced, extracting semantic feature vectors with a dimension of 512; for image data, an improved ResNet-50 model with added channel attention mechanism is used, introducing an SE module in the last convolutional layer to enhance the understanding of key components of training equipment and student operations. The system is capable of extracting action features, producing visual feature vectors with a dimension of 512. For audio data, the MFCC algorithm combined with a bidirectional LSTM model is used to extract temporal acoustic feature vectors containing speech intonation and device operating status, with a dimension of 512. For video data, a 3D-CNN model is used to extract action feature vectors containing spatiotemporal correlations, with a dimension of 512. For sensor data, an LSTM model is used to extract temporal feature vectors containing device operating parameters and operation trajectories, with a dimension of 512. The dimension of feature vectors for each modality is unified to 512, providing a foundation for subsequent cross-modal fusion.

[0013] Furthermore, the data fusion sub-model of the multimodal big data processing module adopts a cross-modal attention fusion algorithm, the formula of which is: ,in For the first Feature vectors of each modality The contribution weights of each modality are determined by the mutual information between the modality's features and the evaluation objective. Normalization yields, Mutual information is obtained by calculating the joint probability distribution and marginal probability distribution of features and evaluation targets. For attention mechanisms, in which The cross-modal context vector is obtained by mean pooling of all modal features. As an element-wise multiplication algorithm, this algorithm dynamically adjusts the contribution of each modality feature through an attention mechanism, achieving deep fusion of multimodal features and significantly improving the representational ability of the fused features.

[0014] Furthermore, the evaluation index weights of the multi-dimensional evaluation system construction module are determined through a subjective and objective weight fusion algorithm, as shown in the formula: ,in The subjective weights obtained using the analytic hierarchy process (AHP) were calculated by pairwise comparison judgment matrices by 5 vocational education experts and 3 industry experts. The objective weights obtained by the entropy weight method are derived by calculating the information entropy and difference coefficient of each indicator data. The balancing coefficient is obtained by weighting the prediction accuracy of the two weighting methods on the validation set. This algorithm combines expert experience with objective data laws, effectively improving the scientific nature and accuracy of the weights.

[0015] Furthermore, the specific implementation steps of the basic evaluation dimensions of the multi-dimensional evaluation system construction module are as follows: First, obtain course plans, teaching outlines, and industry standard documents through the text data acquisition unit; calculate the rationality index of the course setting; and use the Jaccard coefficient to calculate the matching degree between course knowledge points and industry standard knowledge points. ,in It is a collection of course knowledge points. The evaluation process involves three key steps: first, calculating the set of industry-standard knowledge points; second, using data from the teaching management system to calculate the teaching plan completion rate, which is the ratio of actual completed class hours to planned class hours multiplied by the pass rate; third, using data from the teacher management system to calculate the teacher educational structure indicator, which is the proportion of teachers with doctoral, master's, and bachelor's degrees to the total number of teachers; and fourth, using data from the equipment management system to calculate the teaching equipment availability rate, which is the ratio to the total equipment operating time. After each indicator is standardized using the min-max method, the weights are calculated using a subjective and objective weight fusion algorithm to obtain a comprehensive score for the basic evaluation dimensions.

[0016] Furthermore, the specific implementation steps of the professional competence evaluation dimension of the multi-dimensional evaluation system construction module are as follows: First, calculate the theoretical knowledge mastery index using exam paper and homework text data, which is a weighted average of exam scores and homework quality scores; second, calculate the practical skills proficiency index using training video data, which is the cosine similarity between student action features extracted by 3D-CNN and standard action features, multiplied by the normalized value of completion time; then, calculate the professional quality attainment index using enterprise feedback text and behavioral evaluation data, and use the fuzzy comprehensive evaluation method to combine the fuzzy matrix of enterprise feedback scores and behavioral evaluations to obtain the comprehensive score of professional quality; finally, calculate the job suitability index using internship videos and job requirement data, which is the cosine similarity between student skill features and job requirement features. After each index is standardized by z-score, the weights are calculated by combining the subjective and objective weight fusion algorithm to obtain the comprehensive score of the professional competence evaluation dimension.

[0017] Furthermore, the specific implementation steps of the resource adaptation and industry-education integration evaluation dimensions of the multi-dimensional evaluation system construction module are as follows: First, calculate the matching degree index of training resources through training equipment data and professional training objectives, and use the analytic hierarchy process (AHP) to calculate the weighted sum of the matching degree of equipment quantity, type, and demand; second, calculate the resource update timeliness index through the teaching resource update time and industry technology iteration time, which is the normalized value of ; then, calculate the resource utilization rate index through school-enterprise cooperation project data, which is the ratio of (number of students participating × number of projects) to (enterprise investment funds + equipment value); then, calculate the resource utilization rate index through the teaching time of enterprise lecturers and The number of training projects developed is used to calculate the depth of enterprise participation, which is a normalized value of (lecturer teaching hours × 0.5 + number of projects developed × 0.5). Then, the project fit index is calculated by comparing the training project content with job task data, and the matching degree is calculated using the Jaccard coefficient. Finally, the employment quality index is calculated by using graduate employment data, which is a weighted average of (employment rate × 0.3 + average salary × 0.4 + job matching degree × 0.3). After each index is standardized by min-max, the weights are calculated by combining the subjective and objective weights with the weights calculated by the algorithm, and a comprehensive score for the evaluation dimensions of resource matching and industry-education integration is obtained.

[0018] Furthermore, the specific implementation steps of the evaluation result output and guidance module are as follows: First, a multi-dimensional evaluation score radar chart, a bar chart of each indicator score, and an indicator correlation heatmap are generated using visualization tools. The radar chart visually displays the balance of scores across dimensions, the bar chart compares the score differences of each indicator, and the heatmap uses the Pearson correlation coefficient to show the correlation strength between indicators. Second, a finely tuned GPT-4o natural language generation model, combined with the correlation analysis of evaluation scores and indicators, generates a text report containing strengths, weaknesses, and improvement suggestions. Finally, through a rule engine, preset improvement rules are matched to the evaluation weaknesses. For majors with unreasonable curriculum settings, course module reconstruction rules are triggered, outputting suggestions for optimizing course knowledge points. For students' practical skills weaknesses, practical training project optimization rules are triggered, outputting suggestions to increase practical training time and introduce real production projects. For the problem of insufficient industry-education integration, rules for innovation in the school-enterprise joint training model are triggered, outputting suggestions for jointly building training bases and enterprise mentors' teaching. This module achieves efficient transformation of evaluation results into practical guidance through visualization and intelligent analysis.

[0019] On the other hand, a multimodal vocational education system evaluation method is characterized by the following specific steps:

[0020] S1. Multimodal data acquisition: Text, image, audio, video, and sensor data are acquired through various acquisition units and stored in a distributed database;

[0021] S2. Data preprocessing and feature extraction: The preprocessing sub-model cleans and normalizes the data, and the feature extraction sub-model extracts feature vectors for each modality.

[0022] S3. Multimodal data fusion: A fused feature vector is obtained through a cross-modal attention fusion algorithm.

[0023] S4. Evaluation index calculation: The scores of each index are calculated based on the fusion algorithm of fusion features and subjective and objective weights.

[0024] S5. Evaluation Result Analysis and Guidance Generation: Evaluation reports and innovation guidelines are generated through visualization tools and rule engines. This method ensures the scientific rigor, comprehensiveness, and guidance of the evaluation through full-process technical implementation.

[0025] Compared with existing technologies, this multimodal vocational education system evaluation system and method have the following beneficial effects:

[0026] First, by collecting multiple types of data and utilizing a multimodal big data processing module for in-depth preprocessing, feature extraction, and data fusion, this system achieves a comprehensive evaluation of the vocational education system. The multi-dimensional evaluation system construction module combines basic evaluation dimensions, vocational ability evaluation dimensions, resource matching evaluation dimensions, and industry-education integration evaluation dimensions, ensuring the comprehensiveness and scientific nature of the evaluation. This end-to-end technical implementation makes the evaluation results more objective and accurate, providing strong data support for the innovative development of vocational education.

[0027] Second, the system generates multi-dimensional evaluation score radar charts, bar charts of each indicator score, and indicator correlation heatmaps through visualization tools via the evaluation result output and guidance module. This intuitively displays the balance of scores in each dimension and the strength of correlation between indicators. At the same time, combined with a finely tuned GPT-4o natural language generation model, the system can automatically generate text reports containing strengths, weaknesses, and improvement suggestions. Through the rule engine, it matches preset improvement rules to the evaluation weaknesses. This intelligent analysis method enables the evaluation results to be quickly transformed into practical guidance, helping vocational education institutions to adjust teaching strategies and optimize resource allocation in a timely manner, thereby promoting the continuous improvement of vocational education quality.

[0028] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0030] Figure 1 A flowchart for an evaluation system of a multimodal vocational education system;

[0031] Figure 2 A flowchart illustrating the specific implementation process of the evaluation result output and guidance module in a multimodal vocational education system evaluation system;

[0032] Figure 3 This is a flowchart of an evaluation method for a multimodal vocational education system. Detailed Implementation

[0033] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0034] Example 1

[0035] Following the process of the multimodal vocational education system evaluation method, such as Figure 3 As shown, the specific implementation process of this embodiment is as follows: The multi-modal data acquisition module acquires various types of data related to intelligent manufacturing professional training through its various units and stores them in a distributed database. The text data acquisition unit uses a 4K high-definition scanner to collect paper course plans, student training reports, and skills assessment text materials from partner companies. After being converted into editable text using industrial-grade OCR optical character recognition technology, the text is then segmented, entity-based, and semantically annotated using a BERT model with a fine-tuned vocabulary in the vocational education field. Invalid characters, duplicate content, and format noise are removed. Finally, the processed text data is stored in a distributed text database, and a SHA-256 checksum hash value is generated to ensure data integrity.

[0036] The image data acquisition unit uses a global shutter industrial camera with an infrared fill light module to capture images of the operating status of CNC machine tools, robots, and other training equipment, as well as images of students operating the equipment, at a frequency of 30 frames per second in the training workshop. Adaptive median filtering is used to remove image noise, and bilinear interpolation is then used to uniformly scale all images to 224×224 pixels. The audio data acquisition unit uses a directional array microphone to collect interactive voice messages between teachers and students regarding key points of equipment operation, as well as audio information of the training equipment during operation, in classroom teaching and training scenarios. Noise reduction is performed using a wavelet transform algorithm, the audio sampling rate is uniformly set to 16kHz, and MFCC features are extracted for preliminary feature compression. The video data acquisition unit records the entire process of students completing intelligent manufacturing training projects. The sensor data acquisition unit collects the operating parameters of the training equipment and the trajectory data of students operating the equipment.

[0037] The preprocessing sub-model of the multimodal big data processing module cleans and normalizes the collected text, image, audio, video, and sensor data to ensure consistent data format and quality. Then, the feature extraction sub-model extracts feature vectors for each modality: for text data, an improved BERT model incorporating professional terminology from the field of intelligent manufacturing in vocational education is used. This model incorporates corpora from intelligent manufacturing industry standards and training manuals during the pre-training phase, enhancing its understanding of professional semantics, ultimately extracting semantic feature vectors with a dimension of 512; for image data, an improved ResNet-50 model with added channel attention mechanism is used. The model introduces an SE module in the last convolutional layer to enhance feature extraction of key components of the training equipment and student operation actions, extracting visual feature vectors with a dimension of 512. For audio data, the MFCC algorithm combined with a bidirectional LSTM model is used to extract temporal acoustic feature vectors containing teacher and student voice intonation and equipment operating status information, with a dimension of 512. For video data, a 3D-CNN model is used to extract student operation action feature vectors containing spatiotemporal correlations, with a dimension of 512. For sensor data, an LSTM model is used to extract temporal feature vectors containing changes in equipment operating parameters and patterns in student operation trajectories, with a dimension of 512.

[0038] The data fusion sub-model of the multimodal big data processing module employs a cross-modal attention fusion algorithm, with the following formula: ,in For the first Feature vectors of each modality Assign weights to the contributions of each modality. For attention mechanisms, This method employs element-wise multiplication, combining the contribution weights of each modal feature vector with an attention mechanism. It performs element-wise multiplication and fusion processing on the 512-dimensional feature vectors of five modalities: text, image, audio, video, and sensor data. The final result is a fused feature vector that comprehensively reflects various aspects of the intelligent manufacturing professional training system. Figure 1 As shown, the system ensures the continuity of data processing and the scientific nature of evaluation through a complete process design of multimodal data acquisition, processing, fusion, and evaluation analysis.

[0039] Based on a multi-dimensional evaluation system construction module, the scores of each evaluation indicator are calculated using a fusion algorithm that combines fused feature vectors and subjective / objective weights. The formula is as follows: ,in The subjective weights obtained by the analytic hierarchy process (AHP) The objective weights obtained by the entropy weight method. To balance the evaluation criteria, in terms of basic evaluation dimensions, the rationality index of the course design was calculated using text data, and the Jaccard coefficient was used to calculate the matching degree between the course knowledge points and industry standard knowledge points. ,in It is a collection of course knowledge points. The system is a collection of industry standard knowledge points. It calculates the teaching plan completion rate index through data from the teaching management system, the teacher academic qualifications structure index through data from the teacher management system, and the teaching equipment integrity rate index through data from the equipment management system. After the index is standardized by min-max, the scores of the basic evaluation dimensions are obtained by combining the weights.

[0040] In terms of the professional competence evaluation dimension, the following indicators are used: theoretical knowledge mastery index (based on exam papers and assignment text data); practical skills proficiency index (based on training video data); professional competence attainment index (based on enterprise feedback text and behavioral evaluation data using fuzzy comprehensive evaluation); and job suitability index (based on internship videos and job requirement data). All indicators are standardized using z-scores and weighted to obtain a comprehensive score for each professional competence evaluation dimension. In terms of the resource matching and industry-education integration evaluation dimension, the following indicators are used: training equipment matching index (based on the analytic hierarchy process); resource update timeliness index (based on the update time of teaching resources and the iteration time of industry technologies); resource utilization rate index (based on school-enterprise cooperation project data); enterprise participation depth index (based on the teaching hours of enterprise lecturers and the number of training projects developed); project fit index (based on the Jaccard coefficient); and employment quality index (based on graduate employment data). All indicators are standardized using min-max methods and weighted to obtain a comprehensive score for this dimension.

[0041] Through the evaluation result output and guidance module, such as Figure 2As shown, firstly, visualization tools are used to generate a multi-dimensional evaluation score radar chart, bar charts of scores for each indicator, and a heatmap of indicator correlations. The radar chart visually displays the balance of scores across the four dimensions of basic evaluation, professional competence, resource matching, and industry-education integration. The bar charts clearly present the score differences for each sub-indicator, and the heatmap uses Pearson correlation coefficients to show the strength of correlations between different indicators. Subsequently, a finely tuned GPT-4o natural language generation model is used, combined with the results of correlation analysis between evaluation scores and indicators across dimensions, to generate a text report containing the advantages, shortcomings, and targeted improvement suggestions of the intelligent manufacturing professional training system. Finally, a rule engine is used to match pre-defined improvement rules to the shortcomings identified in the evaluation, outputting a reference guide for the innovative development of vocational education.

[0042] Example 2

[0043] According to the established procedure of the evaluation method, such as Figure 3 As shown, the specific implementation process of this embodiment is as follows: The multimodal data acquisition module acquires various types of data related to clinical skills in the nursing profession and stores them in a distributed database. The text data acquisition unit uses a 4K high-definition scanner to collect paper-based course plans, student nursing records, and hospital internship feedback text materials in the nursing profession. After being converted into editable text using industrial-grade OCR optical character recognition technology, the text is segmented, entity recognized, and semantically annotated using a BERT model with a fine-tuned vocabulary in the vocational education field. Invalid characters, duplicate content, and format noise are removed. The processed text data is stored in the distributed text database, and a SHA-256 checksum hash value is generated to ensure the integrity of data transmission and storage.

[0044] The image data acquisition unit uses a global shutter industrial camera with an infrared illumination module to capture images of the operational status of nursing equipment such as infusion pumps and monitors, as well as images of students performing clinical skills such as intravenous infusions and aseptic procedures, at a frequency of 30 frames per second in the nursing training center. Adaptive median filtering is used to remove image noise, and bilinear interpolation is used to uniformly scale all images to 224×224 pixels. The audio data acquisition unit uses a directional array microphone to capture interactive voice conversations between teachers and students regarding nursing operation procedures, as well as audio information of nursing equipment operation, in nursing classroom teaching and clinical training scenarios. Noise reduction is performed using wavelet transform algorithms, the audio sampling rate is uniformly set to 16kHz, and MFCC features are extracted for preliminary feature compression. The video data acquisition unit records the entire process of students completing nursing clinical training projects. The sensor data acquisition unit collects vital sign sensor data from nursing mannequins and trajectory data such as the force and speed of students operating nursing equipment.

[0045] The preprocessing sub-model of the multimodal big data processing module cleans and normalizes the collected data, removes abnormal data and redundant information, and standardizes the data format. Next, feature vectors for each modality are extracted through a feature extraction sub-model: For text data, an improved BERT model incorporating professional terminology from the field of vocational education nursing is used. During the pre-training phase, corpora from nursing industry standards and training manuals are introduced to enhance the understanding of professional semantics, resulting in a semantic feature vector with a dimension of 512. For image data, an improved ResNet-50 model with added channel attention mechanism is used, introducing an SE module in the last convolutional layer to strengthen feature extraction of key components of nursing equipment and student operational actions, resulting in a visual feature vector with a dimension of 512. For audio data, the MFCC algorithm combined with a bidirectional LSTM model is used to extract temporal acoustic feature vectors containing teacher and student voice intonation and the operating status of nursing equipment, with a dimension of 512. For video data, a 3D-CNN model is used to extract feature vectors of student nursing operational actions containing spatiotemporal correlations, with a dimension of 512. For sensor data, an LSTM model is used to extract temporal feature vectors containing simulated human vital sign changes and student operational trajectories, with a dimension of 512.

[0046] Using the data fusion sub-model of the multimodal big data processing module, and employing a cross-modal attention fusion algorithm, this algorithm combines the contribution weights of each modality's feature vectors with an attention mechanism to perform element-wise multiplication and fusion operations on the 512-dimensional feature vectors from five modalities: text, image, audio, video, and sensor data. This yields a fused feature vector that comprehensively reflects all dimensions of the nursing professional clinical skills evaluation system. Figure 1 As shown, the coordinated operation of the various modules of the system enables the efficient transformation from multimodal data to fused features.

[0047] Based on the multi-dimensional evaluation system construction module, the scores of each evaluation indicator are calculated by combining the fusion feature vector and the subjective and objective weight fusion algorithm. In terms of basic evaluation dimensions, the rationality index of curriculum setting is calculated by using Jaccard coefficient through text data, the completion index of teaching plan is calculated by using data from the teaching management system, the academic structure index of teachers is calculated by using data from the teacher management system, and the integrity rate index of teaching equipment is calculated by using data from the equipment management system. After each indicator is standardized by min-max, the comprehensive score of basic evaluation dimensions is obtained by combining the weights.

[0048] In terms of professional competence evaluation, the following indicators are used: theoretical knowledge mastery index (based on exam papers and assignment text data); practical skills proficiency index (based on training video data); professional ethics attainment index (based on hospital feedback text and behavioral evaluation data using fuzzy comprehensive evaluation); and job suitability index (based on internship videos and nursing job requirements data). All indicators are standardized using z-scores and weighted to obtain a comprehensive score for each professional competence evaluation dimension. In terms of resource matching and industry-education integration evaluation, the following indicators are used: training resource matching index (based on the analytic hierarchy process) (based on nursing training equipment data and professional training objectives); resource update timeliness index (based on the update time of teaching resources and the technological iteration time in the nursing industry); resource utilization rate index (based on school-enterprise cooperation project data); enterprise participation depth index (based on the teaching hours of hospital lecturers and the number of training projects developed); project fit index (based on the Jaccard coefficient) (based on training project content and nursing job task data); and employment quality index (based on graduate employment data). All indicators are standardized using min-max and weighted to obtain a comprehensive score for each dimension.

[0049] Through the evaluation result output and guidance module, such as Figure 2 As shown, visualization tools were used to generate a multi-dimensional evaluation score radar chart, bar charts for each indicator score, and a heatmap of indicator correlation. The radar chart visually displays the balance of scores across the four dimensions: basic evaluation, professional competence, resource matching, and industry-education integration. The bar charts compare the score differences of each sub-indicator, and the heatmap uses Pearson correlation coefficients to present the correlation strength between different indicators. Subsequently, a finely tuned GPT-4o natural language generation model was used, combined with the correlation analysis results of evaluation scores and indicators across dimensions, to generate a text report containing the advantages, shortcomings, and improvement suggestions of the nursing professional clinical skills evaluation system. Finally, a rule engine was used to match pre-defined improvement rules to the shortcomings identified in the evaluation, outputting a reference guide for the innovative development of nursing professional vocational education.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A multi-modal vocational education system evaluation system, characterized in that, The system comprises the following components: a multimodal data acquisition module, a multimodal big data processing module, a multidimensional evaluation system construction module, and an evaluation result output and guidance module. The multimodal data acquisition module collects various types of data in vocational education scenarios, including at least three types of data: text data, image data, audio data, video data, and sensor data. The multimodal big data processing module is communicatively connected to the multimodal data acquisition module and has a built-in multimodal big data model to preprocess, extract features, and fuse data from the acquired multi-type data. The multi-dimensional evaluation system construction module communicates with the multi-modal big data processing module to construct a multi-dimensional evaluation index system that includes basic evaluation dimensions, professional ability evaluation dimensions, resource matching evaluation dimensions, and industry-education integration evaluation dimensions, and calculates the quantitative values ​​of each evaluation index based on the integrated multi-modal data. The evaluation result output and guidance module communicates with the multi-dimensional evaluation system construction module, and is used to comprehensively analyze the quantitative values ​​of evaluation indicators, generate a visual evaluation report, and output reference guidance for the innovative development of vocational education based on the evaluation results.

2. The multi-modal career education system evaluation system of claim 1, wherein, The text data acquisition unit of the multimodal data acquisition module uses industrial-grade OCR optical character recognition technology combined with a domain-pre-trained NLP model to achieve the structured conversion of unstructured text. The specific implementation steps are as follows: First, paper lesson plans, training reports, and enterprise feedback text materials are acquired using a 4K high-definition scanner and converted into editable text via OCR recognition. Then, a BERT model with a finely tuned vocabulary in the vocational education field is used to perform word segmentation, entity recognition, and semantic annotation on the text, removing invalid characters, duplicate content, and formatting noise. Finally, the processed text data is stored in a distributed text database, and an SHA-256 checksum is generated. To ensure the integrity of data transmission and storage, the image data acquisition unit uses a global shutter industrial camera with an infrared fill light module to acquire images of equipment operation status and student operation at a frequency of 30 frames per second in the training scenario. The image noise is removed by an adaptive median filtering algorithm, and the images are uniformly scaled to 224×224 pixels using bilinear interpolation. The audio data acquisition unit uses a directional array microphone to acquire teacher-student interaction voice and equipment operation audio in classroom and training scenarios. The audio is denoised by a wavelet transform algorithm, the sampling rate is uniformly set to 16kHz, and MFCC features are extracted for preliminary feature compression.

3. The multimodal vocational education system evaluation system according to claim 1, characterized in that, The multimodal big data processing module employs improved feature extraction strategies for different types of data in its multimodal feature extraction sub-models: For text data, an improved BERT model incorporating professional terminology from the vocational education field is used. By introducing industry standards and training manual corpora during the pre-training phase, the model's understanding of professional semantics is enhanced, extracting semantic feature vectors with a dimension of 512. For image data, an improved ResNet-50 model with channel attention mechanism is used. An SE module is introduced in the last convolutional layer to enhance the feature extraction capabilities of key components of training equipment and student operation actions, extracting visual feature vectors with a dimension of 512. For audio data, the MFCC algorithm combined with a bidirectional LSTM model is used to extract temporal acoustic feature vectors containing speech intonation and equipment operating status, with a dimension of 512. For video data, a 3D-CNN model is used to extract action feature vectors containing spatiotemporal correlations, with a dimension of 512. For sensor data, an LSTM model is used to extract temporal feature vectors containing equipment operating parameters and operation trajectories, with a dimension of 512. The dimension of each modality feature vector is unified to 512.

4. The multimodal vocational education system evaluation system according to claim 1, characterized in that, The data fusion sub-model of the multimodal big data processing module adopts a cross-modal attention fusion algorithm, the formula of which is: ,in For the first Feature vectors of each modality Assign weights to the contributions of each modality. For attention mechanisms, This is element-wise multiplication.

5. The multimodal vocational education system evaluation system according to claim 1, characterized in that, The weights of the evaluation indicators in the multi-dimensional evaluation system construction module are determined by a subjective and objective weight fusion algorithm, and the formula is as follows: ,in The subjective weights obtained by the analytic hierarchy process (AHP) The objective weights obtained by the entropy weight method. This is the balance coefficient.

6. The multimodal vocational education system evaluation system according to claim 1, characterized in that, The specific implementation steps of the basic evaluation dimensions of the multi-dimensional evaluation system construction module are as follows: First, obtain course plans, teaching syllabi, and industry standard documents through the text data collection unit; calculate the rationality index of the course setting; and use the Jaccard coefficient to calculate the matching degree between course knowledge points and industry standard knowledge points. ,in This is a collection of course knowledge points. The evaluation process involves three key steps: first, calculating the set of industry-standard knowledge points; second, using data from the teaching management system to calculate the teaching plan completion rate, which is the ratio of actual completed class hours to planned class hours multiplied by the pass rate; third, using data from the teacher management system to calculate the teacher educational structure indicator, which is the proportion of teachers with doctoral, master's, and bachelor's degrees to the total number of teachers; and fourth, using data from the equipment management system to calculate the teaching equipment availability rate, which is the ratio to the total equipment operating time. After each indicator is standardized using the min-max method, the weights are calculated using a subjective and objective weight fusion algorithm to obtain a comprehensive score for the basic evaluation dimensions.

7. The multimodal vocational education system evaluation system according to claim 1, characterized in that, The specific implementation steps of the vocational ability evaluation dimension of the multi-dimensional evaluation system construction module are as follows: First, calculate the theoretical knowledge mastery index using exam paper and homework text data, which is a weighted average of exam scores and homework quality scores; second, calculate the practical skills proficiency index using training video data, which is the cosine similarity between student action features extracted by 3D-CNN and standard action features, multiplied by the normalized value of completion time; then, calculate the vocational quality attainment index using enterprise feedback text and behavioral evaluation data, and use the fuzzy comprehensive evaluation method to combine the fuzzy matrix of enterprise feedback scores and behavioral evaluations to obtain the comprehensive score of vocational quality; finally, calculate the job suitability index using internship videos and job requirement data, which is the cosine similarity between student skill features and job requirement features. After each index is standardized by z-score, the weights are calculated by combining the subjective and objective weight fusion algorithm to obtain the comprehensive score of the vocational ability evaluation dimension.

8. The multimodal vocational education system evaluation system according to claim 1, characterized in that, The specific implementation steps of the resource adaptation and industry-education integration evaluation dimensions of the multi-dimensional evaluation system construction module are as follows: First, calculate the matching degree index of training resources by using training equipment data and professional training objectives, and use the analytic hierarchy process (AHP) to calculate the weighted sum of the matching degree of equipment quantity, type and demand; second, calculate the resource update timeliness index by using the teaching resource update time and industry technology iteration time, which is a normalized value; then, calculate the resource utilization rate index by using school-enterprise cooperation project data, which is the ratio of (number of students participating × number of projects) to (enterprise investment funds + equipment value); finally, calculate the enterprise participation depth index by using the teaching hours of enterprise lecturers and the number of training projects developed, which is a normalized value of (lecturer teaching hours × 0.5 + number of projects developed × 0.5); Next, the project fit index is calculated by comparing the training project content with the job task data, and the matching degree is calculated using the Jaccard coefficient. Finally, the employment quality index is calculated by the graduate employment data, which is a weighted average of (employment rate × 0.3 + average salary × 0.4 + job matching degree × 0.3). After each index is standardized by min-max, the weights are calculated by combining the subjective and objective weight fusion algorithm to obtain the comprehensive score of the resource adaptation and industry-education integration evaluation dimensions.

9. The multimodal vocational education system evaluation system according to claim 1, characterized in that, The specific implementation steps of the evaluation result output and guidance module are as follows: First, a multi-dimensional evaluation score radar chart, a bar chart of each indicator score, and an indicator correlation heatmap are generated using visualization tools. The radar chart intuitively displays the balance of scores in each dimension, the bar chart compares the score differences of each indicator, and the heatmap uses the Pearson correlation coefficient to show the correlation strength between indicators. Second, a fine-tuned GPT-4o natural language generation model is used, combined with the evaluation score and indicator correlation analysis, to generate a text report containing strengths, weaknesses, and improvement suggestions. Finally, a rule engine is used to match preset improvement rules to the evaluation weaknesses.

10. A method for evaluating a multimodal vocational education system, applicable to the multimodal vocational education system evaluation system described in any one of claims 1-9, characterized in that, The specific steps of this method are as follows: S1. Multimodal data acquisition: Text, image, audio, video, and sensor data are acquired through various acquisition units and stored in a distributed database; S2. Data preprocessing and feature extraction: The preprocessing sub-model cleans and normalizes the data, and the feature extraction sub-model extracts feature vectors for each modality. S3. Multimodal data fusion: A fused feature vector is obtained through a cross-modal attention fusion algorithm. S4. Evaluation index calculation: The scores of each index are calculated based on the fusion algorithm of fusion features and subjective and objective weights. S5. Evaluation Result Analysis and Guidance Generation: Evaluation reports and innovation guidelines are generated through visualization tools and rule engines.

Images