Skill micro certificate comprehensive evaluation system fusing multi-modal learning results

Abstract

The invention discloses a skill micro certificate comprehensive evaluation system fused with multi-modal learning achievements, and relates to the technical field of comprehensive evaluation systems, the skill micro certificate comprehensive evaluation system comprises a text module, an audio and video module and an emotion module, the text module judges answers written by trainees through a homework unit, and a log unit in the text module judges answers written by trainees from answering time, the longest number of input characters and the maximum number of input characters. The audio and video module judges behaviors of the trainees in an examination and brings illegal operations into scores, so that bad behaviors of the trainees in an examination room can be directly displayed by the scores, cheating behaviors of the trainees can be monitored at the same time, the method is more friendly to private enterprises with few faculty and staff, and the experience of the students is improved. The system can better save the labor cost, records the psychological state of the student during answering through the arranged emotion module, can finally evaluate the psychological state of the student during answering, and judges the psychological state of the student in the actual operation of subsequent work, thereby more comprehensively judging the ability of the student in multiple aspects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of comprehensive evaluation system, and particularly relates to a skill micro-certificate comprehensive evaluation system fusing multi-modal learning achievements. BACKGROUND

[0002] As a digital certification carrier focusing on specific ability modules, skill micro-certificate has become a key link between education and career development. However, the traditional evaluation mode has obvious limitations: it excessively relies on single result data, lacks dynamic description of the learning process, and is difficult to accurately reflect the real ability level of learners.

[0003] Specifically, the traditional mode is usually that after the learners complete the written test on the computer, the teaching staff evaluates the scores and grades according to the test results. More importantly, the issuing agencies of skill micro-certificate contain a large number of enterprises, which generally have problems of insufficient number of teaching staff and incomplete equipment configuration, resulting in that the evaluation work can only stay at the single result evaluation level, and there are large loopholes.

[0004] The disadvantages of this single evaluation mode mainly reflect in three aspects: It cannot truly reflect the proficiency of learners in corresponding skills, and the evaluation dimension is too one-sided.

[0005] It cannot reflect the state of mind of learners when answering questions, and for enterprises, a good state of mind is often more important than the test scores, and this key element is completely ignored.

[0006] The small number of teaching staff also easily leads to cheating behavior, making the authenticity of the final scores questionable.

[0007] Therefore, in order to solve the above problems, a skill micro-certificate comprehensive evaluation system fusing multi-modal learning achievements is proposed. SUMMARY

[0008] The main purpose of the present application is to provide a skill micro-certificate comprehensive evaluation system fusing multi-modal learning achievements to solve the problems proposed in the above background.

[0009] To achieve the above purpose, the technical scheme adopted by the present application is as follows: a skill micro-certificate comprehensive evaluation system fusing multi-modal learning achievements, which comprises a text module, a sound and video module, and an emotion module. The text module comprises a homework unit and a log unit, the homework unit tests the professional skill level of the learners through preset professional skill content, and the log unit tests the proficiency of the learners in professional skills. The sound and video module records the sound and video information in the answering process of the learners, detects the irregular behaviors therein to evaluate the quality of the learners. The emotion module collects students' facial expressions in real time during the question-answering process and assesses students' mentality based on a preset facial expression feature analysis model. The system comprehensively evaluates trainees' abilities, character, and mindset through text, audio-visual, and emotional modules.

[0010] The assignment unit is based on preset professional skills content. After the trainees complete all the questions, the assignments are sent to the human review unit for manual review and final review, and the assignment results are finally formed.

[0011] The log unit records the input operations of students when answering questions, including all text and image information such as input, modification, deletion, and copying. The log results are generated after being processed by the standard answer time comparison unit and the standard answer character comparison unit.

[0012] The audio-visual module uses deployed cameras and microphones to collect images and sounds of students answering questions. The audio-visual module includes an image monitoring unit and an audio monitoring unit. The cheating detection unit receives the image and sound information collected by the image monitoring unit and the audio monitoring unit, and judges the student's violation or cheating behavior based on the received information.

[0013] The image monitoring unit uses the student's head position at the start of the exam and after receiving the instruction to look at the screen as the origin of the coordinate system. It sets a limit range for the head's rotation angles upward, downward, leftward, and rightward. The image monitoring unit has a default value. When the student's head rotation angle exceeds the limit range, it is considered a violation and the system will deduct points accordingly. If a student's head rotation angle continues to exceed the limit, points will be deducted for each additional second. If the student continues to exhibit head rotation exceeding the limit after all default points have been deducted, the student will be included in the cheating monitoring unit's monitoring record, and a cheating behavior will be recorded. After the exam, the default value set in the image monitoring unit will be removed to remove all deducted points, and the image monitoring unit score will be generated.

[0014] The sound monitoring unit uses iFlytek ASR and Baidu AI speech-to-text technology to convert the student's speech into text synchronously during the exam. The sound monitoring unit has a keyword library containing keywords, and the sound monitoring unit transmits the information to the keyword retrieval unit. The sound monitoring unit has a default value. The text converted from audio is processed by the keyword retrieval unit. Each time a word from the keyword library appears in the text, it is considered a violation and points are deducted from the default value. If the deduction is zero and a keyword from the keyword library continues to appear in the text, the student is included in the cheating monitoring unit and a cheating behavior is recorded. At the end of the exam, all deductions are removed from the default value of the sound monitoring unit to form the sound monitoring unit score. During the sound monitoring process, the microphone converts sound wave vibrations into electrical signals. The computer, through the audio processing module, converts the electrical signals into digital audio data. By calculating the "instantaneous volume" or "short-term energy" of the audio data, the volume level is numerically monitored. When the volume level exceeds the set loud noise limit, the computer transmits the signal to the cheating detection unit and records a cheating behavior.

[0015] The cheating detection unit is equipped with a cheating database, which contains a database of key behaviors. Using monitoring equipment and computer vision technology, the classroom is modeled, and a dedicated area is set for each seat. Each candidate's dedicated area has spare stationery on the desk. The stationery is located in a designated area on the desk. The body parts of the candidates are monitored in real time. When a candidate's body goes out of the dedicated area, the information is transmitted to the cheating detection unit to record a cheating behavior. After the students finish the exam and submit their papers on the computer, the cheating detection unit uses monitoring equipment to detect the position of the stationery and determine whether the stationery is returned to its original position. If it is not returned, the relevant information is transmitted to the cheating detection unit and a cheating record is made. Then the cheating detection unit is stopped. The cheating detection unit has a default score. Every time a cheating behavior is recorded in the cheating detection unit, a point is deducted from the default score and the information is saved simultaneously. When the default score is zero, if a new cheating record is triggered again, points will continue to be deducted and displayed as negative scores.

[0016] The emotion module relies on the log unit to collect the answering time of each question during the exam, forming a timeline unit for each question. During the answering time of each question, the facial expressions of the students are collected synchronously through the monitoring camera.

[0017] The answer timeline unit identifies emotion labels through a pre-trained CNN model, outputs the dominant emotion and confidence score for that segment, and finally calculates the emotional stability value of the student's answer time using a calculation formula.

[0018] The answer timeline unit generates an overall emotional stability value P through a calculation formula. This value is added to the homework results and log results in the text module, as well as the scores of the image monitoring unit and the sound monitoring unit in the audio-visual module, to obtain the final total score of the exam. The employees are ranked in descending order to obtain their ranking, and the employees in the ranking are tentatively hired according to the set admission rules. The cheating detection unit in the audio-visual module determines the validity of the scores of the candidates to be admitted. If the default score of the corresponding candidate is not full, it is determined that there is cheating behavior, the score is invalidated and the candidate's admission quota is cancelled. Finally, the information of the candidates who did not cheat in the list of candidates to be admitted is transmitted to the score unit for public release and certificates are issued.

[0019] The present invention has the following beneficial effects: 1. In this invention, the assignment unit evaluates the student's written response through the set text module and determines the preliminary score. The log unit in the text module judges the time the student takes to answer the question and the maximum length of the input content characters, thereby judging the student's proficiency in the corresponding knowledge points, so that the teachers and staff can have a more accurate and comprehensive grasp of the students' professional knowledge.

[0020] 2. In this invention, the audio-visual module is set up to judge the behavior of students during the answering process. In corporate exams with few faculty and staff, the monitoring equipment detects the behavior of students during the exam, deducts points in a timely manner for violations, and records and saves suspected cheating behavior. This makes it possible to include the students' exam behavior in the exam results, and to link the students' quality to the exam.

[0021] 3. In this invention, by setting an emotion module, a timeline unit for answering questions is formed based on the log unit, and the facial expressions of students when answering questions are accurately collected, while facial expressions during time periods other than answering questions are discarded. This allows for accurate judgment of the students' mindset when answering questions, recording the students' mindset for each test question, evaluating the overall mindset during the exam, and thus accurately measuring the students' mindset during the exam. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall system architecture of the skills micro-certificate comprehensive assessment system that integrates multimodal learning results according to the present invention; Figure 2 This is a schematic diagram of the text module architecture of the skills micro-certificate comprehensive assessment system integrating multimodal learning results of the present invention; Figure 3 This is a schematic diagram of the audio-visual module of the skills micro-certificate comprehensive assessment system that integrates multimodal learning results according to the present invention; Figure 4 This is a schematic diagram of the emotional module of the skills micro-certificate comprehensive assessment system that integrates multimodal learning results according to the present invention. Detailed Implementation

[0023] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0024] Example 1, please refer to Figures 1 to 2 As shown: A comprehensive assessment system for skills micro-certificates that integrates multimodal learning outcomes, including text, audio-visual, and emotion modules; The text module includes an assignment unit and a log unit. The assignment unit tests the trainees' professional skill level through preset professional skill content, and the log unit tests the trainees' mastery and proficiency of professional skills. The audio-visual module records the sound and video information of students during the question-answering process, and detects any violations to assess the students' quality; The emotion module collects students' facial expressions in real time during the question-answering process and assesses students' mentality based on a preset facial expression feature analysis model; The system comprehensively evaluates trainees’ abilities, character, and mindset through text, audio-visual, and emotional modules.

[0025] The assignment unit provides students with a professional skills test paper on a computer. After students complete the test, the assignment unit sends it to a human unit for manual grading. The professionals are the company's teaching staff or private enterprise technical personnel. The assignment results are formed after the professionals review them. There is at least one review to verify the results of the initial evaluation of the students. The assignment results can account for 60% of the total score of the system. For example, if a percentage system is used, the assignment result can be 60 points. This is to evaluate the students' answers to the professional test questions. At this time, only the correctness of the students' answers can be judged. It is not possible to judge whether the students' input of the correct answers was fluent or whether they have mastered the skills.

[0026] The log unit records the input of students during the answering process, including all text and image information entered, modified, deleted, and copied by the user. The log result is formed after comparison with the standard answer time unit and the standard answer character unit. The specific steps for forming the log result are as follows: Step 1: The log unit has a standard answer time library and a standard answer character library. After a large number of professionals answer each question in the work unit, the answer time and the longest character in the answer are averaged. The average answer time and average longest character for each question are collected to form the standard answer time library: T S ={T S1 T S2 ,…,T SN} and standard answer character library: C S ={C S1 C S2 ,…,T SN The professionals are faculty members or technical personnel from private enterprises; Step 2: Record the time taken by students to answer questions to create a student answer time database: T U ={T U1 T U2 ,…,TUN The system records the longest character count generated by each student when answering each question, forming a student answer character library: C U ={C U1 C U2 C UN}; Standard answer time bank T S Standard answer character library C S The standard time T for each question SN The longest standard character C SN These are expanded by a set multiple to form the default maximum time and default maximum character limit for answering questions. For example, the standard answer time T for answering questions can be... SN The longest character in a standard answer is C. SN The length is increased by 3 times, thus setting the default maximum time and default maximum character. If a student does not respond to some questions, the student's answer time and the longest answer character are set to the corresponding default maximum time and default maximum character. Step 3: Create the time deviation library and character deviation library. The calculation formula is as follows: DT U ={[T U1 -T S1 ] / T S1 ×100%,[T U2 -T S2 ] / T S2 ×100%, …, [T] UN -T SN ] / T SN ×100%}; DC U ={[C U1 -C S1 ] / C S1 ×100%,[C U2 -C S2 ] / C S2 ×100%, …, [C UN -C SN ] / C SN ×100%}; For example, DT can be used U If the corresponding value in DC is greater than 50%, it is marked as abnormal. U If the corresponding value in the log cell is greater than 30%, it is marked as an anomaly. Then, the anomaly is marked and points are deducted from the corresponding default value in the log cell. For example: the log unit can account for 10% of the system score, the overall time deviation and the overall character deviation each account for 5%, and then they are evenly distributed among the answers. When DT U DC UWhen an anomaly occurs, the points allocated to the corresponding answer are deducted. Finally, the points obtained from the time deviation and character deviation of each answer are summed to form the log result, thereby judging the trainee's mastery of professional skills. The trainees are ranked in descending order of the scores in the log result to obtain the ranking of the trainees' overall mastery of professional skills test. For example, when two students have the same homework results, image monitoring unit score, and sound monitoring unit score, the students can be compared again by the log result score obtained by the students. In the log result score, the closer the score is to the default value, the closer the student's answer input is to that of a professional, thus showing their proficiency in knowledge. The assignment results only show the written answers and cannot determine the student's mastery of the knowledge. When a student is proficient in professional skills, their answering time and longest character count will be close to those of professionals. The score in the log results can more intuitively demonstrate the student's professional proficiency.

[0027] Example 2, please refer to Figure 1 , Figure 3 As shown: Based on Embodiment 1, the audio-visual module uses deployed camera and microphone devices to collect images and sounds of students answering questions. The audio-visual module includes an image monitoring unit and an audio monitoring unit, which are monitored by the cheating detection unit. The cheating detection unit is mainly used to monitor students' cheating behavior during exams. The video monitoring unit and the audio monitoring unit determine whether students have any irregular behavior during exams and deduct points for irregular behavior.

[0028] The image monitoring unit establishes its coordinate origin at the head position after the student hears the instruction to look directly at the screen at the start of the exam. Before the student answers the questions, this step is required to record the initial facial expression, head, and body coordinate information. The angle of head rotation (up, down, left, or right) is limited. For example, the maximum rotation angle is 30 degrees. This is a custom angle used to define the student's standard sitting posture for answering questions. This angle can be adjusted according to actual needs, including 30 degrees, 45 degrees, 60 degrees, and 90 degrees. When the student's head rotation angle exceeds 30 degrees, the built-in image monitoring unit deducts points from the default score. The image monitoring unit and the sound monitoring unit can each account for 5% of the system score. Taking a percentage system as an example, each can account for five points. When the angle of head rotation exceeds the limit, 1% of the score will be deducted, that is, one point will be deducted. When the angle of head rotation exceeds the limit, points will be deducted again for every additional second. Less than one second will be counted as one second. When the 5% of the image monitoring unit is deducted, if the student continues to tilt their head beyond the limit, the student will be included in the cheating monitoring unit and recorded once. This means that the student has only five chances to tilt their head beyond the limit. After five times, it will no longer be recorded as a violation, but it will be directly set as cheating, and the exam result will be invalid. When students are taking an exam, they should focus their heads on the computer screen. If their heads frequently turn out of the right angle and remain there for more than five seconds, it can be determined that they are spying on other students and cheating.

[0029] The sound monitoring unit uses iFlytek ASR and Baidu AI speech-to-text technology to synchronously convert the student's speech into text during the exam. It has a keyword library containing keywords such as "see," "help me," "help," and "copy." During the exam, the student should not make any sound except for the sounds of writing and typing on the keyboard. By detecting whether the student's speech triggers words that are commonly used for cheating, the unit can determine whether the student is cheating. The audio-to-text conversion process passes through a keyword retrieval unit. When a word from the keyword database appears in the text, 1% of the score is deducted for each occurrence, i.e., one point is deducted. Once the deducted score exceeds 5%, i.e., five points, no further deductions will be made. If a keyword from the keyword database continues to appear in the text, the student will be included in the cheating detection unit again, and this will be recorded once, rendering the student's exam score invalid. During the exam, students should remain quiet as required by the exam room rules. If a student violates the quiet requirement by frequently uttering sensitive keywords, it can be determined that they are cheating, and their score will be invalidated. During the sound monitoring process, the microphone converts sound wave vibrations into electrical signals. The computer, through the audio processing module, converts the electrical signals into digital audio data. By calculating the "instantaneous volume" or "short-term energy" of the audio data, the volume level is numerically monitored. When the volume level exceeds the internally set loud noise limit, the computer transmits the signal to the cheating detection unit, records it, and invalidates the score. The volume of the students' voices will be monitored. If a student makes a loud noise, it indicates that the student does not respect the examination record and has no sense of shame for disturbing others' examination. This should be treated as cheating.

[0030] The cheating detection unit is equipped with a cheating database, which contains a database of key behaviors, such as: a student leaving their seat, a student extending their limbs into another student's designated area, and a student raising their hand if they suddenly need to use the restroom during the exam. The faculty and staff will then disable the cheating detection for that student. Under normal exam conditions, students should always keep their bodies within their designated area. Using monitoring equipment and computer vision technology, the classroom is modeled, and a dedicated area is set for each seat. Each student's dedicated area has a desk with spare stationery. This stationery should meet the students' needs; students are required to bring their own stationery. The stationery on the desks is for use only and cannot be taken away. The stationery is located in a designated area on the desks. After the exam, the spare stationery must be returned to its original place. The student's head, upper body, upper limbs, and lower limbs are monitored. If a student's body moves outside the designated area, it is considered cheating. The spare stationery prevents students from picking up pens or other items. With spare stationery available, students can be prohibited from picking up items in the exam room. If a student still picks up items, it is marked as cheating by the cheating monitoring unit. After students finish their exams and submit their papers on the computer, the cheating detection unit monitors the location of their stationery through monitoring equipment to see if it has been returned to its proper place. Stationery belongs to the examination room and does not belong to individuals, so it cannot be taken away without permission. If it is not returned to its proper place, the information is sent to the cheating detection unit to be considered as cheating behavior, and then the cheating detection unit is stopped. The cheating detection unit has a default score. This default score is only used as a standard for determining the validity of the score and is not included in the total score. For example, the default score is ten. When a cheating signal is transmitted from the image detection unit or the sound detection unit, the corresponding image or sound is converted into text content, transmitted synchronously and saved, and the default score is deducted once, for example, one point is deducted at a time. When the behavior set in the key behavior library is triggered, the computer transmits and saves the synchronous image and deducts points from the default score again. When the default score reaches zero, if cheating behavior is triggered again, points will continue to be deducted and displayed in the form of negative scores. After the exam, if there is no cheating, the default score in the cheating monitoring unit should be full. When points are deducted, it indicates that the student is suspected of cheating. When a large number of points are deducted, resulting in a negative score, it indicates that the student's conduct is poor.

[0031] Example 3, please refer to Figure 1 , Figure 4As shown: Based on the first embodiment, the emotion module relies on the log unit to collect the answering time of each question during the exam, forming a timeline unit for each question. During the answering time of each question, the facial expressions of the students are collected synchronously through the monitoring camera. Since students may adjust their mindset during the free time of answering the questions, these facial expression changes should be discarded. The system accurately locates the changes in the students' mindset when answering questions, thereby accurately locating and calculating the students' mindset when encountering problems in their future work.

[0032] The answer timeline unit identifies emotion labels using a pre-trained CNN model, outputs the dominant emotion and confidence score for that segment, and finally calculates the emotional stability value of the student's answer time using a formula. The specific steps are as follows: Step 1: Manually assign values ​​to different emotions experienced by trainees during the exam. For example, a calm state can be set as a stable emotion, tension and hesitation can be set as an emotional fluctuation state, and anxiety, restlessness, and lethargy can be set as an unstable state. Assign corresponding base scores from high to low to the calm state, the emotional fluctuation state, and the unstable state; The final sentiment score E for each question is obtained by multiplying the confidence score by the base score for each question. N : E N =Base score for each question × CNN confidence score; The base score can be manually adjusted, assigning a base score to each expression. Step 2: Based on the answer time database T in the log unit U Student answer character library C U Forming a duration weight W N : For example: if the actual duration does not exceed one-third of the standard duration, it indicates that the answer was not taken seriously. N The value is assigned to 0.7; If the actual duration exceeds one-third of the standard duration but does not exceed twice the standard duration, it indicates that the time is normal, and the mood score will not be adjusted. The corresponding W is... N Assign a value of 1.0; If the actual duration exceeds twice the standard duration, it indicates a pause or distraction, corresponding to W. N The value is assigned to 0.8; Then, based on the final emotional score E for each question... N The time weight W for each question N The duration and emotion score (S) for each question are obtained. N =E N ×W N ; Step 3: Based on the above results, calculate the final overall emotional stability value. The calculation formula is as follows: P= , Where P is the overall emotional stability value, and E1, E2, ..., E N To correspond to the final emotion score for each question, W1, W2, ..., WN are the time weights for each question. The lower the score, the more unstable the emotions; the higher the score, the more stable the emotions. During the exam, the final emotional score and time weight for each question will be recorded and submitted together with the final emotional score. The answer timeline unit generates an overall emotional stability value through a calculation formula. This value is then added to the homework results and log results in the text module, and the video monitoring unit and sound monitoring unit scores in the audio-visual module to obtain the final exam paper score. The scores are arranged in the order of the actual results to determine the employee's grade. The cheating monitoring unit in the audio-visual module judges the authenticity of the employee's score. If the default score of the cheating monitoring unit is not full, it indicates that the employee has cheated, and the score is invalidated and deleted. Finally, the score ranking is obtained, and certificates are issued.

[0033] In this invention, a skills micro-certificate comprehensive assessment system integrating multimodal learning outcomes is used. During operation, after the student enters their designated seat in the examination room and sits upright, they follow the computer's instructions, look directly at the computer screen, and have their head and body coordinates collected. The student answers questions sequentially on the computer, using a one-question-one-answer mode, with only one question appearing on the screen at a time. The student enters their answer in the input box below the question. When the student's cursor begins to blink in the input box, a log file is simultaneously opened, recording all the characters entered. The log file counts the input characters. Students may modify or delete text in the input box, but during character counting, modified and deleted characters are still treated as new characters and added together, thus accurately calculating the maximum number of characters appearing in the input box. After submitting the answers to the corresponding questions, the log records the student's answering time and the maximum number of characters in the answer. The monitoring equipment on the computer detects the student's behavior throughout the examination room. The microphone collects the student's facial expressions during the answering time and the student's voice throughout the examination time. After the student finishes answering all the questions, the student tidies up the examination table and leaves the examination room. The computer sends the students' test questions to the assignment unit, where professionals manually grade them to obtain the students' professional skills test results. Then, the log unit is activated, comparing and calculating the time deviation and character deviation by comparing the students' answer time database and the students' answer character database with the standard answer time database and the standard answer character database in the computer. The time and character deviation databases are calculated by comparing the students' thinking time and the fluency of their font input with the thinking time and the fluency of their font input with the professionals. When the deviation value between the two is too large, or when the set anomaly occurs, it indicates that the students are slow in thinking and have not mastered the professional skills. Finally, the default value of the log unit is removed and the points are deducted to form the log result. Then, the audio-visual module judges the angle of the student's head turn and whether the student whispers sensitive words during the exam. It records the violations and saves the corresponding image or text information. The video monitoring unit and the audio monitoring unit deduct points for these violations. The video monitoring unit and the audio monitoring unit have default values. When a student violates the rules, the corresponding default value is deducted. When the default values ​​in the image monitoring unit and sound monitoring unit are exhausted, if a student still violates the rules, the violation will be directly recorded in the cheating monitoring unit. The cheating monitoring unit has a key behavior library with a large number of prohibited behaviors. When a student triggers the corresponding behavior, cheating points will be deducted. The cheating monitoring unit can be set with a default score. One point will be deducted for each cheating behavior. Stationery on the exam desk is stored in a designated area for students to use. After use, students must return the stationery to the designated area. If a student fails to return the stationery as required, points will be deducted from their score in the cheating detection unit. When the student finishes the exam, the default scores of the video and audio detection units are reduced by the deducted points to form corresponding video and audio detection unit scores. This allows the student's score to be correlated with their behavior during the exam, and the student's character can be judged based on the specific score. The score in the cheating detection unit is then used to monitor whether the student's final score is genuine. Then, the emotion module collects the students' facial expressions during the exam, and forms a timeline unit based on the time the students answer the questions. During the corresponding answer time, the timeline unit identifies emotion labels through a pre-trained CNN model, outputs the dominant emotion and confidence score for that period, and finally calculates the emotional stability value of the students during the answer time using a calculation formula. The final score is calculated by adding the final assignment results, log results, video monitoring unit scores, audio monitoring unit scores, and emotional stability scores. The evaluation comprehensively considers factors such as the accuracy and proficiency of mastering professional skills, examination violations, and examination mentality. The cheating monitoring unit determines the authenticity of the score, thereby better evaluating the student's overall ability.

[0034] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A skill microcredential portfolio assessment system that fuses multimodal learning outcomes, characterized in that, The system includes a text module, an audio-visual module, and an emotion module; The text module includes an assignment unit and a log unit. The assignment unit tests the trainees' professional skill level through preset professional skill content, and the log unit tests the trainees' mastery and proficiency of professional skills. The audio-visual module records the sound and video information of the students during the question-answering process and detects any violations to assess the students' quality. The emotion module collects students' facial expressions in real time during the question-answering process and assesses students' mentality based on a preset facial expression feature analysis model. The system comprehensively evaluates trainees' abilities, character, and mindset through text, audio-visual, and emotional modules.

2. The system of claim 1, wherein, The assignment unit is based on preset professional skills content. After the trainees complete all the questions, the assignments are sent to the human review unit for manual review and final review, and the assignment results are finally formed.

3. The system of claim 1, wherein, The log unit records the input operations of students when answering questions, including all text and image information such as input, modification, deletion, and copying. After processing by the standard answer time comparison unit and the standard answer character comparison unit, the log result is formed. The specific steps for forming the log result are as follows: Step 1: The log unit is provided with a standard answer time library T S and a standard answer character library C S . After a large number of professionals answer each question in the work unit, the average answer time and the average longest character are obtained respectively. The average answer time and the average longest character of each question are collected to form a standard answer time library T S ={T S1 , T S2 , …, T SN} and a standard answer character library C S ={C S1 , C S2 , …, C SN}. Step 2: record the answer time of each question of the student, form the student answer time library: T U ={T U1 , T U2 , …, T UN}, record the longest character number of each question of the student, form the student answer character library: C U ={C U1 , C U2 , …, C UN} The standard answering time library T S , the standard answering character library C S , the standard time T SN of each answering question in the standard answering time library T S , the standard longest character C SN are respectively enlarged according to a set multiple to form the default longest time and the default longest character corresponding to the answering question, and when the student does not reply to part of the answering questions, the student's answering time and the longest character of the answering questions are set as the corresponding default longest time and the default longest character. Step 3: Forming a time deviation library DT U and a character deviation library DC U The calculation formula is: DT U { [T U1 -T S1 ] / T S1 x 100%, [T U2 -T S2 ] / T S2 x 100%, …, [T UN -T SN ] / T SN x 100%}; DC U {[C U1 -C S1 ] / C S1 × 100%, [C U2 -C S2 ] / C S2 × 100%, …, [C UN -C SN ] / C SN × 100%}; According to the set abnormal condition, the data in the time deviation library DT U and the character deviation library DC U are judged one by one, and according to the abnormal condition, the default score in the log unit is deducted, and the log result of the log unit is obtained after the default score in the log unit is deducted.

4. The system of claim 1, wherein, The audio-visual module, deployed on a camera and microphone device, collects images and sounds of students answering questions. The audio-visual module includes an image monitoring unit and an audio monitoring unit. The cheating detection unit receives the image and sound information collected by the image monitoring unit and the audio monitoring unit, and judges the student's violation or cheating behavior based on the received information.

5. The system of claim 4, wherein, The image monitoring unit uses the student's head position at the start of the exam and after receiving the instruction to look at the screen as the origin of the coordinate system. It sets a limit range for the head's rotation angles upward, downward, leftward, and rightward. The image monitoring unit has a default value. When the student's head rotation angle exceeds the limit range, it is considered a violation, and the system will deduct points accordingly. If a student's head rotation angle continues to exceed the limit, points will be deducted for each additional second. If the student continues to exhibit head rotation exceeding the limit after all default points have been deducted, the student will be included in the cheating monitoring unit's monitoring record, and a cheating behavior will be recorded. After the exam, the default value set in the image monitoring unit will be removed to remove all deducted points, and the image monitoring unit score will be generated.

6. The system of claim 5, wherein, The sound monitoring unit uses iFlytek ASR and Baidu AI speech-to-text technology to convert the student's speech into text synchronously during the exam. The sound monitoring unit has a keyword library containing keywords, and the sound monitoring unit transmits the information to the keyword retrieval unit. The sound monitoring unit has a default value. The text converted from audio is processed by the keyword retrieval unit. Each time a word from the keyword library appears in the text, it is considered a violation and points are deducted from the default value. If the deduction is zero and a keyword from the keyword library continues to appear in the text, the student is included in the cheating monitoring unit and a cheating behavior is recorded. At the end of the exam, all deductions are removed from the default value of the sound monitoring unit to form the sound monitoring unit score. During the sound monitoring process, the microphone converts sound wave vibrations into electrical signals. The computer, through the audio processing module, converts the electrical signals into digital audio data. By calculating the "instantaneous volume" or "short-term energy" of the audio data, the volume level is numerically monitored. When the volume level exceeds the set loud noise limit, the computer transmits the signal to the cheating detection unit and records a cheating behavior.

7. The system of claim 6, wherein, The cheating detection unit is equipped with a cheating database, which contains a database of key behaviors. Using monitoring equipment and computer vision technology, the classroom is modeled, and a dedicated area is set for each seat. Each candidate's dedicated area has spare stationery on the desk. The stationery is located in a designated area on the desk. The body parts of the candidates are monitored in real time. When a candidate's body goes out of the dedicated area, the information is transmitted to the cheating detection unit to record a cheating behavior. After the students finish the exam and submit their papers on the computer, the cheating detection unit uses monitoring equipment to detect the position of the stationery and determine whether the stationery is returned to its original position. If it is not returned, the relevant information is transmitted to the cheating detection unit and a cheating record is made. Then the cheating detection unit is stopped. The cheating detection unit has a default score. Every time a cheating behavior is recorded in the cheating detection unit, a point is deducted from the default score and the information is saved simultaneously. When the default score is zero, if a new cheating record is triggered again, points will continue to be deducted and displayed as negative scores.

8. The system of claim 7, wherein, The emotion module relies on the log unit to collect the answering time of each question during the exam, forming a timeline unit for each question. During the answering time of each question, the facial expressions of the students are collected synchronously through the monitoring camera.

9. The system of claim 8, wherein, The answer timeline unit identifies emotion labels using a pre-trained CNN model, outputs the dominant emotion and confidence score for that segment, and finally calculates the emotional stability value of the student's answer time using a calculation formula. The specific steps are as follows: Step 1: Manually assign base scores to students based on their different emotions during the exam, using a CNN model to identify the emotion labels. Multiply the confidence score of each question by the base score to obtain the final emotion score for each question. E N = base score per question x CNN confidence; Step 2: Forming the duration weight W according to the answer time library T in the log unit U and the character library C of the student's answers U , forming the duration weight W N : An abnormal short time interval, a normal time interval and an abnormal long time interval are set, the actual answering time of the examinee is judged to belong to which interval, and a corresponding time length weight W is given according to the setting N , Then the final emotion score E of each question is obtained N The time length weight W of each question is obtained N The time length emotion score S of each question is obtained N The calculation formula is: S N =E N ×W N ; Step 3: Based on the above results, calculate the final overall emotional stability value P. The calculation formula is as follows: P= 。 10. The system of claim 9, wherein, The answer timeline unit generates an overall emotional stability value P through a calculation formula. This value is added to the homework results and log results in the text module, as well as the scores of the image monitoring unit and the sound monitoring unit in the audio-visual module, to obtain the final total score of the exam. The employees are ranked in descending order to obtain their ranking, and the employees in the ranking are tentatively hired according to the set admission rules. The cheating detection unit in the audio-visual module determines the validity of the scores of the candidates to be admitted. If the default score of the corresponding candidate is not full, it is determined that there is cheating behavior, the score is invalidated and the candidate's admission quota is cancelled. Finally, the information of the candidates who did not cheat in the list of candidates to be admitted is transmitted to the score unit for public release and certificates are issued.

Images