Examination teaching video and electronic courseware real-time synchronous processing method

Abstract

The invention relates to the technical field of synchronous processing, and discloses an examination teaching video and electronic courseware real-time synchronous processing method, which comprises the following steps: capturing an operation event of an electronic courseware in real time, synchronously collecting teaching data, and generating a corresponding first data packet and a second data packet; performing time-space synchronization on the first data packet and the second data packet to obtain a plurality of synchronization results, and verifying the synchronization results of corresponding categories based on verification models of different event types to obtain a comprehensive verification evaluation value; and judging whether the synchronization result is corrected or not according to the comprehensive verification evaluation value, if yes, generating a correction instruction corresponding to the synchronization result, and if not, generating a structured metadata file and a courseware video. Through dynamic time-space synchronization, multi-dimensional verification and intelligent correction, the automation degree, the synchronization precision and the content quality of synchronous processing of the examination teaching video and the electronic courseware are remarkably improved, and the strict requirements of scenes such as clinical skill assessment on the recording precision of the teaching process are better met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of synchronous processing, in particular to a real-time synchronous processing method for examination teaching video and electronic courseware. BACKGROUND

[0002] In the scenarios of clinical skills training examination (such as OSCE), remote teaching, academic conference, etc., it is usually required to synchronously record the explanation process of the teacher (or examiner) and the electronic courseware (such as PPT) displayed by the teacher, and generate rich media content for playback, review and evaluation.

[0003] In the prior art, the synchronous processing of teaching video and electronic courseware is usually realized by post-editing and synthesizing or using a fixed layout recording and broadcasting system, but these methods have obvious limitations: post-editing and synthesizing requires a lot of manual labor, and the synchronization accuracy depends on the experience of manual operation, which is difficult to ensure the accuracy of time axis alignment; the fixed layout recording and broadcasting system cannot dynamically adjust the presentation mode of video and courseware according to the teaching content, and when the content of the courseware is switched or the teacher makes notes or demonstrates operations, the key information is easily blocked or delayed, and the above-mentioned synchronous processing methods do not contain a verification and correction mechanism, which may result in quality problems such as time axis misalignment and content association confusion of the generated teaching resources, affecting the teaching effect and user experience, and it is difficult to meet the requirement of accurate association of operation steps, courseware key points and video pictures, and it cannot meet the requirement of recording accuracy of teaching process in clinical skills examination. SUMMARY

[0004] To solve the above technical problems, the present application provides a real-time synchronous processing method for examination teaching video and electronic courseware, which generates operation events of the electronic courseware and first and second data packets of teaching data, and performs spatio-temporal synchronization, verifies the synchronous results of different event types based on a verification model, obtains a comprehensive verification evaluation value, judges whether to correct the synchronous results, generates a correction instruction if correction is required, and generates a structured metadata file and courseware video if correction is not required. Through dynamic spatio-temporal synchronization, multi-dimensional verification and intelligent correction, the automation degree, synchronization accuracy and content quality of the synchronous processing of examination teaching video and electronic courseware are significantly improved, ensuring the accurate association of operation steps, courseware key points and video pictures in the teaching process, and better meeting the strict requirement of recording accuracy of teaching process in clinical skills examination, and providing high-quality rich media resources for subsequent teaching evaluation, review and playback.

[0005] In some embodiments of the present application, a real-time synchronous processing method for examination teaching video and electronic courseware is provided, comprising:

[0006] Real-time capture of operation events of the electronic courseware and synchronous collection of teaching data to generate corresponding first data packets and second data packets; Spatiotemporal synchronization of the first data packets and the second data packets to obtain a plurality of synchronization results, checking of the synchronization results of corresponding categories based on a check model of different event types to obtain a comprehensive check evaluation value; Judging whether to correct the synchronization results according to the comprehensive check evaluation value, if yes, generating a correction instruction of the corresponding synchronization results, and if no, generating a structured metadata file and courseware video according to the corresponding synchronization results.

[0007] In some embodiments of the present application, real-time capture of operation events of the electronic courseware and synchronous collection of teaching data to generate corresponding first data packets and second data packets, comprising: Real-time capture and analysis of operation events generated for the electronic courseware application at the operation level based on a courseware analysis agent module running on the teaching terminal; For each captured operation event, generating a first data packet containing the event type, event content and corresponding first timestamp; Parallel collection of video stream, audio stream and screen data stream of the examination based on the recording terminal; Generating corresponding teaching data for the collected video stream, audio stream and screen data stream, the teaching data including video data, audio data and screen image data of the corresponding frame; For each frame of teaching data, generating a second data packet including video data, audio data, screen image data of the corresponding frame and corresponding second timestamp.

[0008] In some embodiments of the present application, before spatiotemporal synchronization of the first data packets and the second data packets, further comprising: Pre-setting a reference clock; Distributing time information of the reference clock to the teaching terminal and the recording terminal, real-time monitoring of time deviation and clock drift rate between the actual clock of each terminal and the reference clock, and generating a synchronization state parameter corresponding to each timestamp; Encapsulating the synchronization state parameter into the data packet of the corresponding timestamp, and performing dynamic time calibration calculation to convert the timestamp in the corresponding data packet into a unified timestamp relative to the reference clock to obtain a reference timestamp of the corresponding data packet; Generating reference timestamps of all data packets and mapping them to a preset reference time axis.

[0009] In some embodiments of the present application, spatiotemporal synchronization of the first data packets and the second data packets, comprising: generate a corresponding synchronization time window parameter for each event type in the first data packet, the synchronization time window parameter including a forward search pre-judgment time length and a backward search confirmation time length; generate a synchronization time window on a reference time axis for each first data packet according to the synchronization time window parameter and the reference timestamp of the first data packet; retrieve the second data packet according to each synchronization time window and the corresponding synchronization condition, and perform spatio-temporal synchronization between the retrieved second data packet and the first data packet of the corresponding synchronization time window to obtain a plurality of synchronization results.

[0010] In some embodiments of the present application, the verification model of different event types includes: a plurality of event types are preset, and a random event type is selected as a target type; obtain historical synchronization logs of the target type, sort the historical synchronization data according to the historical synchronization coefficients in the historical synchronization logs to obtain a historical synchronization data matrix of the target type; determine a plurality of key verification indexes affecting the synchronization effect of the target type according to the historical synchronization data matrix, and set a weight coefficient of the corresponding key verification index according to the influence degree; construct a verification rule of the target type according to the plurality of key verification indexes and the corresponding weight coefficients; construct a verification evaluation model of each key verification index; generate a verification model of the target type according to the verification rule and the plurality of verification evaluation models; generate a verification model of each event type in turn.

[0011] In some embodiments of the present application, the verification evaluation model of each key verification index includes: determine the verification features of each key verification index, and call the corresponding feature extraction algorithm to generate a historical verification feature set of each key verification index; the historical verification feature set includes a plurality of historical verification features, and each historical verification feature is mapped with a corresponding predicted verification evaluation value; set a training sample set according to the historical verification feature set, and use a machine learning algorithm to train the training sample set to obtain a verification evaluation model of the corresponding key verification index; generate a verification evaluation model of each key verification index in turn.

[0012] In some embodiments of the present application, the verification model based on different event types is used to verify the corresponding category of synchronization results to obtain a verification evaluation value, including: determine an event type of each synchronization result, and obtain, according to a plurality of key check indexes of the determined event type, a check feature of each key check index, and a corresponding feature extraction algorithm, a plurality of actual check features of each key check index of each synchronization result for the corresponding event type; input the plurality of actual check features of the same key check index into a check evaluation model of the corresponding key check index, and obtain a corresponding check evaluation value; perform weight processing on the check evaluation values of the plurality of key check indexes based on a check rule of the event type of the synchronization result, and obtain a comprehensive check evaluation value of each synchronization result.

[0013] In some embodiments of the present application, whether to modify the synchronization result is determined according to the comprehensive check evaluation value, including: pre-set a comprehensive check evaluation value threshold; when the comprehensive check evaluation value of the synchronization result is greater than the comprehensive check evaluation value threshold, the synchronization result is not modified; when the comprehensive check evaluation value of the synchronization result is not greater than the comprehensive check evaluation value threshold, the synchronization result is modified.

[0014] In some embodiments of the present application, a modification instruction corresponding to the synchronization result is generated, including: determine a plurality of to-be-modified indexes and corresponding to-be-modified coefficients of the corresponding synchronization result; retrieve, according to the event type of the corresponding synchronization result and the to-be-modified indexes, a plurality of modification strategies of the corresponding synchronization result in a modification strategy library; the modification strategy library contains a plurality of preset to-be-modified coefficients corresponding to each preset to-be-modified index under different event types, and each preset to-be-modified coefficient is mapped with a corresponding modification step and a parameter adjustment range; sort the plurality of modification strategies of the same synchronization result according to a priority rule, and generate a modification instruction corresponding to the synchronization result according to a sorting result.

[0015] In some embodiments of the present application, a structured metadata file and a course video are generated according to the corresponding synchronization result, including: convert and assemble a synchronization result that passes the check into structured data based on a predefined metadata mode; the structured data includes an event type, an event content, a time parameter, a synchronization media reference identifier, and a check confidence; organize a plurality of structured data in a time sequence, generate a structured metadata file, and construct an interactive index data of the structured metadata file; synthesize teaching data in the synchronization result that passes the check into a video file based on a preset synthesis layout; The interaction index data extracted from the structured metadata file is associated with the video file to form a course video.

[0016] The exam teaching video and electronic courseware real-time synchronization processing method has the beneficial effects that, compared with the prior art, the exam teaching video and electronic courseware real-time synchronization processing method has the beneficial effects that: The operation event of the electronic courseware and the first data packet and the second data packet of the teaching data are generated, and the space-time synchronization is performed, the synchronization result of the corresponding category is checked based on the check model of different event types, the comprehensive check evaluation value is obtained, it is judged whether the synchronization result is modified, if necessary, the modification instruction is generated, if no modification is needed, the structured metadata file and the course video are generated. Through dynamic space-time synchronization, multi-dimensional verification and intelligent correction, the automation degree, synchronization accuracy and content quality of the exam teaching video and electronic courseware synchronization processing are significantly improved, the accurate association of the operation steps, courseware key points and video pictures in the teaching process is ensured, and the strict requirements of the clinical skill examination scene on the teaching process recording accuracy are better met. High-quality rich media resources are provided for subsequent teaching evaluation, review playback and the like. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flowchart of an exam teaching video and electronic courseware real-time synchronization processing method in the embodiments of the present application. DETAILED DESCRIPTION

[0018] The specific embodiments of the present application will be further described in detail below in combination with the drawings and embodiments. The following embodiments are used to illustrate the present application, but not to limit the scope of the present application.

[0019] In the description of the present application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements indicated must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0020] The terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more.

[0021] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting" should be understood in a broad sense, for example, can be fixed connection, can also be detachable connection, or integral connection; can be mechanical connection, can also be electrical connection; can be direct connection, can also be indirect connection through intermediate medium, can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0022] As shown in Figure 1 The real-time synchronization processing method for examination teaching video and electronic courseware of the embodiment of the present application comprises: S101: Real-time capture of operation events of electronic courseware and synchronous collection of teaching data, generating corresponding first data packet and second data packet; S102: Spatio-temporal synchronization of the first data packet and the second data packet, obtaining a plurality of synchronization results, checking the synchronization results of the corresponding categories based on the check model of different event types, and obtaining the comprehensive check evaluation value; S103: According to the comprehensive check evaluation value, it is judged whether to modify the synchronization result, if yes, the modification instruction of the corresponding synchronization result is generated, if not, the structured metadata file and the courseware video are generated according to the corresponding synchronization result.

[0023] In some embodiments of the present application, real-time capture of operation events of electronic courseware and synchronous collection of teaching data, generating corresponding first data packet and second data packet, comprises: Based on the courseware analysis agent module running on the teaching terminal, the operation events generated for the electronic courseware application program are captured and analyzed in operation layer; For each captured operation event, a first data packet containing event type, event content and corresponding first timestamp is generated; Based on the recording and broadcasting terminal, the video stream, the audio stream and the screen data stream of the examination are collected in parallel; The corresponding teaching data is generated for the collected video stream, audio stream and screen data stream, and the teaching data comprises video data, audio data and screen image data of the corresponding frame; For each frame of teaching data, a second data packet comprising video data, audio data, screen image data of the corresponding frame and corresponding second timestamp is generated.

[0024] In the present embodiment, the courseware analysis agent module can adapt to the electronic courseware application program, and through the application program interface, a data capture channel is established between the operating system kernel layer and the application layer, and the user's various interactive operations on the courseware are listened to and analyzed in real time, such as slide switching, annotation adding, key marking, formula editing, multimedia playback control, etc.

[0025] In the embodiment, when capturing the operation event, not only the original instruction triggering the event is recorded, but also the event content is further analyzed to ensure that the event content in the first data packet has rich semantic information.

[0026] In the embodiment, the recording and broadcasting terminal adopts a multi-channel data acquisition architecture to configure a high-definition camera to collect a teacher operation demonstration video stream, an omnidirectional microphone array to collect a teaching audio stream, and a screen capture card to collect a courseware display picture data stream of the teaching terminal in real time. When generating the second data packet, key frame marking is performed on the video stream, voice activity detection marking is performed on the audio stream, and picture change area detection marking is performed on the screen data stream, so that each frame of teaching data is attached with scene feature identification, thereby facilitating fast positioning of key content during subsequent synchronous retrieval.

[0027] In the embodiment, the second timestamp and the second timestamp both adopt a homologous local clock calibration mechanism, thereby ensuring that the two types of data packets have comparability in the time dimension.

[0028] In the embodiment, by capturing the operation event and synchronously collecting the teaching data, the first data packet with the event type, specific event content and time parameter and the second data packet with the multi-modal teaching data and corresponding timestamp are generated, thereby providing a precise and comprehensive data basis for subsequent space-time synchronization and realizing source data association of the electronic courseware operation behavior and the teaching audio and video and screen data.

[0029] In some embodiments of the present application, before the space-time synchronization of the first data packet and the second data packet, the following steps are further included: a reference clock is preset; time information of the reference clock is distributed to the teaching terminal and the recording and broadcasting terminal, time deviation and clock drift rate between actual clocks of the terminals and the reference clock are monitored in real time, and a synchronization state parameter corresponding to each timestamp is generated; the synchronization state parameter is encapsulated into the data packet of the corresponding timestamp, dynamic time calibration calculation is performed, the timestamp in the corresponding data packet is converted into a unified timestamp relative to the reference clock, and a reference timestamp of the corresponding data packet is obtained; the reference timestamps of all data packets are generated and mapped to a preset reference time axis.

[0030] In the embodiment, the reference clock refers to a high-precision standard time, and the time information of the reference clock is distributed to the terminals to ensure high uniformity of the teaching terminal and the recording and broadcasting terminal in the time reference.

[0031] In this embodiment, generating synchronization status parameters corresponding to each timestamp means recording the synchronization status parameters at the timestamp in addition to recording the timestamp of each data packet. The synchronization status parameters include the absolute time deviation value measured at the synchronization time of the last operation event, the clock drift rate, and the actual clock time of the most recent successful synchronization operation.

[0032] In this embodiment, performing dynamic time calibration calculation includes calculating time elapsed and drift compensation, calculating the real-time deviation between the current synchronization time and the last synchronization time, thereby converting the timestamp of the data packet to a reference timestamp on the reference time axis. Specifically, the reference timestamp = actual timestamp + time elapsed. The time drift rate plus the absolute time deviation value, where the time elapsed is the actual clock time minus the last synchronization time, is estimated based on the trend of the deviation between the actual clock and the reference clock over time.

[0033] In this embodiment, by converting to a reference timestamp, it can be ensured that the reference timestamps of all data packets are precisely aligned on the preset reference timeline, so that the timestamps of all data packets are unified on the reference timeline, providing an accurate time reference for subsequent spatiotemporal synchronization, and achieving a high degree of spatiotemporal consistency between teaching content and video footage.

[0034] In some embodiments of this application, spatiotemporal synchronization of the first data packet and the second data packet includes: For each event type in the first data packet, a corresponding synchronization time window parameter is generated. The synchronization time window parameter includes the prediction duration for forward lookup and the confirmation duration for backward lookup. The synchronization time window for each first data packet on the reference time axis is generated according to the synchronization time window parameters and the reference timestamp of the first data packet. According to each synchronization time window and the corresponding synchronization conditions, the second data packet is retrieved, and the retrieved second data packet is spatiotemporally synchronized with the first data packet of the corresponding synchronization time window to obtain multiple synchronization results.

[0035] In this embodiment, the synchronization time window parameters are different for different event types, and the synchronization time window parameters are set in advance based on historical data. For high-frequency operation events such as page turning and highlighting, the prediction time is set to 0.5 seconds and the confirmation time is set to 1 second to ensure that the teaching data before and after the operation can be accurately captured. For event types with longer durations, such as pausing and writing on the blackboard during teacher explanation, the prediction time is extended to 2 seconds and the confirmation time is extended to 3 seconds to avoid synchronization omissions due to the large time span between the operation event and the teaching data.

[0036] In this embodiment, the synchronization conditions include the correspondence between the reference timestamp and the synchronization time window, as well as the content matching degree between the operation event content and the screen image data. Specifically, when the reference timestamps of the video data and the screen image data in the second data packet fall within the synchronization time window of the first data packet, the time region of the audio data overlaps with the synchronization time window of the first data packet, and the screen image data contains the courseware content features (such as specific page number identifiers or highlighted text areas) corresponding to the event content in the first data packet, it is determined that the synchronization conditions are met; otherwise, the synchronization conditions are not met.

[0037] In this embodiment, spatiotemporal synchronization refers to establishing a time-dimensional correspondence between the reference timestamp of the first data packet and the reference timestamp of the second data packet that meets the synchronization conditions on a preset reference time axis, and combining the spatial dimension matching relationship between event content and content information such as audio and video data and image data, to accurately match and bind the first data packet representing the operation behavior of electronic courseware with the second data packet containing audio, video and screen images.

[0038] In this embodiment, through this dual synchronization in the spatiotemporal dimensions, the operation events are organically combined with the audio, video, and screen content in the corresponding teaching scenario, ensuring that the subsequently generated metadata files and courseware videos can accurately reflect the synchronization relationship between courseware operations and teaching content presentation during the teaching process.

[0039] In some embodiments of this application, the verification models for different event types include: Several event types are predefined, and one event type is randomly selected as the target type; Obtain the historical synchronization logs of the target type, sort the historical synchronization data according to the historical synchronization coefficients in the historical synchronization logs, and obtain the historical synchronization data matrix of the target type; Based on the historical synchronization data matrix, several key verification indicators that affect the synchronization effect of the target type are determined, and the weight coefficients of the corresponding key verification indicators are set according to the degree of influence. Verification rules for the target type are constructed based on several key verification indicators and their corresponding weighting coefficients; Construct a verification evaluation model for each key verification indicator; Generate a verification model for the target type based on the verification rules and several verification evaluation models; Generate a validation model for each event type in sequence.

[0040] In this embodiment, event types include page turning, annotation, mouse click, writing, and area electronic whiteboard writing.

[0041] In this embodiment, the historical synchronization coefficient is quantified based on the actual matching effect between the operation events and teaching data in the historical synchronization data. It is used to measure the accuracy of historical synchronization processing. The higher the accuracy, the larger the corresponding historical synchronization coefficient, and vice versa. The value range of the historical synchronization coefficient is 0-1.

[0042] In this embodiment, historical synchronization data refers to historical event content data of the target type and historical teaching data within the corresponding historical synchronization time window. The historical event content data is clustered to obtain multiple event content data clusters. The historical teaching data of the same event content data cluster are sorted according to the size of the historical synchronization coefficient to construct the historical synchronization data sub-matrix of the corresponding cluster. The rows of the historical synchronization data sub-matrix are the historical synchronization coefficients, and the columns are the data types. The influence of historical teaching data of different data types on the historical synchronization coefficient is obtained based on the historical synchronization data sub-matrix. The influence of the same data type on the historical synchronization coefficient is weighted and averaged by combining the operation probabilities of multiple event content data clusters to obtain the final influence of each data type.

[0043] In this embodiment, data types with an impact greater than a preset threshold are selected, and then the corresponding key verification indicators are determined. For example, when the data type is screen image data, the key verification indicator is the correlation degree of the courseware screen frames; when the data type is audio data, the key verification indicator is the audio semantic correlation degree; when the data type is video data, the key verification indicator is the teacher video correlation degree. The weight coefficient is set based on the impact degree corresponding to each key verification indicator. The higher the impact degree, the larger the weight coefficient.

[0044] In this embodiment, key verification indicators include courseware screen frame correlation, audio semantic correlation, teacher video correlation, operation event temporal coherence, and data integrity verification indicators. Specifically, courseware screen frame correlation measures the matching degree between the operation event content (such as page numbers after page turning, specific marked areas) in the first data package and the corresponding screen image data in the second data package. Audio semantic correlation involves performing speech recognition and semantic analysis on the audio data in the second data package, and semantically matching the recognition results with the event content (such as the topic of the explanation, operation instructions) of the first data package to obtain semantic similarity. Teacher video correlation analyzes the correlation between the teacher's gestures, pointing, and other actions in the teacher's video stream and the courseware operation events. Operation event temporal coherence verifies whether the time sequence of various operation events (such as courseware switching, adding annotations, questioning and interaction, etc.) during the teaching process conforms to normal teaching logic. Data integrity verification indicators perform integrity checks on key data fields (such as event timestamps, data identifiers, content summaries, etc.) in the first and second data packages to ensure that the basic data upon which synchronous processing is based is not missing or damaged.

[0045] In this embodiment, the verification rule refers to the process and judgment criteria for verifying the synchronization result based on several key verification indicators of the corresponding type and the weight coefficients of each key verification indicator. Specifically, the verification evaluation value is obtained by using the corresponding verification evaluation model based on several key verification indicators. The value is then weighted and summed according to the weight coefficients of each indicator set for the current type to obtain the comprehensive verification evaluation value.

[0046] In this embodiment, the key verification indicators and corresponding weight coefficients for different event types are different. By customizing differentiated verification models for different event types, the effectiveness of various synchronization results can be evaluated more accurately, ensuring the quality of real-time synchronization processing of exam teaching videos and electronic courseware in complex teaching scenarios.

[0047] In some embodiments of this application, a verification evaluation model is constructed for each key verification indicator, including: Determine the verification features of each key verification indicator, and call the corresponding feature extraction algorithm to generate a historical verification feature set for each key verification indicator. The historical verification feature set includes several historical verification features, and each historical verification feature is mapped to a corresponding predicted verification evaluation value. A training sample set is set based on the historical verification feature set, and a machine learning algorithm is used to train the model on the training sample set to obtain the verification evaluation model corresponding to the key verification indicators. The verification evaluation model for each key verification indicator is generated sequentially.

[0048] In this embodiment, the verification features refer to specific features that can reflect the actual performance of key verification indicators. For example, for the key verification indicator of courseware screen frame correlation, its verification features include the pixel change rate of the screen image in the historical correlation time window of the operation event, the matching degree of key content areas (such as title bar, page number mark, and key marked area), and the OCR recognition matching rate of text content and event content in the screen image.

[0049] In this embodiment, the feature extraction algorithm is selected according to the type of the verification feature and set according to the pre-constructed feature-extraction algorithm mapping table. The feature-extraction algorithm mapping table includes several preset verification features, and each preset verification feature is mapped to at least one extraction algorithm. The feature-extraction algorithm mapping table is pre-constructed based on historical synchronization data and expert experience.

[0050] In this embodiment, each historical verification feature is mapped to a corresponding predicted verification evaluation value. The predicted verification evaluation value is calculated based on the historical synchronization coefficient corresponding to the historical verification feature and the contribution of the historical verification feature to the verification results of the key verification indicators. Specifically, the frequency of occurrence of the historical verification feature in the historical synchronization data and its influence weight on the historical synchronization coefficient are first determined. The contribution of the historical verification feature is calculated based on the frequency of occurrence and the influence weight. The contribution is then normalized to obtain the predicted verification evaluation value. The value range of the predicted verification evaluation value is 0-1.

[0051] In this embodiment, a dynamic adjustment mechanism for the predicted verification evaluation value of historical verification features is also included, that is, the predicted verification evaluation value is periodically updated according to the latest synchronous processing data.

[0052] In this embodiment, the machine learning algorithms used include, but are not limited to, support vector machines, random forests, or neural networks. During the model training process, historical verification features in the historical verification feature set are used as inputs, and the corresponding predicted verification evaluation values ​​are used as output labels. The model parameters are optimized through multiple rounds of iterative training until the prediction error of the model is lower than a preset threshold, thereby obtaining a verification evaluation model that can accurately output single-item verification results.

[0053] In this embodiment, after the model training is completed, the performance of the model is verified and evaluated using a test sample set. When the model's accuracy, recall, and other indicators reach the standard, the model is evaluated and verified.

[0054] In this embodiment, by constructing a verification evaluation model, it is possible to achieve intelligent and accurate evaluation of key verification indicators, providing a reliable basis for subsequent verification evaluation value calculation.

[0055] In some embodiments of this application, the synchronization results of corresponding categories are verified based on verification models for different event types to obtain verification evaluation values, including: The event type of each synchronization result is determined. Based on the determined event type, several key verification indicators, the verification features of each key verification indicator, and the corresponding feature extraction algorithm, several actual verification features of each synchronization result for each key verification indicator of the corresponding event type are obtained. Several actual verification features of the same key verification indicator are input into the verification evaluation model of the corresponding key verification indicator to obtain the corresponding verification evaluation value. The event type verification rules based on synchronization results apply weights to the verification evaluation values ​​of several key verification indicators to obtain a comprehensive verification evaluation value for each synchronization result.

[0056] In this embodiment, after determining the event type of each synchronization result, for each key verification indicator under that event type, the actual verification features corresponding to that indicator are extracted from the current synchronization result to be verified. The extraction process of these actual verification features strictly follows the feature extraction algorithm determined in the model building stage for the corresponding key verification indicators to ensure the consistency and comparability of the feature data.

[0057] In this embodiment, the model outputs a verification evaluation value between 0 and 1 by analyzing and calculating the actual verification features. After obtaining the verification evaluation values ​​of all key verification indicators, the model performs a weighted summation of these individual evaluation values ​​according to the weight coefficients of each key verification indicator set in the event type verification rules to obtain a comprehensive verification evaluation value. This comprehensive verification evaluation value is the indicator that measures the overall synchronization quality of the current synchronization result. The higher the value, the more accurate and effective the spatiotemporal synchronization of the first data packet and the second data packet is.

[0058] In some embodiments of this application, determining whether to correct the synchronization result based on the comprehensive verification evaluation value includes: Pre-set the threshold for comprehensive verification and evaluation values; When the comprehensive verification evaluation value of the synchronization result is greater than the comprehensive verification evaluation value threshold, the synchronization result will not be corrected. When the comprehensive verification evaluation value of the synchronization result is not greater than the comprehensive verification evaluation value threshold, the synchronization result is corrected.

[0059] In this embodiment, the comprehensive verification evaluation value threshold is preset based on historical synchronization data and teaching synchronization accuracy requirements. In this application, the comprehensive verification evaluation value threshold is set to 0.85.

[0060] In some embodiments of this application, the generation of correction instructions corresponding to the synchronization results includes: Identify several indicators to be corrected for the corresponding synchronization results and their corresponding coefficients to be corrected; Based on the event type of the corresponding synchronization result and the indicator to be corrected, a search is conducted in the correction strategy library to obtain several correction strategies for the corresponding synchronization result. The correction strategy library contains preset correction coefficients corresponding to preset indicators to be corrected under different event types, and each preset correction coefficient is mapped to a corresponding correction step and parameter adjustment range. Several correction strategies for the same synchronization result are sorted according to priority rules, and correction instructions for the corresponding synchronization result are generated based on the sorting results.

[0061] In this embodiment, the indicator to be corrected refers to a key verification indicator whose verification evaluation value is less than the corresponding preset verification evaluation value threshold. The correction coefficient is set based on the weight coefficient of the corresponding key verification indicator in the corresponding event type and the difference between the verification evaluation value of the indicator to be corrected and the preset verification evaluation value threshold. The specific calculation formula is: Correction coefficient = (Preset verification evaluation value threshold - Current verification evaluation value) × Key verification indicator weight coefficient. For example, if the weight coefficient of a key verification indicator is 0.3, the preset verification evaluation value threshold is 0.8, and the current verification evaluation value is 0.6, then the correction coefficient = (0.8 - 0.6) × 0.3 = 0.06. The larger the value, the higher the degree of correction required for the indicator.

[0062] In this embodiment, the correction strategy library is a pre-built structured database that stores various preset indicators to be corrected that may occur under each event type, as well as the specific correction steps and parameter adjustment ranges corresponding to each preset indicator to be corrected in different preset correction coefficient ranges. It is set according to historical correction logs.

[0063] In this embodiment, the priority rule refers to the comprehensive setting based on the size of the coefficient to be corrected and the weight coefficient of the indicator to be corrected when there are multiple indicators to be corrected and corresponding to multiple correction strategies for the same synchronization result. The higher the coefficient to be corrected and the higher the weight coefficient of the indicator to be corrected, the higher the priority of the correction strategy.

[0064] In this embodiment, when generating a correction instruction, the system extracts the corresponding correction steps and parameter adjustment ranges in sequence according to the sorted correction strategy, and integrates this information into a clear and executable instruction to guide the specific correction operation of the synchronization result.

[0065] In this embodiment, by generating correction instructions, the synchronization results can be adjusted precisely and in a targeted manner, which effectively improves the accuracy and reliability of real-time synchronization processing of exam teaching videos and electronic courseware, and ensures that the corrected synchronization results can meet the preset synchronization quality requirements.

[0066] In some embodiments of this application, a structured metadata file and courseware video are generated based on the corresponding synchronization result, including: Based on a predefined metadata schema, the synchronization results are transformed and assembled into structured data through verification. The structured data includes event type, event content, time parameters, synchronous media reference identifier, and verification confidence level; Multiple structured data are organized in chronological order to generate a structured metadata file, and interactive index data of the structured metadata file is constructed. Based on a preset synthesis layout, the teaching data in the verified synchronization results are synthesized into a video file. The interactive index data extracted from the structured metadata file is associated with the video file to form the courseware video.

[0067] In this embodiment, the predefined metadata schema refers to a structured description framework that conforms to current clinical industry data standards. The event type field clearly identifies the teaching operation category to which the synchronization result belongs; the event content field records detailed information about the operation, such as the target page number in a page-turning event, and the annotation position and content description in an annotation event; the time parameter includes time-series information such as the precise timestamp of the event and its duration; the synchronization media reference identifier is used to establish a mapping relationship with teaching videos and electronic courseware to ensure that the metadata is traceable to the corresponding media resources; and the verification confidence field is filled with the comprehensive verification evaluation value of the synchronization result, which intuitively reflects the synchronization quality.

[0068] In this embodiment, during the construction of interactive index data, the system generates a unique index identifier for each structured data and establishes a timeline index based on time parameters. At the same time, it constructs a retrieval index based on event type and key content keywords, enabling users to quickly locate the corresponding teaching segment in the video by time point, event type, or content keywords.

[0069] In this embodiment, the preset composite layout can be flexibly configured according to the needs of the teaching scenario. For example, the "picture-in-picture" layout embeds the teacher's video window into a designated area of ​​the courseware screen image, while the "split-screen" layout displays the courseware content and the teacher's video side by side.

[0070] In this embodiment, the courseware video contains complete teaching audio and video content, and also supports interactive operations by users during playback through associated interactive index data, such as clicking on key content marked by metadata to jump to the corresponding video location, viewing event details, etc., thereby improving the integrated application effect of exam teaching videos and electronic courseware and the user experience.

[0071] In this embodiment, by constructing metadata files and generating courseware videos, the structured integration and efficient utilization of examination teaching resources are realized, enhancing the interactivity and practicality of teaching resources. This enables clinical learners to more conveniently access, understand, and review teaching content, effectively improving the quality and effectiveness of examination teaching.

[0072] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of this application, and these improvements and substitutions should also be considered within the scope of protection of this application.

Claims

1. A method for real-time synchronous processing of exam teaching videos and electronic courseware, characterized in that, include: Real-time capture of operation events of electronic courseware and synchronous collection of teaching data, generating corresponding first data packet and second data packet; The first and second data packets are spatiotemporally synchronized to obtain multiple synchronization results. Based on the verification model of different event types, the corresponding synchronization results are verified to obtain a comprehensive verification evaluation value. Based on the comprehensive verification evaluation value, determine whether to correct the synchronization result. If yes, generate the corresponding correction instruction for the synchronization result; otherwise, generate a structured metadata file and courseware video based on the corresponding synchronization result.

2. The method for real-time synchronous processing of examination teaching videos and electronic courseware as described in claim 1, characterized in that, Real-time capture of operation events in electronic courseware and synchronous collection of teaching data, generating corresponding first data packet and second data packet, including: Based on the courseware parsing agent module running on the teaching terminal, the operation events generated by the electronic courseware application are captured and parsed in real time at the operation level. For each captured operation event, a first data packet is generated containing the event type, event content, and corresponding first timestamp; The video stream, audio stream, and screen data stream of the examination are collected in parallel using a recording terminal. The system generates corresponding teaching data based on the collected video stream, audio stream, and screen data stream. The teaching data includes video data, audio data, and screen image data of the corresponding frames. For each frame of teaching data, a second data packet is generated, which includes the corresponding frame's video data, audio data, screen image data, and the corresponding second timestamp.

3. The method for real-time synchronous processing of examination teaching videos and electronic courseware as described in claim 2, characterized in that, Before performing spatiotemporal synchronization on the first and second data packets, the following is also included: Preset reference clock; The time information of the reference clock is distributed to the teaching terminal and the recording terminal, and the time deviation and clock drift rate between the actual clock and the reference clock of each terminal are monitored in real time, and the synchronization status parameters corresponding to each timestamp are generated. The synchronization status parameters are encapsulated into a data packet with the corresponding timestamp, and dynamic time calibration calculation is performed to convert the timestamp in the corresponding data packet into a unified timestamp relative to the reference clock, so as to obtain the reference timestamp of the corresponding data packet. Generate reference timestamps for all data packets and map them onto a preset reference timeline.

4. The method for real-time synchronization of examination teaching videos and electronic courseware as described in claim 1, characterized in that, Spatiotemporal synchronization of the first and second data packets includes: For each event type in the first data packet, a corresponding synchronization time window parameter is generated. The synchronization time window parameter includes the prediction duration for forward lookup and the confirmation duration for backward lookup. The synchronization time window for each first data packet on the reference time axis is generated according to the synchronization time window parameters and the reference timestamp of the first data packet. According to each synchronization time window and the corresponding synchronization conditions, the second data packet is retrieved, and the retrieved second data packet is spatiotemporally synchronized with the first data packet of the corresponding synchronization time window to obtain multiple synchronization results.

5. The method for real-time synchronous processing of examination teaching videos and electronic courseware as described in claim 1, characterized in that, The verification models for different event types include: Several event types are predefined, and one event type is randomly selected as the target type; Obtain the historical synchronization logs of the target type, sort the historical synchronization data according to the historical synchronization coefficients in the historical synchronization logs, and obtain the historical synchronization data matrix of the target type; Based on the historical synchronization data matrix, several key verification indicators that affect the synchronization effect of the target type are determined, and the weight coefficients of the corresponding key verification indicators are set according to the degree of influence. Verification rules for the target type are constructed based on several key verification indicators and their corresponding weighting coefficients; Construct a verification evaluation model for each key verification indicator; Generate a verification model for the target type based on the verification rules and several verification evaluation models; Generate a validation model for each event type in sequence.

6. The method for real-time synchronization of examination teaching videos and electronic courseware as described in claim 5, characterized in that, Construct a verification evaluation model for each key verification indicator, including: Determine the verification features of each key verification indicator, and call the corresponding feature extraction algorithm to generate a historical verification feature set for each key verification indicator. The historical verification feature set includes several historical verification features, and each historical verification feature is mapped to a corresponding predicted verification evaluation value. A training sample set is set based on the historical verification feature set, and a machine learning algorithm is used to train the model on the training sample set to obtain the verification evaluation model corresponding to the key verification indicators. The verification evaluation model for each key verification indicator is generated sequentially.

7. The method for real-time synchronization of examination teaching videos and electronic courseware as described in claim 5, characterized in that, The synchronization results of the corresponding categories are verified based on the verification model for different event types to obtain verification evaluation values, including: The event type of each synchronization result is determined. Based on the determined event type, several key verification indicators, the verification features of each key verification indicator, and the corresponding feature extraction algorithm, several actual verification features of each synchronization result for each key verification indicator of the corresponding event type are obtained. Several actual verification features of the same key verification indicator are input into the verification evaluation model of the corresponding key verification indicator to obtain the corresponding verification evaluation value. The event type verification rules based on synchronization results apply weights to the verification evaluation values ​​of several key verification indicators to obtain a comprehensive verification evaluation value for each synchronization result.

8. The method for real-time synchronization of examination teaching videos and electronic courseware as described in claim 7, characterized in that, Whether to correct the synchronization results is determined based on the comprehensive verification evaluation value, including: Pre-set the threshold for comprehensive verification and evaluation values; When the comprehensive verification evaluation value of the synchronization result is greater than the comprehensive verification evaluation value threshold, the synchronization result will not be corrected. When the comprehensive verification evaluation value of the synchronization result is not greater than the comprehensive verification evaluation value threshold, the synchronization result is corrected.

9. The method for real-time synchronous processing of examination teaching videos and electronic courseware as described in claim 1, characterized in that, Generate correction instructions corresponding to the synchronization results, including: Identify several indicators to be corrected for the corresponding synchronization results and their corresponding coefficients to be corrected; Based on the event type of the corresponding synchronization result and the indicator to be corrected, a search is conducted in the correction strategy library to obtain several correction strategies for the corresponding synchronization result. The correction strategy library contains preset correction coefficients corresponding to preset indicators to be corrected under different event types, and each preset correction coefficient is mapped to a corresponding correction step and parameter adjustment range. Several correction strategies for the same synchronization result are sorted according to priority rules, and correction instructions for the corresponding synchronization result are generated based on the sorting results.

10. The method for real-time synchronous processing of examination teaching videos and electronic courseware as described in claim 1, characterized in that, Based on the corresponding synchronization results, structured metadata files and courseware videos are generated, including: Based on a predefined metadata schema, the synchronization results are transformed and assembled into structured data through verification. The structured data includes event type, event content, time parameters, synchronous media reference identifier, and verification confidence level; Multiple structured data are organized in chronological order to generate a structured metadata file, and interactive index data of the structured metadata file is constructed. Based on a preset synthesis layout, the teaching data in the verified synchronization results are synthesized into a video file. The interactive index data extracted from the structured metadata file is synchronized with the video file to form the courseware video.

Images