An academic speech automatic scoring method and system based on multi-modal fusion

Abstract

The present application belongs to the technical field of speech signal processing and natural language processing, and is used to solve the problem that the prior art cannot track and score the cross-page explanation task according to the semantic attribution of speech instead of the time sequence position, and specifically relates to an academic speech automatic scoring method and system based on multi-modal fusion, which comprises the following steps: obtaining speech stream data of a speaker and page content data and page switching time sequence data of a demonstration document; routing the speech semantic block to a current page record area, a delay pointer list of historical footprint particles or a bridge integral area according to the attribution type; at the end of the speech, adjusting the basic score by using the total deduction score and the coherence bonus and outputting the final score; by constructing a cross-page semantic footprint pool to track the unclosed logical predicate of the historical page, the system can allocate the explanation task according to the semantic attribution of speech instead of the time sequence position, and effectively avoids the scoring deviation caused by the cross-page backtracking supplementary speech being wrongly attributed to a new page.

Description

Technical Field

[0001] This invention belongs to the field of speech signal processing and natural language processing technology, specifically an automatic scoring method and system for academic presentations based on multimodal fusion. Background Technology

[0002] In the field of automatic scoring of academic presentations, multimodal fusion-based evaluation methods have become the mainstream technical approach. These methods typically collect the speaker's voice signal and the page content of the presentation document simultaneously. By establishing a temporal correspondence between voice segments and PPT pages, the completeness of the explanation on each page is evaluated separately, and then the overall quality score of the presentation content is obtained by fusion. To achieve the above temporal correspondence, existing systems generally regard PPT page turning events as natural semantic segmentation boundaries, that is, they consider the moment of page turning to mark the end of the explanation task on the previous page and the beginning of the explanation task on the next page, and the subsequent voice stream is assigned to the new page accordingly.

[0003] This processing logic works effectively when the speaker strictly follows the page order; however, academic presentations have a high cognitive load, and speakers often employ non-linear knowledge transfer strategies to enhance their argumentation. A typical scenario exists in practice: after navigating to the experimental results page, the speaker needs to go back and supplement a constraint not detailed in the methods page to clarify the theoretical basis of the experimental design; or after turning the page, they may provide a summary of a complex chart from the previous page. In such cross-page backtracking and supplementation scenarios, the audio content semantically belongs to the old page, but temporally it is located after the page-turning event. Existing systems, lacking the ability to recognize the semantical cross-page existence of audio, forcibly assign such audio to the new page, leading to two consequences: first, a large amount of irrelevant audio is mixed into the new page's explanation, interfering with the effective extraction of explanation features for that page; second, the old page is judged as incomplete due to the lack of key supplementary audio, causing a systematic bias in the structural integrity score. The root cause of this deviation lies in the improper equating of physical page-turning events with the switching boundaries of logical explanation tasks, while the latter often has a temporal misalignment with the former during actual presentations.

[0004] Therefore, there is an urgent need for a mechanism that can track and score cross-page explanation tasks based on the semantic attribution of speech rather than its temporal position. Summary of the Invention

[0005] The purpose of this invention is to provide an automatic scoring method and system for academic presentations based on multimodal fusion; to solve the problem that existing technologies cannot track and score cross-page presentation tasks based on the semantic attribution of speech rather than its temporal position.

[0006] The technical problem to be solved by this invention is: how to provide an automatic scoring method and system for academic presentations based on multimodal fusion that can track and score cross-page explanation tasks based on the semantic attribution of speech rather than temporal position.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] An automatic scoring method for academic presentations based on multimodal fusion includes:

[0009] Acquire the speaker's audio stream data, the presentation document's page content data, and page transition sequence data;

[0010] In response to page switching events, semantic parsing is performed on the content of the current page to extract entities and explicit predicates. The default predicate slots are marked by combining the theoretical predicate templates corresponding to the chapter type, generating a default vector of logical predicates and encapsulating it as historical footprint particles to be stored in the cross-page semantic pool.

[0011] The switched speech stream is semantically segmented to obtain speech semantic blocks. The similarity of the speech semantic block is compared with the semantic vector of the current page and the page vector of each historical footprint particle in the pool. The type of the speech semantic block is determined based on the comparison results.

[0012] Based on the attribution type, the speech semantic block is routed to the current page record area, the delayed pointer list of historical footprint particles, or the bridging integral area;

[0013] For the speech semantic block pointed to by the delayed pointer, extract its core predicate and entity reference, and match them with the predicate category and subject-object reference of the corresponding default predicate slot respectively. If the match is successful, update the slot to be filled.

[0014] At the end of the presentation, the final state of the default predicate slots of each historical footprint particle is traversed. The pre-stored penalty for unfilled slots is converted to positive and weighted according to the particle survival time to obtain the total deduction. At the same time, a coherence bonus is generated based on the data of the bridging integration area. The base score is adjusted using the total deduction and the coherence bonus, and the final score is output.

[0015] The present invention has the following beneficial effects:

[0016] 1. By downgrading PPT page-turning events from semantic boundaries to weak logical events, a cross-page semantic footprint pool is constructed to track unclosed logical predicates of historical pages. This enables the system to allocate explanation tasks based on the semantic attribution of speech rather than its temporal position, effectively avoiding scoring bias caused by cross-page backtracking supplementary speech being incorrectly assigned to a new page.

[0017] 2. By using an adaptive attribution determination mechanism based on extreme value difference, the attribution boundary is dynamically defined by utilizing the similarity distribution characteristics between speech blocks and each candidate page, so that the system can maintain a stable attribution accuracy in speeches with different semantic densities and terminology styles.

[0018] 3. By routing fuzzy attribution states (bridging states) to independent integration regions instead of mandatory classification, ambiguous speech blocks, which are regarded as decision noise in traditional systems, are transformed into positive evidence representing cross-page logical coherence, thus realizing a quantitative assessment of the speaker's higher-order narrative ability.

[0019] 4. By pre-storing penalties and reverse settlement mechanisms at the terminal, the deduction of structural integrity points is delayed until the end of the speech, and the decision on whether to execute the penalty is based on the subsequent supplementary status. This gives the scoring system the ability to correct errors after the fact, and avoids inappropriate penalties for advanced presentation skills due to real-time misjudgment caused by the blank logic in the early stage. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of the method according to Embodiment 1 of the present invention;

[0022] Figure 2 This is a system block diagram of Embodiment 2 of the present invention. Detailed Implementation

[0023] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example 1: As Figure 1 As shown, an automatic scoring method for academic presentations based on multimodal fusion includes:

[0025] Step S1: Obtain the speaker's audio stream data, the page content data of the presentation document, and the page switching sequence data; in response to the page switching event, perform semantic parsing on the page content of the current page to extract entities and explicit predicates, mark the default predicate slots by combining the theoretical predicate templates corresponding to the chapter type, generate the logical predicate default vector, and encapsulate it as historical footprint particles and store it in the cross-page semantic pool.

[0026] like Figure 1 As shown, the method provided in this embodiment pre-builds several core resources before running to support semantic parsing and default slot marking in step S1. The specific construction method of these resources is described below.

[0027] Regarding the predicate semantic category mapping table, this embodiment pre-constructs a mapping relationship from predicate prototypes to semantic categories based on a corpus of academic speeches. Specifically, it collects common academic speech texts and paper abstracts covering mainstream engineering disciplines such as computer science, electrical engineering, and biomedicine, and extracts predicate verbs whose frequency exceeds a set threshold, totaling approximately 120 predicate prototypes. For each predicate prototype, domain experts label it with a corresponding semantic category tag based on its semantic function. The semantic category tag system includes "causal transformation category" (covering semantic functions such as induction, inhibition, transformation, mapping, decomposition, and aggregation). The mapping table is divided into several categories: "structural association" (covering semantics such as connection, inclusion, membership, input to, output to), "comparison" (covering semantics such as higher than, lower than, better than, inferior to, equivalent to, and approaching), "trend conclusion" (covering semantics such as convergence, divergence, stability, improvement, and decline), "problem statement" (covering semantics such as existence, lack, need, and challenge), and "contribution statement" (covering semantics such as proposal, design, implementation, and verification). This mapping table is stored in a key-value pair structure, where the key is the predicate prototype string and the value is the corresponding semantic category label. It is loaded into memory at system startup for subsequent fast lookups.

[0028] The predicate semantic category mapping table covers approximately 120 predicate prototypes. This number is derived from statistics on mainstream engineering and scientific academic lecture corpora: the number of high-frequency predicate verbs (those with a cumulative frequency exceeding 95%) is typically between 80 and 150. These 120 prototypes cover the vast majority of core logical relational verbs in conventional academic expressions. Low-frequency predicates exceeding this number have a negligible contribution to the overall evaluation results statistically, and classifying them as "unknown categories" and excluding them from matching has an impact of less than 3% on the overall recall rate of the system (based on actual measurements on the development set).

[0029] The mapping table does not involve synonym reasoning or fuzzy matching; all predicate matching is a precise lookup operation from string to label. For predicates not included in the database, the system queries the predicate for its hypernym in WordNet. If the hypernym belongs to a set of predicate prototypes already included in the database, it inherits the semantic category label of that prototype. If the hypernym query also fails, the predicate is classified as "unknown category" and does not participate in any default slot matching. That is, predicate verbs of unknown category in this speech block will not trigger closure determination.

[0030] Regarding the expected template for core logical relationships, this embodiment configures corresponding templates for different chapter types. The determination of chapter type is based on the title level information in the PPT outline tree. If the outline is missing, it is matched based on the keywords in the page title text. The matching rules are as follows: if the title contains any keyword from "method, model, algorithm, architecture, design, process", it is classified as a method chapter; if it contains any keyword from "experiment, result, performance, comparison, ablation, evaluation", it is classified as an experiment chapter; if it contains any keyword from "introduction, background, motivation, related work, problem definition", it is classified as an introduction chapter; and the rest are classified as other chapters. For pages classified as other chapters, their theoretical predicate templates only contain the two most basic "existence class" and "description class". The general semantic categories ensure that even if the title information is missing, the page can still participate in the default slot generation process, but the number of generated slots is significantly less than that of pages with explicit chapter type tags. The template for the Methodology chapter contains two semantic category sets: "Causal Transformation" and "Structural Relationship". The template for the Experiment chapter contains two semantic category sets: "Comparison and Contrast" and "Trend and Conclusion". The template for the Introduction chapter contains two semantic category sets: "Problem Statement" and "Contribution Statement". The templates for other chapters only contain two general semantic category sets: "Existence" and "Description". Each template is also stored in a key-value pair structure, where the key is the chapter type identifier and the value is the corresponding semantic category set.

[0031] Regarding the text semantic parsing tool, this embodiment configures a text processing pipeline based on dependency parsing. This pipeline includes a word segmentation module, a part-of-speech tagging module, a named entity recognition module, and a dependency parsing module. The dependency parsing module outputs the dependencies between words in a sentence, identifying the core root predicate verb and the subject-predicate and verb-object relationships between its components. The part-of-speech tagging module distinguishes between nouns and verbs, providing basic labels for subsequent entity extraction and predicate verb extraction. These modules are implemented using an open-source natural language processing library and have been fine-tuned on text corpora from academic lectures to improve the accuracy of recognizing academic terms and formula identifiers.

[0032] Based on the above resource pre-configuration, the specific operation process of step S1 is as follows: When the system captures the PPT page turning event, it detects that the currently displayed page is about to switch away, records the moment as the switching timestamp, and triggers the offline semantic parsing process of the old page; the system first obtains the page content data of the old page, which comes from the text information extracted by optical character recognition from the PPT screen screenshot, including the page title, the main text at all levels, the chart title text, the axis label text, and the text annotation part in the formula.

[0033] The page content data is segmented into sentences using periods, question marks, exclamation marks, and line breaks as segmentation boundaries to obtain the sentence sequence of the page. For each sentence, a text semantic parsing pipeline is sequentially called to perform part-of-speech tagging and dependency parsing, resulting in a part-of-speech tag sequence and dependency relation tree for each sentence. Based on this, two types of information extraction are performed: the first is entity extraction: the system extracts all noun words and phrases based on part-of-speech tags, including common nouns, proper nouns, formula identifiers (variable names appearing in mathematical symbol form), and chart labels (such as "Table 1"). All extracted entities are compiled into the entity set of the page. If no entity is extracted after optical character recognition of the page content data... If the text is effective, the system sets the overall sentence embedding vector of the page to zero and sets the semantic self-consistency flag of the page to true. The zero vector will not become the maximum similarity candidate of any speech block in subsequent similarity comparisons (the similarity between the zero vector and any vector is 0 in the cosine similarity calculation), thus avoiding the page being misclassified as the speech object. The second type is explicit predicate verb extraction: the system identifies the core root predicate verb of each sentence (i.e., the verb whose dependency relationship is the root node) based on the dependency relationship tree, and filters out copulas and light verbs without actual logical connection meaning, retaining the core predicate verbs with action, causality, comparison or transformation semantics, and sets all retained predicate verbs into the explicit predicate verb set of the page.

[0034] After obtaining the entity set and the explicit predicate verb set, the system retrieves the chapter type tag of the page and loads the expected template of the core logical relationship corresponding to the chapter type. The system iterates through each semantic category in the template and converts each verb in the explicit predicate verb set into its semantic category tag through the predicate semantic category mapping table to obtain the explicit semantic category set of the page. For each semantic category in the template, the system searches whether there is a category with the same name in the explicit semantic category set. If there is no match, the semantic category is marked as a default predicate category, and the preset anchor prototype verb under the category is recorded (the predicate prototype with the highest frequency under the category is selected from the mapping table as the anchor).

[0035] For each tagged default predicate category, determine its logical subject and logical object candidates: Identify the core keyword of the page from the page content data by extracting noun phrases from the page title text and removing stop words; if the title is empty, extract the subject from the first sentence of the main text as the core keyword; iterate through each entity in the entity set and count the number of sentences in which each entity and the core keyword co-occur in the original page text as co-occurrence frequency; select the entity with the highest co-occurrence frequency as the logical subject candidate; if there is an entity with the second highest co-occurrence frequency, select it as the logical object candidate; otherwise, set the object candidate to empty (indicating that the default predicate is a unary attribute). (Assertions, not binary relation assertions); If the logical subject candidate of the current default predicate category is empty (i.e., the page entity set is empty or the core keyword is not determined), the system determines that the default predicate category is not worth tracking because it lacks a traceable entity anchor point, skips the generation of the slot, records such a default as "untraceable default" and writes it to the log, but the record does not enter the default predicate slot list and does not participate in subsequent matching verification and deduction settlement; after processing, it continues to traverse the next default predicate category; the default predicate category, anchor prototype verb, logical subject candidate, logical object candidate and the initial "unfilled" filling status identifier are combined into a default predicate slot.

[0036] For example, on a page in the "Methods" section, the page title is "Feature Extraction and Classification Process," and the text includes "After the feature extraction module performs a Fourier transform on the input signal, it transmits the spectral features to the classifier." After parsing, the explicit predicate verbs "to perform" and "to transmit" are extracted. After transformation using a mapping table, the explicit semantic category set includes "Causal Transformation" (corresponding to the semantics of "to perform") and "Structural Association" (corresponding to the semantics of "to transmit"). The template corresponding to this section type contains two expected categories: "Causal Transformation" and "Structural Association." Traversing the template reveals that both categories already appear in the explicit category set; therefore, the default predicate slot for this page is determined. If the list is empty, the page is marked as a semantically self-consistent page; conversely, if the page text is only "feature extraction module connected to classifier", then the explicit predicate verb only contains "connect", and after mapping, only the "structural association class" category is obtained. The "causal transformation class" does not appear in the explicit category set, so the system marks the "causal transformation class" as the default predicate category, extracts "transformation" as the anchor prototype verb, selects "input signal" with the highest co-occurrence frequency with "feature extraction" as the subject candidate from the entities, and "spectral features" as the object candidate, generating a default predicate slot, indicating that the page lacks an explanation of the logical relationship of "what kind of transformation the input signal undergoes to become spectral features".

[0037] The system's sentence embedding model employs a pre-trained Transformer encoder structure. The input is a text sequence truncated to 512 tokens, and the output is a fixed-length vector of 768 dimensions. After pre-training on a general domain corpus, the encoder is further fine-tuned using approximately 100,000 pairs of academic paper abstracts and speech captions. The fine-tuning aims to shorten the cosine distance between the abstract sentence vector and the corresponding semantic expression in the speech sentence. All page content text and speech semantic block text share the same encoder instance to ensure that the vectors output by both are located in the same semantic space.

[0038] After marking all default predicate slots, the system combines the page identifier, chapter type tag, overall sentence embedding vector (obtained by encoding all text on the page using a lightweight sentence encoder), default predicate slot list, and semantically consistent tag into the logical predicate default vector for that page. Then, an active footprint particle is created, whose data structure contains all the contents of the logical predicate default vector, plus an entry timestamp, current decay weight (initially 1.0), and backtracking hit count counter (initially 0). This active footprint particle is then pushed into the cross-page semantic pool, which is a first-in, first-out queue with its capacity dynamically set based on the total number of pages in the PPT.

[0039] If the queue is full, the oldest particle must be discarded. Before discarding, check if the particle meets any of the following conditions: there is a slot in the default predicate slot list whose fill status is still "unfilled" and the particle is not marked as semantically consistent; the particle's backtracking hit count counter is not less than 3 times and there are still unclosed slots.

[0040] If any of the above conditions are met, the system will discard the particle and simultaneously write the particle's page identifier, all unfilled default slot numbers and corresponding pre-deduction values ​​(-0.3 points / slot), current decay weight value, and backtracking hit count counter value into the "Particle Sedimentation Log". This sedimentation log is stored independently of the cross-page semantic pool, and its records will not be lost due to changes in pool capacity. Each record in the sedimentation log has its decay weight value determined at the time of writing (taking the current value at the moment the particle is discarded), and will not be updated subsequently. The discarded particle and its corresponding pending penalty record in the temporary pool will be deleted (to avoid duplicate counting). If the particle does not have any unfilled slots, it will be discarded directly without any additional operations.

[0041] The settlement results of the sedimentation log and the temporary storage pool are summarized into a single total score for structural integrity after step S5, and the two do not overlap. The pending penalty records in the temporary storage pool that have been covered by the sedimentation log are removed when the particle is discarded to avoid the same omission being scored repeatedly.

[0042] At this point, step S1 completes the encapsulation of the logical snapshot of the switched page, providing a structured data foundation for subsequent steps to track cross-page semantic attribution.

[0043] Step S2: Semantically segment the switched speech stream to obtain speech semantic blocks. Compare the similarity of the speech semantic block with the current page semantic vector and the page vector of each historical footprint particle in the pool. Determine the type of the speech semantic block based on the comparison results.

[0044] In step S2, the system performs semantic segmentation and attribution determination on the audio stream acquired after page turning. This step relies on a pre-loaded audio stream cache queue and a text semantic encoder. The encoder uses the same lightweight sentence embedding model as the page vector generated in step S1 to ensure that the audio block and the page are in the same vector space. The specific operation process is as follows.

[0045] The system receives audio signals from the microphone array in real time. After filtering out non-human voice segments by the speech activity detection module, the valid speech segments are sent to the automatic speech recognition engine and converted into a timestamped text stream. This text stream does not immediately participate in the attribution determination but is temporarily stored in the speech buffer queue, waiting for the segmentation conditions to be triggered. The segmentation boundary is determined by two types of signals: one is the syntactic boundary, that is, the presence of terminal punctuation marks such as periods, question marks, and exclamation marks in the recognition result; the other is the prosodic boundary, that is, the silence interval in the continuous speech segment exceeds 0.8 seconds. If either of the two boundaries is met, the segmentation action is triggered, and the text sequence in the buffer queue is extracted into an independent speech semantic block. Each speech semantic block is encapsulated as a data object containing a unique block identifier, the original text content, the start timestamp, and the end timestamp, and then sent to the text semantic encoder to generate the semantic vector of the block.

[0046] Regarding the similarity comparison and classification determination process, the system executes as follows: Let the semantic vector of the current page be... The number of historical footprint particles surviving in the cross-page semantic pool is The page vector of each historical particle is The vector of the current speech semantic block is The system calculates one by one. With the above The cosine similarity between candidate page vectors yields a similarity set. Each similarity score is calculated by dividing the dot product of the two vectors by the product of their respective magnitudes; the maximum similarity value is extracted from this set. With the second largest similarity value and record The corresponding candidate page index.

[0047] The attribution determination rule uses adaptive logic based on extreme value difference, rather than a fixed threshold; system comparison and The difference reflects the affinity gap between the two closest candidate pages in the semantic space for the speech block; if the difference is not lower than the first preset difference threshold of 0.15, the system considers the speech block to have a clear single attribution tendency, marks its attribution type as strong attribution, and the attribution object is... The corresponding candidate page could be the current page or a historical page. If the difference is lower than the second preset difference threshold of 0.08, the system determines that the speech block is in a semantically ambiguous region and marks its classification as bridging. When the difference falls between 0.08 and 0.15, the system further checks... The absolute value: If the value is not lower than the preset absolute value threshold of 0.55, it is treated as a strong affiliation; if it is lower than 0.55, it is treated as a bridged state.

[0048] For example, suppose the similarity between a certain speech block vector and the current page vector is 0.72, and the similarity with the page vector of a certain historical particle is 0.68, while the similarity of the remaining candidate pages is all below 0.40; then , If the difference is 0.04, which is lower than 0.08, the speech block is judged to be in a bridging state, indicating that its semantics simultaneously anchors the current page and the history page, constituting transitional evidence for cross-page knowledge flow; as another example, if the largest similarity value in the similarity set is 0.81, the second largest similarity value is 0.52, and the difference is 0.29, which is not lower than 0.15, then the attribution type is marked as strong attribution, and the attribution object is the page corresponding to 0.81.

[0049] If the above maximum similarity value If the similarity is below the low-value filtering threshold of 0.35, the system will mark the speech block as weak noise regardless of the difference and will not participate in any subsequent assignment routing. The reason for setting this absolute lower limit is that too low similarity means that the speech block has no effective semantic connection with any candidate page. Such segments usually correspond to filler words or inserted sentences that are irrelevant to the content in a speech. Excluding them can avoid noise interference in the subsequent scoring accumulation process. After the above determination, step S2 outputs the assignment type label to step S3 to perform routing.

[0050] The difference threshold (0.15, 0.08), absolute value threshold (0.55), and low value filtering threshold (0.35) in the above judgment rules are derived from the statistical analysis of the speech-page similarity distribution in academic speech corpora. Specifically, 2000 speech-page pairing samples were collected. The mean cosine similarity of pairings belonging to the same logical page (positive examples) was 0.74, and the standard deviation was 0.11. The mean cosine similarity of pairings belonging to different pages (negative examples) was 0.43, and the standard deviation was 0.14. Therefore, 0.55 is statistically significant as a lower bound for positive examples (mean minus approximately 1.7 times the standard deviation). The difference threshold of 0.15 corresponds to the mean of the largest and second largest similarity differences in positive examples (measured as 0.17), rounded to an integer value; 0.08 corresponds to the mean of the difference in negative examples (measured as 0.07), rounded to an integer value; 0.35 is used as the absolute lower limit, which is the median value between the mean similarity of negative examples minus one standard deviation (0.43-0.14=0.29) and the lower limit of positive examples of 0.55, used to exclude noisy speech that has no substantial connection with any page; the above four thresholds can be fine-tuned in their respective vicinity ranges by grid search based on the actual data distribution of the deployment environment before system deployment (the recommended search range is ±20% of each threshold).

[0051] For speech semantic blocks marked as bridging state by the attribution determination rules, before the attribution type label is output to step S3, pre-intensity filtering is performed: the system uses the currently calculated... and Calculate bridging strength value ;like If the intensity is below a preset threshold (0.50 in this embodiment), the attribution type label of the speech block is downgraded from bridging state to weak noise. This label change takes effect before output. When step S3 receives the block, it is directly processed as weak noise and does not enter any routing branch; only when When the value is not lower than 0.50, the bridging state label remains unchanged and is output normally to step S3; the above threshold can be adjusted to the optimal value through grid search on the calibration dataset before system deployment.

[0052] Regarding the strength threshold of 0.50: Statistically, when the cosine similarity between a speech block and a page reaches 0.50 or higher, approximately 78% of the speech block is manually labeled as having a substantial connection with the corresponding page; when the similarity is below 0.50, this proportion drops to approximately 22%. Therefore, 0.50 is used as the lower limit for bridging effectiveness to filter out weak bridging candidates that are not sufficiently related to either page, thus preventing low-value noise from entering the coherence scoring statistics. This threshold can also be calibrated and adjusted within the range of 0.40 to 0.60 before deployment.

[0053] Step S3: Based on the attribution type, route the speech semantic block to the current page recording area, the delay pointer list of historical footprint particles, or the bridging integration area;

[0054] Step S3 receives the attribution type label, corresponding page index, and maximum similarity value output from step S2. The second largest similarity value The routing action is divided into three types of processing paths based on the label, including the storage address of the current speech semantic block.

[0055] When the attribution type is strong attribution, the system checks... The corresponding candidate page index; if the index points to the current page, the system appends the storage address of the speech block to the end of the linear explanation address list of the current page. This list is used by the subsequent basic scoring module to extract explanation evidence for this page; if the index points to a historical footprint particle, the system locates the particle and adds a new record to the delay pointer list inside its data structure. This record only contains the storage address of the speech block and its timestamp, without copying the speech content; at the same time, the backtracking hit count counter of the particle is incremented; no content movement or copying operation is performed here, only the ownership relationship is established through address reference to maintain the uniqueness of the data and save storage overhead.

[0056] The system generates two identical bridging metadata copies, each containing the storage address of the voice block and... The value is divided into two parts: one is mounted to the bridging points area of ​​the current page, and the other is mounted to... The bridging integration region of the corresponding historical footprint particles; both metadata are marked to indicate that they are not included in the core explanation closure count of the corresponding page, but are only used for subsequent continuity bonus statistics.

[0057] When the attribution type is weak noise, the system does not perform any routing operations, directly releases the cache resources of the speech block, and its content does not participate in any subsequent scoring process.

[0058] After the above three types of routing are completed, step S3 outputs the updated data structure, including the linear explanation address list of each page, the delay pointer list and hit counter of each historical footprint particle, and the bridging integration area record between each page and historical particles, for use by steps S4 and S5.

[0059] Step S4: For the speech semantic block pointed to by the delayed pointer, extract its core predicate and entity reference, and match them with the predicate category and subject-object reference of the corresponding default predicate slot. If the match is successful, update the slot to be filled.

[0060] If the candidate subject for the current default predicate slot is found to be empty (this situation should not occur in the normal process and is only used as an exception defense), the system will directly set the filling status of the slot to "filled" (i.e., it is regarded as not needing to be tracked) and record a "zero-element slot automatically closed" log, skipping the subsequent matching verification of the slot.

[0061] Step S4 receives all pointer records that were attached to the delayed pointer list of each historical footprint particle in step S3, and processes them one by one. For each delayed pointer, the semantic self-consistency flag of the historical footprint particle to which the pointer belongs is checked. If the flag is true, it means that the page itself does not have a default predicate slot that needs to be closed. The pointer is marked as "no need to close" and the subsequent processing is skipped. The pointer record is released directly. If the semantic self-consistency flag is false, the system performs the following matching process.

[0062] Using the address stored in the delay pointer, the corresponding speech semantic block text is extracted from the speech cache. Dependency parsing is then performed on the text again to extract the core root predicate verb. If there is no predicate verb in the speech block, the system directly marks the delay pointer as "invalid supplement" and does not update any slot status. If there is a core predicate verb, the system inputs the verb into the predicate semantic category mapping table pre-built in step S1 to query and obtain its semantic category label.

[0063] The system locates slots in the default predicate slot list within the historical footprint particle whose fill status is still "unfilled" and performs matching checks on each one. The matching process is divided into two levels: the first level is core predicate semantic category matching, comparing the semantic category label of the current speech block's core predicate verb with the predicate semantic category recorded in the current default slot; if they match, the first level passes. The second level is entity referential matching; all entity nouns are extracted from the speech semantic block text, and referential phrases (such as "this module," "its," "this structure") are identified. For each referential phrase, the system performs rule-based coreference resolution: it backtracks to the three most recent sentences; if any explicit entity noun in the three sentences matches the gender and singular / plural form of the noun in the sentence, it is considered a core reference. If the explicit entity noun is found within three sentences, the derivation of the derivation phrase is abandoned, and the original phrase is used in subsequent matching. After abandoning the derivation, the derivation phrase is not used as a valid match for any default slot in subsequent entity matching. This means that derivation supplements that span too far cannot be recognized by the system. This design is based on the following engineering trade-offs: derivations that span more than three sentences are already a relatively far semantic jump in the audio stream. If a forced association is established, it may introduce false matching, and the risk of misjudgment is higher than the benefit of missed judgment. The word vector similarity is calculated between the derivation set of entity nouns (including the original explicit entity nouns and the successfully derivationed derivation results) and the "logical subject candidates" and "logical object candidates" recorded in the default slots.

[0064] If a subject candidate has a matching item in the set with a similarity greater than the preset similarity threshold (0.75), the subject referencing match is considered successful. If an object candidate is not empty and has a matching item in the set with a similarity greater than 0.75, the object referencing match is considered successful. For a binary default slot that records both subject and object candidates, a match between at least one of the subject and object candidates is considered a successful second-level test. For a unary default slot that records only subject candidates, a successful second-level test is considered a successful test only if the subject referencing match is successful. When both levels of testing are successful, the system updates the filling status of the default predicate slot to "filled" and records the time difference between the slot's page switching time and the current time as the closure delay duration, while marking the delay pointer as "verified closure". If either level of testing fails, the filling status of the slot remains unchanged, and the delay pointer is marked as "invalid supplement".

[0065] The preset similarity threshold for entity matching is set to 0.75. This value is based on the statistical distribution of similarity of synonymous entity words in the general pre-trained word vector space. The cosine similarity of word vectors of the same entity under different expressions (such as "feature extractor" and "feature extraction module") is usually between 0.70 and 0.85. Even if there is a semantic relationship between different entities (such as "feature extractor" and "classifier"), their similarity is usually between 0.50 and 0.65. The threshold of 0.75 is exactly in the middle of the separation band between these two distributions, which can effectively distinguish between the two situations of "same reference but different expressions" and "different entities".

[0066] For example, a page has a default predicate slot with a predicate semantic category of "causal transformation", a subject candidate of "feature extractor", and an object candidate of "feature vector". The content of the speech block pointed to by a certain delay pointer is "this module performs spectral transformation on the input signal and outputs it". The core predicate verb "transform" is obtained by parsing. After querying the mapping table, its category is "causal transformation", and the first level of verification is passed. Entity extraction yields "the module", "input signal", and "spectrum". Core pointer resolution traces "the module" back to the most recently appearing "feature extractor" in the previous text. If the subject candidate match passes, the second-level test passes, and the slot is updated to be filled. If another speech block contains "the input of the module is the original signal", the core predicate verb is "is" (a copula), which does not belong to any valid semantic category. The first-level test fails, and the slot remains unfilled. If the core predicate category of a speech block matches, but the entity set contains neither "feature extractor" nor "feature vector" and there is no resolution referencing either of them, the second-level test fails, and the slot is not updated. After step S4 is completed, the system delivers the updated default slot filling status of each particle and the list of verified closed delay pointers to step S5.

[0067] While each default predicate slot is updated to "filled" status due to successful matching at both levels, the system performs a delay reward score generation operation based on the closure delay duration of that slot. The closure delay duration is defined as the time difference between the time the slot is switched out of the page and the current effective closure time, denoted as . The unit is seconds; define a first preset duration threshold. Seconds and the second preset duration threshold Seconds; when At the specified time, the first delayed reward score is generated +0.1 points; when At the specified time, a second delayed bonus score of +0.05 points is generated; when At a certain time, no delayed reward score is generated; the delayed reward scores generated by each valid closing event are accumulated into the global delayed reward pool, which is initialized to zero at the beginning of the speech and continues to accumulate during the speech, for use in step S5 when synthesizing the coherence score; the above thresholds are selected based on the following: 30 seconds corresponds to the average explanation time of a complete technical point in an academic speech, and 90 seconds corresponds to the reasonable tolerance limit for cross-page backtracking. Exceeding this time means that the speaker has entered the subsequent content that is far away. At this time, even if the supplementation of the previous omissions occurs, its structural remedial value has been significantly reduced, so no additional incentive is given.

[0068] The criteria for determining the first and second preset time thresholds are as follows: 30 seconds is based on the average explanation time of a single technical knowledge point in an academic presentation (based on sample corpus statistics, the average is 28 seconds), and 90 seconds corresponds to a reasonable review interval spanning three to five knowledge points. Supplements exceeding 90 seconds are significantly behind the original display time of the page in terms of timing. Even if the speaker completes the logical closure at this point, the remedial effect on the audience's cognitive coherence is greatly reduced, so no additional reward is given. The above thresholds can be adaptively adjusted based on the average speaking speed of the target language environment (Chinese / English) before deployment. The benchmark value is based on the typical speaking speed of Mandarin presentations (approximately 200 to 240 words per minute). The reward scores (0.1 points and 0.05 points) are set at one-third and one-sixth of the pre-deduction score (0.3 points), respectively, to ensure that the reward intensity does not exceed a reasonable proportion of the punishment intensity and to prevent excessive incentives for corrective supplements.

[0069] Step S5: At the end of the speech, iterate through the final state of the default predicate slots of each historical footprint particle, turn the pre-stored penalty of the unfilled slots into a positive value and reduce it by weight according to the particle survival time to obtain the total deduction. At the same time, generate a coherence bonus based on the bridging integration area data, and use the total deduction and coherence bonus to adjust the basic score and output the final score.

[0070] Step S5 is triggered when the system captures a speech end event; this event is generated by manual operation (such as clicking the end button) or timer resetting to zero. After receiving this signal, the system immediately stops receiving new voice streams and enters the clearing process; the clearing stage processes two types of data sources: one is the final filling status of the default predicate slots in each historical footprint particle, and the other is the accumulated bridging metadata in each bridging integration area.

[0071] Regarding the processing of the first type of data, in step S1, the system synchronously creates a corresponding number of pending penalty records in the global pending penalty temporary storage pool each time a page is switched; specifically, if a page generates... If there is a default predicate slot, then add a new one to the temporary pool. Each record is associated with the identifier of the page and the number of the corresponding slot, and a pre-deduction value is recorded. In this embodiment, the pre-deduction value is set to -0.3 points. In step S4, whenever the filling status of a default predicate slot is updated from "not filled" to "filled", the system temporarily stores the pending penalty record corresponding to the slot in the temporary storage pool and marks it as erased.

[0072] The pre-deduction of -0.3 points is based on the following: In a percentage-based scoring system, the contribution of the completeness of a single PPT slide to the total score is usually 100 / total number of pages. For a typical 30-page presentation, the weight of a single page is approximately 3.3 points. The number of missing predicate slots on a single page is usually 0 to 3, with an average of about 1.2 (based on sample corpus statistics). The pre-deduction of -0.3 points per slot means that the theoretical maximum deduction when a single page is completely missing is 0.9 points, which is about 27% of the weight of that page (3.3 points). This proportion was deemed reasonable in internal reviews—it reflects the negative impact of omissions without causing a drastic deviation in the total score due to the omission of a single page.

[0073] Read all records in the particle sedimentation log, multiply the pre-deduction value in each record by the fixed attenuation weight value in that record, and then add a penalty for repeated invalid attempts (+0.1 points) based on whether the number of backtracking hits is not less than 3. This penalty is then directly added to the total structural integrity deduction. After processing, the corresponding record in the sedimentation log is marked as "cleared", but the log content is retained for subsequent review.

[0074] The base value for the penalty for repeated invalid attempts is set at 0.1 points, which is one-third of the pre-deduction value of 0.3 points. This means that even if the speaker repeatedly tries to supplement but still cannot meet the matching conditions, in addition to bearing the original deduction for omissions, they must also bear the additional cost of "invalid attempts" equivalent to one-third of the slot deduction, in order to suppress the behavior of covering up omissions with a large number of low-quality repetitive voices; this base value is not adjusted separately after being multiplied by the decay weight.

[0075] At the start of the clearing process, all records in the temporary storage pool that are not marked as erased are iterated through. For each record, the system locates the historical footprint particle to which the record belongs and reads the current decay weight of that particle. This weight is updated in step S2 each time a new speech semantic block is processed, and the update method is as follows: ,in The time difference (in seconds) between the current moment and the moment the particle enters the pool is used; 0.005 is the preset linear decay coefficient; and 0.1 is the lower limit of the weight. The pre-deduction value is multiplied by the current decay weight of the particle to obtain the actual deduction. The system accumulates the actual deductions corresponding to all unerased records to obtain the total structural integrity deduction. .

[0076] For example, a page has a default predicate slot that is not filled before the end of the presentation. The corresponding historical footprint particle for this page has a lifespan of 120 seconds, and its current decay weight... The actual deduction for that slot is... point;

[0077] If a particle's backtracking hit count counter is not less than 3 times and there are still unclosed slots, the system, after multiplying the particle's unfilled slot pre-deduction value by the current decay weight to obtain the regular deduction, additionally calculates a penalty for repeated invalid attempts. This penalty is calculated by multiplying the preset base penalty value (0.1 points in this embodiment) by the particle's current decay weight. This additional penalty, added to the regular deduction, becomes the particle's final total deduction, which is then accumulated into the total structural integrity deduction. For example, the particle's current decay weight... If there are two unfilled slots and the number of hits is 4, then the standard deduction will be applied. Points, additional penalty is The total score for this particle is [score], divide; when At the same time, the total deduction is [number] points. Points are deducted proportionally from regular deductions and additional penalties, and the relative penalties for older pages are not increased due to the presence or absence of additional penalties.

[0078] If a particle has been marked as semantically consistent, no points will be deducted regardless of whether there are holes in its default slot.

[0079] Regarding the processing of the second type of data, the system summarizes the number of valid bridging metadata in all bridging integration areas throughout the entire presentation, denoted as [data missing]. Each bridging metadata entry includes a bridging strength value. The system calculates the arithmetic mean of the intensity value. ,when hour, A direct assignment of 0 results in a bridging synthesis bonus of 0; a preset scaling factor is also applied. Set to 0.08, bridging synthesis score added. In addition to the bridging bonus, the reward points accumulated in the global delayed reward pool in step S4 are also included in the continuity bonus; continuity bonus ,in This is the cumulative value of the global delayed reward pool.

[0080] Preset scaling factor A value of 0.08 keeps the bridging bonus within a reasonable range of the total score; in an academic presentation with 60 audio sentences, the number of effective bridging metadata... The average bridging strength is typically between 3 and 15. Typically between 0.55 and 0.75, the bridging composite score is approximately... Points; this bonus range is superimposed on the cumulative value of the delayed reward pool (usually 0 to 0.5 points), and the total coherence bonus accounts for about 1% to 2% of the total percentage score variation range, which can significantly distinguish the coherence of speeches without interfering with the dominant position of the base score; this coefficient can be adjusted before deployment according to the specific dimensions of the base score in the scoring system.

[0081] The final score synthesis method is as follows: the system obtains the basic content score output by the basic scoring module. (This score is based on the existing page-by-page content scoring results.) The total deduction for structural integrity and the bonus for coherence are added to the base score, i.e. ,in Negative or zero The result is positive or zero; if the calculation result is lower than zero, it is truncated to zero; if it exceeds the full score, it is truncated to the full score. Step S5 also writes all the settlement details into the log, including the final filling status of each particle's default slot, the erase or positive mark of each pending penalty record, and the source and strength of each bridging metadata, for subsequent review and traceability.

[0082] Example 2: Figure 2 As shown, an automatic academic presentation scoring system based on multimodal fusion includes a processor, a memory, and a computer program stored in the memory and executable on the processor. The system also includes an audio input interface (for receiving speaker speech stream data collected by a microphone array) and a data input interface (for receiving page content data and page switching timing data of a presentation document) connected to the processor. When the processor executes the computer program, it implements all the steps of the automatic academic presentation scoring method based on multimodal fusion in Embodiment 1.

[0083] Specifically, the system is deployed on the scoring server. It receives audio signals from the microphone array at the presentation venue in real time via an audio input interface, and simultaneously receives page content data and page-turning event logs from the presentation terminal (such as a computer playing a PPT) via a data input interface. After receiving the above data, the processor performs the following operations in sequence: in response to a page switching event, it performs semantic parsing on the page content of the current page and generates a default vector of logical predicates, which is then encapsulated as historical footprint particles and stored in the cross-page semantic pool; it performs semantic segmentation on the switched audio stream to obtain audio semantic blocks and determines their classification type; based on the classification type, it routes the audio semantic blocks to the current page recording area, the delay pointer list of historical footprint particles, or the bridging integration area; it performs default predicate slot matching verification on the audio semantic blocks pointed to by the delay pointers; and at the end of the presentation, it performs a settlement, converting the pre-stored penalties for unfilled slots into positive ones and weighting them to obtain the total deduction, combining the data from the bridging integration area to generate a coherence bonus, adjusting the basic score, and outputting the final presentation score.

[0084] The system can be deployed as a local server or as a cloud server, receiving audio streams and presentation document data from remote presentation terminals via the network. The specific implementation methods, threshold settings, and data processing rules of each step in the system are completely consistent with the description in Example 1, and will not be repeated here.

[0085] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

[0086] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0087] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. An automatic scoring method for academic presentations based on multimodal fusion, characterized in that, include: Acquire the speaker's audio stream data, the presentation document's page content data, and page transition sequence data; In response to page switching events, semantic parsing is performed on the content of the current page to extract entities and explicit predicates. The default predicate slots are marked by combining the theoretical predicate templates corresponding to the chapter type, generating a default vector of logical predicates and encapsulating it as historical footprint particles to be stored in the cross-page semantic pool. The switched speech stream is semantically segmented to obtain speech semantic blocks. The similarity of the speech semantic block is compared with the semantic vector of the current page and the page vector of each historical footprint particle in the pool. The type of the speech semantic block is determined based on the comparison results. Based on the attribution type, the speech semantic block is routed to the current page record area, the delayed pointer list of historical footprint particles, or the bridging integral area; For the speech semantic block pointed to by the delayed pointer, extract its core predicate and entity reference, and match them with the predicate category and subject-object reference of the corresponding default predicate slot respectively. If the match is successful, update the slot to be filled. At the end of the presentation, the final state of the default predicate slots of each historical footprint particle is traversed. The pre-stored penalty for unfilled slots is converted to positive and weighted according to the particle survival time to obtain the total deduction. At the same time, a coherence bonus is generated based on the data of the bridging integration area. The base score is adjusted using the total deduction and the coherence bonus, and the final score is output.

2. The automatic scoring method for academic presentations based on multimodal fusion according to claim 1, characterized in that, The generation of the default vector for logical predicates specifically includes: loading the corresponding core logical relationship expectation template according to the chapter type of the current page, wherein the template contains at least one predicate semantic category; extracting all explicit predicate verbs in the current page and mapping them to a predefined predicate semantic category mapping table to obtain an explicit category set; traversing each predicate semantic category in the template, and marking it as the default predicate category if it does not appear in the explicit category set; selecting the entity with the highest co-occurrence frequency with the core topic word from the entities in the current page as a logical subject candidate, and if there is an entity with the second highest co-occurrence frequency, it as a logical object candidate; and combining the default predicate category, subject candidate, object candidate, and the initially unfilled filling status identifier into the default predicate slot.

3. The automatic scoring method for academic presentations based on multimodal fusion according to claim 1, characterized in that, The determination of the attribution type based on the comparison results specifically includes: calculating the similarity value between the speech semantic block and each candidate page, wherein the candidate pages include the current page and the historical pages corresponding to each historical footprint particle in the cross-page semantic pool, to obtain a similarity set; extracting the maximum similarity value and the second maximum similarity value in the set; when the difference between the maximum similarity value and the second maximum similarity value is greater than or equal to a first preset difference threshold, it is determined to be a strong attribution and the attribution object is the candidate page corresponding to the maximum similarity value; when the difference is less than a second preset difference threshold, it is determined to be a bridging state, wherein the second preset difference threshold is less than the first preset difference threshold; when the difference is between the second threshold and the first threshold, if the absolute value of the maximum similarity value is greater than or equal to a preset absolute value threshold, it is determined to be a strong attribution, otherwise it is determined to be a bridging state.

4. The automatic scoring method for academic presentations based on multimodal fusion according to claim 1, characterized in that, The predicate matching specifically includes: mapping the core predicate verb of the speech semantic block to a predefined predicate semantic category mapping table to obtain the semantic category label of the core predicate verb; comparing the semantic category label with the predicate semantic category recorded in the default predicate slot; if they are the same, the category matching is determined to be successful; if no core predicate verb is extracted from the speech semantic block, the category matching is directly determined to be unsuccessful.

5. The automatic scoring method for academic presentations based on multimodal fusion according to claim 1, characterized in that, Entity referential matching specifically includes: extracting the set of entity nouns from the speech semantic block, and performing coreference resolution on the referential phrases therein to backtrack to the corresponding explicit entity nouns; comparing the semantic similarity of the backtracked explicit entity nouns and the non-referential entity nouns in the set with the subject references and object references recorded in the default predicate slot; if the subject reference has a similarity greater than a preset similarity threshold, the subject match is determined to be successful; if the object reference exists and has a similarity greater than the preset similarity threshold, the object match is determined to be successful; for binary slots that record both subject and object, referential matching is determined to be successful when at least one of the subject match and object match is successful; for unary slots that only record the subject, referential matching is determined to be successful when the subject match is successful.

6. The automatic scoring method for academic presentations based on multimodal fusion according to claim 1, characterized in that, Converting pre-stored penalties into actual structural deductions specifically includes: when switching pages, creating a corresponding number of pending penalty records in the global temporary storage pool based on the number of default predicate slots for that page, with each record associated with the page's identifier and corresponding slot number, and recording the pre-deduction value; when the filling status of any default predicate slot is updated to filled, searching for the corresponding record in the temporary storage pool and marking it as erased; after the presentation ends, traversing all records in the temporary storage pool that have not been marked as erased, multiplying their pre-deduction values ​​by the current decay weight of the corresponding historical footprint particle, and summing them to obtain the total structural integrity deduction.

7. The automatic scoring method for academic presentations based on multimodal fusion according to claim 1, characterized in that, The method also includes the step of generating a delay reward score: recording the closure delay duration of each default predicate slot from the moment of page switching to the moment the fill status is updated to filled; generating a first delay reward score when the closure delay duration is less than or equal to a first preset duration threshold; generating a second delay reward score when the closure delay duration is greater than the first preset duration threshold and less than or equal to a second preset duration threshold, wherein the second reward score is less than the first reward score; not generating a delay reward score when the closure delay duration is greater than the second preset duration threshold; and accumulating the delay reward scores corresponding to all valid closures into the continuity score.

8. The automatic scoring method for academic presentations based on multimodal fusion according to claim 1, characterized in that, The generation of coherence bonus specifically includes: counting the number of bridging metadata in the bridging score area, wherein the bridging metadata is a storage record of a speech semantic block whose attribution type is determined to be in a bridging state and whose bridging strength value is greater than or equal to a preset strength threshold, and the bridging strength value is the average similarity between the speech semantic block and the two candidate pages that generate the highest and second highest similarity; calculating the average bridging strength value of each of the bridging metadata; multiplying the number of bridging metadata by the average value and then by a preset scaling factor to obtain the bridging comprehensive bonus; and adding the bridging comprehensive bonus to the delay reward score to obtain the coherence bonus.

9. The automatic scoring method for academic presentations based on multimodal fusion according to claim 1, characterized in that, The decay weighting based on particle survival time specifically includes: recording the entry timestamp of each historical footprint particle; whenever a new speech semantic block enters the processing flow, obtaining the time difference between the current global time and the entry timestamp; updating the current decay weight in a linear decay manner according to the time difference, wherein the current decay weight is equal to one minus the product of the preset linear decay coefficient and the time difference, and is not lower than the preset weight lower limit.

10. An automatic academic presentation scoring system based on multimodal fusion, comprising a processor, characterized in that, The processor is used to execute the automatic scoring method for academic presentations based on multimodal fusion as described in any one of claims 1-9.

Images