Lecture note generation model training method, and lecture note generation method and device of lecture question video

Abstract

The embodiment of the invention provides a lecture manuscript generation model training method and a lecture manuscript generation method and device of a question-speaking video, and the method comprises the steps: obtaining a sample question-speaking video, and determining a sample lecture manuscript text based on the sample question-speaking video; sample question information corresponding to the sample question-speaking video is determined, a corresponding sample question-solving thought is obtained based on the sample question information, and the sample question-solving thought is a question-solving thought matched with the sample question-speaking video in at least one candidate question-solving thought of the sample question information; constructing a training sample based on the sample question information, the sample question solving thought and the sample lecture text; and training the initial lecture generation model by using the training sample until a target lecture generation model meeting a training stop condition is obtained. And training the initial lecture generation model to learn based on the input sample question information and the sample problem solving thought, and generating the specific sample lecture text under the sample problem solving thought, so that one question with multiple solutions is realized, and the lecture generation flexibility is higher.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of deep learning technology, and in particular to a method for training a lecture script generation model, a method and apparatus for generating lecture scripts for lecture videos. Background Technology

[0002] With the booming development of online education and remote work, lecture videos have become an important teaching resource. The technology for automatically generating lecture videos has been widely used in education, training, content creation, and enterprise internal knowledge management. The demand for automatically generating high-quality lecture videos is constantly increasing, which can significantly improve content creation efficiency, reduce manual writing time, and ensure the consistency and professionalism of the content.

[0003] In existing technologies, a semi-automatic generation method based on template matching is often used. First, a rich library of lecture script templates is built according to dimensions such as subject, question type, and explanation style. Each template contains a fixed structure and content prompts. Then, key information such as question type and related knowledge points is extracted from the question to be explained. Based on the question analysis results, the most suitable template is matched from the template library, and the lecture script for the lecture video is automatically generated based on the matched template.

[0004] However, the above methods rely heavily on lecture templates, and the creation of these templates mainly depends on professional teachers, which results in high costs and low efficiency. In addition, lecture video scripts generated based on templates can lead to stiff and unnatural presentations. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a method for training a lecture script generation model. One or more embodiments of this specification also relate to a method for generating lecture scripts for lecture videos, a lecture script generation model training device, a lecture script generation device for lecture videos, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the embodiments of this specification, a method for training a lecture note generation model is provided, comprising: Obtain sample lecture videos and determine sample lecture text based on the sample lecture videos; Determine the sample question information corresponding to the sample lecture video, and obtain the corresponding sample problem-solving approach based on the sample question information, wherein the sample problem-solving approach is the problem-solving approach that matches the sample lecture video from at least one candidate problem-solving approach in the sample question information; Training samples are constructed based on the sample question information, the sample problem-solving ideas, and the sample lecture text. The initial lecture generation model is trained using the training samples until a target lecture generation model that meets the training stopping condition is obtained.

[0007] According to a second aspect of the embodiments of this specification, a method for generating lecture scripts for lecture videos is provided, comprising: Obtain information about the target question and the corresponding target problem-solving approach; Based on the target topic information and the target problem-solving approach, the corresponding target lecture text is generated using a target lecture generation model, wherein the target lecture generation model is trained using the lecture generation model training method described above.

[0008] According to a third aspect of the embodiments of this specification, a lecture note generation model training apparatus is provided, comprising: The first acquisition module is configured to acquire sample lecture videos and determine sample lecture text based on the sample lecture videos; The second acquisition module is configured to determine the sample question information corresponding to the sample lecture video, and acquire the corresponding sample problem-solving approach based on the sample question information, wherein the sample problem-solving approach is the problem-solving approach that matches the sample lecture video among at least one candidate problem-solving approach in the sample question information; The training module is configured to construct training samples based on the sample question information, the sample problem-solving ideas, and the sample lecture text, and use the training samples to train the initial lecture generation model until a target lecture generation model that meets the training stopping condition is obtained.

[0009] According to a fourth aspect of the embodiments of this specification, a lecture script generation apparatus for lecture videos is provided, comprising: The third acquisition module is configured to acquire target question information and corresponding target problem-solving ideas; The generation module is configured to generate the corresponding target lecture text based on the target question information and the target problem-solving approach, using a target lecture generation model, wherein the target lecture generation model is trained using the lecture generation model training method described above.

[0010] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the above-described lecture script generation model training method or lecture script generation method for lecture videos.

[0011] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-described lecture script generation model training method or lecture script generation method for lecture videos.

[0012] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described lecture script generation model training method or lecture script generation method for lecture videos.

[0013] One embodiment of this specification provides a method for training a lecture script generation model, which involves: acquiring sample lecture videos and determining sample lecture script text based on the sample lecture videos; determining sample question information corresponding to the sample lecture videos; obtaining corresponding sample problem-solving approaches based on the sample question information, wherein the sample problem-solving approach is a problem-solving approach that matches the sample lecture video from at least one candidate problem-solving approach in the sample question information; constructing training samples based on the sample question information, the sample problem-solving approaches, and the sample lecture script text; and training an initial lecture script generation model using the training samples until a target lecture script generation model that meets the training stopping condition is obtained.

[0014] One embodiment of this specification implements an initial lecture script generation model trained by combining sample question information, sample solution approaches, and sample lecture script text. This allows the initial lecture script generation model to learn to generate specific sample lecture script texts under the input sample question information and sample solution approaches. The trained target lecture script generation model has the ability to automatically generate corresponding lecture scripts for different solution approaches. This improves the efficiency of lecture script generation for video lectures, reduces manual costs, and allows the trained target lecture script generation model to generate different lecture scripts for different solution approaches to the same question, matching them with the given solution approach. This enables multiple solutions to a single question, increases the flexibility of lecture script generation, avoids rigid explanations, and adapts to different solution approaches, preventing explanations beyond the scope of the curriculum. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a lecture generation model training method provided in one embodiment of this specification; Figure 2 This is a flowchart illustrating a method for generating lecture transcripts for a lecture video, as provided in one embodiment of this specification. Figure 3 This is a flowchart illustrating the process of generating a lecture script for a lecture video according to one embodiment of this specification. Figure 4 This is a schematic diagram of the structure of a lecture generation model training device provided in one embodiment of this specification; Figure 5 This is a schematic diagram of the structure of a lecture script generation device for a lecture video provided in one embodiment of this specification; Figure 6This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0016] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0017] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0018] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0019] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0020] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained on a large-scale unlabeled corpus to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and Multimodal Pre-trained Models (MLMs). modalpre (training model), etc.

[0021] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as Text Generation, Text-based Sentiment Classification, Text Summarization, and Machine Translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0022] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0023] Large Language Models (LLMs) are deep learning models trained on massive amounts of text data. They are capable of understanding, generating, and reasoning about human language, and are specifically designed for processing and generating natural language text. LLMs learn the statistical patterns and semantic relationships of language by analyzing trillions of words, with the number of parameters reaching tens or even trillions. This massive number of parameters gives the model strong generalization ability and the ability to handle complex tasks. These models typically have a large number of parameters and complex architectures, enabling them to understand and generate high-quality natural language, thus performing exceptionally well in various tasks.

[0024] Supervised Fine-Tuning (SFT) is a process in machine learning that aims to further optimize a pre-trained model using labeled data specific to a particular task or domain. It involves training the model on a dataset containing correct answers or labels for a specific task or domain, building upon a model already trained on a large amount of unlabeled data with broad language understanding capabilities, so that the model can predict the correct output based on the input data.

[0025] Explanation videos: These are teaching resources that use video as a medium to explain and demonstrate specific problems or knowledge points step by step. They usually combine clear audio explanations, synchronized handwritten notes or animation demonstrations, and key point annotations. The aim is to present the problem-solving ideas, steps, and principles to the viewer in an intuitive and dynamic way. The value of these videos lies in visualizing the thinking process, simulating the feeling of one-on-one tutoring, and helping learners overcome time and space limitations to understand the logic and methods behind the problems. They are suitable for various self-study scenarios such as previewing, reviewing, and answering questions, and are a widely used micro-lesson format in modern digital education.

[0026] It should be noted that with the booming development of online education, the demand for lecture videos, as an important teaching resource, is increasing daily. Traditional manual production of lecture videos suffers from low efficiency, high cost, and long production time, making it difficult to meet the demands for rapid updates and mass production. Automated lecture video generation technology has wide applications in education, training, content creation, and enterprise internal knowledge management. With the popularization of online education and remote work, the demand for automatically generating high-quality lecture notes and whiteboard writing is constantly increasing. This technology can significantly improve content creation efficiency, reduce manual writing time, and ensure the consistency and professionalism of the content.

[0027] One approach involves generating detailed explanations based on the question analysis results, combined with pre-set explanation templates and knowledge graphs. However, these templates need to be customized for different subjects and question types, leading to issues such as rigid explanations, low efficiency, high costs, and long processing times. In other words, a semi-automated generation method based on template matching is used. This approach first constructs a rich library of lecture templates according to dimensions such as subject, question type, and explanation style. Each template contains a fixed structure and content prompts, such as a general template of "Introduction-Problem Statement-Solution-Summary," as well as professional templates for specific subjects. Then, key information about the question to be explained is extracted, such as the question type and related knowledge points. Based on the question analysis results, the most suitable template is matched from the template library, and the lecture video script and whiteboard notes are automatically generated based on the matched template. This semi-automated generation method based on template matching relies heavily on lecture templates, and the creation of these templates mainly depends on professional teachers, thus resulting in high costs and low efficiency. Furthermore, the lecture video scripts and whiteboard notes generated based on templates can lead to rigid explanations.

[0028] Another approach utilizes a large language model to directly generate lecture video scripts and whiteboard notes based on given question information. This method, based on a large language model, achieves direct generation from question to human lecture video scripts and whiteboard notes. By collecting human lecture videos and using supervised fine-tuning methods based on the large language model, it eliminates reliance on templates in the intermediate process, achieving fully automated script and whiteboard note generation. However, a question may have multiple solutions, or students from different grade levels and backgrounds may require explanations tailored to their specific contexts. This method cannot avoid such problems, leading to explanations that go beyond the syllabus.

[0029] Therefore, one embodiment of this specification provides a method for training a lecture script generation model and a method for generating lecture scripts for video lectures. By combining sample question information, sample problem-solving approaches, and sample lecture script text to train an initial lecture script generation model, it achieves the ability to generate lecture scripts and blackboard notes for specific problem-solving approaches. For different problem-solving approaches to the same question, the trained target lecture script generation model can provide different lecture scripts and blackboard notes, matching the given problem-solving approach. Thus, a solution is provided for generating lecture scripts and blackboard notes based on a large model with specific problem-solving approaches. On the one hand, this avoids dependence on explanation templates and knowledge graphs; only a few high-quality human-generated lecture scripts and blackboard notes need to be collected, and high-quality generation of lecture scripts and blackboard notes can be achieved through supervised fine-tuning. On the other hand, for different solutions to questions or students at different grade levels, only the corresponding problem-solving approach needs to be provided to generate corresponding lecture scripts and blackboard notes, solving the problem of explaining beyond the syllabus.

[0030] This specification provides a method for training a lecture script generation model. It also relates to a method for generating lecture scripts for lecture videos, a lecture script generation model training device, a lecture script generation device for lecture videos, a computing device, a computer-readable storage medium, and a computer program product, each of which will be described in detail in the following embodiments.

[0031] See Figure 1 , Figure 1 A flowchart of a lecture generation model training method according to an embodiment of this specification is shown, specifically including the following steps 102-106.

[0032] Step 102: Obtain sample lecture videos and determine sample lecture text based on the sample lecture videos.

[0033] The lecture script generation model training method provided in this embodiment can be applied to the training of lecture script generation models for any subject, such as Chinese, mathematics, or English. This embodiment does not impose any limitations. The trained target lecture script generation model can generate lecture script text with blackboard markings based on the question information and problem-solving ideas, so that the corresponding lecture video can be generated based on the lecture script text with blackboard markings.

[0034] A problem-solving video is a video that focuses on explaining a problem. These videos can cover various subjects, such as mathematics, physics, and chemistry, providing step-by-step explanations and summaries of key concepts. They can also cover English reading comprehension, essay writing skills, grammar analysis, and even computer programming algorithms. This example does not impose any limitations. The video content can be recorded from a real classroom setting, including the teacher and the blackboard, or it can be recorded from an online screen, containing only the blackboard and related materials.

[0035] Sample problem-solving videos are representative, high-quality videos used for model training. Sample lecture transcripts refer to the accurately extracted and structured text records from the sample problem-solving videos, corresponding to the video content. These transcripts clearly and completely reflect the problem-solving approaches, step-by-step instructions, key knowledge points, and oral explanation logic presented in the video. The sample problem-solving videos are the source material (multimedia format), while the sample lecture transcripts are structured data (text format) derived from the videos, carrying their core linguistic information. The process of obtaining the sample lecture transcripts involves transforming the streaming, conversational audio information from the sample problem-solving videos into static, standardized textual knowledge.

[0036] In practice, the first step is to acquire sample lecture videos. There are many ways to acquire these videos, such as downloading them from online resources, obtaining them from educational platforms, or recording them yourself. This embodiment does not impose any limitations on this method. After acquiring the sample lecture videos, the audio can be extracted from them, and then converted into text using a speech recognition tool to obtain the sample lecture transcript.

[0037] In one optional implementation of this embodiment, obtaining sample lecture videos includes: Obtain multiple candidate lecture videos and determine the lecture score, blackboard writing score, and audio score for each candidate lecture video. Sample lecture videos were selected from multiple candidate lecture videos based on explanation scores, blackboard writing scores, and audio scores.

[0038] Specifically, candidate lecture videos refer to the initial collection of original lecture videos that are to be evaluated and screened. These videos may come from various sources, such as teacher recordings, platform uploads, and automatic generation, and their quality varies.

[0039] The explanation dimension refers to the evaluation of the video teaching content and expression in candidate lecture videos. It focuses on the logic, accuracy, clarity, speaking speed, rhythm, and ease of understanding of the explanation. The explanation score is a quantitative indicator of the explanation dimension, used to measure the overall quality score of candidate lecture videos in this dimension.

[0040] The blackboard writing dimension is used to evaluate the visual presentation quality of candidate lecture videos. It focuses on the clarity of the blackboard writing, the rationality of the layout, the neatness of the handwriting, the synchronicity of the text and graphics, the accuracy of the animation, and the effectiveness of visual guidance. The blackboard writing score is a quantitative indicator of the blackboard writing dimension, used to measure the overall quality score of candidate lecture videos in this dimension.

[0041] The audio dimension evaluates the sound quality of candidate presentation videos, focusing on speech clarity, background noise levels, volume stability, sound quality, and the presence of harsh noises or echoes. The audio score is a quantitative metric for the audio dimension, used to measure the overall quality score of the candidate presentation videos in this dimension.

[0042] Sample lecture videos refer to high-quality lecture videos that have been selected after multi-dimensional comprehensive evaluation and screening, and that meet or exceed the predetermined quality standards in the three dimensions of explanation, blackboard writing and audio. They will be used for subsequent model training.

[0043] In one optional implementation, multiple candidate lecture videos are retrieved in batches from a specified resource library, storage platform, or browser. These candidate videos correspond to different topics, and each topic can have multiple candidate videos with different problem-solving approaches or explanation styles. Subsequently, the retrieved candidate lecture videos undergo unified preprocessing, including format standardization, resolution adjustment, and timestamp alignment.

[0044] Then, each candidate lecture video can be evaluated in parallel across three dimensions. Specifically, a speech recognition service is invoked to convert the audio from the candidate lecture video into sample lecture text. Natural language processing techniques are then used to analyze this sample lecture text. For example, a semantic analysis model is used to evaluate the coherence of the explanation logic and the correctness of the problem-solving steps; keyword and knowledge point matching is used to check the accuracy of the content; and the clarity and approachability of the explanation are evaluated by analyzing the speech rate and pause frequency of the speech transcription and combining it with sentiment analysis. The results of the above analysis can be comprehensively calculated using a trained regression model or weighted rules to output a lecture score.

[0045] Computer vision technology is used to process candidate lecture videos, extract text and mathematical symbols from the blackboard, and evaluate visual readability through image sharpness detection and contrast analysis; object detection is used to evaluate the rationality of the graphic layout; by analyzing the drawing trajectory of handwriting and the synchronization of speech, the coordination between the explanation and the blackboard is evaluated. These visual features are input into another evaluation model to calculate the blackboard score.

[0046] The system directly analyzes the audio stream of candidate lecture videos, assesses background noise levels by calculating the signal-to-noise ratio, evaluates volume stability through volume normalization and dynamic range detection, distinguishes human voices from silent / noise segments using speech activity detection, and assesses the purity of human voices. It may also detect the presence of echoes or pops. These audio metrics are synthesized into an audio score using a predefined formula or lightweight model.

[0047] In another optional implementation, for each candidate lecture video, manually annotated scores can be obtained for explanation, blackboard writing, and audio.

[0048] In practice, after obtaining the three-dimensional scores for each candidate lecture video, different weights can be assigned to the lecturing, whiteboard writing, and audio scores based on business needs (e.g., prioritizing content or presentation). A weighted composite score is then calculated for each candidate lecture video. A composite score threshold can be pre-set, and individual dimension score thresholds may also be set to avoid highlighting weaknesses in any particular dimension (e.g., extremely poor audio quality). Finally, the candidate lecture videos are ranked, and the videos that simultaneously meet the composite score threshold and the individual dimension score thresholds and rank highly are selected as the final sample lecture videos. For example, using a 5-point scale, videos with scores exceeding 4 points in all three dimensions are selected as high-quality sample lecture videos.

[0049] In this embodiment of the specification, high-quality videos with excellent performance are selected as sample lecture videos based on three dimensions: explanation dimension, blackboard writing dimension, and audio dimension. Through multimodal evaluation and screening, sample lecture videos can be efficiently, objectively, and quantifiably selected from a large number of heterogeneous candidate lecture videos, improving the efficiency and consistency of sample lecture video selection and providing a high-quality and reliable data foundation for subsequent model training.

[0050] Of course, in actual implementation, in addition to the above-mentioned selection of high-quality sample lecture videos based on the explanation score in the explanation dimension, the blackboard score in the blackboard writing dimension, and the audio score in the audio dimension, other evaluation dimensions can also be used or combined to select sample lecture videos, such as the image smoothness and image clarity in the visual dimension. This specification does not limit this.

[0051] In one optional implementation of this embodiment, the sample lecture text is marked with sample whiteboard markings; determining the sample lecture text based on the sample lecture video includes: Extract the audio of the lecture from the sample lecture videos, and identify the original lecture text from the audio. The original speech manuscript is corrected for errors and / or its fluency is adjusted to obtain candidate speech manuscript texts. Based on the blackboard writing effect in the sample lecture video, sample blackboard writing marks are added to the blackboard writing content in the candidate lecture text to obtain the sample lecture text.

[0052] Specifically, sample blackboard markers are structured tags or metadata inserted into sample lecture text to indicate the location and type of blackboard content. These markers are used to label the corresponding blackboard content within the sample lecture text. For example, the tag corresponding to "exam point" in the blackboard is "". <keypoints> or< / keypoints> The label corresponding to "Explanation" on the blackboard is "". <analysis> or< / analysis>Or it may be marked with text tags such as "Formula derivation should appear here", "The corresponding graphic explanation animation begins here", or "This is the key content to be marked", which are used to synchronously restore the visual whiteboard content in the video in subsequent applications. The sample lecture text is the text obtained by marking the candidate lecture text, which is used as label data for subsequent model training, so that the model has the ability to generate lecture text with whiteboard markings.

[0053] The audio explanation refers to the audio track containing the speaker's voice, which is separated or extracted from the sample lecture video. The original lecture text refers to the preliminary text result generated directly after the audio explanation is recognized by automatic speech recognition technology, without correction or processing. It may usually contain problems such as recognition errors, colloquial vocabulary, repetitions, and inaccurate punctuation.

[0054] Text correction refers to identifying and correcting typos in the original speech manuscript. Text fluency adjustment refers to optimizing the text to better conform to language habits, improve readability, coherence, and naturalness of expression, including grammatical correction and logical connection optimization.

[0055] The blackboard writing effect refers to the visual presentation information of the blackboard in the sample lecture video, including dynamic and static visual elements such as text writing, formula derivation, graphic drawing, animation demonstration, highlighting of key points, and color changes, as well as their temporal information. The blackboard writing content refers to the text portion of the candidate lecture script that corresponds to blackboard writing actions or visual presentations during the sample lecture video.

[0056] In practice, after obtaining sample lecture videos, the audio of the lectures can be extracted and the audio can be processed by text recognition to obtain the original lecture text. Then, the original lecture text can be corrected or its fluency adjusted, or both can be adjusted simultaneously, to obtain candidate lecture texts. Text correction involves correcting typos in the original lecture text, while fluency adjustment can include grammatical correction, optimization of logical connections, and naturalization of tone, etc. Specifically, text correction and fluency adjustment can be performed using a text processing model, which can be a large language model.

[0057] In addition, after obtaining the candidate lecture text, sample blackboard markers can be added to the blackboard content in the candidate lecture text based on the blackboard effect in the sample lecture video to obtain the sample lecture text, so that the model can be trained based on the sample lecture text carrying the sample blackboard markers.

[0058] In the embodiments of this specification, optimizing the original lecture text to obtain candidate lecture texts can generate high-quality sample data, reduce the training cost of the model, enable the model to learn feature patterns faster and more accurately, and result in a model with better performance. The lectures generated using this model are of higher quality. Furthermore, adding sample whiteboard markers to the sample lecture texts provides basic data for subsequent model training, enabling the training model to generate target lecture texts with whiteboard markers.

[0059] Of course, in actual implementation, in addition to the above-mentioned method of extracting the audio of the explanation to obtain the original lecture text, correcting errors and / or adjusting the text fluency to obtain candidate lecture text, and then marking the sample whiteboard, the sample lecture video can also be directly input into the trained visual recognition model for content analysis, and directly output the sample lecture text with the sample whiteboard marking. This specification does not limit this.

[0060] In one optional implementation of this embodiment, based on the blackboard writing effect in the sample lecture video, sample blackboard writing marks are added to the blackboard writing content in the candidate lecture text to obtain the sample lecture text, including: Extract the original whiteboard image from the sample lecture video, and perform denoising and / or sharpening processing on the original whiteboard image to obtain the sample whiteboard image; Based on the blackboard structure of the sample blackboard image, sample blackboard markers are added to the blackboard content in the candidate lecture text to obtain the sample lecture text.

[0061] Specifically, the original whiteboard image is a direct crop from the sample lecture video, without any processing. Denoising is achieved through algorithms or techniques to remove noise and blur from the image, improving its clarity. Sharpening is an image processing technique that enhances edges and details, making the image clearer. The sample whiteboard image refers to a clear, high-quality whiteboard image obtained after a series of enhancement processes, including denoising and / or sharpening, from the original whiteboard image.

[0062] The blackboard structure refers to the layout and logical relationships of the blackboard content analyzed and identified from sample blackboard images. The blackboard content refers to the text portion of the candidate lecture script corresponding to the time of the explanation and the time when the blackboard writing action occurs in the video. For example, when explaining "Step 1, let the unknown be X", the text "Let the unknown be x" appears handwritten in the video at the same time, then this text is the "blackboard content".

[0063] In practice, after acquiring sample lecture videos, images from these videos are extracted as the original whiteboard images. These original images undergo denoising or sharpening, or both, to obtain processed sample whiteboard images. Then, for each sample whiteboard image, in-depth visual content understanding is performed to identify its whiteboard structure and add sample whiteboard markers. Specifically, text recognition technology is used to extract the whiteboard content from each sample whiteboard image, and visual algorithms are used to analyze its layout and logical relationships to parse the whiteboard structure. Sample whiteboard markers are then inserted into the corresponding positions in the sample lecture text, thus encoding the visual whiteboard effect into the sample lecture text, facilitating model learning to generate lecture text with whiteboard markers.

[0064] In this embodiment of the specification, the structure of the sample whiteboard image is obtained by performing visual structured analysis on the sample whiteboard image. Based on the whiteboard structure, sample whiteboard markers are added to the positions of the whiteboard content in the candidate lecture text to obtain the sample lecture text. The dynamic whiteboard in the video is transformed into static and clear structured markers in the sample lecture text, providing a data foundation for subsequent model training and improving the model training effect and efficiency.

[0065] Step 104: Determine the sample question information corresponding to the sample lecture video, and obtain the corresponding sample problem-solving approach based on the sample question information. The sample problem-solving approach is the problem-solving approach that matches the sample lecture video among at least one candidate problem-solving approach in the sample question information.

[0066] Specifically, sample question information refers to the specific question information extracted from the sample question-solving video, such as the text description (question stem), subject, knowledge point tags, difficulty level, etc.

[0067] Problem-solving strategies refer to the thought processes that guide the problem-solving process when solving problems (especially structured problems such as those in mathematics, logic, and programming). A problem-solving strategy is a framework of solutions or methods, not involving specific calculation or operational details, but rather answering the question at a high level: "What methods should be used, what principles should be followed, and what stages should be involved?" A problem-solving strategy is the mental bridge connecting the problem conditions and the solution objective, determining the direction of subsequent specific reasoning steps.

[0068] Candidate solutions refer to a collection of multiple, reasonable solutions or thought processes that exist in theory or practice for the same sample problem. For example, a geometry proof problem may have multiple candidate solutions such as the "synthetic method," the "analytic method," and the "vector method."

[0069] Sample problem-solving approaches refer to the problem-solving approaches that are verified to be consistent with the actual explanation process in the sample problem-solving video, from among multiple possible problem-solving methods (candidate problem-solving approaches) associated with the sample problem information.

[0070] In practice, the specific problem information of the sample problem explained in the sample problem-solving video can be identified. Based on the sample problem information, at least one candidate solution approach can be determined. Then, the sample solution approach that matches the content explained in the sample problem-solving video can be selected from the at least one candidate solution approach.

[0071] In one optional implementation of this embodiment, obtaining the corresponding sample problem-solving approach based on sample question information includes: Generate at least one candidate solution approach corresponding to the sample question information using a generative model; and / or, search for at least one corresponding candidate solution approach in the question-solving database based on the sample question information; Perform content consistency verification on at least one candidate problem-solving approach and sample problem-solving videos; At least one candidate problem-solving approach that is consistent with the content of the sample lecture video is identified as the sample problem-solving approach.

[0072] Specifically, a generative model refers to an artificial intelligence model that can automatically generate corresponding content based on input information. For example, the generative model can be a large language model (LLM) or a sequence generation model, which can infer problem-solving strategies, i.e., candidate problem-solving ideas, based on the question information.

[0073] A problem-solving database is a structured storage system that contains a large number of problems and their associated, verified solutions. This database may be built in the form of a relational database, knowledge graph, or vector database, and supports efficient retrieval based on problem features (such as text and knowledge point tags).

[0074] Content consistency verification refers to the process of evaluating whether the candidate problem-solving approach is consistent with the actual content of the sample problem-solving video through calculation and logical comparison. Its output is usually a quantitative consistency score or a binary judgment (consistent / inconsistent).

[0075] In practice, based on the sample question information, at least one candidate solution approach can be obtained by automatically reasoning through the generation model or by retrieving from the solution database. Subsequently, the candidate solution approach is subjected to multimodal content alignment analysis with the sample lecture video. By comparing the semantic similarity and logical consistency between the actual explanation logic and steps parsed from the video and the description of the candidate solution approach, the content consistency verification is completed. The candidate solution approach generated by the model that passes the content consistency verification is determined as the sample solution approach of the sample lecture video.

[0076] In the embodiments of this specification, by dynamically generating or retrieving multi-source candidate problem-solving approaches and performing automated, multi-modal content consistency verification with sample problem-solving videos, it is possible to efficiently and accurately identify and determine sample problem-solving approaches that are consistent with the content of sample problem-solving videos from among many possible methods and strategies. This provides basic data for subsequent model training and facilitates the training of the model's ability to generate explanatory texts for problem-solving videos under specific problem-solving approaches.

[0077] Step 106: Construct training samples based on sample question information, sample problem-solving ideas, and sample lecture text. Use the training samples to train the initial lecture generation model until the target lecture generation model that meets the training stopping condition is obtained.

[0078] Specifically, training samples are a set of structured data pairs used to train a machine learning model. A training sample is a triple containing input ("sample question information" and "sample problem-solving approach") and expected output ("sample lecture text (which may include sample whiteboard markings)"). The learning objective of the model is to generate a lecture that is as close as possible to the expected output based on the input.

[0079] The initial speech generation model is a general pre-trained large language model, serving as the starting point for training. Its performance has not yet been optimized for a specific task. Training termination conditions are pre-defined objective criteria used to determine when training ends. Common conditions include reaching a pre-defined number of training epochs or the loss value reaching a predetermined threshold. The target speech generation model refers to the final model, optimized to a stable state after supervised fine-tuning of the initial model on the training sample set. It possesses the ability to generate speech text with blackboard markings based on a given topic and solution approach.

[0080] For example, the data format of the training samples can be as follows: ## Sample Question Information: xxxx; ## Sample Solution Approach: xxxx; ## Sample lecture text: xxxx (carrying sample whiteboard markings).

[0081] The model input is: ## Sample Question Information: xxxx; ## Sample Solution Approach: xxxx.

[0082] The model's expected output is: ## Sample Question Information: xxxx; ## Predicted Speech Text xxxx (carrying prediction whiteboard markings).

[0083] It should be noted that, using the training samples constructed above, the initial lecture script generation model is trained through supervised fine-tuning. This enables the model to learn the ability to generate lecture scripts for corresponding problem-solving videos based on the problem information and specific solution approaches. Thus, using supervised fine-tuning, only the problem and solution approach are needed to output the lecture script (which can include whiteboard annotations) in one step, eliminating the need for manual creation of lecture templates. This achieves fully automated generation of lecture scripts, with no pre-requisite dependencies and low cost. Furthermore, the model can be trained to generate different lecture scripts for different solution approaches, achieving lecture script generation at the solution approach level, rather than the problem level, avoiding issues such as explaining beyond the scope of the topic or failing to provide multiple solutions to a single problem.

[0084] In one optional implementation of this embodiment, the initial speech generation model is trained using training samples until a target speech generation model that meets the training stopping condition is obtained, including: Input the sample question information and sample problem-solving ideas from the training samples into the initial lecture script generation model to obtain the predicted lecture script text, which includes the predicted text content and the predicted whiteboard markings. Obtain the sample lecture text from the training samples, where the sample lecture text includes the sample text content and the labeled sample whiteboard marks; Based on the predicted text content and predicted whiteboard markings of the predicted lecture text, as well as the sample text content and sample whiteboard markings of the sample lecture text, the model loss value is determined. The model parameters of the initial lecture generation model are adjusted based on the model loss value. If the training stopping condition is not met, the operation steps of obtaining sample lecture videos are returned until the training stopping condition is met, and the target lecture generation model is obtained.

[0085] Specifically, the predicted lecture text is the lecture text with predicted whiteboard markings, which is determined by the initial lecture generation model based on the sample question information and sample problem-solving ideas.

[0086] In practice, the sample question information and sample problem-solving approaches from the training samples can be input into the initial lecture script generation model to obtain the predicted lecture script text, which includes the predicted text content and predicted whiteboard markings. Then, based on the predicted text content and predicted whiteboard markings of the predicted lecture script text, as well as the sample text content and labeled sample whiteboard markings in the sample lecture script text, the model loss value is calculated. The model parameters of the initial lecture script generation model are then adjusted based on the model loss value, and it is checked whether the adjusted initial lecture script generation model meets the training stopping condition. If the training stopping condition is not met, the process returns to obtaining the sample lecture video, obtaining the next sample lecture video, and continuing the supervised fine-tuning training of the initial lecture script generation model until the training stopping condition is met, thus obtaining the target lecture script generation model, which can then be deployed to the scenario of generating lecture videos. In this scenario, the target topic and target solution approach are input into the target lecture generation model. The target lecture generation model can output the target lecture text with blackboard markings under the target solution approach. Subsequently, a lecture video under the target solution approach can be generated based on this target lecture text with blackboard markings.

[0087] It should be noted that through iterative training and continuous optimization, the accuracy and adaptability of the speech generation model are continuously improved, ensuring that the target speech generation model can generate different speech texts for different problem-solving approaches, thereby achieving accurate and efficient speech generation at the problem-solving approach level.

[0088] The lecture script generation model training method provided in this specification combines sample question information, sample problem-solving approaches, and sample lecture script text to train an initial lecture script generation model. This allows the initial lecture script generation model to learn to generate specific sample lecture script text under the input sample question information and sample problem-solving approaches. The trained target lecture script generation model has the ability to automatically generate corresponding lecture scripts for different problem-solving approaches. This not only improves the efficiency of lecture script generation for lecture videos and reduces manual costs, but also allows the trained target lecture script generation model to generate different lecture scripts for different problem-solving approaches to the same question, matching them with the given problem-solving approaches, achieving multiple solutions to one question, and making the lecture script generation more flexible, avoiding rigid explanations, and adapting to different problem-solving approaches to avoid explaining beyond the scope of the topic.

[0089] See Figure 2 , Figure 2 A flowchart is shown of a method for generating lecture scripts for a lecture video according to an embodiment of this specification, specifically including the following steps 202-204.

[0090] Step 202: Obtain the target question information and the corresponding target problem-solving approach.

[0091] In practice, the target question information refers to the specific question information specified in the task scenario for which the explanation is to be generated. The target problem-solving approach refers to the specific problem-solving strategy selected or planned for the target question information and intended to be used to generate the explanation content. It clarifies the core strategy and logical framework to be explained and is a key instruction controlling the direction of the generated content. It may be specified by the user or automatically recommended or generated based on the question.

[0092] In one optional implementation of this embodiment, obtaining the target question information and the corresponding target problem-solving approach includes: In response to the first lecture script generation task, retrieve the target problem information and target solution approach carried in the first lecture script generation task. The first lecture script generation task instructs the generation of target lecture script text corresponding to the target solution approach based on the target problem information; or... In response to the second lecture script generation task, the target question information carried in the second lecture script generation task is obtained; based on the target question information, the corresponding target problem-solving approach is generated through the generation model, wherein the second lecture script generation task is used to instruct the generation of the target lecture script text corresponding to the target question information.

[0093] Specifically, the first lecture script generation task refers to a lecture script generation request that explicitly specifies the problem-solving approach. This task instruction directly carries the target problem information and the target problem-solving approach selected by the user. Its core requirement is: to generate a lecture script for a given problem according to the given specific problem-solving approach. It is a task mode of "knowing the approach and generating on demand".

[0094] The second lecture script generation task is a lecture script generation request without specifying a solution approach. The task instruction only carries the target question information. Its core requirement is to automatically determine the solution approach and generate the corresponding lecture script for the given question. It is a task mode of "unknown approach, fully automatic generation".

[0095] In actual implementation, when a first lecture script generation task carrying target question information and a specified target solution approach is received, the trained target lecture script generation model is directly invoked to generate the corresponding target lecture script text based on the given question and solution approach. When a second lecture script generation task carrying only target question information is received, the solution approach generation module or the generation model is invoked to automatically infer, retrieve, or generate a target solution approach suitable for the question. Then, the solution approach and question information are fed into the trained target lecture script generation model, and the corresponding target lecture script text is finally output.

[0096] In the embodiments of this specification, for tasks with clearly specified problem-solving approaches, a lecture text conforming to a specific problem-solving approach can be quickly generated. For tasks that only provide a question, the planning and generation of a problem-solving approach can be completed automatically, and a complete lecture text can be produced accordingly. Through a dual-path intelligent task processing mechanism, high flexibility and strong automation of lecture text generation are achieved, reducing the user's usage threshold and flexibly adapting to different levels of application scenarios.

[0097] Step 204: Based on the target topic information and the target problem-solving approach, generate the corresponding target lecture text using the target lecture generation model. The target lecture generation model is trained using the lecture generation model training method described above.

[0098] It should be noted that after obtaining the target question information and the target problem-solving approach, the target question information and the target problem-solving approach are input into the target lecture script generation model. The target lecture script generation model then infers the target question information according to the target problem-solving approach and generates the target lecture script text under the target problem-solving approach.

[0099] In one optional implementation of this embodiment, the target lecture text is marked with a target whiteboard mark; after generating the corresponding target lecture text based on the target topic information and the target problem-solving approach using the target lecture generation model, the method further includes: Based on the target topic information and the target lecture text with target whiteboard markings, a corresponding lecture video is generated. The target topic information is the topic information of any candidate topic among the candidate topics, and at least one candidate solution approach for each candidate topic generates a corresponding lecture video. A lecture video database is constructed based on each candidate question and at least one solution approach. The lecture video database includes each candidate question, and each candidate question has at least one solution approach corresponding to a different lecture video.

[0100] Specifically, target whiteboard tags refer to structured instruction tags embedded in the target lecture text, used to identify the whiteboard content within the target lecture text, thereby enabling the generation of whiteboard notes. The lecture video database is a structured storage system used to organize and manage massive amounts of lecture videos. Using the question as the index unit, it allows retrieval of multiple lecture videos under different problem-solving approaches.

[0101] In practice, any candidate question can be used as a target question. Based on the target question information and the corresponding target problem-solving approach, a target lecture text carrying target blackboard markings can be generated. This target lecture text can then be converted into audio, and a corresponding target blackboard image can be generated based on the target blackboard markings in the target lecture text. The target blackboard image and audio are then integrated to obtain the corresponding lecture video. Each candidate question can serve as the target question, and lecture videos with different problem-solving approaches can be generated offline in advance through the above process, thus constructing a lecture video database.

[0102] Of course, in actual implementation, a lecture video for a specific topic can also be generated online in real time. The target topic information of the target topic uploaded or selected by the user can be obtained. Based on the specific target problem-solving approach (uploaded by the user or automatically generated by the large model), the corresponding target lecture text can be generated through the target lecture generation model, and the corresponding lecture video can be generated and fed back to the user in real time.

[0103] In the embodiments of this specification, for a large number of candidate questions, corresponding lecture videos can be generated based on at least one problem-solving approach, and a lecture video database can be constructed. Each candidate question in the lecture video database is associated with lecture videos of different problem-solving approaches, realizing centralized storage and efficient retrieval of video resources with multiple solutions to a single question, and improving the output efficiency, coverage, and organizational logic of lecture video resources.

[0104] In an optional implementation of this embodiment, the method further includes: In response to a request to retrieve the video explanation for the question to be explained, retrieve the user attribute information of the requesting user; Retrieve video explanations corresponding to at least one solution approach for the problem to be explained from the video explanation database; The video tutorials for each problem-solving approach are matched with user attribute information to determine the target video tutorial for the problem to be explained.

[0105] Specifically, a request to obtain a video explanation for a specific topic is a service request initiated by a user (student or teacher) with the purpose of obtaining a video explanation for that specific topic (i.e., the topic to be explained).

[0106] User attribute information is structured data used to describe the characteristics of the requesting user and serves as the basis for personalized recommendations. It may include the user's grade level, level of historical knowledge (such as knowledge graph mastery), common error types, learning style preferences (such as preference for intuitive demonstrations or logical deduction), device environment (such as screen size), etc.

[0107] The target video for a problem refers to the video recommended to the current user who meets their personalized needs after matching and filtering, among the different problem-solving approaches for the problem to be explained. If there are multiple solution videos for a problem, the target video is the solution video that matches the user's solution approach.

[0108] In practice, when responding to a user's request for a lecture video, the system can parse the user's attribute information and retrieve the corresponding lecture videos for each solution approach to the question from the lecture video database. Then, it constructs user feature vectors and video feature vectors. The user feature vectors incorporate user attribute information, while the video feature vectors extract content features (such as solution approach type and knowledge point coverage), presentation features (such as animation ratio, speaking speed, and whiteboard density), and statistical features (such as historical playback completion rate and average viewing time) from the lecture video's metadata. Next, it can use a pre-trained matching model or a configurable rule engine to calculate the similarity between the user feature vector and each video feature vector in the multi-dimensional space, generating personalized matching scores for the lecture videos corresponding to each solution approach. The system then sorts and selects lecture videos with higher scores that exceed the confidence threshold as target lecture videos. If none of the targets are met, a fallback strategy (such as selecting a general lecture video) is activated.

[0109] It should be noted that the video explanations for each problem-solving approach can be matched with user attribute information to select target video explanations for problem-solving approaches that match the user's personalized needs. This achieves video recommendation at the problem-solving approach level rather than the question level, avoiding issues such as explanations beyond the syllabus or the inability to provide multiple solutions to a single problem, thus achieving accurate recommendations.

[0110] The lecture script generation method for problem-solving videos provided in this specification, through a trained target lecture script generation model, can generate target lecture script text for a specific problem-solving approach based on target problem information and a given target problem-solving approach. This achieves the ability to automatically generate corresponding lecture scripts for different problem-solving approaches, which not only improves the efficiency of lecture script generation for problem-solving videos and reduces manual costs, but also allows for the generation of different lecture scripts for different problem-solving approaches using the trained target lecture script generation model, matching them with the given problem-solving approach, realizing multiple solutions to one problem, and making the lecture script generation more flexible, avoiding rigid explanations, and adapting to different problem-solving approaches to avoid explanations beyond the scope of the curriculum.

[0111] The following is in conjunction with the appendix Figure 3 Taking the method for generating lecture notes for problem-solving videos provided in this manual as an example in the application of mathematics, this paper further explains the method for generating lecture notes for problem-solving videos. Figure 3The flowchart illustrates a method for generating lecture transcripts for a lecture video according to an embodiment of this specification, which specifically includes the following steps.

[0112] Step 302: Obtain multiple candidate lecture videos under the mathematics subject, and determine the explanation score, blackboard writing score, and audio score of each candidate lecture video; based on the explanation score, blackboard writing score, and audio score, select sample lecture videos from the multiple candidate lecture videos.

[0113] Step 304: Extract the audio of the lecture from the sample lecture video, and identify the audio to obtain the original lecture text; perform text correction and / or text fluency adjustment on the original lecture text to obtain candidate lecture text; based on the blackboard writing effect in the sample lecture video, add sample blackboard writing marks to the blackboard writing content in the candidate lecture text to obtain the sample lecture text.

[0114] Step 306: Determine the sample math problem information corresponding to the sample lecture video, generate at least one candidate solution approach corresponding to the sample math problem information through a generative model, and / or search for at least one candidate solution approach in the solution database based on the sample math problem information; perform content consistency verification on at least one candidate solution approach and the sample lecture video; determine the at least one candidate solution approach that is consistent with the content of the sample lecture video as the sample solution approach.

[0115] Step 308: Construct training samples based on sample math problem information, sample problem-solving ideas, and sample lecture text. Use the training samples to train the initial lecture generation model until a target lecture generation model that meets the training stopping condition is obtained.

[0116] Step 310: In response to the first lecture note generation task under the mathematics subject, obtain the target mathematics problem information and target problem-solving ideas carried in the first lecture note generation task; or, in response to the second lecture note generation task under the mathematics subject, obtain the target mathematics problem information carried in the second lecture note generation task; based on the target mathematics problem information, generate the corresponding target problem-solving ideas through the generation model.

[0117] Step 312: Based on the target math problem information and the target lecture text with target blackboard markings, generate the corresponding lecture video using the target lecture generation model.

[0118] Step 314: Based on the video lectures for each candidate math problem and at least one solution approach, construct a math problem lecture video database. The math problem lecture video database includes each candidate math problem, and each candidate math problem has at least one solution approach corresponding to a different video lecture.

[0119] Step 316: In response to the request to obtain the explanation video of the math problem to be explained, obtain the user attribute information of the requesting user; obtain the explanation video corresponding to at least one solution approach of the math problem to be explained from the math explanation video database; match the explanation videos of each solution approach with the user attribute information to determine the target explanation video of the math problem to be explained, and send the target explanation video back to the requesting user.

[0120] One embodiment of this specification implements training an initial lecture script generation model by combining sample math problem information, sample problem-solving approaches, and sample lecture script text under the math subject. This allows the initial lecture script generation model to learn from the input sample math problem information and sample problem-solving approaches, generating specific sample lecture script text under that sample problem-solving approach. The trained target lecture script generation model has the ability to automatically generate corresponding lecture scripts for the same math problem under the math subject with different problem-solving approaches. This improves the efficiency of lecture script generation for video lectures, reduces manual costs, and, using the trained target lecture script generation model, can generate different lecture scripts for different problem-solving approaches to the same math problem, matching them with the given problem-solving approach, achieving multiple solutions to one problem, increasing the flexibility of lecture script generation, avoiding rigid explanations, and adapting to different problem-solving approaches, thus avoiding explanations beyond the syllabus.

[0121] Corresponding to the above method embodiments, this specification also provides embodiments of a lecture note generation model training device. Figure 4 A schematic diagram of a lecture generation model training device according to one embodiment of this specification is shown. Figure 4 As shown, the device includes: The first acquisition module 402 is configured to acquire sample lecture videos and determine sample lecture text based on the sample lecture videos; The second acquisition module 404 is configured to determine the sample question information corresponding to the sample lecture video, and acquire the corresponding sample problem-solving approach based on the sample question information, wherein the sample problem-solving approach is the problem-solving approach that matches the sample lecture video among at least one candidate problem-solving approach in the sample question information. Training module 406 is configured to construct training samples based on sample question information, sample problem-solving ideas and sample lecture text, and use the training samples to train the initial lecture generation model until the target lecture generation model that meets the training stopping condition is obtained.

[0122] Optionally, the first acquisition module 402 is further configured as follows: Obtain multiple candidate lecture videos and determine the lecture score, blackboard writing score, and audio score for each candidate lecture video. Sample lecture videos were selected from multiple candidate lecture videos based on explanation scores, blackboard writing scores, and audio scores.

[0123] Optionally, the sample lecture text is annotated with sample whiteboard markings; the first acquisition module 402 is further configured to: Extract the audio of the lecture from the sample lecture videos, and identify the original lecture text from the audio. The original speech manuscript is corrected for errors and / or its fluency is adjusted to obtain candidate speech manuscript texts. Based on the blackboard writing effect in the sample lecture video, sample blackboard writing marks are added to the blackboard writing content in the candidate lecture text to obtain the sample lecture text.

[0124] Optionally, the first acquisition module 402 is further configured as follows: Extract the original whiteboard image from the sample lecture video, and perform denoising and / or sharpening processing on the original whiteboard image to obtain the sample whiteboard image; Based on the blackboard structure of the sample blackboard image, sample blackboard markers are added to the blackboard content in the candidate lecture text to obtain the sample lecture text.

[0125] Optionally, the second acquisition module 404 is further configured as follows: Generate at least one candidate solution approach corresponding to the sample question information using a generative model; and / or, search for at least one corresponding candidate solution approach in the question-solving database based on the sample question information; Perform content consistency verification on at least one candidate problem-solving approach and sample problem-solving videos; At least one candidate problem-solving approach that is consistent with the content of the sample lecture video is identified as the sample problem-solving approach.

[0126] Optionally, training module 406 is further configured as follows: Input the sample question information and sample problem-solving ideas from the training samples into the initial lecture script generation model to obtain the predicted lecture script text, which includes the predicted text content and the predicted whiteboard markings. Obtain the sample lecture text from the training samples, where the sample lecture text includes the sample text content and the labeled sample whiteboard marks; Based on the predicted text content and predicted whiteboard markings of the predicted lecture text, as well as the sample text content and sample whiteboard markings of the sample lecture text, the model loss value is determined. The model parameters of the initial lecture generation model are adjusted based on the model loss value. If the training stopping condition is not met, the operation steps of obtaining sample lecture videos are returned until the training stopping condition is met, and the target lecture generation model is obtained.

[0127] The lecture script generation model training device provided in this specification includes a first acquisition module, a second acquisition module, and a training module. These modules interact and cooperate to train an initial lecture script generation model by combining sample question information, sample problem-solving approaches, and sample lecture script text. This allows the initial lecture script generation model to learn from the input sample question information and sample problem-solving approaches, generating specific sample lecture script text under those approaches. The trained target lecture script generation model possesses the ability to automatically generate corresponding lecture scripts for different problem-solving approaches. This improves the efficiency of lecture script generation for video lectures, reduces manual costs, and allows for the generation of different lecture scripts for different problem-solving approaches to match the given approach, enabling multiple solutions to a single problem. This increases the flexibility of lecture script generation, avoids rigid explanations, and adapts to different problem-solving approaches, preventing explanations from exceeding the scope of the topic.

[0128] The above is a schematic scheme of a lecture note generation model training device according to this embodiment. It should be noted that the technical solution of this lecture note generation model training device and the technical solution of the lecture note generation model training method described above belong to the same concept. For details not described in detail in the technical solution of the lecture note generation model training device, please refer to the description of the technical solution of the lecture note generation model training method described above.

[0129] Corresponding to the above method embodiments, this specification also provides embodiments of a lecture script generation device for lecture videos. Figure 5 This specification illustrates a schematic diagram of a lecture script generation device for a lecture video according to one embodiment. Figure 5 As shown, the device includes: The third acquisition module 502 is configured to acquire target question information and the corresponding target problem-solving approach; The generation module 504 is configured to generate the corresponding target lecture text based on the target question information and the target problem-solving approach, using the target lecture generation model. The target lecture generation model is trained using the lecture generation model training method described above.

[0130] Optionally, the third acquisition module 502 is further configured as follows: In response to the first lecture script generation task, retrieve the target problem information and target solution approach carried in the first lecture script generation task. The first lecture script generation task instructs the generation of target lecture script text corresponding to the target solution approach based on the target problem information; or... In response to the second lecture script generation task, the target question information carried in the second lecture script generation task is obtained; based on the target question information, the corresponding target problem-solving approach is generated through the generation model, wherein the second lecture script generation task is used to instruct the generation of the target lecture script text corresponding to the target question information.

[0131] Optionally, the target presentation text is annotated with target whiteboard markers; the device also includes a building module configured to: Based on the target topic information and the target lecture text with target whiteboard markings, a corresponding lecture video is generated. The target topic information is the topic information of any candidate topic among the candidate topics, and at least one candidate solution approach for each candidate topic generates a corresponding lecture video. A lecture video database is constructed based on each candidate question and at least one solution approach. The lecture video database includes each candidate question, and each candidate question has at least one solution approach corresponding to a different lecture video.

[0132] Optionally, the device also includes a matching module configured to: In response to a request to retrieve the video explanation for the question to be explained, retrieve the user attribute information of the requesting user; Retrieve video explanations corresponding to at least one solution approach for the problem to be explained from the video explanation database; The video tutorials for each problem-solving approach are matched with user attribute information to determine the target video tutorial for the problem to be explained.

[0133] The lecture script generation device for video lectures provided in this specification includes a third acquisition module and a generation module. The modules of this device interact and cooperate. Through the trained target lecture script generation model, it can generate target lecture script text for a specific problem-solving approach based on the target question information and the given target problem-solving approach. This realizes the ability to automatically generate corresponding lecture scripts for different problem-solving approaches, which not only improves the efficiency of lecture script generation for video lectures and reduces manual costs, but also allows for the generation of different lecture scripts for different problem-solving approaches using the trained target lecture script generation model, matching them with the given problem-solving approach, realizing multiple solutions to one question, and making the lecture script generation more flexible, avoiding rigid explanations, and adapting to different problem-solving approaches to avoid explanations beyond the scope of the subject.

[0134] The above is a schematic scheme of a lecture script generation device for a lecture video according to this embodiment. It should be noted that the technical solution of this lecture script generation device and the technical solution of the lecture script generation method described above belong to the same concept. For details not described in detail in the technical solution of the lecture script generation device, please refer to the description of the technical solution of the lecture script generation method described above.

[0135] Figure 6A structural block diagram of a computing device according to one embodiment of this specification is shown. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and a database 650 is used to store data.

[0136] The computing device 600 also includes an access device 640, which enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 640 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0137] In one embodiment of this specification, the above-described components of the computing device 600 and Figure 6 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 6 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0138] The computing device 600 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 600 can also be a mobile or stationary server.

[0139] The processor 620 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned lecture script generation model training method or lecture script generation method for lecture videos.

[0140] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solution of the above-mentioned lecture script generation model training method or lecture script generation method for lecture videos. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-mentioned lecture script generation model training method or lecture script generation method for lecture videos.

[0141] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described lecture script generation model training method or lecture script generation method for lecture videos.

[0142] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the above-described lecture script generation model training method or lecture script generation method for lecture videos. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the above-described lecture script generation model training method or lecture script generation method for lecture videos.

[0143] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described lecture script generation model training method or lecture script generation method for lecture videos.

[0144] The above is an illustrative example of a computer program in this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solutions of the above-described lecture script generation model training method or lecture script generation method for topic videos. Details not described in detail in the computer program's technical solution can be found in the descriptions of the technical solutions of the above-described lecture script generation model training method or lecture script generation method for topic videos.

[0145] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0146] Computer instructions include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in computer-readable media can be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0147] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0148] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0149] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for training a lecture script generation model, characterized in that, include: Obtain sample lecture videos and determine sample lecture text based on the sample lecture videos; Determine the sample question information corresponding to the sample lecture video, and obtain the corresponding sample problem-solving approach based on the sample question information, wherein the sample problem-solving approach is the problem-solving approach that matches the sample lecture video from at least one candidate problem-solving approach in the sample question information; Training samples are constructed based on the sample question information, the sample problem-solving ideas, and the sample lecture text. The initial lecture generation model is trained using the training samples until a target lecture generation model that meets the training stopping condition is obtained.

2. The lecture note generation model training method according to claim 1, characterized in that, The acquisition of sample lecture videos includes: Obtain multiple candidate lecture videos and determine the lecture score, blackboard writing score, and audio score for each candidate lecture video. Based on the explanation score, the blackboard writing score, and the audio score, the sample lecture video is selected from the multiple candidate lecture videos.

3. The lecture note generation model training method according to claim 1, characterized in that, The sample lecture text is marked with sample whiteboard markings; determining the sample lecture text based on the sample lecture video includes: Extract the audio explanations from the sample lecture videos, and identify the audio explanations to obtain the original lecture transcript text; The original speech manuscript is corrected and / or its fluency is adjusted to obtain candidate speech manuscript texts. Based on the blackboard writing effect in the sample lecture video, sample blackboard writing marks are added to the blackboard writing content in the candidate lecture text to obtain the sample lecture text.

4. The lecture note generation model training method according to claim 3, characterized in that, The step of adding sample blackboard markers to the blackboard content in the candidate lecture text based on the blackboard effects in the sample lecture video to obtain the sample lecture text includes: Extract the original whiteboard image from the sample lecture video, and perform noise reduction and / or sharpening processing on the original whiteboard image to obtain the sample whiteboard image; Based on the blackboard structure of the sample blackboard image, sample blackboard markers are added to the blackboard content in the candidate lecture text to obtain the sample lecture text.

5. The lecture note generation model training method according to claim 1, characterized in that, The process of obtaining the corresponding sample problem-solving approach based on the sample question information includes: The model generates at least one candidate solution approach corresponding to the sample question information; and / or, the model searches for at least one candidate solution approach in the question-solving database based on the sample question information. Perform content consistency verification on the at least one candidate problem-solving approach and the sample problem-solving video; The candidate problem-solving approach that is consistent with the content of the sample lecture video is identified as the sample problem-solving approach.

6. The lecture note generation model training method according to any one of claims 1-5, characterized in that, The step of training the initial speech generation model using the training samples until a target speech generation model that meets the training stopping condition is obtained includes: The sample question information and sample problem-solving ideas in the training samples are input into the initial lecture script generation model to obtain the predicted lecture script text, wherein the predicted lecture script text includes the predicted text content and the predicted whiteboard markings. Obtain the sample lecture text from the training samples, wherein the sample lecture text includes sample text content and labeled sample whiteboard marks; Based on the predicted text content and predicted whiteboard markings of the predicted lecture text, as well as the sample text content and sample whiteboard markings of the sample lecture text, the model loss value is determined. The model parameters of the initial lecture generation model are adjusted based on the model loss value. If the training stopping condition is not met, the operation step of obtaining the sample lecture video is returned until the training stopping condition is met, and the target lecture generation model is obtained.

7. A method for generating lecture scripts for video presentations, characterized in that, include: Obtain information about the target question and the corresponding target problem-solving approach; Based on the target topic information and the target problem-solving approach, a corresponding target lecture text is generated using a target lecture generation model, wherein the target lecture generation model is trained using the lecture generation model training method described in any one of claims 1-6.

8. The method for generating lecture transcripts for lecture videos according to claim 7, characterized in that, The process of obtaining the target question information and the corresponding target problem-solving approach includes: In response to the first lecture script generation task, the target question information and target problem-solving approach carried in the first lecture script generation task are obtained, wherein the first lecture script generation task is used to instruct the generation of target lecture script text corresponding to the target problem-solving approach based on the target question information; or... In response to the second lecture script generation task, the target question information carried in the second lecture script generation task is obtained; based on the target question information, the corresponding target problem-solving approach is generated through the generation model, wherein the second lecture script generation task is used to instruct the generation of the target lecture script text corresponding to the target question information.

9. The method for generating lecture transcripts for lecture videos according to claim 7, characterized in that, The target lecture text is marked with target whiteboard markings; after generating the corresponding target lecture text based on the target question information and the target problem-solving approach using the target lecture generation model, the process further includes: Based on the target question information and the target lecture text carrying the target whiteboard mark, a corresponding lecture video is generated. The target question information is the question information of any candidate question among the candidate questions, and at least one candidate solution for each candidate question generates a corresponding lecture video. Based on the candidate questions and at least one solution approach, a question-and-answer video database is constructed. The question-and-answer video database includes the candidate questions, and each candidate question has at least one solution approach corresponding to a different question-and-answer video.

10. The method for generating lecture transcripts for lecture videos according to claim 9, characterized in that, The method further includes: In response to a request to retrieve the video explanation for the question to be explained, retrieve the user attribute information of the requesting user; Obtain, from the lecture video database, at least one solution approach for the problem to be explained corresponding to a lecture video; The video tutorials for each problem-solving approach are matched with the user attribute information to determine the target video tutorial for the problem to be explained.

11. A training device for a lecture script generation model, characterized in that, include: The first acquisition module is configured to acquire sample lecture videos and determine sample lecture text based on the sample lecture videos; The second acquisition module is configured to determine the sample question information corresponding to the sample lecture video, and acquire the corresponding sample problem-solving approach based on the sample question information, wherein the sample problem-solving approach is the problem-solving approach that matches the sample lecture video among at least one candidate problem-solving approach in the sample question information; The training module is configured to construct training samples based on the sample question information, the sample problem-solving ideas, and the sample lecture text, and use the training samples to train the initial lecture generation model until a target lecture generation model that meets the training stopping condition is obtained.

12. A lecture script generation device for lecture videos, characterized in that, include: The third acquisition module is configured to acquire target question information and corresponding target problem-solving ideas; The generation module is configured to generate the corresponding target lecture text based on the target question information and the target problem-solving approach, using a target lecture generation model, wherein the target lecture generation model is trained using the lecture generation model training method described in any one of claims 1-6.

13. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the lecture script generation model training method according to any one of claims 1-6 or the lecture script generation method for lecture videos according to any one of claims 7-10.

14. A computer-readable storage medium, characterized in that, It stores computer-executable instructions, which, when executed by a processor, implement the steps of the lecture script generation model training method according to any one of claims 1-6 or the lecture script generation method for the lecture video according to any one of claims 7-10.

15. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by a processor, implements the steps of the lecture script generation model training method according to any one of claims 1-6 or the lecture script generation method for the lecture video according to any one of claims 7-10.

Images