Course file generation method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a course file generation method and device, electronic equipment and storage medium, and relates to the technical field of computers. The method comprises: obtaining audio data and video data of a target course; generating a course text of the target course according to the audio data; intercepting a course image in the video data; and generating a course file corresponding to the target course according to the course text and the course image. The method provided by the present disclosure can enable a teacher to quickly generate a course file corresponding to the target course after the course ends without additional operations, thereby improving the teaching efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for generating course documents. Background Technology

[0002] With the increasing maturity of artificial intelligence-related technologies, more and more jobs are being replaced by machines, and traditional teaching methods are undergoing tremendous changes. A large number of intelligent teaching tools have been introduced in all aspects, from pre-class preparation, in-class teaching, and post-class resource organization. In particular, the popularization of artificial intelligence technology has greatly improved the efficiency and quality of teaching.

[0003] However, although the teaching tools in the relevant technologies are sophisticated, the production process of teaching resources requires calling different programs or applications at each step, and teachers need to operate them personally. This not only results in high learning costs but also lacks integration of repetitive processes, making it time-consuming and labor-intensive. Summary of the Invention

[0004] In view of this, the present disclosure provides a method, apparatus, electronic device and storage medium for generating course documents, which can automatically construct course documents and improve teachers' teaching efficiency.

[0005] Firstly, a method for generating course files is provided, including: acquiring audio and video data of a target course; generating course text for the target course based on the audio data; extracting course images from the video data; and generating a course file corresponding to the target course based on the course text and course images.

[0006] In some embodiments, capturing a course image from video data includes: monitoring the grayscale value change between the current frame image and the previous frame image in the video data; if the grayscale value change meets a preset capture condition, then capturing the current frame image from the video data, wherein the course image includes the current frame image.

[0007] In some embodiments, generating course text for a target course based on audio data includes: transcribing the audio data to obtain a transcribed text of the audio data; and translating the transcribed text into a translated text in at least one language, wherein the course text includes at least one of the transcribed text and the translated text.

[0008] In some embodiments, generating a course file corresponding to a target course based on course text and course images includes: extracting knowledge text from the course text based on a knowledge extraction model; and generating a course file corresponding to the target course based on the knowledge text and course images.

[0009] In some embodiments, generating a course file corresponding to a target course based on knowledge text and course images includes: obtaining a first timestamp of the knowledge text in audio data and a second timestamp of the course images in video data; and concatenating the knowledge text and course images based on the first and second timestamps to generate a course file corresponding to the target course.

[0010] In some embodiments, knowledge text and course images are spliced ​​together based on a first timestamp and a second timestamp to generate a course file corresponding to the target course, including: comparing the first timestamp and the second timestamp; if the difference between the first timestamp and the second timestamp meets a preset difference range, then the knowledge text and course images are spliced ​​together to generate a course file corresponding to the target course.

[0011] In some embodiments, after acquiring the audio and video data of the target course, the method further includes: aligning the timelines of the audio data and the video data.

[0012] Secondly, a course file generation device is provided, comprising: an acquisition module for acquiring audio data and video data of a target course; a transcription module for generating course text of the target course based on the audio data; an extraction module for extracting course images from the video data; and a generation module for generating a course file corresponding to the target course based on the course text and course images.

[0013] Thirdly, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of the first aspect by executing the executable instructions.

[0014] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method of the first aspect described above.

[0015] The course file generation method provided in this disclosure acquires audio and video data of a target course; generates course text for the target course based on the audio data; extracts course images from the video data; and generates a course file corresponding to the target course based on the course text and the course images. This method allows teachers to quickly generate course files corresponding to the target course after the course ends without additional steps, improving teaching efficiency. It also allows students to browse course-related knowledge without repeatedly playing teaching videos, greatly facilitating their use of teaching resources. Attached Figure Description

[0016] Figure 1 A schematic diagram of the system architecture of a course document generation method according to an embodiment of this disclosure is shown.

[0017] Figure 2 The diagram shows a flowchart of a course document generation method according to an embodiment of this disclosure.

[0018] Figure 3 A schematic diagram of an application scenario in an embodiment of this disclosure is shown.

[0019] Figure 4 A schematic diagram of the structure of a course document generation device according to an embodiment of the present disclosure is shown.

[0020] Figure 5 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0021] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0022] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0023] With the development of computer technology, digital courses are now commonplace in the education field. However, the functions of course document generation tools in these technologies are rather fragmented, only including features such as audio and video recording, live streaming, speech-to-text conversion, and translation. They do not fully consider the complete needs of teachers using teaching tools during classroom instruction, nor their needs for creating related teaching resources after class. To create more complex course resources, multiple tools need to be combined to truly achieve the creation of course documents containing various resources. This not only increases the learning cost for teachers regarding the creation tools but is also time-consuming and laborious.

[0024] In view of this, the solution provided in this disclosure can generate course text from the audio data of the target course and extract course images from the video data of the target course. Then, based on the course text and course images, the course file corresponding to the target course can be generated, so that teachers can directly obtain the course file containing text and image resources after the course ends.

[0025] The following will combine Figure 1 This describes the operating environment of the course file generation method in the embodiments of this disclosure.

[0026] Figure 1 A schematic diagram of an exemplary system architecture for a course document generation method or apparatus applied to embodiments of this disclosure is shown. Figure 1 As shown, the system architecture 100 may include a client 101, a resource center 102, core services 103, and infrastructure 104.

[0027] It should be noted that client 101 can be deployed on terminal devices, such as mobile phones, smart TVs, tablets, laptops, or personal computers (PCs).

[0028] Specifically, client 101 can be an application client or a browser client, etc., used to implement functions such as account login, start and stop recording, screen recording, audio recording, and video keyframe capture.

[0029] In some embodiments, the client 101 can also realize the function of real-time transcription / translation of subtitles on the terminal device by calling the core service 103.

[0030] Resource Center 102 is a management platform for recorded resources, which can be deployed on a server. Resource Center 102 is primarily used for collecting, storing, managing, and applying recorded teaching resources (video and audio data). It supports teachers in editing recorded resource content and also supports schools in managing their teaching resource repositories. Furthermore, it provides resource sharing services, enabling teachers and students to use shared resources for pre-class preparation, review, and teaching research.

[0031] Core service 103, employing a microservice architecture, provides relevant business interfaces and unifies the calling methods for different service interfaces (including speech-to-text / translation engine, large file upload service, multilingual subtitle upload and distribution service, knowledge point extraction service, etc.). For example, core service 103 can be deployed on the same server as resource center 102.

[0032] Infrastructure 104, including databases, servers, network equipment, and other facilities, is used to provide hardware support and data security for the operation of the above clients 101, resource center 102, and core services 103.

[0033] Despite Figure 1In this disclosure, the client 101, resource center 102, core service 103, and infrastructure 104 are presented as separate modules. Those skilled in the art will understand that these modules can be implemented as individual hardware devices or integrated into one or more hardware devices. As long as the principles described herein are implemented, the specific implementation methods of different hardware devices should not be considered as limiting the scope of protection of this disclosure.

[0034] The following will describe the exemplary implementation method in detail with reference to the accompanying drawings and embodiments.

[0035] First, this disclosure provides a method for generating course documents, which can be executed by any electronic device with computing capabilities.

[0036] Figure 2 This diagram illustrates a flowchart of a course document generation method according to an embodiment of the present disclosure, such as... Figure 2 As shown in the embodiments of this disclosure, the course file generation method includes the following steps.

[0037] S201, Obtain the audio and video data of the target course.

[0038] It should be noted that the target course can be any course that can be explained through audio or video. For example, the target course can be taught in person, online, or as a pre-recorded course file.

[0039] In some embodiments, when the target course is taught in person, audio data can be collected via a microphone at the teaching location, and video data can be collected via a camera at the teaching location. When the target course is taught online, the audio and video data of the target course can be recorded via computer equipment (e.g., a device deployed with a live streaming client). When the target course is taught from a pre-recorded course file, the audio and video data of the target course can be directly extracted from the course file. This disclosure does not limit the scope of the embodiments.

[0040] In some embodiments, to ensure that the acquired audio and video data correspond one-to-one, after acquiring the audio and video data of the target course, the timelines of the audio data and the video data can be aligned to improve the quality of the course files generated in subsequent steps.

[0041] S202, Generate the course text for the target course based on the audio data.

[0042] In some embodiments, audio data can be transcribed to obtain a transcribed text of the audio data. Simultaneously, to meet the needs of multilingual teaching, the transcribed text can also be translated into a translated text in at least one language. In this case, the course text includes at least one of the transcribed text and the translated text.

[0043] It should be noted that the transcription of audio data can be performed using speech recognition models from related technologies. By inputting audio data into a speech recognition model, the text sequence with the highest probability can be predicted, i.e., the transcribed text. This disclosure will not elaborate further on this.

[0044] It should be noted that the translated text can be a translation of any language different from the transcribed text, which can be achieved through machine translation technology, and this disclosure will not elaborate further on this aspect.

[0045] For example, the transcription of audio data can be performed in real time during the teaching process. In addition, the transcribed text can be converted into audio output using a text-to-speech (TTS) model, such as the Fastspeech model, etc. This disclosure does not limit this.

[0046] By translating the transcribed text and synthesizing the translated text into speech, the needs of multilingual teaching and simultaneous interpretation can be met, thereby broadening the audience of the target course and its course materials.

[0047] S203, extract course images from video data.

[0048] It should be noted that the course images can be images from the video data showing changes in the teaching scene or content, illustrating the content of various scenes throughout the teaching process in image form. The number of course images captured can be one or multiple, and this embodiment does not limit this.

[0049] In some embodiments, course images can be extracted based on the grayscale value changes between adjacent frames in the monitored video.

[0050] Specifically, it can monitor the change in grayscale value between the current frame and the previous frame in the video data. If the change in grayscale value meets the preset cropping conditions, the current frame image is cropped from the video data, and the course image includes the current frame image.

[0051] In some embodiments, the preset cropping condition can be a grayscale value change threshold. If the grayscale value difference between the current frame image and the previous frame image exceeds the preset grayscale value change threshold, it is considered that the content displayed in the current frame image has changed significantly. At this time, the current frame image can be cropped and used as a course image.

[0052] In some embodiments, the preset cropping condition can be a sudden change in grayscale value. In this case, by analyzing the grayscale value change trend between the current frame and the previous frame, it can be determined whether the grayscale value of the current frame is at a peak or trough of the video data grayscale value curve, thus determining whether a sudden change in grayscale value has occurred. When a sudden change in grayscale value occurs, it is considered that the content displayed in the current frame has changed significantly, and the current frame can be cropped and used as the course image.

[0053] In some embodiments, when cropping course images, a second timestamp of the course image in the video data can be recorded simultaneously for later use.

[0054] S204: Generate the course file corresponding to the target course based on the course text and course images.

[0055] In some embodiments, for course text, knowledge text can be extracted in advance using a knowledge extraction model. Then, based on the knowledge text and course images, a course file corresponding to the target course can be generated. The knowledge text consists of the key content related to the subject knowledge points taught in the course.

[0056] It should be noted that the knowledge extraction model is a type of model in natural language processing technology. The knowledge extraction model can first perform preprocessing operations on the course text, such as word segmentation, stop word filtering, and word sense disambiguation. Then, it extracts textual features from the course text and combines them with a subject-specific semantic ontology based on pre-set teaching and research experience to obtain the relationship between subject knowledge points and textual features, thereby extracting the knowledge text from the course text.

[0057] In some embodiments, the method for generating a course file corresponding to a target course based on knowledge text and course images may be as follows: obtaining a first timestamp of the knowledge text in the audio data and a second timestamp of the course images in the video data; and concatenating the knowledge text and course images based on the first and second timestamps to generate a course file corresponding to the target course.

[0058] Specifically, the first timestamp and the second timestamp can be compared. If the difference between the first timestamp and the second timestamp meets the preset difference range, the knowledge text and course image are stitched together to generate the course file corresponding to the target course.

[0059] It should be noted that the first timestamp can be generated during the transcription of the audio data. When extracting knowledge from the course text, the timestamp corresponding to the knowledge text can be extracted simultaneously and used as the first timestamp, which is the time point in the audio data where the knowledge text is explained.

[0060] It should be noted that the second timestamp can be generated when capturing course images. When capturing course image 5, the timestamp of that frame in the video data can be obtained simultaneously and used as the second timestamp, i.e., the time point when the course image is displayed in the video data.

[0061] In some embodiments, the preset difference range can be a time range, such as 0 minutes to 3 minutes, to determine whether the knowledge text corresponds to the course image. In other words, the greater the difference in their timestamps, the better.

[0062] The smaller the correlation, the higher the correlation between the two. Therefore, knowledge text and course images with high correlation can be combined to obtain a course file containing both text and images.

[0063] The method provided in this disclosure can automatically generate course files containing text and image resources, wherein...

[0064] It displays the knowledge points mentioned in the target course and the course-related metadata, which greatly enriches the form of teaching resources and makes it easier for teachers to create other formats of resources based on course documents, such as slides and documents, thereby improving teaching efficiency.

[0065] 5. In some embodiments, the generated course files can also be displayed in the form of QR codes.

[0066] Users can share the content by scanning the QR code with their smartphones or other mobile devices.

[0067] The course file generation method provided in this disclosure includes: acquiring audio and video data of a target course; generating course text for the target course based on the audio data; and extracting course content from the video data.

[0068] Images; based on course text and course images, a course file corresponding to the target course is generated for students to study and consult. The method provided in this disclosure enables teachers to quickly generate course files corresponding to the target course, improving teaching efficiency and allowing students to browse course-related knowledge without repeatedly playing teaching videos, greatly facilitating students' use of teaching resources.

[0069] To facilitate understanding, the following will combine... Figure 3 This describes one application scenario of the present disclosure.

[0070] like Figure 3 As shown, in one application scenario disclosed herein, a course can be recorded live using five external hardware devices, including a camera and microphone. Third-party video sources can also be interspersed during the recording process.

[0071] (For example, recorded live stream sources, other playback links, etc.).

[0072] The client can pre-set the specified languages ​​in multilingual courses, as well as set the subtitle style, course file saving method, etc.

[0073] During recording, the client can also transcribe and translate the recorded audio data, extract course images from the recorded video data, and control the recorded screen (e.g., turn the screen display on / off, adjust subtitle styles, etc.). Students can then access the client to watch the live course, and the simultaneous interpretation technology can meet the learning needs of students speaking different languages.

[0074] After recording, the client can upload the audio and video data to the resource center, which will then automatically generate course files based on the methods provided herein. Teachers can then modify and fine-tune the course files through the resource center. The client can then share the generated course files with students for them to view.

[0075] The embodiments disclosed herein greatly facilitate teachers in generating digital course resources related to the course, improve teaching efficiency, and also greatly facilitate students in using course-related digital resources before, during, and after class.

[0076] Based on the same inventive concept, this disclosure also provides a course document generation device, as shown in the following embodiment. Since the principle by which this device embodiment solves the problem is similar to that of the above-described method embodiment, the implementation of this device embodiment can refer to the implementation of the above-described method embodiment, and repeated details will not be elaborated further.

[0077] Figure 4 This diagram illustrates the structure of a course document generation device according to an embodiment of the present disclosure, such as... Figure 4 As shown, the course document generation device 400 includes: an acquisition module 401, a transcription module 402, an extraction module 403, and a generation module 404.

[0078] Specifically, the acquisition module 401 is used to acquire the audio and video data of the target course. The transcription module 402 is used to generate the course text of the target course based on the audio data. The extraction module 403 is used to extract course images from the video data. The generation module 404 is used to generate the course file corresponding to the target course based on the course text and course images.

[0079] In some embodiments, the cropping module 403 is further configured to monitor the grayscale value change between the current frame image and the previous frame image in the video data; if the grayscale value change meets the preset cropping conditions, the current frame image in the video data is cropped, and the course image includes the current frame image.

[0080] In some embodiments, the transcription module 402 is further configured to: transcribe the audio data to obtain a transcribed text of the audio data; and translate the transcribed text into a translated text in at least one language, wherein the course text includes at least one of the transcribed text and the translated text.

[0081] In some embodiments, the generation module 404 is further configured to extract knowledge text from the course text based on a knowledge extraction model; and generate a course file corresponding to the target course based on the knowledge text and the course image.

[0082] In some embodiments, the generation module 404 is further configured to obtain a first timestamp of the knowledge text in the audio data and a second timestamp of the course image in the video data; and to splice the knowledge text and the course image according to the first timestamp and the second timestamp to generate a course file corresponding to the target course.

[0083] In some embodiments, the generation module 404 is further configured to compare the first timestamp and the second timestamp; if the difference between the first timestamp and the second timestamp meets a preset difference range, then the knowledge text and the course image are spliced ​​together to generate a course file corresponding to the target course.

[0084] In some embodiments, the acquisition module 401 is further configured to perform alignment processing on the timeline of the audio data and the timeline of the video data.

[0085] It should be noted that the course document generation device provided in the above embodiments is only illustrated by the division of the above functional modules when used for course document generation. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the course document generation device and the course document generation method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0086] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0087] The following reference Figure 5 To describe an electronic device 500 according to such an embodiment of the present disclosure. Figure 5 The electronic device 500 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0088] like Figure 5 As shown, the electronic device 500 is manifested in the form of a general-purpose computing device. The components of the electronic device 500 may include, but are not limited to: at least one processing unit 510, at least one storage unit 520, and a bus 530 connecting different system components (including storage unit 520 and processing unit 510).

[0089] The storage unit stores program code, which can be executed by the processing unit 510, causing the processing unit 510 to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0090] In some embodiments, the processing unit 510 may perform the following steps of the above method embodiments: acquiring audio data and video data of the target course; generating course text of the target course based on the audio data; extracting course images from the video data; and generating a course file corresponding to the target course based on the course text and course images.

[0091] Storage unit 520 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 5201 and / or cache memory 5202, and may further include a read-only memory (ROM) 5203.

[0092] Storage unit 520 may also include a program / utility 5204 having a set (at least one) program module 5205, such program module 5205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0093] Bus 530 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0094] Electronic device 500 can also communicate with one or more external devices 540 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with the electronic device 500, and / or with any device that enables the electronic device 500 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 550. Furthermore, electronic device 500 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 560. Figure 5As shown, network adapter 560 communicates with other modules of electronic device 500 via bus 530. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0095] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0096] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the methods described above is stored thereon. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code, which, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0097] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0098] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0099] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0100] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0101] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0102] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0103] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0104] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method for generating course documents, characterized in that, include: Obtain the audio and video data of the target course; The audio data includes real-time audio data during the recording process, and the video data includes real-time video data during the recording process. The timelines of the audio data and the video data are aligned. Based on the audio data, generate the course text for the target course; Monitor the change in grayscale value between the current frame and the previous frame in the video data; If the grayscale value change meets the preset cropping conditions, then the current frame image in the video data is cropped, and the course image in the video data includes the current frame image; The text features in the course text are extracted using a knowledge extraction model. Based on a pre-set subject semantic ontology, the relationship between subject knowledge points and text features is determined. Based on the relationship, knowledge text in the course text is extracted. Obtain the first timestamp of the knowledge text in the audio data and the second timestamp of the course image in the video data; If the difference between the first timestamp and the second timestamp meets a preset difference range, the knowledge text and the course image are spliced ​​together to generate a course file corresponding to the target course.

2. The method according to claim 1, characterized in that, The step of generating the course text of the target course based on the audio data includes: The audio data is transcribed to obtain the transcribed text of the audio data; The transcribed text is translated into a translated text in at least one language, and the course text includes at least one of the transcribed text and the translated text.

3. A course document generation device, characterized in that, include: The acquisition module is used to acquire the audio and video data of the target course; The audio data includes real-time audio data during the recording process, and the video data includes real-time video data during the recording process; the timelines of the audio data and the video data are aligned. The transcription module is used to generate the course text of the target course based on the audio data; The cropping module is used to monitor the grayscale value change between the current frame image and the previous frame image in the video data; if the grayscale value change meets the preset cropping conditions, the current frame image in the video data is cropped, and the course images in the video data include the current frame image; The generation module is used to extract text features from the course text using a knowledge extraction model, determine the association between subject knowledge points and text features based on a preset subject semantic ontology, extract knowledge text from the course text according to the association, obtain the first timestamp of the knowledge text in the audio data and the second timestamp of the course image in the video data, and if the difference between the first timestamp and the second timestamp meets a preset difference range, then the knowledge text and the course image are concatenated to generate the course file corresponding to the target course.

4. An electronic device, characterized in that, include: processor; and memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 2 by executing the executable instructions.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 2.

Images