Course file generation method and device, terminal equipment and storage medium

Abstract

The invention provides a course file generation method, a course file generation device, terminal equipment and a computer readable storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring audio data acquired by a microphone in the terminal equipment and audio data and image data of a video played in the terminal equipment; the audio data collected by the microphone and the audio data of the played video are subjected to audio mixing processing to obtain audio mixing data, the audio mixing processing is used for integrating the audio data collected by the microphone and the audio data of the played video, and the audio mixing data comprise the audio data collected by the microphone and the audio data of the played video; recording the sound mixing data and the image data to obtain a course video; performing identification processing on the audio data collected by the microphone to obtain a course text; and generating a course file according to the course video and the course text. Based on the scheme, the accuracy of the subtitles in the micro-course video can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a method for generating a course file, an apparatus for generating a course file, a terminal device, and a computer-readable storage medium in the field of computer technology. Background Art

[0002] With the continuous development of artificial intelligence, traditional teaching methods are also developing, and online teaching methods have emerged; for example, micro-class videos; micro-class videos are popular with students due to their flexibility and repeated viewing characteristics. In order to improve the quality and efficiency of students' learning through micro-classes, subtitles can be added to micro-class videos to improve the readability of teachers' course explanations.

[0003] In the related art, after the micro-class video is recorded, corresponding explanation subtitles can be added to the micro-class video; currently, the accuracy of the explanation subtitles in the micro-class video is low. Therefore, how to improve the accuracy of the subtitles in the micro-class video has become an urgent problem to be solved. Summary of the invention

[0004] The present application provides a method for generating a course file, a device for generating a course file, a terminal device and a computer-readable storage medium, which can improve the accuracy of subtitles in micro-course videos.

[0005] In a first aspect, a method for generating a course file is provided, the method comprising: obtaining audio data collected by a microphone in a terminal device and audio data and image data of a video played in the terminal device, wherein the audio data collected by the microphone does not include audio data of the video played; performing mixing processing on the audio data collected by the microphone and the audio data of the video played to obtain mixed data, wherein the mixing processing is used to integrate the audio data collected by the microphone and the audio data of the video played, and the mixed data includes the audio data collected by the microphone and the audio data of the video played; recording the mixed data and image data to obtain a course video; performing recognition processing on the audio data collected by the microphone to obtain a course text; and generating a course file based on the course video and the course text.

[0006] In an embodiment of the present application, in the process of generating a course file (i.e., a micro-course video), mixed data is obtained by mixing audio data collected by a microphone and audio data of a video played in a terminal device, and the mixed data and image data of the terminal are recorded to obtain a corresponding course video; since the audio data collected by the microphone does not include audio data of the video played, when the audio data collected by the microphone is recognized and generated, the audio data of the video played in the terminal device will not be recognized, so that the audio data of the video played in the terminal device will not interfere with the text recognized by the audio data collected by the microphone, so that the recognized course text only includes the text corresponding to the audio data collected by the microphone, but does not include the text corresponding to the audio data of the video played in the terminal device, thereby improving the accuracy of the subtitles in the micro-course video.

[0007] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: after obtaining the audio data collected by the microphone in the terminal device, it also includes: copying the audio data collected by the microphone to obtain first data and second data, wherein the first data and the second data are the same; mixing the audio data collected by the microphone and the audio data of the played video to obtain mixed data, including: mixing the first data and the audio data of the played video to obtain mixed data; and recognizing the audio data collected by the microphone to obtain a course text, including: recognizing the second data to obtain a course text; wherein the mixing process and the recognition process are performed simultaneously.

[0008] In an embodiment of the present application, by copying and processing the audio data collected by a microphone to obtain two identical audio data (i.e., first data and second data), it is possible to achieve mixing of the first data and the audio data of the played video while also identifying the second data, thereby synchronously generating the course video and course text, and improving the real-time generation of subtitles in the course file.

[0009] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, the second data is identified and processed, and the method includes: obtaining function words in the second data; removing the function words in the second data; and identifying and processing the second data without the function words.

[0010] In an embodiment of the present application, since the function words in the second data are removed when the second data is recognized, the second data to be recognized does not contain function words and only includes explanation text related to the course, thereby further improving the accuracy of the subtitles in the course file.

[0011] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, the method also includes: removing function words in the course text to obtain a target text; generating a course file based on the course video and the course text, including: generating a course file based on the target text and the course video.

[0012] In an embodiment of the present application, by removing function words from the course text, the target text from which the function words are removed only contains explanation text related to the course, thereby further improving the accuracy of the subtitles in the course file.

[0013] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, the course file includes multiple course sub-files, the course text includes multiple course sub-texts, and the course video includes multiple course sub-videos; generating a course file based on the course video and the course text, the method includes: segmenting the course text to obtain multiple course sub-texts; editing the course video based on the multiple course sub-texts to obtain multiple course sub-videos; generating multiple course sub-files based on the multiple course sub-texts and the multiple course sub-videos.

[0014] In an embodiment of the present application, multiple course sub-files can be generated based on multiple course sub-texts and multiple course sub-videos. Since the course sub-files can be generated based on the course sub-texts and the course sub-videos corresponding to the course sub-texts, the generated course files can meet the user's needs for listening to lectures or be edited according to the key content of the course, thereby improving the flexibility of course sub-file generation.

[0015] In combination with the first aspect and the above-mentioned implementation manner, in some implementation manners of the first aspect, the method further includes: detecting whether the ordering of multiple segmentations of the course text meets the preset ordering conditions; if it is detected that the ordering of the multiple segmentations does not meet the preset ordering conditions, adjusting the ordering of the multiple segmentations to meet the preset ordering conditions; or, if it is detected that the ordering of the multiple segmentations does not meet the preset ordering conditions, sending a reminder message to the user, so that the user adjusts the ordering of the multiple segmentations to meet the preset ordering conditions according to the reminder message; wherein the reminder message is used to prompt the user that the ordering of the multiple segmentations does not meet the preset ordering conditions.

[0016] In an embodiment of the present application, if it is detected that the order of multiple word segments does not meet the preset ordering conditions, the order of the multiple word segments can be adjusted to meet the preset ordering conditions, that is, when the order of multiple word segments is disordered, the disordered order is adjusted to the correct order, so that the sentences in the reading course text can be more fluent, further improving the accuracy of the subtitles in the micro-class video.

[0017] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, audio data collected by a microphone in a terminal device is obtained, and the method includes: obtaining audio data collected by the microphone according to a preset duration; or, obtaining audio data collected by the microphone according to a preset page number interval, wherein the preset page number interval is determined by the size of the image data.

[0018] In an embodiment of the present application, obtaining audio data collected by a microphone according to a preset duration or preset page number interval can avoid uniformly obtaining volume data collected by the microphone after the course video recording is completed, which may easily result in an excessive amount of audio data and reduce the speed of recognizing audio data. Therefore, obtaining audio data collected by a microphone according to a preset duration or preset page number interval can reduce the amount of processing when recognizing audio data each time, thereby improving the efficiency of recognizing audio data.

[0019] In combination with the first aspect and the above-mentioned implementation manners, in some implementation manners of the first aspect, the method further includes: resampling the second data.

[0020] In an embodiment of the present application, the second data can be resampled so that the sampling rate of the second data after the resampled processing is reduced, which is more suitable for fast speech recognition scenarios; since the size of the second data is reduced by resampling, the recognition of the second data is faster, thereby improving the efficiency of generating subtitles in micro-class videos.

[0021] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, a course file is generated based on the course video and the course text, and the method includes: obtaining a first timestamp of the audio data collected by the microphone, and obtaining a second timestamp of the mixed data; determining the difference between the first timestamp and the second timestamp; controlling the audio data and the mixed data collected by the microphone to align according to the difference; and generating a course file based on the image data, the course text and the aligned mixed data.

[0022] In an embodiment of the present application, after the audio data and the mixed data collected by the microphone are aligned, it can be shown that the time axes of the audio data and the mixed data collected by the microphone are one-to-one corresponding. Since the mixed data and the image data in the video played by the terminal device are recorded at the same time, it can be shown that the time axes of the image data in the video played by the terminal device, the course text and the aligned mixed data are also one-to-one corresponding, thereby ensuring that the time of the image data, audio data and subtitles in the course file are synchronized, avoiding the possibility of the image data, audio data and subtitles in the course file being out of synchronization.

[0023] In a second aspect, a device for generating a course file is provided, and the device includes: an acquisition module: used to acquire audio data collected by a microphone in a terminal device and audio data and image data of a video played in the terminal device, wherein the audio data collected by the microphone does not include audio data of the video played; a mixing module: used to mix the audio data collected by the microphone and the audio data of the video played to obtain mixed data, wherein the mixing process is used to integrate the audio data collected by the microphone and the audio data of the video played, and the mixed data includes the audio data collected by the microphone and the audio data of the video played; a recording module: used to record the mixed data and image data to obtain a course video; an identification module: used to identify the audio data collected by the microphone to obtain a course text; a generation module: used to generate a course file based on the course video and the course text.

[0024] In combination with the second aspect, in some implementations of the second aspect, the device also includes: a copy module, after obtaining the audio data collected by the microphone in the terminal device, the copy module is used to copy the audio data collected by the microphone to obtain first data and second data, wherein the first data and the second data are the same; mixing the audio data collected by the microphone and the audio data of the played video to obtain mixed data, the mixing module is specifically used to: mix the first data and the audio data of the played video to obtain mixed data; and, recognize the audio data collected by the microphone to obtain a course text, the recognition module is specifically used to recognize the second data to obtain a course text; wherein the mixing process and the recognition process are performed simultaneously.

[0025] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the second data is identified and processed, and the identification module is specifically used to: obtain function words in the second data; remove function words in the second data; and identify and process the second data without function words.

[0026] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the device also includes: a removal module, used to remove function words in the course text to obtain a target text; generate a course file based on the course video and the course text, and the generation module is specifically used to: generate a course file based on the target text and the course video.

[0027] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the course file includes multiple course sub-files, the course text includes multiple course sub-texts, and the course video includes multiple course sub-videos; a course file is generated based on the course video and the course text, and the generation module is specifically used to: segment the course text to obtain multiple course sub-texts; edit the course video based on the multiple course sub-texts to obtain multiple course sub-videos; and generate multiple course sub-files based on the multiple course sub-texts and the multiple course sub-videos.

[0028] In combination with the second aspect and the above-mentioned implementation manner, in some implementation manners of the second aspect, the device also includes: a detection module, used to detect whether the order of multiple segmentations of the course text meets the preset sorting conditions; if it is detected that the order of multiple segmentations does not meet the preset sorting conditions, the order of the multiple segmentations is adjusted to meet the preset sorting conditions; or, if it is detected that the order of multiple segmentations does not meet the preset sorting conditions, a reminder message is sent to the user, so that the user adjusts the order of the multiple segmentations to meet the preset sorting conditions according to the reminder message; wherein the reminder message is used to prompt the user that the order of the multiple segmentations does not meet the preset sorting conditions.

[0029] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, audio data collected by a microphone in a terminal device is obtained, and the acquisition module is specifically used to: obtain the audio data collected by the microphone according to a preset duration; or, obtain the audio data collected by the microphone according to a preset page number interval, wherein the preset page number interval is determined by the size of the image data.

[0030] In combination with the second aspect and the above-mentioned implementation manners, in some implementation manners of the second aspect, the device further includes: a resampling module, configured to perform resampling processing on the second data.

[0031] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, a course file is generated based on the course video and the course text, and the generation module is specifically used to: obtain a first timestamp of the audio data collected by the microphone, and obtain a second timestamp of the mixed data; determine the difference between the first timestamp and the second timestamp; control the alignment of the audio data and the mixed data collected by the microphone according to the difference; generate a course file based on the image data, the course text and the aligned mixed data.

[0032] In a third aspect, a terminal device is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the terminal device executes the method in the first aspect or any possible implementation of the first aspect.

[0033] In a fourth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0034] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a schematic diagram of a scenario for generating a micro-course video provided in an embodiment of the present application;

[0036] Figure 2 It is a schematic diagram of the principle of generating a course file provided in an embodiment of the present application;

[0037] Figure 3 It is a flow chart of a method for generating a course file provided in an embodiment of the present application;

[0038] Figure 4 is a schematic diagram of a display method of subtitles on a terminal device screen provided by an embodiment of the present application;

[0039] Figure 5 It is a flowchart of another method for generating a course file provided in an embodiment of the present application;

[0040] Figure 6 It is a structural schematic diagram of a device for generating a course file provided in an embodiment of the present application;

[0041] Figure 7 It is a structural diagram of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0043] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0044] Figure 1 It is a schematic diagram of a scenario for generating a micro-course video provided in an embodiment of the present application.

[0045] For example, Figure 1 As shown, Figure 1 (a) includes a first interface 110, which may represent a trigger interface for starting to record a micro-class video. The interface may include a "start recording" button for triggering to start recording a micro-class video; Figure 1 (b) includes a second interface 120, and the second interface 110 may represent an interface during the recording of a micro-class video, and the interface may include information such as the recording duration; Figure 1 (c) includes a third interface 130, which may represent a trigger interface for ending the recording of a micro-class video, and the interface may include an "end recording" button for triggering the end of the recording of the micro-class video; Figure 1 (d) includes a fourth interface 140, which can represent an interface for playing a micro-class video after recording, and includes a micro-class video and corresponding subtitles. In addition, the first interface 110, the second interface 120, the third interface 130 and the fourth interface 140 are all displayed in the terminal device, and the terminal device can be used to collect the lecturer's (for example, a teacher) explanation of the course, as well as the audio data (which may include the content of the course or the audio data played by other software) and image data played by itself, and generate a micro-class video based on these audio data and image data.

[0046] It should be understood that the terminal device includes but is not limited to devices such as smartphones, tablet computers, video playback devices, and notebooks. In addition, the operating system of the terminal includes but is not limited to a system deeply developed based on the Android system, an IOS system developed by Apple, including a system deeply developed based on the IOS system or other systems. In other words, any device capable of collecting audio data and image data, and processing audio data and image data to generate micro-class videos can be used as a terminal device, and the embodiments of the present application do not limit this.

[0047] In the related art, when making a micro-class video, the lecturer can click on the start recording mark in the terminal device (for example, the first interface 110), so that the terminal device starts collecting the lecturer's explanation of the course, as well as the audio data and image data played by itself from the current moment (for example, the second interface 120); after the recording is completed, the lecturer can click on the end recording mark in the terminal device (for example, the third interface 130) The end recording button ends the recording of the micro-class video.

[0048] In order to facilitate the listeners (e.g., students) to learn better through micro-class videos, the audio data about the explanation content in the micro-class video can be extracted, and the extracted audio data can be identified to obtain the corresponding text; afterwards, the text is added to the micro-class video to generate the corresponding explanation subtitles of the course, for example, as shown in the fourth interface 140, while the micro-class video is played on the left side of the terminal device, the lecturer's explanation subtitles of the course can be displayed on the right side of the terminal device, so that the listeners can understand the course explanation content more clearly and intuitively, and improve the learning effect. However, since the micro-class video may include the audio data played by the terminal device itself, the subtitles may contain subtitles corresponding to the audio data played by the terminal device itself, and not just the subtitles corresponding to the lecturer's explanation of the course content, which is easy to cause interference to the listeners when watching the subtitles of the course explanation.

[0049] In order to solve the above problems, the present application provides a method for generating a course file, an apparatus for generating a course file, a terminal device and a computer-readable storage medium, which are specifically described in the following content.

[0050] Figure 2 It is a schematic diagram of the principle of course file generation provided in the embodiment of the present application.

[0051] For example, Figure 2 As shown, Figure 2 Specifically include the following:

[0052] S201 is the audio data collected by the microphone in the terminal device, S202 is the audio data played by the video of the terminal device, and S203 is the image data played by the video of the terminal device.

[0053] S204 encodes the audio data collected by the microphone through the audio encoder A, and inputs the audio data collected by the microphone after being encoded by the audio encoder A into the network speech recognition service module 205 or the local speech recognition module 206 for recognition to obtain the text corresponding to the audio data collected by the microphone.

[0054] And, S207 mixes the audio data collected by the microphone and the audio data played by the video, inputs the mixed audio data (which can be recorded as "mixed data") into the audio encoder B, S208 encodes the mixed data through the audio encoder B, and S209 outputs the encoded mixed data; at the same time, S210 encodes the input image data through the video encoder, and S211 outputs the encoded image data; then, S212 generates a micro-class video from the encoded mixed data and image data. Further, S213 generates a micro-class file through the micro-class video and text.

[0055] Figure 3 is a flow chart of a method for generating a course file provided in an embodiment of the present application. Figure 1 Executed by the terminal device in.

[0056] For example, Figure 3 As shown, the method 300 includes the following implementation process:

[0057] S310, obtaining audio data collected by a microphone in a terminal device and audio data and image data of a video played in the terminal device.

[0058] The audio data collected by the microphone does not include the audio data of the played video, or it can be said that the audio data collected by the microphone only includes the explanation sound.

[0059] It should be understood that the audio data of the video played in the terminal device refers to the audio data directly obtained from the system of the terminal device, and is not the audio data collected by the microphone; that is, the audio data of the video played does not need to be collected by the microphone, but can be directly retrieved from the system of the terminal device.

[0060] Optionally, if the audio data collected by the microphone includes the explanation sound and the audio played by the terminal device, the explanation sound can be extracted separately based on the teacher's voice feature information.

[0061] Optionally, the audio data collected by the microphone may include the sound of one or more teachers explaining the course at the current moment, and the explanation sound may also include the sound of students and teachers interacting during the lecture; the time of playing the video in the terminal device is synchronized with the time of the explanation sound. Therefore, the audio data and image data of the video played in the terminal device are also synchronized with the time of the explanation sound. In addition, the audio data of the video played may include the audio played by itself (for example, background music or message reminders), and the audio data can be directly obtained through the internal of the terminal device.

[0062] Optionally, the audio data collected by the microphone may be acquired according to a preset duration; or, the audio data collected by the microphone may be acquired according to a preset page number interval.

[0063] The preset page interval is determined by the size of the image data. Since the size of the image data can correspond to the number of pages, for example, if the size of an image data is 60kb, the corresponding page number is 5 when the size of the image data is 300kb. Also, the preset duration can represent a fixed duration, for example, 5S, or 10S, which is not limited in the embodiment of the present application.

[0064] For example, obtain the audio data collected by the microphone within a fixed duration of 5S (which can be recorded as "1.aac, 2.aac...n.aac"), and the duration of the previous collection and the next collection is continuous; or, obtain the audio data collected by the microphone corresponding to every 5 page numbers (which can be recorded as "P1.aac, P2.aac...Pn.aac").

[0065] In an embodiment of the present application, obtaining audio data collected by a microphone according to a preset duration or preset page number interval can avoid uniformly obtaining volume data collected by the microphone after the course video recording is completed, which may easily result in an excessive amount of audio data and reduce the speed of recognizing audio data. Therefore, obtaining audio data collected by a microphone according to a preset duration or preset page number interval can reduce the amount of processing when recognizing audio data each time, thereby improving the efficiency of recognizing audio data.

[0066] S320, mixing the audio data collected by the microphone and the audio data of the played video to obtain mixed data.

[0067] The audio mixing process may be used to integrate the audio data collected by the microphone and the audio data of the played video, and the audio mixing data may include the audio data collected by the microphone and the audio data of the played video.

[0068] Exemplarily, the audio data collected by the microphone and the audio data of the played video can be integrated into a stereo track or a mono track, so that the mixed data obtained after the mixing process can include the audio data collected by the microphone and the audio data of the played video.

[0069] S330, recording the mixed audio data and the image data to obtain a course video.

[0070] Exemplarily, the mixed audio data and the image data in the video played in the terminal can be recorded to obtain the video at the current moment (which can be recorded as a "course video"). As the recording time increases, a course video of a certain length can be obtained.

[0071] S340, recognize and process the audio data collected by the microphone to obtain the course text.

[0072] Exemplarily, the audio data corresponding to the explanation sound collected by the microphone can be recognized and processed to obtain one or more characters corresponding to the audio data, and these characters are combined into text (which can be recorded as "course text").

[0073] It should be understood that in order to ensure the synchronous generation of the course video and the course text, S320 and S340 can be performed simultaneously; in addition, the embodiments of the present application do not exclude the execution steps of S320 and S340 being to execute S320 first and then S340, or to execute S340 first and then S320.

[0074] Optionally, after obtaining the audio data collected by the microphone, the audio data collected by the microphone can also be copied to obtain first data and second data, wherein the first data and the second data are the same. The audio data collected by the microphone and the audio data of the played video are mixed to obtain mixed data, including: mixing the first data and the audio data of the played video to obtain mixed data; and the audio data collected by the microphone is recognized to obtain the course text, including: recognizing the second data to obtain the course text; wherein the mixing process and the recognition process are performed simultaneously.

[0075] By copying the audio data collected by the microphone, two identical audio data can be obtained, for example, the same first data and second data; then one data (for example, the first data) and the audio data of the played video can be mixed to obtain mixed data after mixing; at the same time, the other data (for example, the second data) can be input into the video player. Figure 2 The network speech recognition service module or the local speech recognition module shown performs recognition to obtain the course text corresponding to the second data.

[0076] In an embodiment of the present application, by copying and processing the audio data collected by the microphone to obtain two identical audio data (i.e., first data and second data), it is possible to achieve mixing of the first data and the audio data of the played video while also identifying the second data, thereby synchronously generating the course video and course text, thereby improving the real-time generation of subtitles in the course file.

[0077] Since the audio data is too large, the recognition processing may be slow. Therefore, the second data can be resampled so that the sampling rate of the second data after the resampled processing is reduced, which is more suitable for fast speech recognition scenarios. Since the size of the second data is reduced by resampling, the recognition of the second data is faster, thereby improving the efficiency of generating subtitles in micro-class videos.

[0078] Optionally, noise elimination processing can also be performed on the second data to avoid interference caused by audio data that is not the voice of the explanation collected by the microphone to the recognition processing.

[0079] Optionally, optimization processing can also be performed on the second data. For example, convert dialects into standard Mandarin, or optimize the loudness of the second data so that the loudness is the same, to avoid uneven sound sizes in the finally output voice.

[0080] In one possible implementation, when performing recognition processing on the second data, function words in the second data can also be obtained; the function words in the second data can be removed; and recognition processing can be performed on the second data after removing the function words.

[0081] Exemplarily, during the process of performing recognition processing on the second data, one or more function words included in the second data can be obtained. For example, modal particles such as "ah", "la", and "ne", etc.; the obtained function words such as "ah" are removed from the second data, and recognition processing is performed on the second data after removing the function words such as "ah", that is, the second data after removing the function words only contains audio data related to the course. Specifically, when not removing function words from the second data, the recognized course text may be "In this class, we learn addition ah", while when removing function words from the second data, the recognized course text is "In this class, we learn addition", that is, the recognized course text only contains the explanatory text related to the course.

[0082] In the embodiments of the present application, since function words in the second data are removed when performing recognition processing on the second data, there are no function words in the second data for recognition, and only the explanatory text related to the course is included, thereby further improving the accuracy of the subtitles in the course file.

[0083] In another possible implementation, it is detected whether the sorting of multiple word segments of the course text meets a preset sorting condition; if it is detected that the sorting of the multiple word segments does not meet the preset sorting condition, the sorting of the multiple word segments is adjusted to meet the preset sorting condition; or, if it is detected that the sorting of the multiple word segments does not meet the preset sorting condition, a reminder message is sent to the user so that the user adjusts the sorting of the multiple word segments to meet the preset sorting condition according to the reminder message.

[0084] Among them, the preset sorting condition can represent the arrangement order of the subject, predicate, and object, and can be obtained through training of a convolutional network model. The embodiments of the present application do not limit this.

[0085] Among them, the reminder message is used to prompt the user that the sorting of the multiple word segments does not meet the preset sorting condition. The reminder message can be to display the part where the sorting of the multiple word segments does not meet the preset sorting condition on the terminal screen, or through voice reminder, or text reminder, etc. The embodiments of the present application do not limit this.

[0086] Exemplarily, a trained convolutional network model can be used to detect whether the order of multiple participles (for example, ) included in the course text meets the preset ordering conditions; if it is detected that the order of "We learn addition in this class" does not meet the preset ordering conditions, the order of "We learn addition in this class" can be adjusted to "We learn addition in this class", that is, when the order of multiple participles is disordered, the disordered order is adjusted to the correct order, so that the order adjustment of multiple participles can meet the preset ordering conditions.

[0087] Alternatively, if it is detected that the order of "In this lesson we learn addition" does not meet the preset ordering conditions, a text message can be sent to the user, allowing the user to adjust the order of multiple participles according to the text message, that is, when the order of multiple participles is disordered, the disordered order is adjusted to the correct order, so that the order adjustment of multiple participles can meet the preset ordering conditions.

[0088] In an embodiment of the present application, if it is detected that the order of multiple word segments does not meet the preset ordering conditions, the order of the multiple word segments can be adjusted to meet the preset ordering conditions, that is, when the order of multiple word segments is disordered, the disordered order is adjusted to the correct order, so that the sentences in the reading course text can be more fluent, further improving the accuracy of the subtitles in the micro-class video.

[0089] S350, generates course files based on course videos and course texts.

[0090] Exemplarily, the obtained course video and course text may be synthesized to form a complete file (which may be recorded as a "course file"), thereby generating a course file (which may also be recorded as a "micro-course video").

[0091] Furthermore, the generated course files may be saved in the terminal device, and / or the generated course files may be uploaded to a cloud server.

[0092] Optionally, after obtaining the course text, function words in the course text may be removed to obtain a target text; and a course file may be generated according to the course video and the course text, including: generating a course file according to the target text and the course video.

[0093] For example, after obtaining the course text, the course file is detected. It is detected that the course text includes one or more function words, such as modal particles "ah", "la", and "ne", etc. The detected function words such as "ah" are removed from the course text, and the course text after removing the function words is used as the target text. Specifically, when not removing the function words from the course text, the course text may be "In this class, we learn addition ah", while when removing the function words from the course text, the course text is "In this class, we learn addition", that is, the target text only contains the explanatory text related to the course.

[0094] In the embodiments of the present application, by removing the function words from the course text, the target text after removing the function words only contains the explanatory text related to the course, thereby further improving the accuracy of the subtitles in the course file.

[0095] Optionally, the first timestamp of the audio data collected by the microphone can be obtained, and the second timestamp of the mixed audio data can be obtained; the difference between the first timestamp and the second timestamp is determined; according to the difference, the audio data collected by the microphone and the mixed audio data are controlled to be aligned; according to the image data, the course text, and the aligned mixed audio data, a course file is generated.

[0096] Exemplarily, a certain time point (for example, the first timestamp) in the audio data collected by the microphone is obtained as 02:15, and a certain time point (for example, the second timestamp) in the mixed audio data is obtained as 02:14. It can be determined that the difference between the first timestamp and the second timestamp is 00:01. If the difference is less than or equal to the preset difference (for example, 00:05), then the audio data collected by the microphone and the mixed audio data can be aligned. After the time points of the audio data collected by the microphone and the mixed audio data are aligned, it can be shown that the time point of the course text and the time point of the aligned mixed audio data are also aligned; since the mixed audio data and the image data in the video played by the terminal device are recorded simultaneously, it can further be shown that the image data, the course text, and the aligned mixed audio data in the video played by the terminal device are also aligned. Thus, the image data, the course text, and the processed mixed audio data in the video played by the terminal device are synthesized to generate a course file.

[0097] In an embodiment of the present application, after the audio data and the mixed data collected by the microphone are aligned, it can be shown that the time axes of the audio data and the mixed data collected by the microphone are one-to-one corresponding. Since the mixed data and the image data in the video played by the terminal device are recorded at the same time, it can be shown that the time axes of the image data in the video played by the terminal device, the course text and the aligned mixed data are also one-to-one corresponding, thereby ensuring that the time of the image data, audio data and subtitles in the course file are synchronized, avoiding the possibility of the image data, audio data and subtitles in the course file being out of synchronization.

[0098] Optionally, when playing a course file, the course text can be played in correspondence with the course video, that is, the timeline of the course text and the timeline of the course video are one-to-one corresponding, and when playing the course file, the course file can be used as an annotation to explain the corresponding content of the course video; it can also be used as subtitles or barrage to explain the corresponding content of the course video, and the comparison in the embodiments of the present application is not limited.

[0099] Exemplarily, a course file may include multiple course sub-files, a course text may include multiple course sub-texts, and a course video may include multiple course sub-videos; the course text may be segmented to obtain multiple course sub-texts; the course video may be edited based on the multiple course sub-texts to obtain multiple course sub-videos; and multiple course sub-files may be generated based on the multiple course sub-texts and the multiple course sub-videos.

[0100] For example, the course text can be segmented according to a preset duration or a preset page number interval to obtain course sub-text 1 (which can be recorded as "1.txt"), course sub-text 2 (which can be recorded as "2.txt"), course sub-text 3 (which can be recorded as "3.txt")...course sub-text n (which can be recorded as "n.txt"); then based on the obtained course sub-text 1, course sub-text 2, course sub-text 3...course sub-text n, the course video is edited to obtain course sub-video 1, course sub-video 2, course sub-video 3...course sub-video n corresponding to the multiple course sub-texts, wherein course sub-text 1 corresponds to course sub-video 1, course sub-text 2 corresponds to course sub-video 2, course sub-text 3 corresponds to course sub-video 3...course sub-text n corresponds to course sub-video n. It should be understood that the time point at which course sub-text n corresponds to course sub-video n is corresponding.

[0101] Furthermore, course sub-text 1, course sub-text 2, course sub-text 3, ... course sub-text n and corresponding course sub-video 1, course sub-video 2, course sub-video 3, ... course sub-video n can be synthesized to obtain course sub-file 1, course sub-file 2, course sub-file 3, ... course sub-file n. For example, course sub-text 1 and course sub-video 1 can be synthesized to obtain course sub-file 1, and so on, which will not be described in detail here.

[0102] Optionally, after the course file is generated, the course file may be segmented according to a preset duration or a preset page number interval, thereby obtaining a plurality of course sub-files.

[0103] In an embodiment of the present application, multiple course sub-files can be generated based on multiple course sub-texts and multiple course sub-videos. Since the course sub-files can be generated based on the course sub-texts and the course sub-videos corresponding to the course sub-texts, the generated course files can meet the user's needs for listening to lectures or be edited according to the key content of the course, thereby improving the flexibility of course sub-file generation.

[0104] For example, when playing a course file, the subtitles in the course file can be as follows: Figure 1 The fourth interface 140 shown in (d) is displayed on the right side of the terminal device screen, and can also be displayed as shown in FIG. Figure 4 The fifth interface 410 shown in (a) is displayed on the left side of the terminal device screen; Figure 4 The sixth interface 420 shown in (b) is displayed at the bottom of the terminal device screen, or can also be displayed at the top of the terminal device screen; or, it can also be displayed as shown in FIG. Figure 4 The seventh interface 430 shown in (c) and Figure 4 The eighth interface 440 shown in (d) in FIG. 1 displays subtitles in chronological order, and the display time of the subtitles corresponds to the time when the audio data collected by the microphone is output.

[0105] It should be understood that the subtitles can be displayed in other ways on the terminal device screen, and the screen ratio of the subtitles when displayed on the terminal device screen and the screen ratio of the image when the course file is played can be adjusted according to user needs, and the embodiments of the present application do not limit this.

[0106] exist Figure 3In the method 300 shown, in the process of generating a course file (i.e., a micro-course video), mixed data is obtained by mixing the audio data collected by the microphone and the audio data of the video played in the terminal device, and the mixed data and the image data of the terminal are recorded to obtain the corresponding course video; since the audio data collected by the microphone does not include the audio data of the video played, when the text generated by recognizing the audio data collected by the microphone, the audio data of the video played in the terminal device will not be recognized, so that the audio data of the video played in the terminal device will not interfere with the text recognized by the audio data collected by the microphone, so that the recognized course text only includes the text corresponding to the audio data collected by the microphone, but does not include the text corresponding to the audio data of the video played in the terminal device, thereby improving the accuracy of the subtitles in the micro-course video.

[0107] It should be noted that the solution provided in the embodiments of the present application is only introduced for the generation of course files, but it does not affect the implementation of the solution of the present application in similar scenarios, such as live broadcast scenarios, or song recording scenarios, etc.

[0108] Figure 5 is a flow chart of another method for generating a course file provided in an embodiment of the present application. Figure 1 Executed by the terminal device in.

[0109] For example, Figure 5 As shown, the method 500 includes the following implementation process:

[0110] S501, detecting that a user triggers a start recording button of a course file.

[0111] For example, the terminal device can detect that when the lecturer clicks Figure 1 The start recording button in the first interface 110 shown in (a) above is clicked, and when it is detected that the start recording button is triggered, the course file generation process is entered.

[0112] S502, obtaining audio data collected by a microphone, and S503, obtaining audio data played by a terminal device, and S504, obtaining image data played by a terminal device. It should be understood that S502, S503 and S504 are performed synchronously.

[0113] For example, the microphone in the terminal device can collect the current explanation sound, such as pulse modulation coding (PCM) data; and the terminal device's own audio data when playing a video, such as background music; and the terminal device's own image data when playing a video, such as color coding data. After obtaining the audio data collected by the microphone, S504 can be executed.

[0114] S505: Copy and process the audio data collected by the microphone to obtain first data and second data.

[0115] Exemplarily, the acquired audio data collected by the microphone is copied to obtain completely identical first data and second data.

[0116] S506, input the second data into audio encoder A.

[0117] Exemplarily, the second data is input into the audio encoder A, and the second data encoded by the audio encoder A is input into Figure 2 The network speech recognition service module or the local speech recognition module shown is used to execute S506.

[0118] S507, performing recognition processing on the encoded second data to obtain the course text.

[0119] Exemplarily, the second data encoded by the audio encoder A is subjected to text recognition processing to obtain text corresponding to the second data, thereby obtaining the course text corresponding to the second data. For example, the second data is segmented and output according to a preset time length or preset page number interval, and a plurality of recognized course subtexts are obtained, namely, course subtext 1, course subtext 2, course subtext 3, ... course subtext n.

[0120] After the audio data collected by the microphone is copied to obtain the first data, S508 may be executed.

[0121] S508: Mix the first data and the audio data of the video playback to obtain mixed data.

[0122] Exemplarily, the first data and the audio data of the video playback may be mixed, that is, the first data and the audio data of the video playback may be integrated to obtain mixed data.

[0123] S509: Input the mixed audio data into audio encoder B.

[0124] Exemplarily, the mixed data obtained through the mixing process can be input into the audio encoder B.

[0125] It should be noted that audio encoder A and audio encoder B can use different configurations. For example, audio encoder A can use 16000 sampling rate monophonic encoding to make the volume of the encoded second data smaller and more suitable for network transmission and speech recognition; while audio encoder B can use 44100 sampling rate dual-channel encoding to ensure the clarity of the audio signal, thereby improving the playback effect of the mixed data. In other words, the sampling rate of audio encoder B is higher than the sampling rate of audio encoder A.

[0126] S510, inputting image data into a video encoder.

[0127] Exemplarily, the image data of the video playback acquired in S503 may be input into a video encoder.

[0128] S511, recording the encoded mixed audio data and image data to obtain a course video.

[0129] Exemplarily, the mixed audio data encoded by the audio encoder B in S509 and the image data encoded by the video encoder in S510 are recorded to obtain a course video.

[0130] It should be understood that S506 and S507, and S508, S509 and S510, and S511 are performed synchronously, thereby ensuring the real-time generation of the course video and course text.

[0131] S512, synthesizing the course text and the course video to generate a course file.

[0132] Exemplarily, the course text obtained in S507 and the course video obtained in S511 may be synthesized to obtain a course file, which may be saved and displayed.

[0133] S513, segmenting the course file to obtain a plurality of course sub-files.

[0134] Exemplarily, after a course file is generated, it can be edited according to the listening needs or key contents of the course, thereby generating multiple course sub-files.

[0135] S514, saving and displaying multiple course sub-files.

[0136] Exemplarily, after a plurality of course sub-files are generated, the plurality of course sub-files may be saved and / or displayed.

[0137] It should be understood that Figure 5 All the steps involved in Figure 3 The corresponding steps have been introduced in the introduction and will not be repeated here.

[0138] Figure 6 It is a structural diagram of a device for generating course files provided in an embodiment of the present application.

[0139] For example, Figure 6 As shown, the device 600 includes:

[0140] Acquisition module 610: used to acquire audio data collected by a microphone in a terminal device and audio data and image data of a video played in the terminal device, wherein the audio data collected by the microphone does not include audio data of the video played;

[0141] Mixing module 620: used for mixing audio data collected by the microphone and audio data of the played video to obtain mixed data, wherein the mixing process is used to integrate the audio data collected by the microphone and the audio data of the played video, and the mixed data includes the audio data collected by the microphone and the audio data of the played video;

[0142] Recording module 630: used to record the mixed audio data and the image data to obtain a course video;

[0143] Recognition module 640: used to recognize and process the audio data collected by the microphone to obtain the course text;

[0144] Generating module 650: used to generate course files according to course videos and course texts.

[0145] Optionally, the device 600 also includes: a copy module 660, after obtaining the audio data collected by the microphone in the terminal device, the copy module 660 is used to copy the audio data collected by the microphone to obtain first data and second data, wherein the first data and the second data are the same; the audio data collected by the microphone and the audio data of the played video are mixed to obtain mixed data, and the mixing module 620 is specifically used to: mix the first data and the audio data of the played video to obtain mixed data; and, recognize the audio data collected by the microphone to obtain a course text, and the recognition module 640 is specifically used to recognize the second data to obtain a course text; wherein the mixing process and the recognition process are performed simultaneously.

[0146] In a possible implementation, the second data is subjected to recognition processing, and the recognition module 640 is specifically used to: obtain function words in the second data; remove the function words in the second data; and perform recognition processing on the second data from which the function words are removed.

[0147] Optionally, the device 600 also includes: a removal module 670, which is used to remove function words in the course text to obtain a target text; generate a course file based on the course video and the course text, and the generation module 650 is specifically used to: generate a course file based on the target text and the course video.

[0148] In one possible implementation, the course file includes multiple course sub-files, the course text includes multiple course sub-texts, and the course video includes multiple course sub-videos; the course file is generated based on the course video and the course text, and the generation module 650 is specifically used to: segment the course text to obtain multiple course sub-texts; edit the course video based on the multiple course sub-texts to obtain multiple course sub-videos; generate multiple course sub-files based on the multiple course sub-texts and the multiple course sub-videos.

[0149] Optionally, the device 600 also includes: a detection module 680, which is used to detect whether the ordering of multiple segmentations of the course text meets the preset ordering conditions; if it is detected that the ordering of multiple segmentations does not meet the preset ordering conditions, the ordering of the multiple segmentations is adjusted to meet the preset ordering conditions; or, if it is detected that the ordering of multiple segmentations does not meet the preset ordering conditions, a reminder message is sent to the user, so that the user adjusts the ordering of the multiple segmentations to meet the preset ordering conditions according to the reminder message; wherein the reminder message is used to prompt the user that the ordering of the multiple segmentations does not meet the preset ordering conditions.

[0150] In one possible implementation, audio data collected by a microphone in a terminal device is obtained, and the acquisition module 610 is specifically used to: obtain audio data collected by the microphone according to a preset duration; or obtain audio data collected by the microphone according to a preset page number interval, wherein the preset page number interval is determined by the size of the image data.

[0151] Optionally, the device 600 further includes: a resampling module 690, configured to perform resampling processing on the second data.

[0152] In one possible implementation, a course file is generated based on the course video and course text, and the generation module 650 is specifically used to: obtain a first timestamp of the audio data collected by the microphone, and obtain a second timestamp of the mixed data; determine the difference between the first timestamp and the second timestamp; control the alignment of the audio data and the mixed data collected by the microphone according to the difference; generate a course file based on the image data, the course text and the aligned mixed data.

[0153] It should be noted that the course file generation device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when executing the course file generation method. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0154] In addition, the device for generating course files provided in the above embodiment and the method for generating course files embodiment belong to the same concept, so for details not disclosed in the device embodiment of this specification, please refer to the embodiment of the method for generating course files mentioned above in this specification, which will not be repeated here.

[0155] Figure 7 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application.

[0156] For example, Figure 7 As shown, the terminal device 700 includes: a memory 710 and a processor 720, wherein the memory 710 stores an executable program code 7101, and the processor 720 is used to call and execute the executable program code 7101 to perform a method for generating a course file.

[0157] The present application can divide the functional modules of the terminal device according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0158] In the case of dividing each functional module according to each function, the terminal device may include: an acquisition module, a mixing module, a recording module, a recognition module, a generation module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0159] The terminal device provided in this application is used to execute the above-mentioned method for generating a course file, and thus can achieve the same effect as the above-mentioned implementation method.

[0160] In the case of using an integrated unit, the terminal device may include a processing module and a storage module. The processing module may be used to control and manage the actions of the terminal device. The storage module may be used to support the terminal device to execute mutual program codes and data.

[0161] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits disclosed in conjunction with the present application. The processor may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0162] The present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method of any of the above embodiments are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD (Digital Video Disc), a CD-ROM (Compact Disc Read-Only Memory), a micro drive and a magneto-optical disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), an EPROM (Erasable Programmable Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory), a DRAM (Dynamic Random Access Memory), a VRAM (Video Random Access Memory), a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0163] The present application also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement a method for generating a course file in the above-mentioned embodiment.

[0164] In addition, the terminal device provided in the embodiments of the present application may specifically be a chip, component or module, and the terminal device may include a connected processor and memory; wherein the memory is used to store instructions, and when the terminal device is running, the processor may call and execute instructions so that the chip executes a method for generating a course file in the above-mentioned embodiment.

[0165] Among them, the terminal device, computer-readable storage medium, computer program product or chip provided in this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0166] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0167] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0168] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for generating a course document, characterized in that: The method comprises: Acquire audio data collected by a microphone in a terminal device and audio data and image data of a video played in the terminal device, wherein the audio data collected by the microphone does not include the audio data of the video played; Performing a mixing process on the audio data collected by the microphone and the audio data of the played video to obtain mixed data, wherein the mixing process is used to integrate the audio data collected by the microphone and the audio data of the played video, and the mixed data includes the audio data collected by the microphone and the audio data of the played video; Recording the mixed audio data and the image data to obtain a course video; Recognize and process the audio data collected by the microphone to obtain the course text; Generate a course file based on the course video and course text.

2. The method according to claim 1, characterized in that After obtaining the audio data collected by the microphone in the terminal device, the method further includes: Copying the audio data collected by the microphone to obtain first data and second data, wherein the first data and the second data are the same; The mixing the audio data collected by the microphone and the audio data of the played video to obtain the mixed data includes: mixing the first data and the audio data of the played video to obtain the mixed data; and, The performing recognition processing on the audio data collected by the microphone to obtain the course text includes: performing the recognition processing on the second data to obtain the course text; The mixing process and the recognition process are performed synchronously.

3. The method according to claim 2, characterized in that The performing the identification process on the second data includes: Acquire function words in the second data; removing the function words in the second data; The recognition process is performed on the second data from which the function words are removed.

4. The method according to claim 1, characterized in that: Also includes: removing function words from the course text to obtain a target text; Generating a course file according to the course video and course text includes: The course file is generated according to the target text and the course video.

5. The method according to any one of claims 1 to 3, characterized in that The course file includes a plurality of course sub-files, the course text includes a plurality of course sub-texts, and the course video includes a plurality of course sub-videos; and generating the course file according to the course video and the course text includes: Segmenting the course text to obtain a plurality of course sub-texts; According to the plurality of course sub-texts, the course video is edited to obtain a plurality of course sub-videos; A plurality of the course sub-files are generated according to the plurality of the course sub-texts and the plurality of the course sub-videos.

6. The method according to any one of claims 1 to 4, characterized in that Also includes: Detecting whether the order of multiple word segments of the course text meets the preset ordering conditions; If it is detected that the order of the multiple word segments does not meet the preset order condition, the order of the multiple word segments is adjusted to meet the preset order condition; or, If it is detected that the order of the multiple segmented words does not meet the preset ordering condition, a reminder message is sent to the user, so that the user adjusts the order of the multiple segmented words to meet the preset ordering condition according to the reminder message; The reminder information is used to prompt the user that the order of the multiple word segments does not meet the preset order condition.

7. The method according to claim 1, characterized in that The step of obtaining audio data collected by a microphone in the terminal device includes: Acquire the audio data collected by the microphone according to a preset time length; or, The audio data collected by the microphone is acquired according to a preset page number interval, wherein the preset page number interval is determined by the size of the image data.

8. The method according to claim 2, characterized in that: Also includes: The second data is resampled.

9. The method according to any one of claims 1 to 4 or 7 or 8, characterized in that Generating a course file according to the course video and course text includes: Obtaining a first timestamp of the audio data collected by the microphone, and obtaining a second timestamp of the mixed audio data; determining a difference between the first timestamp and the second timestamp; Controlling the audio data collected by the microphone and the mixed data to perform alignment processing according to the difference; The course file is generated according to the image data, the course text and the aligned mixed audio data.

10. A device for generating a course document, characterized in that: The device comprises: An acquisition module, used to acquire audio data collected by a microphone in a terminal device and audio data and image data of a video played in the terminal device, wherein the audio data collected by the microphone does not include the audio data of the video played; A mixing module, used for mixing the audio data collected by the microphone and the audio data of the played video to obtain mixed data, wherein the mixing process is used to integrate the audio data collected by the microphone and the audio data of the played video, and the mixed data includes the audio data collected by the microphone and the audio data of the played video; A recording module, used for recording the mixed audio data and the image data to obtain a course video; A recognition module, used to recognize and process the audio data collected by the microphone to obtain a course text; A generation module is used to generate a course file based on the course video and course text.

11. A terminal device, characterized in that: The terminal device comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the terminal device executes the method as claimed in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 9 is implemented.

Images