Method and system for generating micro-video content for foreign language learning, and learning method thereof

Abstract

A method (10) for generating micro-video content (7; 7.1, 7.2, 7.3) for language learning, the method being computer-implemented by extracting long-form videos from one or more video content sources (11) and comprising the steps of: - searching and identifying (12) from the relevant video content sources (11) a plurality of video content that contain the same target word or predefined phrase; - extracting from each identified video content a video clip (4) that contains the relevant target word or phrase; - enriching each extracted video clip with metadata and a contextualized test related to the relevant target word or phrase to provide micro-video content for learning.

Description

Detailed Description of the Invention

[0001] [Technical field] The present invention relates to a method and system for generating micro-video content for foreign language learning, and also to a foreign language learning system and language learning method using the method.

[0002] PRIOR ART The demand for foreign language learning is increasing worldwide, especially in the professional sector. Learners are looking for more motivation, personalized learning methods and prefer methods that have minimal impact on their personal time. Video-based learning platforms are now widespread and can be accessed on computers and mobile devices.

[0003] These videos provide audiovisual context to aid in language comprehension and learning, and help understand certain nuances of the language, such as the use of politeness expressions and the distinction between various accents.

[0004] However, the video format also has its drawbacks: each video contains several minutes of dialogue, not all of which may be useful to the learner, the difficulty level may be uneven, and some uncommon expressions may be included. Additionally, "flash cards" using the spaced repetition method have been used as a memorization tool: the spaced repetition algorithm optimizes the timing of flash card review and reduces the frequency of flash card review as the learner masters the material, thus preventing forgetting.

[0005] For example, US 2020 / 0143704 A1 (Liao et al), published May 7, 2020, discloses a computer system for improving video-based language learning, including a video server for storing videos in a learning language, and a meta-database for utterance translation in the video, character profiles, and matching with a learner profile.

[0006] The system can generate microcontent to better illustrate and contextualize the usage of new words and phrases to learners. However, even with these existing systems, learners struggle to organize their memorization and review, and are at high risk of forgetting what they have learned.

[0007] US 2018 / 0293912 Al(Ni), published on October 11, 2018, discloses a method for learning English using video clips, which implements a process of providing video clips at multiple time intervals over a specific period of time.

[0008] US 2021 / 0014575 Al (Selfors), published on January 14, 2021, discloses a method for learning a foreign language using video, whereby subtitles are displayed in a first target language or a second native language when the video is viewed. When a pause event occurs, subtitles in the target language and the native language are displayed simultaneously to facilitate comprehension.

[0009] During the pause, the user can choose to receive additional information indicating usage and context related to the target language subtitle. The user can navigate to the previous or next subtitle while the video is paused with the additional information. This method allows the user to generate video loops of self-definable duration and generate video clips by searching an entire database of subtitle texts.

[0010] The document US2019 / 0108773 (Wyner et al.) discloses a computerized method and system for teaching a foreign language. A user can generate flash cards containing words and phrases to be studied and select images to display during review. The invention relates to improved methods and systems for identifying and retrieving images for display, as well as improved methods for presenting flash cards for words with multiple definitions and improved configuration of a database for storing flash card data for multiple users.

[0011] US 2014 / 0134576 Al (Edge et al.) discloses a personalized language learning system and method for learning a language in a customizable manner for learners. The system and method define a learner model that contains personalized information about the learner and a language model that describes language-specific information. The combination of the learner model and the language model is used to assist the learner in learning the language. Specifically, the learner model and the language model are used to generate content for flashcards that are displayed to the learner. The learner's responses are used to update both the learner model and the language model. It also allows the learner to play games while focusing on specific language skills.

[0012] US 10,854,106 B2 (Geller et al.) discloses a computerized system for language learning that uses repetitive training to familiarize the learner with targeted words and language rules. The repetitive training presents training phrases based on specific words and rules, and the intervals between encounters with those words and rules vary based on the learner's previous correct, incorrect, or partially correct responses. This targeted reinforcement determines how to respond to the training phrases. A learning record helps the system identify aspects of language elements that the learner has not yet mastered.

[0013] US20120164609 (Kehoe) discloses a bilingual acquisition system that trains the auditory processing areas of a user's brain to recognize phonemes, syllables, and words in a foreign language. The system presents videos of words and phrases pronounced at a slow and normal speed, and provides an interface that allows the user to input the phonemes and words heard, providing immediate correct or incorrect feedback. The system utilizes spaced repetition learning.

[0014] Although video is a particularly interesting medium for understanding real language environments, as Liao et al. point out, learning from long videos has a problem: it is difficult for learners to review. Watching the same long video over and over again and memorizing the few useful phrases it contains is a very arduous task. Even if learners can extract the useful parts themselves, for example using Liao et al.'s system, it is difficult for them to organize their own review when they identify a hundred interesting phrases in ten videos. The main technical challenge with this problem lies in the difficulty of effectively reviewing and memorizing the vocabulary found in foreign language videos.

[0015] The main objective of the present invention is to provide a new method for more efficiently generating video content for foreign language learning, and another objective of the present invention is to provide a learning method for implementing these video contents.

[0016] DISCLOSURE OF THEINVENTION The object of the invention is achieved with a computerized process for generating micro-video content for language learning from a long-form video, the process comprising the following steps: - Search and identify multiple video contents from applicable sources that contain a specific target word or phrase belonging to a specified language. - Extract video clips that contain the specified target words or phrases from each video content. - Enrich each video clip with metadata and contextualized quizzes based on relevant target words or phrases. The resulting video clips constitute micro-video content for learning.

[0017] Enriching the data includes adding metadata including a contextualized quiz. The video clips thus enriched form the educational micro-video content (7) that is presented to the learner who selected the micro-video content for the first time and then again after each time the learner successfully completes the quiz after an increasing time interval ΔT determined by the spaced repetition algorithm.

[0018] The present invention proposes to facilitate memorization and eliminate administrative review by splitting long-form videos into video clips and providing quizzes on these clips. As mentioned by Liao et al., there is a process for generating micro-video content from single words, but the method of the present invention combines this micro-content generation mechanism with a spaced repetition learning system to solve the aforementioned technical problems of long-form videos.

[0019] The process of the present invention combines an automatic quiz generation algorithm with a spaced review algorithm.

[0020] An auto-generation algorithm automatically generates multiple quiz variations for each piece of content. There is no human intervention in this quiz generation process and quizzes can be generated for videos with any dialogue. Quizzes are cleverly designed to, for example, hide target words in the question text, require users to select the correct translation, or ask for correct pronunciation (and many more). Each quiz variation is presented based on the user's request, depending on their level and goals.

[0021] The objective of the present invention is achieved by using a spaced repetition based revision algorithm to organize the microcontent to be reviewed automatically by the learner every day, minimizing the possibility of forgetting and optimizing the use of time. The algorithm eliminates the need for the learner to open and review specific microcontent or videos by himself. The learner is provided with an already optimized review sequence of microcontent every day. Multiple iterations were performed to find the optimal settings of the algorithm, setting the most suitable revision interval for each quiz variation to most rapidly promote the learner's progress.

[0022] The search and identification step may be advantageously configured to identify video content that fits a particular context and contains pre-defined target words or phrases, and may include pre-filtering the video content based on multiple criteria with pre-filtering including analysis of audio transcription data or associated metadata of the video content.

[0023] The audio transcription data and / or metadata associated with the video content may be provided directly from the source of the relevant video content, which may advantageously be provided by an external video distribution service that offers an application programming interface (API).

[0024] Further, in accordance with the present invention, a system is proposed for generating micro-video content using one or more video content sources extracted from a long-form video, comprising: - A means of searching and identifying multiple video content sources that contain a particular target word or phrase. - A means for extracting, from each identified piece of video content, video clips that contain the corresponding target word or phrase. - A means to generate micro-video content for learning by adding metadata and contextualized tests (quizzes) based on relevant target words or phrases to each extracted video clip.

[0025] The reinforcing means is arranged to add an automatically generated contextualised test, such that the thus-reinforced video clip forms a micro-video content for learning, which test is provided for the first time to the learner who selected the micro-video content and thereafter again after an increasing time interval ΔT determined by the spaced repetition algorithm, each time the learner passes the test.

[0026] The search and identification means may be configured to identify video content that contains pre-defined target words or phrases that fit a particular context.

[0027] The video content searching and identification means may further include means for pre-filtering the video content based on a number of pre-defined filtering criteria, and means for analyzing audio transcription data and associated metadata of the video content.

[0028] According to another aspect of the present invention, there is provided a language learning method using micro-video content, the method being implemented in a communication device used by a learner and including the steps of: - In response to a learner making a request based on a particular target word or phrase, providing a list of micro-video content in the remote storage space that contains the word or phrase. - Select the micro content from the list. - View selected micro-video content on a communication device. - Run the included contextualized quizzes (tests) and enter answers from learners. - Processing the entered answers and providing a time indication for the next viewing of the micro-video content based on a spaced repetition algorithm. - Providing a display interface on the learner's communication device and suggesting that the learner review the video content just before they forget.

[0029] The learning method utilizes a spaced repetition algorithm to plan the sequence of review based on each learner's performance. Additionally, a pre-stage may be included to provide multiple video courses to the learner. Each course contains a group of micro-video content related to a particular topic.

[0030] Systems for language learning have also been proposed, including: - A set of micro-video content containing specific target words or phrases, the video content being accessible from a remote storage space. - one or more communication devices equipped with a digital application implementing the learning process according to the invention, said communication devices being connected to a storage space of micro-video content via a communication network.

[0031] Each communication device - Providing a list of relevant micro-video content when a learner requests it based on a specific target word or phrase. - Select the micro content from the list. - View automatically generated tests. - Based on your answers, we calculate review intervals to help you re-watch tests that contain the target words and phrases at the right time. - Display an interface to allow the learner to review the material again.

[0032] The microcontent generation process of the present invention allows the generation of flashcards (micro-quiz style questions and answers) from long videos. A video contains many phrases of various difficulty levels, and the video flashcard format allows the required phrases to be studied in isolation at the appropriate level. This format can be reviewed more effectively using spaced repetition algorithms.

[0033] These micro-quizzes may be generated manually by teachers or administrators, or automatically via algorithms that hide target words in subtitles and require learners to identify them in the audio track.

[0034] Learning focuses on target words, enabling learners to discover how the same target word is used in different contexts and develop a "feel" for the word.

[0035] Furthermore, the microcontent generation process according to the present invention can utilize a video distribution service operator interface module. This interface relies on the public API of the video distribution service to index a large amount of content of suitable quality for learning, and uses specific algorithms to optimally index among the billions of videos available. To ensure quality, this module includes a pre-filtering function that rejects videos that may be inappropriate (vulgar language, religious discussions, low vocabulary density, excessively long phrases, etc.) by analyzing the video transcription and other metadata.

[0036] Description of the drawings FIG. 1 is a schematic diagram showing an example of a foreign language learning system according to the present invention.

[0037] [FIG. 2] FIG. 2 shows an example of a process for generating micro-video contents for foreign language learning according to the present invention.

[0038] FIG. 3 shows an example of a foreign language learning process according to the present invention.

[0039] [Figure 4] Figure 4 shows example screenshots for searching micro-video content by keywords.

[0040] (Definition of terms) Library: A digital storage space where video flash cards are stored in a structured way. Courses: A collection of video flashcards related to a particular topic Communication devices: devices connected to one or more communication networks, such as smartphones, tablets, and personal computers. Video flashcards: micro-content adapted for learning on smartphones, tablets and personal computers Indexer: A digital tool that searches videos for target words or expressions Quizzes: Questions and surveys designed to test specific knowledge or skills Spaced Repetition: A neuroscience-based memory technique in which a learner is presented with an element of the material, such as a quiz, again just before they are about to forget it.

[0041] (Detailed Description) Referring to FIG. 1 , a foreign language learning system 1 according to the present invention includes a set of personal communication devices C1, C2, ..., Ck used by a number of learners S1, S2, ..., Sk, which are connected to a cloud-type remote storage space 33 via a communication network 30.

[0042] Each personal communication device C1, C2, ..., CK receives a software application (including a front-end) dedicated to the learning process downloaded from an application provider.

[0043] The system 2 for generating video flash cards of micro content comprises a dedicated server 34 (backend) and a storage space 32 connected via a communication network 30 to a cloud 33 containing a database for storing the video channels 11 and the micro content.

[0044] This production system 2 is dedicated to generating video flash cards 7.1, 7.2, …, 7.a associated with target words or phrases detected in video clips 4.1, 4.2, …, 4.n extracted from video content delivered from a video channel 11.

[0045] These video flash cards 7.1, 7.2, ..., 7.n may be classified into courses 9.1, 9.2, ..., 9.n that are provided to the learners. In practice, these video flash cards and courses are stored in a library L in the cloud 33. Each learner S1, S2, ..., Sk can browse the course catalog available in the learning server 30 and select a course that matches his / her learning objectives. The learner then receives a series of flash cards related to the selected course in sequence on his / her communication device to be reviewed by spaced repetition.

[0046] The use of Courses is not mandatory and learners and teachers can also search for video contexts based on target words. The use of Courses is an optional feature that allows teachers to organize flashcards more systematically and add more meaning to learning.

[0047] An embodiment of a flash card production process according to the present invention will now be described in more detail with reference to Figure 2. The production process 10 includes a step 12 of sequentially searching a video channel for videos containing a target word or phrase. The search 12 may be performed using the video channel's API.

[0048] The detected videos based on the target words or phrases are indexed (3) to generate video clips 4 that are relevant to the target words or phrases. These segments are then processed by a pre-filtering algorithm 6, for example natural language processing (NLP).

[0049] From these indexed and prefiltered video clips, video flashcards7 (or video cards, also called micro-video content) are generated (13) containing quizzes on content related to the target words and phrases. These quizzes are provided in audio and / or scripted form in relation to the content of the video flashcards. These quizzes may be generated manually by a teacher or administrator, or automatically by asking learners to practice translation.

[0050] The video flashcards 7 thus generated can be stored in a dedicated library L located in the cloud 33. These flashcards can also be pre-organized into courses 9.1, 9.2, 9.3 such as "Telephone communication", "Food vocabulary" and "Place vocabulary" or can be stored unorganized in the library L. Each course 9.1, 9.2, 9.3 includes multiple video flashcards 7.1, 7.2, 7.3 dedicated to the theme of that course.

[0051] Based on Fig. 3, an embodiment of the learning process according to the invention is described. This learning process 20 includes an initial step 21 where the learner S or the teacher selects a flashcard based on a specific keyword or phrase directly from the library L (as shown in Fig. 4 below), or selects a course from among the courses 9.1, 9.2, 9.3 available from the video flashcard library of the library L. If none of the above courses are used for a specific topic, the system can also suggest microcontent by default based on target keywords according to the learner's level.

[0052] Based on Figure 4, the software application downloaded to the learner S's communication device provides a user interface 40 for directly searching for video flashcards from the library L stored in the cloud 33. This is done by entering a keyword or phrase, such as "patent", into an input window 41 and operating a search button or icon 42.

[0053] In response to the request "patent," a list of relevant video flashcards 43, 44, 45 is displayed along with an excerpt of the index content for each listed video clip. Learner S can then select from this list the flashcard he or she wishes to view.

[0054] The selection step 21 is followed by a step 22 of viewing the selected video flash card on the communication device used by the learner S. This is followed by a step 26 of answering a quiz or test contained in the video flash card. The learner S is asked to answer one or more questions related to the target content of the video flash card (26). The answers entered by the learner S can take multiple forms, such as voice, alphanumeric, or selection from multiple answer options, and are processed (24) by a software application (front end) that has been pre-downloaded onto the communication device used by the caller.

[0055] The spaced repetition algorithm23 has been implemented in a dedicated software application and can provide useful information on the state of the art regarding spaced repetition systems (SRS), in particular the paper "Optimizing the long-term acquisition of knowledge components by adaptive spacing algorithms for learning" (Benoit Choffin, University of Paris-Saclay, 2021, NNT: 2021-UPASG001 -03216648).

[0056] The spaced repetition algorithm 23 automatically determines the timing of the presentation of microcontent and the prompting to answer the quiz based on previous answers.

[0057] Learner S is asked to retry the quiz after a certain time interval ΔT has elapsed (25). If learner S fails the quiz, he or she is asked to retry the quiz after one minute (27).

[0058] For example, if a learner answers a quiz correctly 20 times, the time interval ΔT may be set to one year or more. In practice, many users stop using the system before that. During each review, the learner's memory is strengthened. In practice, it is believed that memory becomes almost stable when the time interval is 10 days or more.

[0059] The learning process suggests to the learner S to review the video microcontent just before forgetting through a dedicated interface displayed on the communication device. It uses a spaced repetition algorithm to plan the scheduling of review based on the learner's performance.

[0060] Without being limited to the embodiment described herein, many other configurations are possible without departing from the scope of the present invention, in particular other configurations of the quiz answering process described herein may be implemented. [Brief description of the drawings]

[0061] [Figure 1] 1 illustrates a schematic diagram of an example of a foreign language learning system according to the present invention; [Diagram 2] 1 illustrates an example of a process for generating micro-video content for foreign language learning according to the present invention. [Diagram 3] 1 illustrates an example of a foreign language learning process according to the present invention. [Figure 4] 1 shows example screenshots of searching micro-video content by keyword.

Claims

1. A method (10) for extracting long videos from a large amount of video content (11) and using them to generate micro-video content for language learning (7; 7.1, 7.2, 7.3) by computer. The method includes the following steps: - searching and identifying (12) from the video content (11) that contains the same target word or phrase in the language; - Extracting from each identified video content a video clip (4) that contains the corresponding target word or phrase. - adding metadata and an automatically generated contextualized quiz to each extracted video clip (4) to produce learning micro-video content (7), which is presented for the first time to a learner who selects the corresponding micro-video content and then re-presented at increasing time intervals ΔT each time the learner successfully completes the quiz;

2. 2. The method of claim 1, wherein the searching step is configured to identify video content containing a target word or phrase that corresponds to a particular context (10).

3. 3. The method of claim 1 or 2, wherein identifying the video content includes pre-filtering the video content based on a number of predefined filtering criteria and analysing a transcription of audio data or associated metadata of the relevant video content.

4. 10. A method as claimed in any preceding claim, wherein audio transcription data and / or metadata associated with video content are provided directly from the source of said video content.

5. 10. A method according to any preceding claim, wherein the source of the video content is provided by an external video distribution service and comprises an Application Programming Interface (API).

6. A system (2) for generating micro-video content (7.1, 7.2, …7n) from long-form video. The system uses video from one or more sources (11) and performs a generation process according to any of the preceding claims, including: - A means for identifying, from said video content source, multiple video contents that contain the same target word or phrase. - A means for extracting video clips containing the target word or phrase from the identified video content. - means for adding metadata and automatically generated contextualized tests (quizzes) to the extracted video clips, where the enrichment means are configured to be provided to the learner for the first time after studying the selected micro-video content (7) and thereafter again according to increasing time intervals ΔT each time the learner successfully completes the test.

7. 7. The generating system (2) according to claim 6, characterized in that the identifying means is adapted to identify video content containing a target word or phrase adapted to a particular context.

8. The generation system (2) of claim 6 or 7, characterized in that the video content searching and identification means further comprises means for pre-filtering the video content according to a plurality of defined pre-filtering criteria and means for analyzing a transcription of audio data of the video content and associated metadata.

9. A language learning method (20) using micro-video content (7.1, 7.2, ... 7n) generated according to any of claims 1 to 5, implemented in a communication device used by a learner, comprising the following steps: - providing a list of micro-video content containing a target word or phrase in a particular language from a remote storage space in response to a request made by the learner based on the target word or phrase; - Select the micro content from the provided micro content list. - Watch selected micro-video content. - Implement contextualized tests (quizzes) within micro-video content and have learners enter their answers. - Uses a spaced repetition algorithm that processes the answers entered and prompts you when to view the test again. - Displaying an interface on the learner's communication device that suggests reviewing the micro-video content just before it is likely to be forgotten, based on a spaced repetition algorithm that plans a review schedule based on the learner's performance.

10. A language learning method as claimed in any preceding claim, further comprising an initial step of offering to the learner a number of courses including micro-video content of each group.

11. A language learning system that includes: Each micro-video content is associated with a specific target word or phrase and includes a pre-identified video clip, and these micro-video content are accessible from a micro-video content storage space. - one or more communication devices equipped with a dedicated digital application implementing the aforementioned method, said communication devices being connected via a communication network that is connected to a storage space for micro-video content. - each communication device responding to a learner's request based on a target word or phrase in a particular language by providing a list of micro-video content from remote storage that includes the target word or phrase; - Select the micro content from the provided micro content list, - Watch selected micro-video content; - prompting the learner for input of a response from said micro-video content, including an automatically generated contextualized test; - Using a repetition interval algorithm that processes the answers entered and provides time indications for viewing the test again. - Displaying an interface that suggests to the learner to review the micro-video content just before they are likely to forget it, based on a repetition interval algorithm that plans a review schedule based on the learner's performance.

Images