Information synthesis generation system, method, and computer program based on such a system

Abstract

The invention relates to an information synthesis system comprising: - at least one information receiving means (R1, R2) for receiving text (T), audio (A) and / or video (V) files; - a means (C) for converting a multimedia file (A, V) into text in order to obtain at least one converted file (FC); - a first artificial intelligence (AI) model for analyzing the text file (T) and / or the converted file (FC), and for generating a mental map (CM) on the basis thereof, the first artificial intelligence (AI) model being constructed by machine learning from construction data comprising reference files (T1, T2, …, Ti, A1, A2, …, Ai, V1, V2, …, Vi), as well as corresponding reference mental maps (CM1, CM2, … CMi); the invention further relates to a method and a computer program based on such a system.

Description

Information synthesis generation system, method and program based on such a system

[0001] The invention relates to the field of information synthesis generation systems, in particular the generation of mind maps and podcasts.

[0002] Managing and assimilating large amounts of information from diverse sources presents a major challenge in many fields. Mind maps are effective tools for visualizing, organizing, and memorizing this information. However, manually creating mind maps can be time-consuming and require specialized expertise.

[0003] The same difficulty is found in the synthesis of audio information, particularly speeches, oral lectures, etc.

[0004] Some mind mapping solutions use generic artificial intelligence such as a GPT4 module. This is notably the case with the document “How to Effortlessly Generate Mind Maps from Various file formats with MindMap AI” dated April 11, 2024. Unfortunately, the resulting mind maps are crude and flawed, leading to nonsense. These mind maps are unusable. Indeed, it is not possible to have mind maps in a predetermined format.

[0005] One objective of the invention is to remedy the defects of the prior art, and in particular to propose a system enabling the synthesis of information from different possible sources to generate usable mind maps and / or podcasts incorporating information from said sources.

[0006] To achieve this objective, the invention proposes an information synthesis system comprising: - at least one means for receiving information receiving at least one text file, at least one audio file and / or at least one video file; - a means for converting multimedia files into text converting said audio file and / or said video file into text to obtain at least one converted file; - a first artificial intelligence model analyzing said text file and / or said converted file, and generating a mind map based on said text file and / or said converted file, the first artificial intelligence model being built by machine learning from construction data comprising reference text files, reference audio files and / or reference video files, as well as corresponding reference mind maps.

[0007] Advantageously, the first artificial intelligence model synthesizes information from various sources to generate mind maps in a format that matches reference mind maps and incorporates the original information. Unlike prior art solutions using generic artificial intelligence, this invention uses an artificial intelligence specifically trained to create mind maps based on reference mind maps used in training.

[0008] According to one variant, the construction data for the first artificial intelligence model comes from search engine documents and conversational artificial intelligence information.

[0009] This allows for the refinement of artificial intelligence during machine learning.

[0010] According to one variant, the first artificial intelligence model is a mistral type model, preferably of the mistral7B type.

[0011] This allows for a model adapted to mind maps with easy and scalable language.

[0012] According to one variant, the information synthesis system further includes a second artificial intelligence model that analyzes the text file and / or the converted file and generates a synthesized audio file based on synthesized or explicit information from said text file or converted file.

[0013] This allows information to be summarized in podcast format. This type of podcast is offered in the prior art, but the invention proposes, in a variant, to produce this podcast while also displaying the mind map to facilitate the listener's understanding of the podcast.

[0014] According to one variant, the information synthesis system further includes a third artificial intelligence model that analyzes the text file and / or the converted file and generates an explanatory video file per avatar based on information from said text file or converted file.

[0015] This allows information to be summarized in the form of an explainer video using an avatar. This type of video is available in the prior art, but the invention proposes, in a variant, to create this video while also displaying a mind map to facilitate audience comprehension.

[0016] The invention further relates to a method of information synthesis for an information synthesis system according to one of the preceding claims, comprising: - an information reception step in which at least one text file, at least one audio file and / or at least one video file is received; - a multimedia file to text conversion step in which said audio file and / or said video file is converted into text to obtain at least one converted file;- a mind map generation step in which said text file and / or said converted file is analyzed, and a mind map is generated based on said text file and / or said converted file, using a first artificial intelligence model, the first artificial intelligence model being built by machine learning from construction data including reference text files, reference audio files and / or reference video files, as well as corresponding reference mind maps.;

[0017] According to one variant, the first artificial intelligence model is a mistral type model, preferably of the mistral7B type.

[0018] According to one variant, the information synthesis process further includes a synthetic audio file generation step in which said text file and / or said converted file is analyzed, and a synthetic audio file is generated based on synthesized information from said text file or said converted file, by means of a second artificial intelligence model.

[0019] According to one variant, the information synthesis process further includes a step of generating an explanatory video file per avatar in which said text file and / or said converted file is analyzed, and an explanatory video file per avatar is generated based on information from said text file or said converted file, by means of a third artificial intelligence model.

[0020] Another object of the invention relates to a computer program comprising program code instructions for executing the steps of the information synthesis process according to the invention, when said program is running on a computer.

[0021] The invention will now be described on the basis of the attached figures illustrating different embodiments of the invention, among which: - schematically illustrates an information synthesis system according to a preferred embodiment of the invention; - schematically illustrates a training of the first artificial intelligence model of the system of the invention; - [] schematically illustrates an example of a mind map produced by the system of the invention; - [] schematically illustrates the podcast generation function in a preferred embodiment of the invention; - [] schematically illustrates the explanatory video generation function by avatar in a preferred embodiment of the invention.

[0022] The invention relates to an innovation in the field of artificial intelligence, specifically an AI model, preferably Mistral7B, optimized for the automatic generation of mind maps from various types of documents, including text documents (e.g., PDFs), videos, and audio files. The model has been refined using a dataset developed with the aid of GPT-4 to improve its information comprehension and structuring capabilities.

[0023] At the heart of this innovation is an AI model based in the preferred variant on Mistral7B, an advanced language model designed to process and understand large volumes of text.

[0024] Regarding the specific formation of the dataset, it is carried out using GPT-4. This dataset contains examples of source documents (PDF, video and audio transcripts) as well as the corresponding mind maps, manually generated to serve as a reference.

[0025] Regarding the training of the first artificial intelligence (AI) (illustrated in), a conversational artificial intelligence, such as a GPT4 module (GPT) with prompts provided or generated by this module, can be used to obtain different topics, and each topic can be developed into different reference texts T1, T2, …, T i and we also create reference mind maps for grades 4, 5, ..., grade 6 i corresponding manually (i.e., by a human operator).

[0026] Alternatively or in combination, the reference texts T1, T2, …, T ican be written manually just like the reference mind maps for grades 4, 5, ..., CM i .

[0027] The reference text can be a converted file text.

[0028] The same operation can be performed in a training session using reference videos V1, V2, …, V i or reference audios A1, A2, …, A i ., searched by conversational artificial intelligence on one or more search engines, and converted into FC converted files.

[0029] Regarding the fine-tuning process, to adapt the Mistral7B model to the specific task of generating mind maps, a fine-tuning process has been put in place.

[0030] More specifically, the inventors used GPT4 to generate a non-exhaustive list of topics. They then created a system to process the various documents they retrieved from the Google search engine. TM In this way, the inventors were able to create a database of documents to transcribe in order to obtain their textual content. They then fed these documents into GPT4, instructing it to generate the associated mind maps. Ultimately, the inventors performed a local distillation of GPT4 into the Mistral7B model to obtain a custom mind map generation model.

[0031] The Mistral7B model was trained on this dataset to learn how to extract key concepts from documents and organize them into mind maps.

[0032] The invention includes multimodal processing. Indeed, the system is capable of processing different types of documents: - PDF: the model performs an extraction and analysis of the text contained in PDF files (or other formats): this will be referred to as text file T; - Videos: the model performs a use of transcription techniques to convert the audio content into text: this will be referred to as video file V; - Audios: the model performs a conversion of audio files into text via speech recognition systems: this will be referred to as audio file A.

[0033] Regarding file reception, methods for receiving information R1 and R2 are planned. This could involve a URL input interface allowing the download of the corresponding file, for example, from YouTube. TM or on a search engine. Alternatively, or in combination, it could be an interface for loading such a file.

[0034] For processing text files, a text selection module or an optical character recognition module can be used to retrieve the raw text information from the file. This raw text information is then analyzed, as appropriate, by the first artificial intelligence and the second artificial intelligence.

[0035] For processing audio files, a speech-to-text module can be used to obtain raw textual information (FC converted file) which is analyzed as mentioned above.

[0036] For video processing, an audio track extraction module can be used for the file, and then the audio track is processed as explained above.

[0037] In the illustrations, the conversion of audio or video files is carried out by a C conversion method.

[0038] The generation process comprises three steps: - a first step of data extraction, in which information is retrieved and converted from the source documents T, A, V; - a second step of analysis and structuring, in which key concepts and hierarchical organization are identified; - a third step of generating a mind map CM, in which the visual mind map is created based on the previous analysis.

[0039] Specifically regarding mind map generation, the model generates mind maps structured by context and concepts, and including contextual links between concepts. Mind maps can be hierarchically organized in a specific way, for example, by heading T, categories C1, C2, subcategories S1, S2, etc., as illustrated in the example.

[0040] The system can be used in various fields, such as: - education: aiding learning and revision by structuring course information; - research: organizing research data and publications; - business: structuring information for presentations and meetings.

[0041] The advantages include saving time by automating the creation of mind maps; accessibility by facilitating the understanding and memorization of complex information; and versatility by the ability to process documents of different types and formats.

[0042] This innovation offers an automated and intelligent solution for generating mind maps, thereby improving efficiency and access to information. By integrating advanced natural language processing and multimodal conversion techniques, the system represents a significant advancement in the field of information organization and visualization.

[0043] The invention also relates to podcast generation, illustrated in. We can refer to it as an AS synthesized audio file.

[0044] To this end, a second artificial intelligence model, AI2, analyzes the text file T and / or the converted file FC, and generates a synthesized audio file AS based on synthesized information from said file T, FC. A text synthesis model can be used for this purpose, for example, a GPT4 module with a text synthesis prompt.

[0045] The invention further relates to the generation of explainer videos using avatars, such as a video of the Synthesia solution. TM illustrated in. We can refer to this as an explainer video file per avatar VE. Reference A relates to the avatar.

[0046] To this end, a third artificial intelligence model, AI3, analyzes the text file T and / or the converted file FC, and generates an explainer video file per avatar VE based on synthesized information from said file T, FC. A known solution can be used specifically for this aspect.

[0047] The invention can incorporate a keyword-based mind map search engine to find matches.

[0048] It can also integrate a conversational robot (or "chatbot" in English) to extract information from a document and discuss it between the user and said robot.

Claims

Information synthesis system comprising: - at least one information reception means (R1, R2) receiving at least one text file (T), at least one audio file (A), and / or at least one video file (V); - a multimedia file-to-text conversion means (C) converting said audio file (A) and / or said video file (V) into text to obtain at least one converted file (FC); - a first artificial intelligence (AI) model analyzing said text file (T) and / or said converted file (FC) and generating a mind map (CM) based on said text file (T) and / or said converted file (FC), the first artificial intelligence (AI) model being built by machine learning from construction data comprising reference text files (T1, T2, … ... i ), reference audio files (A1, A2, …, A i ) and / or reference video files (V1, V2, …, V i), as well as reference mind maps (CM1, CM2, … CM i ) corresponding. Information synthesis system according to the preceding claim, characterized in that the construction data of the first artificial intelligence (AI) model are derived from search engine documents and conversational artificial intelligence information. Information synthesis system according to any one of the preceding claims, characterized in that the first artificial intelligence (AI) model is a mistral type model, preferably of mistral7B type. Information synthesis system according to any one of the preceding claims, further comprising a second artificial intelligence model (AI2) analyzing the text file (T) and / or the converted file (FC) and generating a synthesized audio file (AS) based on synthesized or explicit information from said text file (T) or said converted file (FC). Information synthesis system according to any one of the preceding claims, further comprising a third artificial intelligence model (AI3) analyzing the text file (T) and / or the converted file (FC) and generating an explanatory video file per avatar (VE) based on information from said text file (T) or said converted file (FC). A method for synthesizing information for an information synthesis system according to any one of the preceding claims, comprising: - an information reception step in which at least one text file (T), at least one audio file (A), and / or at least one video file (V) is received; - a multimedia file-to-text conversion step in which said audio file (A) and / or said video file (V) is converted into text to obtain at least one converted file (FC); - a mind map generation step in which said text file (T) and / or said converted file (FC) is analyzed, and a mind map (CM) is generated based on said text file (T) and / or said converted file (FC), using a first artificial intelligence (AI) model, the first artificial intelligence (AI) model being built by machine learning from construction data comprising reference text files (T1, T2, …, T i), reference audio files (A1, A2, …, A i ) and / or reference video files (V1, V2, …, V i ), as well as reference mind maps (CM1, CM2, … CM i ) corresponding. Information synthesis method according to the preceding claim, characterized in that the first artificial intelligence (AI) model is a mistral type model, preferably of type mistral7B. Information synthesis method according to any one of claims 6 to 7, further comprising a synthesis audio file (AS) generation step in which said text file (T) and / or said converted file (FC) is analyzed, and a synthesis audio file (AS) is generated based on synthesized information from said text file (T) or said converted file (FC), by means of a second artificial intelligence model (AI2). Information synthesis method according to any one of claims 6 to 8, further comprising a step of generating an explanatory video file by avatar (VE) in which said text file (T) and / or said converted file (FC) is analyzed, and an explanatory video file by avatar (VE) is generated based on information from said text file (T) or said converted file (FC), by means of a third artificial intelligence model (AI3). Computer program comprising program code instructions for executing the steps of the information synthesis process according to any one of claims 6 to 9, when said program is running on a computer.