Information synthesis generation system, method and program based on such a system

Abstract

The invention relates to an information synthesis system comprising: - at least one information reception means (R1, R2) receiving text (T), audio (A), and / or video (V) files; - a multimedia file (C) conversion means (C) to text to obtain at least one converted file (FC); - a first artificial intelligence (AI) model analyzing the text file (T) and / or the converted file (FC) and generating a mind map (CM) based thereon, the first artificial intelligence (AI) model being built by machine learning from construction data comprising reference files and corresponding reference mind maps (CM1, CM2, … CMi). The invention further relates to a method and a program based on such a system. (Fig. 1)

Description

Title of the invention: Information synthesis generation system, method and program based on such a system

[0001] The invention relates to the field of information synthesis generation systems, in particular the generation of mind maps and podcasts.

[0002] Managing and assimilating large amounts of information from diverse sources presents a major challenge in many fields. Mind maps are effective tools for visualizing, organizing, and memorizing this information. However, manually creating mind maps can be time-consuming and require specialized expertise.

[0003] The same difficulty is found in the synthesis of audio information, in particular speeches, oral lectures, etc.

[0004] Some mind mapping solutions use common artificial intelligence such as a GPT4 module. Unfortunately, the resulting mind maps are crude and erroneous, leading to nonsense. These mind maps are unusable.

[0005] One objective of the invention is to remedy the defects of the prior art, and in particular to propose a system enabling the synthesis of information from different possible sources to generate usable mind maps and / or podcasts containing information from said sources.

[0006] To achieve this objective, the invention proposes an information synthesis system comprising: - at least one means of receiving information that receives at least one text file, at least one audio file and / or at least one video file; - a means of converting multimedia files to text, converting said audio file and / or said video file into text to obtain at least one converted file; - a first artificial intelligence model analyzing said text file and / or said converted file, and generating a mind map based on said text file and / or said converted file, the first artificial intelligence model being built by machine learning from construction data including reference text files, reference audio files and / or reference video files, as well as corresponding reference mind maps.

[0007] Advantageously, the first artificial intelligence model makes it possible to synthesize information from various possible sources to generate mind maps that are in a format corresponding to the reference mind maps, and which reproduce the initial information.

[0008] According to one variant, the construction data of the first artificial intelligence model are derived from search engine documents and conversational artificial intelligence information.

[0009] This allows for the refinement of artificial intelligence during machine learning.

[0010] According to one variant, the first artificial intelligence model is a mistral type model, preferably of mistral7B type.

[0011] This allows us to have a model adapted to mind maps with an easy and evolving language.

[0012] According to one variant, the information synthesis system further comprises a second artificial intelligence model analyzing the text file and / or the converted file and generating a synthesized audio file based on synthesized or explicit information from said text file or from said converted file.

[0013] This allows information to be summarized in podcast format. This type of podcast is offered in itself in the prior art, but the invention proposes, in a variant, to produce this podcast while also displaying the mind map to facilitate the listener's understanding of the podcast.

[0014] According to one variant, the information synthesis system further includes a third artificial intelligence model analyzing the text file and / or the converted file and generating an explanatory video file per avatar based on information from said text file or from said converted file.

[0015] This allows information to be synthesized in the form of an explanatory video using an avatar. This type of video is available in the prior art, but the invention proposes, in a variant, to produce this video while also displaying the mind map to facilitate the audience's understanding of the video.

[0016] The invention further relates to an information synthesis method for an information synthesis system according to the invention, comprising: - an information reception step in which at least one text file, at least one audio file and / or at least one video file is received; - a multimedia file to text conversion step in which said audio file and / or said video file is converted into text to obtain at least one converted file; - a mind map generation step in which the text file and / or the converted file are analyzed, and a mind map is generated based on the text file and / or the converted file, using an initial artificial intelligence model, the first artificial intelligence model being built by machine learning from construction data including reference text files, reference audio files and / or reference video files, as well as corresponding reference mind maps.

[0017] According to one variant, the first artificial intelligence model is a mistral type model, preferably of mistral7B type.

[0018] According to one variant, the information synthesis process further includes a synthesis audio file generation step in which said text file and / or said converted file is analyzed, and a synthesis audio file is generated based on synthesized information from said text file or said converted file, by means of a second artificial intelligence model.

[0019] According to one variant, the information synthesis process further includes a step of generating an explanatory video file per avatar in which said text file and / or said converted file is analyzed, and an explanatory video file per avatar is generated based on information from said text file or said converted file, by means of a third artificial intelligence model.

[0020] Another object of the invention relates to a computer program comprising program code instructions for executing the steps of the information synthesis process according to the invention, when said program is running on a computer.

[0021] The invention will now be described on the basis of the accompanying figures illustrating different embodiments of the invention, including: - [Fig.l] schematically illustrates an information synthesis system according to a preferred embodiment of the invention; - [Fig.2] schematically illustrates a training of the first artificial intelligence model of the system of the invention; - [[Fig.3]] schematically illustrates an example of a mind map produced by the system of the invention; - [[Fig.4]] schematically illustrates the podcast generation function in a preferred embodiment of the invention; - [[Fig.5]] schematically illustrates the function of generating explanatory video by avatar in a preferred embodiment of the invention.

[0022] The invention relates to an innovation in the field of artificial intelligence, specifically an AI model, preferably Mistral7B, optimized for the automatic generation of mind maps from various types of documents, including text documents (e.g., PDFs), videos, and audio files. The model has been refined using a dataset developed with the aid of GPT-4, in order to improve its information comprehension and structuring capabilities.

[0023] The core of this innovation is an AI model based in the preferred variant on Mistral7B, an advanced language model designed to process and understand large volumes of text.

[0024] Regarding the formation of the dataset specifically, it is carried out using GPT-4. This dataset contains examples of source documents (PDF, video and audio transcripts) as well as the corresponding mind maps, manually generated to serve as a reference.

[0025] Regarding the training of the first artificial intelligence AI (illustrated in [Fig.2]), a conversational artificial intelligence, such as a GPT4 module (GPT) with prompts proposed or generated by this module, can be used to obtain different topics, and each topic can be developed into different reference texts Ti, T2, ..., Tb and reference mind maps CMb CM2, ..., CM are also produced manually (i.e. by a human operator).

[0026] Alternatively or in combination, the reference texts Ti, T2, ..., T; can be written manually just like the reference mind maps CMb CM2, ..., CM;

[0027] The reference text may be converted file text.

[0028] The same operation can be carried out in a training session using reference videos Vb V2, ..., V; or reference audios Ab A2, ..., Ab, searched for by conversational artificial intelligence on one or more search engines, and converted into FC converted files.

[0029] Regarding the fine-tuning process, to adapt the Mistral7B model to the specific task of generating mind maps, a fine-tuning process has been implemented.

[0030] More specifically, the inventors used GPT4 to generate a non-exhaustive list of topics. The inventors then created a system to process the various documents they retrieved from the Google™ search engine. In this way, the inventors were able to create a database of documents to transcribe in order to obtain their textual content. The inventors then fed these documents into GPT4, instructing it to generate the associated mind maps. Finally, the inventors performed a local distillation of GPT4 into the Mistral7B model to obtain a custom mind map generation model.

[0031] The Mistral7B model was trained on this dataset to learn to extract key concepts from documents and organize them into mind maps.

[0032] The invention includes multimodal processing. Indeed, the system is capable of processing different types of documents: - PDF: the model performs an extraction and analysis of the text contained in PDF files (or other formats): we will refer to it as a text file T; - Videos: the model uses transcription techniques to convert audio content into text: we will refer to it as a V video file; - Audios: the model performs a conversion of audio files into text via speech recognition systems: we will refer to audio file A.

[0033] Regarding file reception, means for receiving information RI, R2 are provided. This could be a URL input interface allowing the corresponding file to be downloaded, for example from Youtube™ or a search engine. Alternatively, or in combination, it could be an interface for loading such a file.

[0034] For processing text files, a text selection module or an optical character recognition module can be used to retrieve the raw text information from the file. This raw text information is then analyzed, as appropriate, by the first artificial intelligence and the second artificial intelligence.

[0035] For the processing of audio files, a speech-to-text module can be used to obtain raw textual information (FC converted file) which is analyzed as mentioned above.

[0036] For video processing, an audio track extraction module can be used for said file, and then the audio track is processed as explained above.

[0037] In the illustrations, the conversion of audio or video files is carried out by a C conversion means.

[0038] The generation process comprises three steps: - a first step of data extraction, in which information is retrieved and converted from source documents T, A, V; - a second step of analysis and structuring, in which key concepts and hierarchical organization are identified; - a third step in generating the CM mind map, in which the visual mind map is created based on the previous analysis.

[0039] Regarding the generation of mind maps specifically, the model generates mind maps structured by context and concepts, and including contextual links between concepts. The mind maps can be hierarchically organized in a specific way, for example, divided into heading T, categories Cl, C2, subcategory SI, S2, etc., as illustrated in [Fig. 3].

[0040] The system can be used in various fields, such as: - education: aids to learning and revision by structuring course information; - research: organization of research data and publications; - Business: structuring information for presentations and meetings.

[0041] The advantages include saving time by automating the creation of mind maps; accessibility by facilitating the understanding and memorization of complex information; and versatility by the ability to process documents of different types and formats.

[0042] This innovation offers an automated and intelligent solution for generating mind maps, thereby improving efficiency and access to information. By integrating advanced natural language processing and multimodal conversion techniques, the system represents a significant advance in the field of information organization and visualization.

[0043] The invention further relates to podcast generation, illustrated in [Fig.4]. This can be referred to as an AS synthesized audio file.

[0044] To this end, a second artificial intelligence model AI2 analyzes the text file T and / or the converted file FC, and generates a synthesized audio file AS based on synthesized information from said file T, FC. A text synthesis model can be used for this purpose, for example, a GPT4 module with a text synthesis prompt.

[0045] The invention further relates to the generation of explainer videos using avatars, such as a video of the Synthesia™ solution, illustrated in [Fig. 5]. This can be referred to as an avatar explainer video file (VE). The reference Av refers to the avatar.

[0046] To this end, a third artificial intelligence model AI3 analyzes the text file T and / or the converted file FC, and generates an explainer video file per avatar VE based on synthesized information from said file T, FC. A known solution can be used for this specific aspect.

[0047] The invention can incorporate a keyword-based mind map search engine to find matches.

[0048] It can also integrate a conversational robot (or “chatbot” in English) to extract information from a document and discuss it between the user and said robot.

Claims

Demands

1. A computer-implemented information synthesis system comprising: - at least one information receiving means (RI, R2) receiving at least one text file (T), at least one audio file (A), and / or at least one video file (V); - a multimedia-to-text conversion means (C) converting said audio file (A) and / or said video file (V) into text to obtain at least one converted file (FC); - a first artificial intelligence (AI) model analyzing said text file (T) and / or said converted file (FC) and generating a mind map (CM) based on said text file (T) and / or said converted file (FC), the first artificial intelligence (AI) model being built by machine learning from construction data comprising reference text files (Tb, T2, ..., Tb), reference audio files (Ai, A2, ..., Aj), and / or reference video files (Vi, V2, ..., Vj)., Vi), as well as corresponding reference mind maps (CMb CM2, ... CM;).

2. Information synthesis system according to the preceding claim, characterized in that the construction data of the first artificial intelligence (AI) model are derived from search engine documents and conversational artificial intelligence information.

3. Information synthesis system according to any one of the preceding claims, characterized in that the first artificial intelligence (AI) model is a mistral type model, preferably of mistral7B type.

4. Information synthesis system according to any one of the preceding claims, further comprising a second artificial intelligence model (AI2) analyzing the text file (T) and / or the converted file (FC) and generating a synthesized audio file (AS) based on synthesized or explicit information from said text file (T) or said converted file (FC).

5. Information synthesis system according to any one of the preceding claims, further comprising a third artificial intelligence model (AI3) analyzing the text file (T) and / or the file converted (FC) and generating an explanatory video file per avatar (VE) based on information from said text file (T) or said converted file (FC).

6. Computer-implemented information synthesis method for an information synthesis system according to any one of the preceding claims, comprising: - an information reception step in which at least one text file (T), at least one audio file (A) and / or at least one video file (V) is received; - a multimedia file-to-text conversion step in which said audio file (A) and / or said video file (V) is converted into text to obtain at least one converted file (FC);- a mind map generation step in which said text file (T) and / or said converted file (FC) are analyzed, and a mind map (CM) is generated based on said text file (T) and / or said converted file (FC), by means of a first artificial intelligence (AI) model, the first artificial intelligence (AI) model being built by machine learning from construction data comprising reference text files (Tb T2, ..., T;), reference audio files (Ai, A2, ..., Ai) and / or reference video files (Vi, V2, ..., V;), as well as corresponding reference mind maps (CMb CM2, ... CM;).

7. Information synthesis method according to the preceding claim, characterized in that the first artificial intelligence (AI) model is a mistral type model, preferably of mistral7B type.

8. Information synthesis method according to any one of claims 6 to 7, further comprising a synthesis audio file (AS) generation step in which said text file (T) and / or said converted file (FC) is analyzed, and a synthesis audio file (AS) is generated based on synthesized information from said text file (T) or said converted file (FC), by means of a second artificial intelligence model (AI2).

9. A method for synthesizing information according to any one of claims 6 to 8, further comprising a step of generating an explanatory video file by avatar (VE) in which said file is analyzed

10. text (T) and / or said converted file (FC), and an explanatory video file per avatar (VE) is generated based on information from said text file (T) or said converted file (FC), using a third artificial intelligence model (AI3). Computer program comprising program code instructions for executing the steps of the information synthesis process according to any one of claims 6 to 9, when said program is running on a computer.

Images