Smart cognitive notebook integrating generative ai personalized assistance for students and professionals

Abstract

The present invention, called the Smart Cognitive Notebook, represents a significant advancement in the field of education and professional productivity. The system is designed to provide a comprehensive solution to the challenges faced by users of traditional notebooks. The Smart Cognitive Notebook addresses the limitations of conventional physical notebooks by offering a variety of advanced features. This invention is conceived to enhance organization, collaboration, and accessibility of information for users. Central to the system's functionality is the Artificial Intelligence Agent, a key component that drives interactivity and personalization of the user experience. This agent is expertly designed to interpret user messages, answer questions, review content, and provide personalized responses, all while taking into account the user's preferences and age group. Additionally, the system offers personalized quizzes generated based on the notebook's content, allowing users to test their knowledge interactively. Intelligent flashcards complement this functionality, making the review and memorization of key concepts more efficient. The Smart Cognitive Notebook further stands out for its versatility, seamlessly integrating with physical notebooks, and fully digital versions, and offering access via web and mobile applications.

Description

Smart Cognitive Notebook Integrating Personalized Generative AI Assistance for Students and Professionals

[0001] The present invention will be of multidisciplinary use, especially in the educational and professional segment and is related to a methodological innovation that, bets on a software or application, with a focus on intelligent systems, specifically on an intelligent notebook system with Generative Artificial Intelligence.

[0002] The limitations of traditional physical notebooks extend beyond their physical constraints, impacting various aspects of the user experience and productivity. The lack of a systematic search function in physical notebooks poses a significant challenge, as users struggle to quickly locate specific information within extensive handwritten content. This problem intensifies as the volume of notes increases, hindering efficiency and generating frustration.

[0003] The manual nature of organizing handwritten notes results in disorganization, compromising user efficiency. Users often resort to flipping through pages or creating ad hoc organizational systems, leading to a lack of structure. This confusion makes it difficult for users to quickly access relevant information when needed.

[0004] Collaboration with handwritten notes is hampered by the cumbersome process of transferring ideas from physical notebooks to digital platforms. This introduces friction, potentially resulting in lost context, misinterpretation, and a decline in collaboration efficiency. The lack of seamless integration between analog and digital modes becomes a barrier to effective teamwork.

[0005] Traditional notebooks offer a one-way interaction, where information is transferred from the user to the paper without any dynamic feedback. This limitation reduces opportunities for enhanced learning and ideation. An AI-powered assistant has the potential to provide real-time insights, suggestions, and corrections, transforming the note-taking experience into a more interactive and iterative process.

[0006] Physical notebooks lack the inherent version control and backup features common in digital platforms. Losing a notebook, damaging it, or accidentally deleting critical notes can lead to irreversible data loss. An AI-powered solution can automatically generate backups and versions, ensuring data integrity and security, providing users with peace of mind.

[0007] As users accumulate more notes and ideas, the scalability of physical notebooks becomes a practical concern. Carrying a large collection of notebooks everywhere is impractical. An AI assistant can address this challenge by effortlessly switching between multiple notebooks and providing a centralized repository for all notes, ensuring accessibility without the physical burden.

[0008] In an era where digital tools are essential for personal and professional workflows, physical notebooks often operate in isolation. Integrating handwritten notes with digital calendars, task managers, or collaborative platforms requires manual effort, introducing inefficiencies and the potential for data discrepancies. An AI-powered assistant can bridge this gap by facilitating seamless integration with various digital tools.

[0009] Physical notebooks don't offer easy remote access, presenting challenges when users need to consult their notes outside of their immediate physical environment. Manually transcribing or digitizing content becomes a tedious task. An AI-powered assistant addresses this by providing digital access to handwritten notes from anywhere, enhancing the flexibility and accessibility of information recorded on paper.

[0010] Patent BR7800903U, filed under the title "Construction provisions introduced into school notebooks for elementary education," addresses structural innovations implemented in school notebooks for elementary education. These modifications aim to provide an ergonomic design, replacing the need for multiple notebooks traditionally used for different subjects.

[0011] The goal is to facilitate logical note-taking during classes, while reducing the burden carried by children.

[0012] On the other hand, patent BRMU8800496U2, filed under the title "Layout introduced in school notebook with game," does not establish connections between the physical and digital notebooks. Furthermore, it does not present a defined process, associated means of use, or a digital tool for its implementation.

[0013] Given the gaps identified in the current state of the art, the development of a new approach is imperative. This approach seeks to fill these gaps, as will be detailed below.

[0014] This highlighted innovation not only reinforces its relevance in the educational landscape but also highlights its multidisciplinary approach. The smart notebook system, combined with an intelligent tutor powered by generative AI, presents a flexible and customizable proposal, aimed at both students and professionals from various fields.

[0015] This solution goes beyond meeting educational needs, encompassing a wide range of disciplines. By integrating generative artificial intelligence into the smart notebook, it provides an adaptable and personalized approach for users, especially those involved in educational and professional activities.

[0016] The system is designed to be accessible through interfaces on mobile platforms or the web, providing users with the ability to formulate questions interactively and obtain instant answers.

[0017] Interactions in the system can be introduced in a variety of ways, including audio, text, or images, expanding the user's communication options.

[0018] The system's unique feature lies in the personalized responses, which not only address questions comprehensively but also offer detailed explanations, covering concepts in subjects such as mathematics, chemistry, physics, and related fields step by step.

[0019] The proposed methodology serves as an introduction to an innovative language system, where flexibility and customization are key elements. Users can use the mobile app or the web version, entering questions via audio, text, or image, all facilitated by an interactive interface.

[0020] In a preferred implementation, the insertion of questions occurs intuitively, whether through audio, text or image, both through the web platform and the mobile application.

[0021] In a preferred realization, the integration of advanced technologies, such as Retrieval Augmented Generation (RAG), enriches the user experience by ensuring contextually relevant responses.

[0022] In a preferred embodiment, the incorporation of Optical Character Recognition (OCR) enriches the user experience, enabling the use of the technology to extract handwritten content through a simple photograph of a notebook. OCR is a technology that converts handwritten documents, captured in images, into searchable and editable data.

[0023] In a key achievement, the system's ability to automatically link a subject to captured content through a Natural Language topic identification module stands out. This functionality adds an additional layer of organization to the system, facilitating categorization by specific subject topics. This makes it easier to search for and retrieve specific information related to each topic.

[0024] In a key aspect, users of the presented methodologies will receive academic and professional support, with knowledge delivered appropriate to their intellectual level and age. This broad and adaptable approach positions innovation as a comprehensive and personalized tool for enhancing learning and professional development.

[0025] In a preferred embodiment, the user can link their physical notebook to the digital one during registration using a QR code, data matrix or similar code using an image capture device. Fig. 1

[0026] [Fig. 1] Figure 1 illustrates a block diagram that represents the general Flow of Operation. Fig.2

[0027] [Fig. 2] Figure 2 illustrates a block diagram representing the System Insertion Module. Fig. 3

[0028] [Fig. 3] Figure 3 illustrates the Recovery Augmented Generation (RAG) Module Diagram. Fig.4

[0029] [Fig. 4] Figure 4 illustrates the Transformation Module Diagram. Fig. 5

[0030] [Fig. 5] Figure 5 illustrates a diagram of the Conversation Module Flow. Fig.6

[0031] [Fig. 6] Figure 6 illustrates the Language Customization Module. Fig. 7

[0032] [Fig. 7] Figure 7 illustrates the Questionnaire Module. Fig.8

[0033] [Fig. 8] Figure 8 illustrates the Question Generation Module. Fig.9

[0034] [Fig. 9] Figure 9 illustrates the Educational Management Module. Fig.10

[0035] [Fig. 10] Figure 10 illustrates the QR and Data Matrix codes on the notebook sheets for authentication. Fig.11

[0036] [Fig. 11] Figure 11 illustrates an example of a brand logo on a notebook sheet. Fig. 12

[0037] [Fig. 12] Figure 12 illustrates an example of a Page Identification, Acquisition and Scaling process with Computer Vision Algorithms.

[0038] The present invention discloses an Intelligent Cognitive Notebook system composed of 12 modules accessed, directly or indirectly, through the General Operation Flow through different types of input devices. In a brief description:

[0039] The Smart Cognitive Notebook service can be accessed through data input / output devices such as cell phones, laptops, desktop computers, and any other device that has access to basic data input / output functionality through text and images, with an internet connection via a browser or app store. Through the device interface, users can register, purchase a license, and access the product's features.

[0040] Insertion Module: Manages the ways in which the User enters content into the cognitive notebook and its proper processing from multimedia formats to text format ready to be consumed by other services within the platform.

[0041] Conversation Module: Responsible for interacting with the user through the exchange of messages in text, audio, or image format. It processes user input to identify the nature of received messages.

[0042] Input Recognition Module: Belonging to the Conversation Module, this module takes care of the recognition of the input type and the subsequent categorization within a knowledge field of the Input received within the chat conversation.

[0043] Resolution Module: Part of the Conversation Module and responsible for processing user questions and providing accurate and relevant answers using databases, machine learning models, and mathematical and analytical modeling and calculation tools.

[0044] Language Personalization Module: Another part of the Conversation Module. This module manages and applies language personalization to the user, taking into account parameters such as age and education.

[0045] Questionnaire Module: Manages the process of creating and displaying different types of questionnaires that can be completed by the user. These questionnaires are created based on content extracted from the notebook entered during the Input Module.

[0046] Educational Management Module: Part of the Quiz Module. This module is responsible for the User's learning cycle as part of the Smart Cognitive Notebook service ecosystem. It manages assessment reviews, study sessions for reinforcement, and the continuation of a Quiz's learning cycle or its completion if the user wishes to end the activity.

[0047] Transformation Module: Handles the transformation of the original notebook text inserted during the Insertion Module. In this context, transformation means using the notebook's textual content as a basis for some operation to be performed on this text and saving it as an alternative version of the original text. Examples include transforming an original text into audio; a summary of the original text; or translating it into another language.

[0048] RAG Module: Part of the Conversation Module. This module manages search through a Retrieval-Augmented Generation mechanism, which compartmentalizes the context used by the model into fractions of its original content in a vector database and feeds excerpts to the LLM model to reduce artifacts and hallucinations.

[0049] Questionnaire Module: Part of the Questionnaire Module. This module is responsible for creating a multiple-choice questionnaire. The questionnaire is generated through a LLM that uses the user's scanned notebook as one of the parameters for generating the questionnaire.

[0050] Flashcard Module: This module is a functional extension of the Knowledge Assessment Module, with features similar to the Quiz Module, but adapted to the Flashcard format. Flashcards are cards used for memorization, containing a single question and an associated answer. The goal of this format is to encourage the user to formulate the answer in their own words before checking the solution presented on the Flashcard. Like the Quiz module, Flashcards are generated using a Language Model (LLM), using the content of the digitized notebook as the main parameter for personalized creation.

[0051] Question Generation Module: This module is simultaneously part of the Flashcard and Quiz Modules. This module is responsible for generating the questions that will be used for both Flashcards and Quizzes.

[0052] The examples shown here are intended only to exemplify one of the numerous ways of carrying out the invention, however, without limiting its scope.

[0053] The method will be used as follows: when purchasing the license, User 1.2 will create their login and password (Figure 1.1) through the web portal or mobile application 1.3.

[0054] Once registration is complete, through an access token system, the User will have the option to link their physical notebook to the digital one through a QR code, data matrix, AZTEC, or other system of symbols, pictograms, or similar codes present on a notebook page (Figure 10) or by scanning a logo of a specific brand using an image capture device (Figure 11). This process is performed by Insertion Module 2.1. Once a digital notebook is created through Insertion Module 2.1, it can be interacted with through the following modules: Transformation Module 4.1, Conversation Module 5.1, and Questionnaire Module 7.1.

[0055] Identification through scanning: If the input method of the analog sheet to a digital format is through scanning a QR code, data matrix, AZTEC or other system of symbols, pictograms or similar codes present on a notebook sheet, as in the example illustrated in figure 10, the user's device is used to identify the positioning through an algorithm for recognizing the codes and interpreting them so that they can be properly validated by our system.

[0056] Computer vision identification: If the input method is computer vision-based, without the use of QR codes or similar technologies, the system uses a method to identify the details of the company logo printed on the notebook, as exemplified in Figure 11. The logo can be identified in any position on the page, such as at the top or bottom, ensuring flexibility in analysis and recognition. This logo is analyzed by the tool and compared with the authenticated logos stored in the database, ensuring its recognition and verification. Logo identification can be done exclusively through visual elements, alphanumeric elements, or a combination of both, covering all possible cases. Therefore, only notebooks from manufacturers previously registered in our database are allowed for scanning services.

[0057] After authentication in Authentication Modes Step 2.3, the process moves on to Text Recognition Step 2.4. This mode performs the OCR process in phases. The first phase involves identifying and then acquiring and sizing the page using computer vision algorithms to define the text area on the page and identify margin boundaries (Figure 12). The purpose of this initial step is to determine the page dimensions and establish the working area for subsequent steps in the recognition process.The second phase performs an initial reading of all text on the page within the previously delimited area; this first reading is then treated and refined in order to reduce noise and reading errors; finally, the treated text is processed and rendered so that it is as presentable as possible for the user, for example, but not limited to, formatting in visual standards such as ASCIIMath, LaTeX, Markup or similar.

[0058] Through Transformation Module 4.1, User 1.2 can interact with the digitized version of their notebook. This interaction occurs through the text extracted from each page that User 1.2 inserts through Insertion Module 2.1. Each interaction User 1.2 makes with the text creates a distinct version of that text according to the modification methods specified, but not limited, by Transformation Modes 4.4.

[0059] Data transformation modes: Text-to-speech conversion transforms the base text into an audio format read by a specialized AI. Summarization summarizes the given content across the entire sheet or section of the notebook into a few paragraphs. Translation translates the given text into one of the languages ​​supported in the menu. Oracle allows the User (1.2) to ask any question related to the content of a specific text. The answer will be extracted directly from the text itself, ensuring that the information provided is based on the available material. Reducer works similarly to Summarization, but instead of summarizing the text into a few paragraphs, it condenses it into just two sentences.

[0060] Through Conversation Module 5.1, User 1.2 can interact directly with the AI ​​through a chat window.

[0061] The method also includes an AI chatbot fully integrated with the Cognitive Notebook, which is made available to the user. The chatbot is accessed through the Conversation Module (5.1) and acts as a virtual assistant capable of understanding different types of messages.

[0062] If the message is an image, it is interpreted by a Computer Vision 5.9 model; if it is an audio file, it is interpreted by another Text-to-Speech 5.8 model. In both cases, a textual description is extracted at the end. This textual description is treated in the same way as if the message had been sent in text format. It is then qualified and classified 5.7 to fit into a subject category.

[0063] If the message is a document like a PDF, DOCX, or similar, it's handled in a different workflow. The file is stored in our 5.10 vector database, and this document later provides context so that chat questions can be answered more accurately, using the sent file to search for answers.

[0064] Resolution Module 5.5 is triggered following Input Recognition Module 5.4. During this module, the processed input is received and treated according to its qualification so that a response can be generated by the chatbot.

[0065] The entry point for Resolution Module 5.5 is through a stage of verifying the category qualification between subjects that can be classified as numerical or textual operations, or, in general terms, whether the subject can be classified as more pertinent to the exact or human sciences.

[0066] If the subject is not considered as Exact, it is passed to our Retrieval Augmented Generation (RAG) 3.1 system and, subsequently, to our LLM 5.11 model. The predictive nature of the LLM associated with the RAG makes this case more suitable for purely conversational or textual questions while reducing hallucinations caused by a pure LLM.

[0067] If the subject is considered to be Exact Sciences, it is passed to a second answer system: a Model Answer. Unlike the LLM, the answers provided by the Model Answer are actually calculated and not produced according to its statistical model, as is the case with an LLM.

[0068] In both cases, just before the end of Resolution Module 5.5, the response is processed and formatted so that it is displayed in the best way for the user to understand in the chat during the Response Processing step 5.13.

[0069] The Language Personalization Module 6.1 is responsible for handling the language so that it is as appropriate as possible to the particularities of User 1.2. It can be handled by several factors, including, but not limited to, their age range and language.

[0070] After customizing the language, the Conversation Module only has to perform the Output steps 5.6 (displaying the generated content to the User 1.2) and the database registration steps 2.5 to save the generated data.

[0071] Another tool the system offers is personalized questionnaires based on the content of the user's notebook. At any time, a user can request a questionnaire, which initiates Knowledge Assessment Module 7.1.

[0072] The Knowledge Assessment Module (7.1) uses an instance of the digitized notebook within the Intelligent Cognitive Notebook (4.2) to generate two types of assessments available in the system: Quiz and Memory Card. The Quiz follows a multiple-choice question-and-answer format, while the Memory Card presents questions accompanied by images generated by Artificial Intelligence, with the aim of encouraging memorization during study.

[0073] In both cases, the first step is to use the Question Generation Module (8.1). For the Quiz type assessment, after generating the questions, the questionnaire is presented to the User (1.2). The system displays each generated question individually and waits for the user's response. Upon completion, the user is evaluated based on their responses, and a detailed report on their performance is presented.

[0074] If the assessment is performed using a Memory Card, an image is generated based on the context and topic of the questionnaire. 7.9. Unlike the questionnaire, the Memory Card does not wait for a user response in the system. It only displays the correct answer to the question if the user chooses to reveal it.

[0075] The Question Generation Module 8.1 is responsible for generating the questions for Quizzes and Flashcards within the system. It uses the context provided by the textual components of a digitized notebook to select chunks of text and generate questions based on the context of these chunks when fed into a LLM. Using prompt engineering techniques, we ensure that a response pattern always returns a set of questions and answers, which are then used to finalize the generation of the questions for the quiz.

[0076] The Educational Management Module 9.1 uses generative AI to enhance the learning process and is triggered after a quiz evaluation is generated and presented to the user. This module operates as a continuous feedback loop for personalized learning. When activated by a 7.6 Assessment, the generative AI system analyzes potential areas for improvement 9.2 in the user's knowledge, considering the context of the completed quiz and the history of their responses in previous assessments. Based on this analysis, the AI ​​generates an Improvement Recommendation 9.3, which highlights areas of knowledge that can be strengthened, aligned with the user's performance history. Furthermore, the system offers the opportunity to monitor learning progress by inviting the user to a personalized study session. If the invitation is accepted, the generative AI creates content summaries 9.4 specifically focused on the areas in which the user experiences greatest difficulty.After the study session, a new Quiz is generated, allowing progress to be reassessed. If the User chooses not to participate in the study session, the workflow ends and the Quiz ends normally. This approach ensures a dynamic, personalized, and continuous learning process, driven by the application of generative AI in diagnosis and the creation of adapted study content.

[0077] Individuals with experience in the field will recognize the relevance of the knowledge presented and will be able to apply the invention in the described modalities, as well as in other variants within the limits of the attached claims.

Claims

Method of a smart notebook system with an assistant powered by Artificial Intelligence for students and professionals, characterized by comprising: a Generative Artificial Intelligence agent, designed to interpret user messages and answer questions in a personalized way, considering age group, education and context; review content and provide detailed explanations; an integrated scanning and processing system, which links physical content, such as handwritten notebooks, to a digital platform accessible on mobile devices and desktops; a multimodal input module, capable of recognizing and processing data in text, audio and image, automatically categorizing them through Natural Language Processing (NLP) algorithms;a quiz and flashcard generation module that uses language models (LLM) to create interactive and personalized content based on user material, with multiple-choice quizzes and intelligent flashcards that include AI-generated images to facilitate memorization; an LLM-based translation model that performs accurate and contextually relevant translations between languages, adapted to the semantics of sentences; an automated categorization system that classifies entries (text, exercises, or notes) into topics or disciplines, creating thematic indexes to facilitate navigation and organization; a multimodal interaction system that allows communication with AI via audio, text, or images, synchronizing the entered data with other functionalities, such as categorization and indexing; offering remote access to digitized handwritten content, ensuring flexibility and accessibility for consultation and study from any location.; Advanced method of a smart notebook system with an assistant powered by Artificial Intelligence for students and professionals, characterized by understanding: the ability to process and answer questions related to specific subjects, such as mathematics, physics and chemistry, offering detailed and contextual answers; a speech-to-text conversion module, which captures nuances of the user's speech with high precision, optimizing the transcription of oral content; an advanced automatic indexing system, which organizes notes and topics into subjects and subtopics, facilitating navigation and search; a text translation and summary functionality, which adapts complex information to the user's context, simplifying understanding; a visual and auditory pattern recognition module, which transforms images and audio into processable text, expanding the possibilities of interaction with the content;a system for categorizing and organizing notes, which uses NLP algorithms to automatically create navigable thematic indexes.; Method of an authentication system of a physical sheet or device, characterized by comprising: capturing an image of a logo associated with a brand or of a QR code, data matrix or similar code using an image capture device; processing the captured image using computer vision techniques to extract unique characteristics of the logo or code; comparing the extracted characteristics with a predefined database of authenticated logos or linked codes; generating an output indicating the result of the authentication; the logo may include visual patterns, geometric shapes, specific colors or combinations of these characteristics to facilitate authentication; the capture of the logo or code is performed using a smartphone camera or other optical device;computer vision techniques include deep learning algorithms or convolutional neural networks to identify the logo or decode the code; authentication can be done by reading a brand logo or a QR code, data matrix or similar code present on a notebook sheet; authentication is validated through a second stage, which includes the analysis of additional graphic patterns present on the same object; authentication system, which comprises: an image capture device to obtain the logo or code; a processing module configured to execute computer vision algorithms; a database containing validated logos and codes; and an output module to display the authentication result.;

Images