Recording and recalling of relevant information

Abstract

The invention relates to a method for providing at least one item of information, the method comprising: using at least one input interface of a first computing device to obtain user interface elements of a current digital environment and, based on a similarity between the current digital environment and a previous digital environment, using at least one output interface of a second computing device to provide at least one obtained item of information based on user interface elements of the previous digital environment.

Description

Recording and recalling relevant information

[0001] This disclosure falls within the domain of data processing and computer systems.

[0002] In a professional context, in particular, the time spent searching for relevant information related to a routine computer task is very significant.

[0003] In professional environments, a significant amount of time is spent searching for relevant information related to ongoing tasks that is not immediately available. This information can come from various sources such as local files on a PC, remote servers, or web pages.

[0004] Although some search engines can facilitate this search task, they are often limited to specific servers and do not allow extensive searching across different types of servers or storage, whether local, remote, or secure.

[0005] There is a need for a computer-based tool to facilitate the search for relevant information. Summary

[0006] This disclosure improves the efficiency of existing systems in information processing and management.

[0007] A method for providing at least one piece of information is proposed, the method comprising: obtaining, through at least one input interface of a first computer device, user interface elements of a current digital environment, and, based on a similarity between the current digital environment and a previous digital environment, providing, through at least one output interface of a second computer device, at least one piece of information obtained based on user interface elements of the previous digital environment.

[0008] Depending on the embodiments, the second device may be different from the first device, or the first and second devices may correspond to the same device.

[0009] According to another aspect, a computer device is proposed comprising at least one microprocessor configured to: obtain user interface elements from a current digital environment via at least one input interface of said computer device, and provide, based on a similarity between the current digital environment and a previous digital environment, via at least one output interface of said computer device, at least one piece of information obtained based on user interface elements of the previous digital environment.

[0010] Depending on the embodiment, the computer device may include an input interface and / or an output interface, or the input interface and / or the output interface may be coupled to the computer device. In one embodiment, the computer device may include a port capable of temporarily and / or removably connecting an input interface and / or an output interface. A secondary display connected to the computer device via such a port represents an example of an input interface and / or an output interface coupled to the computer device.

[0011] The computer system is capable of implementing the proposed process in any of its embodiments.

[0012] According to another aspect, a system is proposed comprising: at least one first computing device including at least one microprocessor configured to obtain user interface elements from a current digital environment via at least one input interface of at least one first computing device, and at least one second computing device including at least one microprocessor configured to provide, based on a similarity between the current digital environment and a previous digital environment, via at least one output interface of said at least one second computing device, at least one piece of information obtained based on user interface elements of the previous digital environment.

[0013] The system is capable of implementing the proposed process in any of its embodiments.

[0014] In one embodiment, the computer device (or the first and / or second computer device) is a communication terminal.

[0015] In one embodiment, a communication terminal may comprise a plurality of data processing circuits. In such an example, a data processing circuit of the terminal may be designated by the term "computer device" and may be configured specifically for implementing the proposed method. This approach illustrates a possible distinction between the computer device, which may have a specific role, and the terminal, which may include such a computer device and various other functionalities.

[0016] In another aspect, a computer program is proposed that includes instructions for implementing all or part of a process as defined herein when executed by a processor. In another aspect, a non-transient, computer-readable recording medium is proposed on which such a program is recorded.

[0017] The proposed technique can help improve the user experience by automatically providing potentially relevant information within the current environment, extracted from previous digital environments. This technique is designed to help anticipate user needs while limiting (or even eliminating) manual user intervention, thus offering information proactively and contextually.

[0018] This approach can help limit (for example, reduce) the time users need to search for information compared to manual search techniques or less automated, traditional systems. Such time savings can contribute to improved user efficiency and productivity compared to such techniques. It can also enhance the user experience, making everyday tasks easier to complete.

[0019] Compared to existing auto-completion systems that are limited to brief text entries and take little or no account of the overall digital environment, the proposed technique can help provide better adaptation to users' contextual needs. The information provided can, for example, be more precise and relevant, thus reducing errors and increasing operational efficiency.

[0020] By limiting information overload and focusing solely on providing targeted information, the proposed technique can help optimize the use of computing resources compared to systems that generate an untargeted flow of information. The proposed technique can, at least in some embodiments, accommodate the separation of the same computing environment for multiple uses, such as personal or professional. In this way, the provision of targeted information can be achieved while respecting specific criteria, such as the protection of professional information and the user's privacy.

[0021] The proposed technique does not require any connection to a word processing / document entry application or any system environment. In particular, in at least some embodiments, it may require no adaptation of the system environment or of applications running within the computer environment. Nor is it limited to intra-document operation. It also does not require the manual delimitation of a screen area by a pointing or selection device. Indeed, the proposed technique provides contextual search functionality without being dependent on any application currently in use.

[0022] In summary, in at least some embodiments, the proposed technique can help improve (for example, significantly) access to information for users of computer equipment. By automatically capturing, storing, and analyzing a user's digital environment, the proposed technique can help, in at least some embodiments, to provide relevant contextual information to the user at the appropriate time, limiting or avoiding, for example, tedious manual searches. This can, for example, result, in at least some embodiments, in increased productivity, reduced errors, and an improved overall user experience.

[0023] The features described in the following paragraphs may optionally be implemented independently of each other or in combination with each other.

[0024] In one embodiment, the process includes filtering user interface elements from the current digital environment.

[0025] Such an implementation can enable targeted selection of the information to be processed, thereby limiting informational noise and helping, for example, to improve responsiveness. In a medical setting, for instance, filtering can be designed to exclude sensitive or irrelevant information for the current task, thus respecting the confidentiality and relevance of the displayed data. Furthermore, filtering can prevent the processing of redundant or out-of-context data, focusing resources on critical information.

[0026] In one embodiment, obtaining user interface elements from the current digital environment includes capturing a current image of a screen of the computing device displaying said current digital environment.

[0027] In one embodiment, the process includes obtaining user interface elements from the previous digital environment.

[0028] In one embodiment, obtaining user interface elements from the previous digital environment includes capturing a previous image rendered on a screen of the computing device displaying said previous digital environment.

[0029] In one embodiment, obtaining the user interface elements of the current digital environment includes extracting text from the current captured image by applying an optical character recognition (OCR) algorithm to the current captured image.

[0030] In one embodiment, obtaining the user interface elements from the previous digital environment includes extracting text from the captured previous image by applying an optical character recognition (OCR) algorithm to the captured previous image.

[0031] In one embodiment, obtaining user interface elements from the current digital environment includes slicing at least a portion of the captured current image into a plurality of information blocks, each information block comprising at least one user interface element from the current digital environment.

[0032] In one embodiment, obtaining user interface elements from the previous digital environment includes slicing at least a portion of the captured previous image into a plurality of information blocks, each information block comprising at least one user interface element from the current digital environment.

[0033] The segmentation can be carried out based on at least one criterion such as the arrangement of elements on at least one portion of the captured image (current or previous), the presence of at least one visual separator on at least one portion of the captured image, or a semantic analysis of the textual content of at least one portion of the captured image.

[0034] In one embodiment, the method includes storing the user interface elements of the current digital environment and / or storing the user interface elements of the previous digital environment in a database (for example, a relational database).

[0035] In one embodiment, the method includes storing at least one contextual data of at least one user interface element of the current digital environment and / or at least one user interface element of the current digital environment in the database.

[0036] Examples of contextual data may include, in some embodiments, a previous image of a computer device screen displaying the previous digital environment. The captured previous image is then divided into information blocks. At least one of these information blocks contains at least one user interface element from the previous digital environment. In such an example, it is possible to store in the database, in addition to the information blocks and / or the user interface elements contained within these blocks, at least one piece of contextual data associated with at least one such information block and / or user interface element, for example, the date and time of the capture, the identifier of an active application (corresponding to the captured image), the position of the relevant information block on the screen, and / or keywords extracted from the content of the relevant information block.

[0037] In one embodiment, the similarity between the current digital environment and the previous digital environment takes into account relationships between elements of the current digital environment and elements of the previous digital environment within their respective contexts. In other words, in such an embodiment, the similarity considers, at a minimum, a collective comparison between a plurality of elements of the current digital environment and a plurality of elements from the previous digital environment. This does not preclude the similarity also considering an individual comparison between an element of the current digital environment and an element of the previous digital environment.

[0038] Such an implementation can help anticipate current or future needs. For example, in a routine task like replying to an email, it is helpful to recall relevant information such as the details of a previously viewed file or URL. Considering relationships between user interface elements in the current digital environment can facilitate the inference of useful information for the user. Furthermore, by considering the relationships between interface elements in a previous digital environment, it is possible to identify similar patterns or structures that can help provide relevant information or tailor information suggestions based on that previous digital environment (context).Such an implementation can therefore help to provide consistent and relevant information, even if the current and previous environments are not identical, in anticipation of the expression of a need by the user.

[0039] The processing of user interface elements from the previous digital environment can refer, for example, to an analysis of user interactions or repeated interaction patterns, taking into account, for instance, text entered by the user, the frequency of button clicks, the order in which windows are opened, or the selection of an option from a menu. Such an analysis can, for example, make it possible to determine important or priority actions in the previous digital environment, to anticipate expected actions in the current environment, and to provide useful information for implementing these expected actions.

[0040] In one embodiment, the method includes using a database (e.g., a relational database) to store user interface elements from the previous digital environment. For a previous digital environment associated with a given instant or time interval, an environment identifier can be linked to a list of user interface elements captured at that time, either in their native form or after processing. The user interface elements can also be linked to additional information, such as their position on a screen in the case of graphical elements, or keywords extracted or inferred from their content. The user interface elements can also be linked to temporal and system information, such as the date and time of capture, the applications active at the time of capture, and other relevant system elements.The previous digital environment can be indexed, for example using a full document indexing mode or "full index" in English, or according to specific criteria such as time-based or application-related indexes.

[0041] The speed of delivery of the information obtained is a criterion that can prove particularly important for professional applications requiring real-time operation, and in this context the use of a relational database can facilitate a quick and accurate retrieval of historical data.

[0042] In one embodiment, the method includes using a pre-trained machine learning model to identify a relationship between user interface elements of the previous digital environment.

[0043] Integrating a machine learning model can help improve the ability to interpret and predict user needs based on collected data. One example use case is a customer support system where the model predicts customer issues based on previous input, thereby improving the speed and accuracy of responses.

[0044] In one embodiment, the determination of the similarity between the current digital environment and a previous digital environment is performed by the machine learning model. The machine learning model can, for example, be trained on a dataset comprising pairs of digital environments and information deemed relevant to each pair. The model can be configured to learn to identify correlations between user interface elements and the information deemed relevant, and to use these correlations to predict the relevance level of the stored information for the current digital environment.

[0045] In one embodiment, the process includes: obtaining prior information based on the user interface elements of the prior digital environment, prioritizing the prior information obtained based on a criterion of relevance relative to the current digital environment, and providing, by at least one output interface, at least one of said prioritized prior information.

[0046] Prioritizing retrieved data based on its current relevance can help improve the effectiveness of the information presented. For example, in a multitasking work environment, older information can be prioritized based on the proximity of deadlines or the importance of tasks, thus aiding in time management and productivity.

[0047] In one embodiment, prioritization is adjusted based on user interaction or lack of interaction with information previously provided by at least one output interface.

[0048] The use of reinforcement learning techniques can enable the continuous refinement of information relevance. For example, if a user frequently interacts with certain types of information, prioritizing them more can help improve (e.g., optimize) the user experience by further reducing the time spent searching for relevant information. Conversely, information retrieved by the system but never selected can be progressively dropped. One possible way to implement a reinforcement learning technique is to associate a level of usefulness with a piece of information and to adjust that level based on user feedback, obtained, for example, via a human-computer interface during and / or after the information is provided.

[0049] In one embodiment, the output interface of the second device is connected to a display device and the method includes displaying, by the display device, the information provided via a contextual user interface.

[0050] A contextual user interface can be, for example, a pop-up window or a sidebar. The way the information is displayed can take into account user interface elements, such as a currently running application or user interaction within the user interface, like a click, a selection, or a viewing time.

[0051] In one embodiment, the process includes: obtaining, via a human-machine interface, at least one interaction with the information provided, and providing the at least one interaction obtained to the machine learning model.

[0052] Interaction with the information provided can, for example, take place in the contextual user interface mentioned earlier.

[0053] In one embodiment, obtaining user interface elements is triggered periodically and / or taking into account the current digital working environment.

[0054] Periodically obtaining user interface elements ensures their regular and systematic collection, which can be used to build a rich and comprehensive database or training set. This regularity can help capture even subtle variations or gradual changes in the digital environment, creating a detailed history that can be used to analyze long-term trends.

[0055] Triggering based on the current digital work environment can enable targeted and contextually relevant user interface elements to be collected. This approach can help ensure that the database contains information directly related to the user's current activities, thus limiting or avoiding clutter with irrelevant data and optimizing storage resources.

[0056] Other features, details, and advantages will become apparent upon reading the detailed description below and analyzing the attached drawings, on which: Fig. 1

[0057] shows, in an example of implementation, a flowchart of a process suitable for implementing the proposed technique. Fig. 2

[0058] shows, in an example of implementation, a diagram of a device suitable for implementing the proposed technique. Fig. 3

[0059] shows, in an example of implementation, a diagram of a system suitable for implementing the proposed technique. Fig. 4

[0060] shows, in an example of implementation, a functional architecture diagram of a system enabling the implementation of the proposed technique.

[0061] In drawings, identical reference numbers designate identical elements or elements having similar functions.

[0062] Some specific terms are now clarified for a better understanding of the proposed technique.

[0063] A digital environment includes all user interface elements present on a computer or digital device at a given moment or within a given time interval.

[0064] These user interface elements can be either interactive or passive, depending on the embodiment, and can include, but are not limited to, at least one graphical user interface (GUI) element such as a window, icon, menu, button, scroll bar, and side panel; at least one multimedia element such as an image, video, animation, graphic, diagram, and sound; at least one text document displayed on a screen, such as an email, presentation, form, and article, and / or at least one subdivision of such a text document, for example, a paragraph or block of text; at least one user interaction, such as text input via a physical or virtual keyboard, voice command, mouse movement, touch gesture, click, scroll, swipe, or any other form of interaction with the device; and at least one running application.and / or at least one element of such an application such as an interface, a window, an open and active tab, a toolbar, and / or at least one underlying data element of such an element such as an open file, a header, session states and an activity log, at least one notification, at least one alert, at least one displayed or audible message, for example a pop-up, a system alert, an instant messaging notification, or a security warning.

[0065] User interface elements may include one of the elements listed above or an aggregation or combination of at least two of the elements listed above.

[0066] The digital environment can also include contextual data not rendered on a user interface (e.g. not displayed, or not directly readable by OCR;such as "system" data), which may include, but are not limited to: at least one state of an open application, such as a running process, a scheduled task, a synchronization state, or a consumed resource; at least one active piece of data in such an application, such as a recently modified document, an ongoing discussion, an online transaction, or a real-time data stream; at least one date and / or time, for example, an exact time, a duration of use, a time zone, or a specific period of a day or week; at least one geographic location, for example, GPS coordinates, an IP address, a connected Wi-Fi access point, or a predefined geographic area; at least one state of a network connection, such as signal strength, connection type (Wi-Fi, Ethernet, cellular), available bandwidth, or a recent or ongoing interruption.

[0067] Contextual data may include one of the elements listed above or an aggregation or combination of at least two of the elements listed above.

[0068] A digital environment can be implemented on a variety of computing or digital devices capable of handling and displaying interactive and passive user interfaces. These devices include, but are not limited to, desktop computers, laptops, tablets, smartphones, embedded devices, virtual and augmented reality devices, and more. A digital environment can be implemented on a single device or across multiple devices.

[0069] A current digital environment refers to the present or recent state of a user's digital environment, captured at a given moment or during a specific time interval. This current digital environment may reflect, in particular, the interactive and contextual elements and data available and active on the user's device at the time of capture.

[0070] A previous digital environment, on the other hand, refers to a past state of the digital environment, captured at a previous time and stored for later use or comparison. This previous digital environment can include, among other things, user interface elements and contextual data that were present and active at that time, and can be used to keep track of the user's digital activity for future analysis or recall.

[0071] Capturing the current digital environment is a recording or acquisition of user interface elements (and optionally contextual data) present in the digital environment at a given time or within a given time interval.This may include, but is not limited to: a snapshot, which records the complete or partial visual display of one or more screens; the recording of text input through a probe or "keylogger," capturing everything the user types on a keyboard; the capture of real-time audio or video, recording, for example, a stream from a microphone, camera, speaker, earphone, or any other connected sensor or device; content intended to be streamed to a screen (e.g., a video) or via an audio device (e.g., a podcast); the recording of touch or gesture interactions, such as finger movements on a touchscreen or mouse clicks; the acquisition of the states of open applications, including active data within those applications; and the collection of other contextual data, such as the time, GPS location, or network connection status.

[0072] Information filtering is the process of selecting or excluding certain data from among all captured or stored information, or prior to its capture and / or storage, according to predefined or dynamic criteria. Filtering criteria can include: a whitelist or a blacklist, defining the inclusion or exclusion of information based on its membership in a predefined list (by prior or dynamic configuration, for example), such as a predefined list of keywords, the user's geographic location at the time of capture, a specific time of day, date, or period.

[0073] Capturing the past digital environment is based on the same principle as capturing the current digital environment, except that it was performed at a previous time and at least a portion of the captured elements (or information extracted from the captured elements) is stored in such a way that it can be referred to later (for example, the captured elements may be filtered before storage). This capture can include the same types of data and interactions, recorded at past times, and can be used to compare these stored elements with the current digital environment or to recall relevant information.

[0074] Extracting information from a set of user interface elements in a digital environment involves isolating, selecting, or retrieving a specific portion of the information contained within those elements, for example, for later use or specific analysis. For instance, text data extraction refers to retrieving a specific sentence, paragraph, and / or keyword from a text document. Graphic element extraction refers to isolating an image, icon, and / or graphic from a user interface or visual document. Multimedia content extraction is the retrieval of a segment from a video or audio file.It is also possible to extract structured data, such as numerical values ​​and / or specific fields from a spreadsheet and / or a database displayed on the screen, and / or to extract user interactions, that is, specific actions by a user, such as clicking a button and / or selecting an option from a drop-down menu. In short, extracted information is a specific portion of the content of a previous digital environment, isolated to meet a particular need.

[0075] Deriving information from a set of user interface elements in a digital environment involves generating and / or calculating new information from the captured data or elements. This derived information may result from analyzing, transforming, and / or combining the original elements (i.e., the user interface elements captured directly from the digital environment before any modification or transformation by the process described in this application), and may include, but is not limited to: retrieving associated information from a database related to extracted information as defined herein; and generating a summary or synthesis of a text document or set of texts.a calculation of statistics, for example averages or totals from numerical data extracted from a spreadsheet and / or database, a generation of metadata, such as labels or categories, from the analysis of the content of an image, video, and / or text, a visual transformation, by applying filters and / or modifications to a captured image and / or video, thus generating a modified version of the original element, a behavioral analysis of the user, by the use of models and / or the determination of trends from user interactions captured in a previous digital environment.

[0076] Information derivation can be implemented by one of the actions listed above or by a combination of at least two of the actions listed above.

[0077] The actions listed above can be implemented in parallel, that is, independently of one another, or sequentially. As an example of sequential implementation, an image captured in the current digital environment can first be modified by a visual filter to improve its sharpness, and this modified version of the image can then be used as input for object recognition analysis to generate metadata describing the objects present in the image. In this case, the original captured image undergoes a transformation, and then the transformed image (which is itself new information) serves as the basis for another transformation, so that implementing this sequence of transformations results in generating yet another new piece of information (the metadata describing the objects present in the image).

[0078] Information derived from a set of user interface elements of a digital environment is therefore new information, created directly or indirectly from the original elements of the previous digital environment, for example to offer added value and / or a deeper understanding of this data.

[0079] In the context of the proposed technique, the term "obtain" should be interpreted broadly and inclusively. When referring to "obtaining user interface elements from a typical digital environment," this can mean one or more actions, including: visually capturing elements displayed on the screen (via screenshot or the use of an optical character recognition module), receiving data from other systems or sensors (such as navigation or file management sensors), extracting information from active documents or applications, or retrieving specific elements via calls to services or databases, or a combination of at least two of the actions listed above. The term "obtained" thus includes the concepts of transmission, reception, capture, extraction, retrieval, derivation, and enrichment, among others.Similarly, the expression "provision of information obtained based on user interface elements of the previous digital environment" encompasses various methods of information acquisition. This information can be stored in a database, referenced from previously captured information blocks, derived from the analysis of similar contexts, and / or supplemented from multiple information sources.

[0080] Similarity between two digital environments is a measure or assessment of the correspondence or resemblance between those environments. Similarity can be, for example, textual, contextual, temporal, functional, and / or visual. Textual similarity refers to the resemblance of texts present in both environments; for example, similar sentences from a literal and / or semantic point of view in two emails, or recurring keywords in different documents. Contextual similarity refers to the correspondence of situations and / or usage scenarios in the two environments, such as working on the same type of project or using the same application at different times. Temporal similarity refers to the correspondence related to similar times, such as performing a task at the same time of day and / or on the same date, but in different digital environments.Functional similarity refers to the resemblance of functions and / or actions performed in both environments, for example, carrying out similar tasks with different applications or tools. Visual or graphical similarity refers to the resemblance of visual elements or graphical interfaces between environments, such as the use of the same colors, fonts, or screen layouts.In general, in the context of the proposed technique, it may be implemented (during a comparison for example) an assessment of the similarity of the current digital environment with one or more previous digital environments to extract or derive information considered relevant from at least one digital environment at least partially similar to the current digital environment, and provide at least some of this information to the user, proactively, for example to aim to anticipate the user's needs, or to provide them with recommendations adapted to their needs.

[0081] This similarity can be determined according to various criteria. For example, one similarity criterion is co-occurrence, that is, the presence of one or more identical elements in both environments. Other quantitative similarity criteria can include similarity values ​​(used as thresholds), similarity distances or metrics such as Euclidean distance, Jaccard distance, or Levenshtein distance, correlation analysis results, statistical tests, similarity coefficients such as Pearson's correlation coefficient or Spearman's rank correlation coefficient, and so on. For example, each captured context can correspond to a set of n-grams of varying lengths calculated from the text blocks contained in each context, and the similarity between the two contexts is a Jaccard distance between the two sets of n-grams.

[0082] To assess the similarity between a current digital environment and a previous digital environment, various technical means can be implemented. Among these means, the use of a relational database and an autoencoder are two particularly effective approaches.

[0083] A relational database allows you to structure and store user interface elements and contextual data captured in different digital environments. This structure can facilitate the management of relationships between the various stored elements and data, and thus help to perform quick and accurate comparisons when evaluating similarities. For example, a relational database can link text documents, user interactions, and contextual data such as date and time, making it possible to find elements similar to those present in the current digital environment.

[0084] An autoencoder is a type of neural network used to learn compressed vector representations of input data. These compressed vector representations can then be used in place of the original inputs to calculate similarity, for example, using Euclidean distance or cosine similarity. In the context of evaluating similarities between two digital environments, an autoencoder can be used to analyze the complex relationships between user interface elements and the contextual data of the captured digital environments. By learning to recognize similar patterns or structures, the autoencoder can facilitate the identification of relevant similarities, even when these similarities are not immediately apparent to a human.

[0085] Alternatively, a clustering-based approach, or principal component analysis, can be used to assess the similarity between two digital environments.

[0086] Providing information refers to the process by which extracted or derived information is presented or made available to the user. This can be done through various means, including: displaying a notification or pop-up window containing the information on the user's screen; suggesting an action or content in the user interface based on the relevant information; integrating the information into an ongoing workflow, where it is automatically applied or suggested; and presenting the information as text, an image, a link, sound, or any other suitable multimedia format.

[0087] The provision of information can be implemented by one of the above means or by a combination of at least two of the above means.

[0088] The provision of information can be implemented on a variety of computing devices that present the information to the user in an appropriate manner depending on the context: for example, computer, tablet or smartphone monitors or screens, projection or display devices, intelligent voice assistants, etc.

[0089] Reinforcement learning is a continuous process by which a system adjusts its models or decisions based on the results obtained and interactions with the user. For example, if a given piece of information is frequently used by the user, the system can increase the relevance of that type of information for similar contexts in the future. Conversely, information that is rarely used may become less relevant.

[0090] The phrase "based on" in the proposed technique, as in the phrase "providing information obtained based on user interface elements from the previous digital environment," should be interpreted more broadly than simply "based on" or "from." This expression implies a more flexible and adaptable relationship between past and present contextual elements. For example, information can be provided in response to a direct match between similar elements in the current and past environments, but also based on more complex contextual analyses that take into account user preferences, deadlines, or repeated interactions. Thus, "based on" includes not only direct links but also indirect or associative influences, allowing the user's needs to be anticipated based on past trends or behaviors.

[0091] This disclosure relates to a technique that relies on a similarity between a current digital context and at least one past digital context to provide information obtained from user interface elements of that past digital context or contexts and that may be considered relevant to the current digital context.

[0092] Unlike traditional automatic completion systems that focus primarily on predicting words based on immediately preceding text sequences, the proposed technique takes into account a context encompassing not only the text being written but also other user interface elements, and may further include contextual data such as application status, previous interactions, geographical location, and network connection status.

[0093] Unlike existing systems that are often tied to specific applications, the proposed technique can operate independently of any specific application and interact with multiple applications simultaneously for a more seamless integration of contextual information.

[0094] While existing systems are limited to a single type of data, typically textual, the proposed technique can capture a wider range of information, potentially including screenshots, audio recordings, keystrokes, and touch interactions. This allows for the creation of a comprehensive picture of the user's digital activity, thus facilitating recommendations or actions based on a more complete user context.

[0095] The proposed technique is not limited to processing the current digital context in real time but also takes into account analytical elements of past digital environments. Thanks to this, it can not only provide information based on the current digital context but also draw lessons from historical digital contexts, offering a longitudinal perspective that improves the personalization and relevance of the information provided to the user.

[0096] Thus, the proposed technique differs from the state of the art, particularly from existing automatic completion systems and document retrieval systems.

[0097] Reference is now made to the diagram, which represents a possible example of a flowchart for a process suitable for implementing the proposed technique. This flowchart shows different logical modules, each defined by a specific function: a module 10 for periodic triggering, a module 11 for triggering on request, a module 12 for capturing the digital environment, a module 13 for transforming into information blocks, a module 14 for analyzing the content of the information blocks, a module 15 for searching for triggering or inhibiting context elements, a module 16 for making decisions regarding information storage, a module 17 for storing information, a module 18 for halting the cycle related to information storage, a module 19 for making decisions regarding information provision, a module 20 for obtaining a list of stored information, and a module 21 for halting the cycle related to information provision.Module 22 for classifying the list of stored information, module 23 for filtering the list of stored information, module 24 for providing at least one piece of information from the list of stored information, module 25 for obtaining user interaction, and module 26 for learning reinforcement.

[0098] A main module (not shown) can be provided to coordinate the actions of one or more of said modules, in particular as detailed later in this document according to a cycle of storing user interface elements from the current digital context and according to a cycle of providing information obtained based on user interface elements from a previous digital context.

[0099] This list of modules is indicative and provided for illustrative purposes only. Variations are possible, allowing those skilled in the art to adapt the proposed technique to the specific needs of each application. Examples of possible variations are detailed later in this document.

[0100] For the purposes of describing the modules, unless otherwise stated, we are at a current moment defined as the beginning of a cycle of obtaining user interface elements from a current digital environment and the potential provision of at least one piece of information based on a similarity between the current digital environment and a previous digital environment.

[0101] Module 12, the digital environment capture module, is responsible for capturing user interface elements present in the current digital environment, such as active windows, open documents, and user interactions. This module can be configured to capture various types of data: a visual snapshot of the screen, specific contextual data (such as application status or notifications), audio or video streams, cursor movements, clicks, and so on. The capture performed by Module 12 can thus form a data sample, which can be time-stamped and made accessible, at least temporarily, for subsequent processing such as that implemented by Modules 13 through 15.

[0102] The visual snapshot of the screen captured by module 12 corresponds to the screen's state at the current moment. For elements such as audio or video streams, the capture refers to a time interval starting at the current moment and covering a defined duration. This allows for capturing not only the frozen image of the present moment, but also the dynamic context in which events unfold.

[0103] Module 12 can also implement a filtering function, where only certain user interface elements present in the current digital environment are captured, or a masking function, where certain information is intentionally omitted or hidden before capture. For example, Module 12 can be configured to capture only user interface elements specified in a whitelist, such as certain keywords, document types, or specific applications. Conversely, a blacklist can exclude unwanted elements, such as confidential information or sensitive content. This filtering can be based on predefined criteria such as file type, associated metadata, or keywords present in the text or window titles.Alternatively, Module 12 can be configured to use artificial intelligence techniques to recognize and capture only specific content, such as faces, text in a particular language, or predefined graphic elements. This content recognition can help automate filtering without requiring predefined lists, dynamically adapting to the content relevant to the user. Module 12 can be configured to adjust filtering based on context, such as the time of day, the user's location, or current activity. For example, the module can be configured to capture only windows opened during working hours and ignore those opened outside of those hours.This type of filtering can be particularly useful for complying with privacy policies or reducing informational noise by capturing only what is relevant at a given time. Alternatively, or in addition to filtering, Module 12 can include a masking function that anonymizes or hides certain parts of the screen or contextual data before capture. For example, personal information such as names, addresses, or payment details can be blurred or replaced with wildcards. This masking can be applied automatically based on rules defined by the user or the system administrator. Module 12 can be configured to capture data only when the user actively interacts with certain interface elements, such as clicking a window or typing text.This allows captures to focus on meaningful actions and minimize the amount of redundant or unnecessary data. Module 12 can be configured to capture only information from certain applications or types of applications (such as web browsers, text editors, or communication software), excluding other applications deemed irrelevant for analysis or storage.

[0104] Depending on the functions selected for module 12, the capture of user interface elements can be carried out in a flexible, selective and secure manner while adapting to different usage contexts and specific user needs.

[0105] The triggering of a capture can be automated by the triggering module 10 iteratively (for example periodically via regular time intervals) and / or initiated by the triggering module 11 on a user request in response to specific events, such as a user action or a change in digital context.

[0106] The periodic trigger module 10 automates the capture performed by module 12 at regular, predefined intervals. This module establishes a repetitive time cycle, for example, every ten seconds, where at each cycle (or "step"), module 12 is activated to capture the current digital environment. At each periodic trigger, a snapshot of the screen or contextual data is captured (for example, immediately upon triggering), while stream captures (audio, video) begin at the moment of triggering and continue for a specified duration less than or equal to the capture step.

[0107] Request-triggered Module 11 allows you to capture the digital environment in response to a specific user action or request, whether explicit or implicit. For example, an explicit request might be clicking a "Capture" button in the user interface, while an implicit request could be triggered by a specific user action, such as typing a keyword or opening a new application. When a capture is triggered on request, its implementation can begin immediately upon initiation of the request. For instance, a screenshot is taken at the precise moment of the request, and stream captures also begin at that instant to cover the ongoing event.

[0108] In addition to periodic and on-demand triggering, Module 12 can be configured to capture information when specific events occur, such as opening a new application, receiving a critical notification, or a network state change. This event-driven capture allows for the documentation of key moments in the user's digital environment.

[0109] Information Block Transformation Module 13 takes the data captured by Module 12 and segments it into distinct units of information, called information blocks. This segmentation facilitates subsequent processing, such as analyzing, storing, or retrieving information. For example, Module 13 can be configured to extract specific portions of information from the captured elements, such as splitting a document into paragraphs, extracting sections from a data table, isolating images or portions of images, segmenting videos, and so on. Each information block can be identified, cataloged, and thus made ready for further processing. Module 13 can be configured to group information blocks by topic or context.For example, information related to the same project or task type can be grouped together, facilitating its processing and subsequent retrieval. This grouping can be based on keywords, metadata, or semantic content analysis. Module 13 can be configured to perform contextual segmentation, where information blocks are created based on the user's activity or the application used. For example, all items captured while using an email program can be grouped together, separately from items captured while using a word processor. Module 13 can apply further filtering during segmentation, excluding information blocks deemed irrelevant or redundant.For example, repetitive text sections or similar images can be ignored to avoid unnecessarily cluttering the database. It can also be stipulated that certain types of information blocks likely to appear on a user interface, such as menu ribbons or directory listings, are systematically considered irrelevant.

[0110] Module 14, the content analysis module for information blocks, operates after the segmentation performed by Module 13. It analyzes each block to extract patterns, contextual relationships, or information relevant to the user. Module 14 can use optical character recognition (OCR), text analysis, pattern recognition, and / or contextual correlation techniques to understand the content of the information blocks. For example, it can identify keywords, frequent phrases, or connections between different blocks based on their content or context. Module 14 can incorporate machine learning models that adapt the analysis based on the user's past interactions. For example, if the user frequently interacts with certain types of information blocks, the module can adjust its algorithms to give more weight to those types of content in the future.Module 14 can also categorize information blocks based on their relevance, urgency, or perceived importance to the user. For example, critical or frequently used information can be highlighted for faster processing or recall. Module 14 can be configured to link different information blocks together based on context, creating dynamic associations. For instance, information blocks captured while using the same application can be linked together, facilitating their simultaneous retrieval in similar contexts.

[0111] Module 15 identifies, for example within information blocks, context elements called "storage triggers and / or inhibitors" that can respectively trigger or inhibit an information storage action. These storage trigger and / or inhibitor context elements can be associated with different levels of granularity: they can apply to all information blocks captured in a given context, to a specific subset of blocks, or to a single information block. Furthermore, each storage trigger or inhibitor context element can be associated with a weight, ranging from absolute influence (such as the weight of a storage trigger and / or inhibitor context element that must always be stored or never be stored) to intermediate levels (such as a greater or lesser probability of a storage requirement).

[0112] Module 15, for example, can scan blocks of information for specific keywords, dates, locations, or other contextual elements that might justify executing or not executing a storage action. For instance, detecting a due date or a critical mention might lead to prioritizing the storage of the corresponding information for future use. Such detection can rely on predefined rules (based on keywords or regular expressions, etc.). These rules can also be used during the training phase of a model. Once the model is trained, detection can be performed directly by the model itself, using relationships learned during the training phase.A machine or semi-automatic learning model can thus be used, with the effect, for example, of helping, at least in some embodiments, to improve the accuracy and efficiency of detection. Module 15 can also adjust its search criteria based on user preferences or changes in the digital context. For example, it can give more weight to contextual elements related to an ongoing project or a recently started task, thereby favoring the storage of the most relevant information for these contexts. By integrating predictive analytics algorithms, Module 15 can help anticipate future storage needs based on observed trends, such as an increase in the frequency of certain types of data, and proactively decide to store them.Furthermore, Module 15 can assess the redundancy of extracted information to avoid storing data that is already present, thus helping to limit (e.g., optimize) the use of storage resources. Module 15 can be configured to apply filtering to exclude irrelevant or sensitive items, in accordance with policies defined by the user or an organization. This can include obtaining storage-inhibiting context elements with absolute influence, resulting in the exclusion of confidential data or the application of additional masking.

[0113] Information storage decision module 16 is configured to receive trigger and / or inhibitor context elements for an information storage action, identified by module 15. Module 16 uses these context elements to decide, based on this information, whether to store or not store all or part of the information blocks or their extracted content. To make its decision, module 16 can process the trigger and inhibitor context elements for storage using several methods. For example, if an absolutely influential storage inhibitor context element is detected for a specific piece of data, module 16 can be configured to systematically decide not to store the relevant information block, which is equivalent to a filtering function.Alternatively, in such a situation, the module can be configured to store the block in a masked form, thus applying a masking function to hide sensitive information while retaining a partial record. Module 16 can evaluate storage trigger and inhibitor context elements with relative influence by assigning them points based on their importance. For example, a storage trigger context element such as a critical deadline can add points, while a storage inhibitor context element such as information redundancy can subtract them. The final storage decision can then be made, for example, based on the total score: above a certain fixed or dynamic threshold, the information is stored; below it, it is not.Module 16 can also apply a weighting or cost function to the relative influence storage trigger and inhibitor context elements, in a manner known per se. Module 16 can also use probabilistic methods, where the storage decision is based on a calculated probability that the information will be useful in the future, taking into account the relative influence storage trigger and inhibitor context elements provided by Module 15. External modules, using, for example, language modeling (LLM) and / or reinforcement learning techniques, can be used to enhance the collection of storage trigger and / or inhibitor context elements by Module 15 and / or the storage decision-making process by Module 16.

[0114] Information storage module 17 handles the organization and storage of information blocks validated by module 16 (for example, in a database). Module 17 can be configured to ensure that information is not only stored but also structured, for example, to be easily accessible for further processing. For instance, module 17 can index information blocks based on criteria such as content type and / or associated metadata (date, time, source application). Module 17 can apply security measures before storage, such as encrypting sensitive data or implementing access controls. This helps restrict access to stored information (for example, ensuring that only authorized personnel can access stored information), in accordance with privacy and data protection policies.Storage can, for example, be implemented in a document system allowing storage and searching in blocks, or in a knowledge base.

[0115] Module 18, the information storage cycle termination module, manages the end of the storage process once the relevant decisions have been made by Module 16 and executed by Module 17. This module simply ensures the end of the information capture and processing cycle, pending the next capture cycle. Module 18 can, for example, trigger actions to prepare Modules 16 and 17 for the next cycle.

[0116] Modules 12 through 18 work in a coordinated manner according to a cycle related to information retrieval. A typical iteration of the cycle begins when a capture is triggered (for example, just before, during, or just after), whether by module 10 (periodic triggering) or module 11 (request-based triggering). Once the capture is performed by module 12, blocks of information are obtained, possibly transformed, and analyzed by modules 13 and 14. Module 15 then intervenes to identify contextual elements that trigger and / or inhibit storage and that may influence the storage decision. Module 16 then decides whether or not to store the blocks of information, and / or to filter or hide them based on the analyzed contextual elements. The cycle ends when module 16 has made a decision regarding the storage of the blocks of information. If storage is decided, module 17 stores the validated blocks.The cycle can also end earlier, without storage, if module 16 determines that the information is redundant or irrelevant. Module 18 can then, for example, close the information retrieval process for the current iteration. Of course, this closure may not be immediate in embodiments where other actions are implemented during an iteration of the information retrieval process.

[0117] Furthermore, modules 15 and 19 through 26 work in a coordinated manner according to a cycle for providing information based on the similarity between the current digital environment and a previous digital environment, and based on the user interface elements of the previous digital environment. An iteration of the cycle begins when module 15 determines, for the information blocks from the current digital environment, triggering or inhibiting contextual elements that could influence the provision of information. These elements are then passed to module 19, which makes a decision regarding the provision of information. If provision is decided, module 20 extracts a list of relevant information from the database, and modules 22, 23, and 24 manage the classification, filtering, and presentation of this information.Module 25 collects user interactions, and Module 26 adjusts the delivery criteria based on past interactions. The cycle can also end earlier, without providing information, if Module 19 determines that such provision is not relevant given the current digital environment. Module 21 can then, for example, close the information delivery process for the current iteration. Of course, this closure may not be immediate in embodiments where other actions are implemented during an iteration of the information retrieval process.

[0118] The cycles related to information retrieval and information delivery can operate in conjunction. For example, it can be stipulated that the completion of the information retrieval process by module 18 for the current iteration will trigger the completion of the information delivery process by module 21 for the same iteration. In other words, the action of module 21 can be controlled by that of module 18. Conversely, it is also possible for the action of module 18 to be controlled by that of module 21, such that the completion of the information delivery process directly influences the completion of the storage process for the same current iteration.

[0119] Alternatively, the information retrieval and delivery cycles can operate independently, without any interdependence between the two. For example, the closing criteria for the information retrieval process and the information delivery process can be distinct. The implementation of the information retrieval cycle can also be desynchronized from the implementation of the information delivery cycle.

[0120] Module 15 is responsible for analyzing information blocks obtained from the current digital environment to identify contextual elements that can either trigger or inhibit an information delivery action, referred to as "delivery triggers and / or delivery inhibitors." These delivery triggers and / or delivery inhibitors can manifest in various forms, such as keywords, user events, changes in the digital environment, or temporal criteria. Module 15 focuses primarily on analyzing information blocks extracted from the current environment capture to identify delivery triggers and delivery inhibitors that influence the decision to initiate or not initiate an information delivery action.However, to facilitate the identification of contextual elements that trigger and / or inhibit provision, module 15 can have access not only to information blocks obtained from the current digital environment, but also, for example, to information blocks obtained from previous digital environments, to a document database, and / or to a history of information provided during previous iterations of the provision cycle.

[0121] Contextual triggers for information delivery are cues present in the current digital environment that indicate information delivery may be relevant. These triggers can include a variety of factors such as specific keywords, user events, changes in application state, or temporal signals. Module 15 can evaluate these factors to determine if the context justifies initiating an information delivery cycle. For example, Module 15 can analyze the content of open applications or documents to identify specific keywords or terms that have historically been associated with useful information.For example, in an environment where a user is viewing a financial document containing terms like "balance sheet," "budget," or "forecast," Module 15 can identify these words as contextual triggers for information provision, suggesting that providing additional information about financial statements, past projections, or previous financial reports may be relevant. Module 15 flags these terms to signal that the context is favorable for potentially providing information. The application context is another possible contextual trigger. Module 15 can be configured to analyze which applications are currently active and what tasks the user is performing.For example, if the user is working in a project management application or accounting software, Module 15 can interpret this as a signal that providing information could be helpful in supporting that specific task, such as past reports, project deadlines, or invoices. The user's actions within the digital interface can also trigger the provision of information. For instance, if the user performs a search in document management software or navigates to a file related to a specific project, this can be interpreted by Module 15 as a contextual trigger indicating that the current context may require additional information. This analysis can be extended to simpler actions, such as clicking a specific button or navigating within a given application.Module 15 can incorporate temporal factors into its context assessment. For example, if it detects that a deadline is approaching in a document or that a scheduled event in the user's calendar is imminent, these elements can serve as contextual triggers for information delivery. If the current digital environment shows intense activity just before a deadline, Module 15 can signal that the context is favorable for providing additional information to assist the user in managing that deadline. Module 15 can also rely on patterns of repeated activity. If the user regularly performs certain actions in a specific context (for example, reviewing a report at the end of each month to close the books), Module 15 can learn to associate these moments with cycles of providing relevant information.These recurring supply trigger contextual elements allow the system to anticipate user needs without an explicit request being necessary.

[0122] Module 15 can also be tasked with identifying provision-inhibiting context elements—that is, elements that indicate providing information would be inappropriate or unnecessary in the current context. These provision-inhibiting context elements serve to filter (i.e., in this case, eliminate) situations where a provisioning action might cause distractions, be redundant, or interfere with ongoing activity. For example, Module 15 can analyze blocks of information captured in the current environment to determine if certain information is already present or has recently been provided. If the digital environment already contains information that would have been recalled (such as an open document that corresponds to a possible provision), the module can consider this a provision-inhibiting context element, signaling that providing information might be redundant in that context.If the current digital environment indicates that the user is engaged in a specific task unrelated to previously stored and / or provided information, Module 15 can identify this context as a provision-inhibiting element. For example, if the user is working on a given project using drawing or graphic design software, but the information available for that project is exclusively related to financial management, Module 15 can be configured to detect the absence of relevant information and identify this absence as a provision-inhibiting element. Module 15 thus ensures that the information provided is appropriate for the current task. Module 15 can also incorporate privacy policies that prevent the provision of sensitive information in certain contexts.If confidential data is present in the current digital environment, such as personal information or protected data, Module 15 can flag provision-inhibiting contextual elements indicating that providing information in that context would be inappropriate or risk a potential leak. Module 15 can also determine that providing information is no longer relevant due to the age of the captured information. For example, if a project was closed several months ago, and there is no contextual element in the current digital environment that justifies recalling information related to that project, Module 15 may consider this a provision-inhibiting contextual element and decide that no provision is necessary.Module 15 can be configured to operate modularly, meaning that contextual triggers and inhibitors can be applied globally or partially. For example, Module 15 can be configured to inhibit the delivery of certain information while allowing other information to be delivered within a given context. This allows for adaptation to situations where some information may still be relevant while other information is no longer so, thus facilitating the delivery of information more tailored to the user's specific needs and / or constraints. In some variants, Module 15 can detect if the user is engaged in a critical or highly focused task (e.g., during a presentation or online meeting). In such cases, irrelevant information or untimely reminders can be disruptive to the user.Module 15 can inhibit the provision of information to avoid interrupting these important moments.

[0123] Module 15 can incorporate machine learning algorithms to dynamically adjust the detection of context elements that trigger and inhibit delivery. For example, it can learn from previous interactions that certain user actions, such as navigating a specific project or opening a particular file type, are consistent context elements that trigger delivery for certain information deliveries. Conversely, it can adjust its context elements that inhibit delivery based on repeated user behavior (for example, ignoring reminders in specific contexts). Module 15 can be configured to adapt to specific user preferences.A user can choose to enable certain delivery-triggering contextual elements (such as date-based alerts or critical events) while disabling others (such as reminders related to secondary tasks). This customization allows for fine-tuning of delivery-triggering and / or delivery-inhibiting contextual elements based on each individual's usage profile.

[0124] A configuration interface (not shown) can be provided, for example, via a module launched during the initial use of the proposed technique and / or accessible at any time. The configuration interface can allow the user to specify the types of information to be captured and extracted, and their retention period, as well as the types of information to never be captured (related, for example, to privacy or confidential projects). In one variant, the configuration information is stored in a file, and pre-configuration profiles can be offered. Module 15 can be configured to access the settings defined via the configuration interface, thus allowing the user or an organization to manage various storage parameters, thereby helping to ensure confidentiality and compliance with regulatory obligations, such as the GDPR.

[0125] Module 15 can be configured to analyze user behavior in real time to adjust triggers and inhibitors. For example, if it detects an acceleration of user actions (such as a series of rapid clicks or intensive use of an application), this can be interpreted by Module 15 as a signal of urgency or high concentration, and inhibitors can be activated to prevent distractions. Conversely, a period of relative inactivity can trigger an action to recall relevant information.

[0126] Module 15 can leverage the frequency of captures to establish correlations between recurring elements in the digital environment. For example, if a sales report and a project tracking document frequently appear together, Module 15 can associate this recurrence with a trigger to suggest providing information related to these documents in similar future situations.

[0127] Module 19, the information provision decision-making module, is configured to receive the triggering and / or inhibiting contextual elements for an information provision action, identified by Module 15. Module 19 uses these contextual elements to decide, based on this information, whether or not to provide the information. To make its decision, Module 19 can process the triggering and inhibiting contextual elements using several possible methods, similar to those described in the context of the storage / non-storage decision made by Module 16.

[0128] Information delivery cycle termination module 21 intervenes to stop an information delivery cycle iteration if module 19 determines that no information delivery is relevant. Module 21's role can include ensuring efficient management of IT resources by closing the delivery process for the current cycle iteration as soon as a negative decision is made by module 19. For example, module 21 can be configured to release resources related to potential information delivery and prepare for the next cycle iteration. Module 21 can also be configured to manage cycle termination following information delivery and / or in cases of unforeseen interruptions.For example, if the digital environment changes drastically before the provision of information has been completed, it may be provided, by a module (not shown) for detecting such a change in the digital environment, that module 21 interrupts a provision of information initially decided by module 19 to avoid situations of inconsistency.

[0129] When Module 19 determines that information is relevant, Module 20, which retrieves a list of stored information, is activated to query the database and obtain a list of potentially useful information based on stored user interface elements from previous digital environments. For example, Module 20 can extract a list of relevant information from information blocks captured during previous cycles and stored in the database. This information might include documents, reports, contextual data, or any other stored element that could assist the user in their current task. This extraction can be based, among other things, on similarities between the current digital environment and past environments.For example, if the user is working on a specific project and documents related to that project have been accessed in similar contexts, Module 20 will include them in the list of relevant information. Module 20 can also be configured to leverage external or document databases. For instance, numbered documents (such as invoices or contracts) can be pre-stored, and if a document contains a specific identifier that matches the current environment, then Module 20 can be configured to retrieve that document. Module 20 can also be configured to provide only extracts of information in contexts where it is unnecessary to retrieve entire documents. For example, it can extract only the paragraphs or sections of a document that contain information directly relevant to the current task.

[0130] Once the information list is generated by module 20, module 22, which sorts the resulting list, intervenes to classify the information based on its estimated relevance to the user (in the current environment). Module 22 establishes, for example, a hierarchy of the list's information based on criteria such as contextual relevance (directly related to the current task), age (more recent information may be considered more relevant), frequency of use (information consulted regularly may be prioritized), and user responsiveness (information with which the user has had repeated interactions, such as clicks). Module 22 can optionally integrate a dynamic sorting system that adjusts the relevance of the information in real time based on user interactions.Module 22 can allow (in certain embodiments) the user to define (for example, beforehand via a configuration module, or dynamically) customized relevance criteria. For example, a user can give more weight to information related to specific projects or short-term deadlines.

[0131] Once the information in the resulting list has been sorted by module 22, module 23, which filters the resulting list, intervenes to refine it and retain only the information required. For example, the filtering process may involve removing information from the resulting list that is deemed irrelevant, redundant, and / or inappropriate in the current context.Module 23 applies dynamic filtering rules to reduce the list of information to be provided, based on criteria such as: redundancy: if some information has already been provided recently or is already visible in the current digital environment, it may be considered redundant and removed from the list in some embodiments; temporal relevance: information that is too old, although initially classified, may in some embodiments be filtered if its relevance decreases over time, especially if more recent versions exist; confidentiality and / or sensitivity: filtering rules may be provided in some embodiments to exclude sensitive or private information, based on privacy policies, so that, for example, medical or financial information is not recalled in professional contexts.Lists of prohibited keywords, or prohibited regular expressions, can for example be used for this functionality.

[0132] External modules, using for example language models (LLM) and / or reinforcement learning techniques, can be used to improve the assessment of the similarity between the current digital context of use and a previous digital context from which relevant information is obtained and / or to determine the relevance of such information given the current digital context.

[0133] Module 23 can be configured to adjust filtering based on the user's current digital context. For example, during a meeting or presentation to third parties, filtering can, in certain embodiments, be stricter to avoid providing information that might be considered irrelevant or inappropriate.

[0134] Module 23 can incorporate machine learning algorithms that learn user preferences over time. For example, if certain information is consistently ignored or irrelevant in specific contexts, the module can be configured to automatically adjust its filtering criteria to avoid providing that information in the future.

[0135] Module 24, responsible for providing at least one piece of information from the stored information list, is tasked with actually delivering the information to the user after sorting and / or filtering (optional in some embodiments). These types of information can, for example, be determined during system configuration: they can be snippets, small blocks of text, or snippets containing typed information such as names, dates, references, keywords of specific interest, URLs, etc.

[0136] The choice of the amount of information to be provided may depend on the implementation. Thus, in some implementations, it may depend on several factors, such as the relevance of the information in the current digital context, the nature of the interface used to provide the information, user preference, a threshold, etc. For example, when the user is focused on a specific task, module 24 can be configured in some implementations to limit the information provided to only one or two items to avoid distracting the user.In a context where the list contains too much relevant information to be provided all at once, module 24 can schedule the provision of this information over several successive supply cycles, depending on the evolution of the digital context over time or in response to a user action (activation of a "next" button for example).

[0137] Module 24 can also manage how information is delivered to the user, taking into account the type of device used and the current digital context. Module 24 can use various methods to present information in a non-intrusive yet effective way. For example, information can be presented as contextual notifications that appear in a corner of the screen, or, for more complex information requiring interaction, pop-up windows can open to display multiple options. Within a pop-up window, some information can be clickable (for links) or copyable to a clipboard (for other information).If the user is working in a specific document or application, information can be inserted directly into the application, for example, as suggestions in word processing software or references in a table. Information delivery is not limited to visual interfaces. Module 24 can also, for example, provide information via a voice assistant or through tactile or vibratory cues. In this case, the information is delivered using speech synthesis. Such implementations can be particularly useful when the user cannot visually interact with a screen, for example, while driving a vehicle, performing tasks that require the use of hands, using the screen for other purposes, or to accommodate a user's visual impairment.For example, the voice assistant can remind users of upcoming appointments, tasks, or provide practical information such as directions, depending on their needs. As another example, Module 24 can integrate information via connected devices embedded in a vehicle, where the information can be displayed on a dashboard screen or provided through the vehicle's audio system. The information provided can include appointment reminders or instructions based on the destination and driving context, all while avoiding distractions. Module 24 can also utilize wearable devices such as smartwatches. For example, a reminder for a meeting or task can appear on a smartwatch as a vibration or a discreet visual notification.For example, Module 24 can be configured to tailor information delivery to the user's display preferences, which can be set via a dedicated configuration interface. For instance, the size, position, display duration, refresh rate, and / or scrolling speed of notifications can be adjusted based on preferences or the type of task being performed. Module 24 can also include a smart display that adjusts how information is presented based on the context. For example, in a highly focused environment (such as a presentation or meeting), notifications can be more discreet or delivered only via a voice assistant to minimize interruptions.

[0138] It is understood that information is provided to the user via one or more user interface elements. Therefore, it can be stipulated that the capture operations implemented by module 12 automatically exclude user interface elements used for providing information to the user, which is managed by module 24.

[0139] Module 25, the user interaction acquisition module, monitors and collects user interactions with the provided information, for example, to adjust future system behavior. Such implementations can contribute to improving the relevance of the information provided in subsequent cycles. Module 25 can record explicit interactions, such as clicking a notification, opening a pop-up window, and / or using a voice command. Alternatively or in addition, Module 25 can record implicit interactions in certain implementations. For example, a lack of user interaction with the provided information can be interpreted as an implicit non-engagement interaction, indicating that the information was ignored. The same applies to a delayed interaction.The assessment of whether an interaction is late can be defined absolutely, for example, beyond a certain duration (used as a threshold), or relatively, compared to the average or usual duration of user interactions with the information provided or with a particular type of information provided. Based on the interactions collected, Module 25 can adjust the provision of future information. Such embodiments can help the process of this request respond more appropriately to user needs. For example, in some embodiments, if certain information is consistently ignored in a specific context, Module 25 can signal this behavior to other modules to lower the priority of this information in subsequent cycles. Module 25 can be configured to accept interactions via different interfaces.For example, when information is provided via a voice assistant, the user can interact by giving voice commands to request more details, dismiss reminders, or obtain additional information. For example, the user can say "Show me the meeting details" or "Cancel this reminder" to interact with the system. If the information is provided on a connected device such as a watch, the user can interact via touch gestures, such as swiping to dismiss a notification or performing specific gestures to open additional details about the information provided. When a user is in a vehicle, Module 25 can be configured to capture interactions via the dashboard or the vehicle's integrated voice system. For example, the user can verbally request information about their appointments while driving or confirm a task via a command on the dashboard.Module 25 can also analyze user behavior in real time to adjust information delivery. For example, if the user seems focused on a specific task and the module detects low engagement with the information provided (such as not reacting to notifications or closing them quickly), this may signal a need to reduce the amount of information recalled or adapt the delivery method (such as switching from a visual notification to a voice notification).

[0140] Module 26, the learning reinforcement module, intervenes to dynamically adjust the delivery criteria based on the user's past interactions, thereby improving the relevance and personalization of the information provided in future cycles. Module 26 collects data on user interactions captured by Module 25 and analyzes it to detect usage patterns. For example, in some embodiments, if it finds that certain information is accessed more frequently than other information in similar contexts, Module 26 can increase the priority of that information in the future.Module 26 can incorporate a continuous learning model, where the system continuously learns and adjusts its delivery criteria based on interactions accumulated over a time window (e.g., a long-term time window), without needing to reset or relearn user preferences. The window size can be defined manually or through machine learning, and can range from a few hours to several weeks or months. For example, Module 26 can also anticipate future user needs based on a number of previous cycles (e.g., a large number of cycles based on day, hour, or week). Such an approach can help identify recurring or cyclical actions.For example, module 26 can predict, in some embodiments, that particular information will likely be needed at a specific time or in a recurring context (such as the end of the month for financial reports).

[0141] The example embodiment shown in the detail diagram represents a possible flowchart of a process suitable for implementing the proposed technique. Alternatively, the process can be simplified, reducing it to the combination of an input interface and an output interface.

[0142] The input interface can be reduced to a module configured to obtain user interface elements from the current digital environment. This module can be a simple routine that intercepts elements visible on the screen, such as active windows or open documents, or that receives information already captured and / or processed directly from an external system.

[0143] The output interface can be limited to a module configured to decide whether or not to provide information relating to a previous digital environment to the user by applying a similarity criterion (predefined for example) between the current digital environment and stored information relating to the previous digital environment.

[0144] Such an embodiment offers the advantage of requiring reduced hardware and software resources, making the process lighter and more responsive while retaining the functionalities of obtaining user interface elements from the current digital environment and providing information.

[0145] The proposed technique can be implemented in a single computer device or in a system comprising a plurality of computer devices, each fulfilling specific functions.

[0146] For example, a single computing device can be configured to integrate all the functions necessary to obtain user interface elements from a common digital environment, to process these elements, and to provide information.

[0147] Any suitable hardware and / or software may be used for the practical implementation of said device. In general, although aspects of the proposed technique may be described in this document as a process, device, system, procedure, method, or technique, it should be noted that the proposed technique may also cover computer memory that can be connected to a processor possibly connected to a communication interface, the memory storing instructions which, when executed by such a processor, enable the implementation of the processes, devices, systems, procedures, methods, or techniques described in this document.

[0148] Examples of devices suitable for implementing the proposed technique include, for example, but not limited to, personal computers, cloud servers, virtualization platforms, mobile devices such as smartphones or tablets, embedded systems for example in vehicles, augmented reality headsets, etc.

[0149] Such a device 30 includes, as illustrated in the figure, the following elements: an input interface 33 coupled (connected) in particular to a source 35, an output interface 34 coupled in particular to a human-machine interface 36, a processor 31 or processing unit, and a memory 32 storing instructions in the form of a computer program.

[0150] In this implementation, when these instructions are executed by the processor, they allow the processor to control the input interface to obtain user interface elements from a typical digital environment.

[0151] Furthermore, in this embodiment, when these instructions are executed by the processor, they enable the processor to control, based on a similarity between the current digital environment and a previous digital environment, the output interface to provide at least one piece of information obtained based on user interface elements of the previous digital environment.

[0152] The input source 35 can correspond to various types of inputs depending on the context of use. For example, it could be a standard user interface (keyboard, mouse, touchscreen), an image source (camera, screenshot, or multiple screens), an audio input (microphone for capturing voice commands or ambient sounds), or contextual sensors (such as a time clock, a GPS location sensor, or environmental sensors that collect data on brightness or temperature). In other cases, the source may also include live video feeds or information from surveillance or recording systems. The source may include a preprocessing circuit configured, for example, to filter, compress, or transform captured signals before they are transmitted to the processor 31 via the input interface 33, in order to optimize their subsequent processing by the processor 31.For example, a preprocessing circuit can be configured to improve the quality of captured data (audio, video, image for example) and / or to extract specific features, such as visual patterns or acoustic signatures.

[0153] Human-machine interface 36 can refer to a wide range of devices that enable interaction with the user. This includes display screens (monitors, touchscreens, dashboards in vehicles), projection systems, virtual or augmented reality headsets, audio devices (speakers, earphones) for reproducing sound information, and haptic feedback systems that provide tactile feedback. This interface can also integrate more specific devices such as voice commands, visual guidance systems, or virtual assistants, depending on the user's needs and the context. Human-machine interface 36 may include a post-processing circuit configured, for example, to prepare and adapt signals for presentation to the user.For example, in the case of a visual display, a post-processing circuit can be configured to adjust the resolution, contrast, or apply graphic filters before presenting the information to the user. For an audio output, it can be configured to optimize sound clarity or adjust the volume.

[0154] The instructions may also cause, when executed by the processor, the implementation of the functions of one or more of modules 10 to 26.

[0155] Alternatively, the input interface and output interface can be provided by two separate devices configured to communicate with each other, either directly or via an intermediate device.

[0156] For example, a system for implementing the proposed process may include, as illustrated in the figure, the following elements: a first computer device 30 configured for obtaining user interface elements from a current digital environment and including at least the input interface 33, and a second computer device 40 configured for providing relevant information to the user and including at least the output interface 34, and a direct or indirect communication channel between the first device and the second device.

[0157] In such a configuration, the system may include at least one processor 31, 41 connected to at least one memory 32, 42, for example, within the first device and / or the second device and / or an intermediate device. The first device, the second device, and / or the intermediate device may be configured to handle additional functions to optimize information processing and its delivery to the user. These functions may include, for example, intermediate processes for transforming, filtering, or classifying information obtained from the current digital environment, or processes related to analyzing the user's needs and preferences.

[0158] The first device can also be configured to analyze and process the captured information before transmitting it to the second device. For example, it can include modules implementing functions for transforming information blocks (module 13), content analysis (module 14), or searching for triggering or inhibiting contextual elements (module 15).

[0159] The second device can also include functions related to the personalization and prioritization of information. For example, it can be equipped with modules responsible for classification (module 22), filtering (module 23), or personalization of delivery methods (module 24).

[0160] An intermediate device, if used, can act as a processing unit dedicated to data storage and management. For example, such a device can implement the functions of modules 16 (storage decision), 17 (information storage), and 18 (storage cycle stop).

[0161] Furthermore, the intermediate device, or the first or second device, or a combination thereof, can also support adaptive learning functions via Module 26 of learning reinforcement. For example, by monitoring user interactions and dynamically adjusting the criteria for providing information, the device can refine the information provided in real time, helping to improve the user experience (e.g., through a more personalized and relevant experience at each iteration of the cycle).

[0162] Reference is now made to the one that illustrates an example of a system suitable for implementing the proposed technique, in an example of an implementation.

[0163] The illustrated system has a functional architecture composed of a set of functional modules: a main module 50, a screen capture module 51, an image analysis module 52, including for example an optical character recognition module 53 and an image block cutting module 54, an activation and / or inhibition decision module 55, a storage module 56, a search and recommendation module 57, including for example a similarity search module 58 and a block selection module 59, and a human-machine interface 60, including for example a display management module 61 and a user feedback management module 62.

[0164] The main module 50 coordinates the actions of the other modules in the system. It ensures that each module communicates and works in a synchronized manner for the smooth running of operations.

[0165] The screen capture module 51 is responsible for regularly capturing copies of one or more screens of a user device. Captures are performed, for example, every T seconds, thus providing a dynamic view of the current digital environment. The captured images are then transmitted to the image analysis module 52 for processing. The screen capture module 51 represents a specific implementation of the digital environment capture module 12 described in [reference missing], in which the current digital environment includes an image of the user's screen.

[0166] The image analysis module 52, similar to modules 13 and 14, is responsible for analyzing captured images. It can be implemented in several ways, with varying levels of complexity. For example, module 52 might include an optical character recognition (OCR) module 53 configured to extract text blocks from the captured images. Module 52 might further include an image block-slicing module 54 configured to receive as input one or more text blocks, previously extracted by module 53 from a captured image, and to output a new segmentation of the captured image into text blocks. Module 54 might, for example, take into account predefined rules and / or predefined parameters to perform the segmentation.Module 54 can, for example, be configured to identify multiple themes within a single input text block and to decompose that text block into smaller blocks. For instance, the decomposition parameters can cause Module 54 to break down certain blocks into atomic information, such as names, URLs, references, links, etc. Conversely, Module 54 can be configured to group multiple input blocks that relate to the same theme.

[0167] Alternatively, module 52 can include an automatic segmentation module trained by deep learning, using numerous example images and associated text blocks, to perform end-to-end automatic segmentation of a captured image into blocks. The automatic segmentation module can, for example, be based on a computer vision model coupled with a Transformer-like architecture.

[0168] Module 52 may include a filtering module for extracted blocks, allowing selection of the blocks of information actually retained. For example, the filtering module may be configured to automatically recognize and prune certain text fields such as menu ribbons, headers, footers, directory lists, etc. When Module 52 includes an optical character recognition (OCR) module 53 and a block-based image segmentation module 54, filtering may be implemented before OCR, or after OCR and before block-based image segmentation, or after block-based image segmentation. When Module 52 includes a deep-learning-trained automatic segmentation module, filtering may be implemented before or after automatic segmentation, or the automatic segmentation module may be trained to perform segmentation and filtering simultaneously.

[0169] Module 52 can also be configured to retrieve block-related metadata, such as the position of blocks on the screen or text styles appearing in blocks, or to retrieve contextual information from captured images such as the date, the user's geographic location, or the version and / or language of the operating system being used.

[0170] Module 55, the activation and / or inhibition decision module, analogous to module 15, is responsible for determining in real time whether the system's recording and recall functions should be active or not. Module 55 can, for example, take into account rules (e.g., predefined) and / or parameters (e.g., predefined) that define the conditions under which the system's recording and recall functions are active and / or the conditions under which they are inactive.

[0171] In one example, these functions are active if and only if: certain trigger keywords and / or certain allowed applications are detected on the screen at the current time, and no inhibitor keyword and / or no prohibited application is detected on the screen at the current time.

[0172] Triggering keywords can include, for example, project names, technical vocabulary, identifiable references with regular expressions, URLs, etc. Inhibiting keywords can be grouped by default into a blacklist covering private use cases of the workstation (medical appointment, urgent school / personal message, banking application, etc.).

[0173] The Storage Module 56 is responsible for managing the recording and organization of blocks, metadata, and contextual information in a persistent database. Within the database, each block is associated with the digital environment at the time the image from which it originates was captured. Each block can also be associated with a utility level. A block's utility level is likely to change over time, influencing the probability that the block will be recalled in similar future contexts. While the example detailed here primarily concerns the processing of image blocks, the Storage Module 56 can also be configured to store blocks other than image blocks. For example, blocks of the current environment can be obtained by temporally segmenting an audio or video stream captured during a time interval corresponding to the current moment.Module 56 can, for example, take into account rules (predefined, for instance) and / or parameters (predefined, for instance), defining, for example, the size of the database, the nature of the data stored in it, and / or the retention period for the stored data. Module 56 can also be configured to pre-inject external document sources into the database to enrich the available information, such as document repositories and / or project-specific files.

[0174] Module 57, the search and recommendation module, is responsible for searching for and / or selecting relevant information based on the current digital environment. Module 57 may, for example, include a similarity-based search module 58, which identifies previous digital environments similar to the current one. These previously identified digital environments are then sorted by similarity. Module 57 may also include a block selection module 59, which selects, from among the blocks associated with the previous digital environments identified by module 58, the most useful blocks that are not redundant with each other or with the information already present in the current digital context.Parameters (predefined for example) and / or rules (predefined for example) can also be applied to prioritize blocks containing certain types of information, such as links, references and / or names of people.

[0175] Alternatively or in addition, module 57, the search and recommendation module, may include one or more autoencoders pre-trained to associate blocks of previous digital environments with a vector representation (or "embedding") of those blocks. A vector representation, extracted from the blocks by an autoencoder, constitutes, in a condensed form, an abstraction of the significant and relevant features of those blocks for the intended task. Since this representation is not directly observable in the input data, it is described as "latent" or "hidden."This or these auto-encoders can be used to associate blocks of the current digital environment with a set of latent vector representations of blocks from previous digital environments, in a relational way, and to identify, on the basis of this association, the previous digital environments most similar to the current digital environment.

[0176] Module 57 can also be configured to take into account a date and / or time indicated in a block of a previous digital environment, and automatically recall one or more blocks of that previous digital environment when the current date and / or time is the one indicated.

[0177] The human-machine interface 60, similar to the information delivery module 24, is responsible for interacting with the user. The human-machine interface 60 may include a display management module 61, which presents the selected blocks as snippets or other summaries. The human-machine interface 60 may also include a user feedback management module 62, which records the user's interactions with the presented blocks. The human-machine interface 60 can also be configured to launch other applications when the provided block includes a link or a document to be opened. Interactions can also be used to adjust the utility level of the blocks: for example, interacting with a presented block may increase its utility level, while not interacting with a presented block may decrease its utility level.Optionally, it may be planned to cyclically purge the database of blocks (and associated digital environments) whose level of utility is considered to be very low or zero.

[0178] Finally, the system can be adapted to manage multiple users in a shared environment. For example, specific settings can be applied to distinguish one user's digital environment from another's, using metadata such as user IDs or access times. This ensures that the information provided to each user is relevant to their individual digital environment.

[0179] Some examples of use cases are now provided.

[0180] The first use case involves replying to an email.

[0181] Alice receives an email from a new contact, Bob. On the screen, Bob's digital signature (name, address, phone number, etc.) appears. This signature is captured, along with a response deadline in the email body (the day after tomorrow...). To reply to this email, Alice opens a file and views the associated document: the link to this file is captured. Similarly, when Alice searches the internet to verify other necessary information, her browser's URL is also captured. All these associations are stored and linked together, including the contact's name and email address, as well as the response deadline.

[0182] Alice has other urgent matters to attend to, and interrupts her response to Bob.

[0183] The process outlined in this application can help Alice return to her task in several ways:

[0184] The next day, Alice spontaneously starts an email to Bob. As she types his email address, the relevant information (Bob's email signature, the response deadline, the link to the file she had opened on the subject, and the URL she had consulted) appears in a pop-up window in the corner of the screen. Alice can then view or copy / paste this information to quickly get back into the flow of her work.

[0185] In some embodiments, the process of this application is capable of sending a reminder to Alice, starting from the response deadline that has been captured and automatically recognized as a date, either on the same day or the day before. In this case, a pop-up window titled, for example, "Reminder" appears, displaying the paragraph in which this date was captured, Bob's name, and other contextual information (signature, link to the document, URL).

[0186] A second use case involves retrieving context following a tele-training session.

[0187] Alice completed an online training course two weeks ago on a specialization in labor law. During this course, various information was displayed on her screen, such as excerpts from legal texts, case studies, and other example situations. Today, Alice receives an email from a colleague asking for advice on a problem similar to one she covered in her course two weeks ago. The similar situation discussed in the course, along with the relevant legal texts, is presented in pop-up windows.

[0188] A third use case involves retrieving information about a product.

[0189] Alice performs internet searches (for example, on a smartphone) and retrieves several pieces of information about herself, such as her price, and various technical details that correspond to keywords she wants to retrieve.

[0190] Subsequently, Alice begins writing a report on this product; previously viewed information relevant to her writing context is displayed and offered for copy / paste as she goes along, or simply as information to be taken into consideration during the writing process.

[0191] These technical solutions can be applied in various types of services where they can contribute to improving information management, operational efficiency, and / or user experience. Examples of possible service types include office automation, accounting, telemedicine, digital marketing, IT security, industrial maintenance, and more.

[0192] This disclosure is not limited to the examples described above, which are merely examples, but encompasses all the variations that a person skilled in the art can consider in the context of the protection sought.

Claims

Method for supplying at least one piece of information, the method comprising: obtaining, by at least one input interface of a first computer device, user interface elements of a current digital environment, and based on a similarity between the current digital environment and a previous digital environment, supplying, by at least one output interface of a second computer device, at least one piece of information obtained based on user interface elements of the previous digital environment. Method according to the preceding claim, comprising filtering of user interface elements of the current digital environment. A method according to any one of the preceding claims, comprising the use of a database storing the user interface elements of the previous digital environment. A method according to any one of the preceding claims, comprising the use of a pre-trained machine learning model to identify a relationship between user interface elements of the previous digital environment. A method according to any one of the preceding claims, comprising: obtaining prior information based on the user interface elements of the prior digital environment, prioritizing the prior information obtained based on a criterion of relevance relative to the current digital environment, and providing, by at least one output interface, at least one of said 'prioritized prior information'. A method according to claim 5, wherein the prioritization is adjusted based on user interaction or lack of interaction with information previously provided by at least one output interface. A method according to any one of the preceding claims, wherein the acquisition of user interface elements is triggered periodically and / or taking into account the current digital working environment. A method according to any one of the preceding claims, wherein the second computer device is different from the first computer device, or the first computer device and the second computer device correspond to the same device. Computer device comprising at least one microprocessor configured to: obtain user interface elements from a current digital environment via at least one input interface of said computer device, and provide, based on a similarity between the current digital environment and a previous digital environment, via at least one output interface of said computer device, at least one piece of information obtained based on user interface elements of the previous digital environment. Computer device according to claim 9, wherein said computer device is a communication terminal. System comprising: at least one first computing device comprising at least one microprocessor configured to obtain user interface elements from a current digital environment via at least one input interface of at least one first computing device, and at least one second computing device comprising at least one microprocessor configured to provide, based on a similarity between the current digital environment and a previous digital environment, via at least one output interface of said at least one second computing device, at least one piece of information obtained based on user interface elements of the previous digital environment. Computer program comprising instructions for the implementation of all or part of the method according to any one of claims 1 to 8, when this program is executed by a processor. Non-transient, computer-readable recording medium on which a computer program according to claim 12 is recorded.

Images