Recommendation of text data during acquisition

Abstract

It is proposed an information processing apparatus for providing assistance to the acquisition (S13) of requested text data, the method comprising:-establishing a first semantic graph (S5) based at least on the requested text data,-obtaining text data (S2) from data delivered by at least one source, and-establishing at least a second semantic graph (S4) based on the obtained text data, -performing a search (S6) of at least partial similarities between the first semantic graph and the second semantic graph to recognize at least some of the required text data in the obtained text data (S7), and-generating a recommendation based on the recognized text data.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing by various computer applications. Background Art

[0002] In everyday life, especially professional life and especially in the context of "digital enterprise", users need to perform diverse digital processes. In particular, they can contribute to multi-application or multi-platform projects (whose execution may require multiple applications (or multiple platforms)).

[0003] For example, a user may be required to open an application, search for information useful for a common task, and then open a second application and enter text data by copying some of the previously found information.

[0004] In another example where a user is reading a document or message in a conversation, the user may trigger a parallel action in another business application to enter text data related to the viewed document or message (or vice versa).

[0005] User management of these multiple applications can be cumbersome, and thus a computerized component is sought for facilitating navigation between these applications. Summary of the invention

[0006] The present invention improves this situation.

[0007] To this end, a method implemented by an information processing device is proposed, comprising:

[0008] - obtaining text data (S2) based on data delivered by at least one source, and

[0009] - Based on semantic similarities between at least some of the obtained data and at least some of the data required by at least one currently running computer application, issuing recommendations related to the required data, at least one user interface of the device allowing interaction with the at least one currently running computer application.

[0010] In at least one embodiment, the method may include recognizing at least a portion of the required text data in the obtained text data (S7), and the recommendation is based on the recognized text data.

[0011] In at least one embodiment, the requested data is data currently being written by a user of the device.

[0012] In at least one embodiment, the recommendations include suggestions for elements to correct and / or fill in the required data.

[0013] In at least one embodiment, the semantic similarity considers similarity between a first semantic graph established for the obtained data and a second semantic graph established for the required data.

[0014] Therefore, in at least one embodiment, the method includes:

[0015] - establishing a first semantic graph based at least on the required text data,

[0016] - obtaining text data from data delivered by at least one source, and establishing at least a second semantic graph based on the obtained text data,

[0017] - performing a search for at least partial similarity between the first semantic graph and the second semantic graph to identify at least part of the required text data in the obtained text data, and

[0018] - Make recommendations based on the recognized text data.

[0019] Here, the term "building a semantic graph" is understood to mean implementing a semantic analysis (rather than a simple lexical analysis) that structures the text data into, for example, a tree structure. Thus, the aforementioned similarity search may include identifying one or more branches that include common elements (e.g., nodes) in the first graph and the second graph. Building such a graph generally allows building a structure for the required text data to easily identify text data from a source that may correspond to these required data.

[0020] The data transmitted by the source is considered reliable and can therefore constitute reference data. Therefore, the data from the first figure can be filled in and / or corrected based on the data from the second figure. The method then provides assistance to the user to facilitate the user to enter reliable text data, which can be required by an application such as in the embodiments shown below.

[0021] In at least one embodiment, the method may then include:

[0022] - establishing a first semantic graph based at least on the text data to be verified,

[0023] - performing a search for at least partial similarity between the first semantic graph and the second semantic graph to identify at least part of the text data to be verified in the obtained text data, and

[0024] - In case the data from the recognized portion differs from the corresponding obtained text data, a correction recommendation is issued.

[0025] Thus, in at least some embodiments, such implementations may assist, for example, in detecting errors in requested data that is typically in the input process, and corrections for those errors may then be recommended.

[0026] As described above, the required data may be required by, for example, a currently running computer application that is requesting the required data. In at least one embodiment, the method may then also include collaborating with such a computer application. For example, such an application may manage the presentation of an online form on an active web page, and will therefore wait for (i.e., request) data entered by a user. According to another example, such an application may convert image data currently being captured by a camera into text data (via optical character recognition). These images may represent, for example, characters currently being entered by a user on paper (e.g., a paper form) or a whiteboard, particularly handwritten characters. For example, in this case, it is advantageous to implement verification of text data already entered by the user to suggest corrections or even recommend new data to be entered.

[0027] To generate these recommendations, in one embodiment, the method may include:

[0028] - manage the human-machine interface connected to the information processing device, and

[0029] - sending a message corresponding to the recommendation via a human-machine interface.

[0030] The message may trigger a suggestion of data to be entered, proposing to the user text data to be provided to the computer application. Figure 6 An example of such an embodiment is shown in Figure 6 A window with helpful suggestions is shown that opens (in the form of a "pop-up window") when a user attempts to enter data, and if the user accepts the suggestions made, the online form is filled with the required data.

[0031] Depending on the embodiment, the human-machine interface may be an interface integrated into the device or an interface coupled to the device via wired and / or wireless communication means.

[0032] Thus, in some exemplary embodiments, the human-machine interface may include a screen displaying at least a first window specific to a computer application, and the method includes:

[0033] - controlling the recommendation to be displayed in a second window on the screen simultaneously with the first window.

[0034] Such embodiments may help improve the ergonomics of the user interface, at least in some embodiments, with recommendations being displayed (eg, transparently) eg, "on top" of a window specific to the current application.

[0035] Additionally or alternatively, in some embodiments, the above recommendation may directly include the recognized text data, and the method includes:

[0036] -Provide the recognized text data to the computer application.

[0037] Thus, for example, text data requested by an application is provided directly to the application without user intervention. Figure 6 In the example shown, the data of the destination city ("Rennes") and the travel date ("January 7, 2022") are filled directly in the online form. The user can of course correct this data later if necessary.

[0038] exist Figure 6 In this example, the required data appears in Figure 2 In the image shown, it corresponds to, for example, an instant messaging chat. This image is considered here as the source of reference data, and Figure 3 An online form to be filled in (eg, accessible via a web page) corresponds to an application requesting the required data.

[0039] Semantic analysis of the required data can produce the first graph mentioned above, and Figure 2 Semantic analysis of the chat in can produce the second figure mentioned above.

[0040] In at least one embodiment, the first semantic graph and the second semantic graph are tree structures and include:

[0041] * nodes representing predicates, and

[0042] *Leaves that represent text data and are responses to predicates.

[0043] The method may then include:

[0044] - identifying nodes having a common predicate between said first graph and said second graph, and

[0045] - select leaves from nodes with a common predicate in the second graph,

[0046] The selected leaf corresponds to the portion of text data required by the obtained text data.

[0047] Thus, in such an embodiment, a search for common nodes is performed in the graph to infer leaves from those nodes in the second graph and corresponding to relevant responses in order to provide the requested data.

[0048] exist Figure 4 and Figure 5 In the example of , common nodes under the d / date-entity branch corresponding to future destination dates can be identified. Figure 4 The date in the second figure gives the Figure 5Therefore, the date (e.g., the default date) of the first graph, February 10, 2022, must be corrected to the date given by the second graph, January 7.

[0049] For example, these first semantic graph and the second semantic graph may be in the form of an Abstract Meaning Representation (AMR) computer language.

[0050] The data delivered by the source may be in different forms. For example, these may be image data containing text, and the method may then include:

[0051] - Process these image data through character recognition to obtain text data.

[0052] Thus, the reference text data may originate from images in any format, such as digital photographs or digital videos.

[0053] Additionally or alternatively, in some embodiments, the data communicated by the source may be audio data, and the method may include:

[0054] - Processing the audio data to detect speech signals and convert the speech signals into text data.

[0055] In this case, the reference text data are derived, for example, from an audio recording.

[0056] Additionally or alternatively, in some embodiments, the data communicated by the source may be from a computerized device for capturing handwritten characters, and the method may include:

[0057] -Implement character recognition on handwritten characters to convert the input characters into text data.

[0058] For example, the reference text data here may come from a connected whiteboard or graphics tablet with handwritten character recognition, or some other source.

[0059] Additionally or alternatively, in some embodiments, the required text data itself may come from analysis of the aforementioned type of image data, or from audio content, or from a file acquired through handwritten character recognition.

[0060] Of course, the reference data and / or the requested data may come from an editable text file (e.g. in "doc" or "docx" format) or from a non-editable text file (e.g. in "pdf" format). In one embodiment, at least part of the obtained text data may be stored in a memory of the information processing device together with data relating to the type of source from which they were transmitted. Thus, for example, in the event of a conflict between corresponding responses given by different sources, certain reference data (e.g., non-editable reference data) may "take precedence" over other reference data (e.g., over editable text that may typically contain input errors). According to another example, reference data from a trusted source, such as an official source (e.g., an institutional site or a government site), may take precedence over reference data from any other source.

[0061] In one embodiment, data transmitted by at least one source is stored in a memory of the information processing device within a first time period (which may be selected and, for example, configurable) after the method was last implemented using the method.

[0062] Such an embodiment allows the most frequently consulted data to be kept in memory (eg in a buffer) and reference data that has not been used since the aforementioned first period of time to be erased.

[0063] In one embodiment, the method may include implementing an artificial intelligence programmed to learn relevant recommendations based on user feedback and, after learning, select relevant recommendations to issue based on the recognized text data.

[0064] Thus, for example, if there may be multiple leaves when there are more than two graphs to compare, artificial intelligence can help immediately eliminate leaves corresponding to recommendations that would be rejected by the user.

[0065] According to another aspect, there is provided a computer program comprising instructions for implementing all or part of a method as defined herein when the program is executed by a processor. According to another aspect, there is provided a non-transitory computer readable storage medium having such a program stored thereon.

[0066] According to another aspect, an information processing apparatus is provided, comprising a processing circuit configured to implement the method according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Other features, details and advantages will become apparent upon reading the following detailed description and analyzing the accompanying drawings, in which:

[0068] Figure 1

[0069] [ Figure 1] shows an example of a method according to one possible embodiment.

[0070] Figure 2

[0071] [ Figure 2 ] shows an example of a reference data source used to establish the "second graph" defined above.

[0072] Figure 3

[0073] [ Figure 3 ] shows an example of a data acquisition context, here an application is executing and requesting text data.

[0074] Figure 4

[0075] [ Figure 4 ] shows that from Figure 2 The "second graph" created by the sources shown.

[0076] Figure 5

[0077] [ Figure 5 ] shows that from Figure 3 A “first graph” (as defined above) is built with the data required for the context.

[0078] Figure 6

[0079] [ Figure 6 ] shows interface management for providing recommendations to users.

[0080] Figure 7

[0081] [ Figure 7 ] shows an example of an information processing device for implementing the method. DETAILED DESCRIPTION

[0082] Figure 1 The method for implementing the purpose of the present application by an information processing device is shown by way of example and in a non-limiting manner. Figure 1 During a first step S1, a context of use is detected. This context can be detected, for example, if an application requiring text data is currently running (test in step S11), and, for example, if a user of the device has recently viewed or is still viewing a document (test in step S12). The document can be a text in a non-editable format (e.g., "pdf"), or a digital image, or a text in an editable format, etc. Here, "recently" is understood to mean that the viewing of the document occurred, for example, less than an hour ago, or less than a few days ago, depending on the relevance of the document (as shown below with reference to steps S8 and S9).

[0083] In this example, as part of the execution of the application in step S11, the user may enter text data. For example, the application may ask questions via the human-machine interface of the device and expect a user response to be entered. As an example, this may be an online form to be filled in by the user, as shown below with reference to Figure 3 Therefore, in step S13, the application requests text data (TXT1).

[0084] Of course, determining the context in steps S11 and S12 is an optional example and is therefore indicated by a dotted line. In a variant, the user can write information, for example, on a whiteboard or blank paper, and a camera (e.g., a camera on smart glasses) captures the information being written. Within the meaning of step S1, the context can also be identified, and recognition of the characters written on the paper or board can be performed to:

[0085] - determine the text data the user is currently writing, and

[0086] - Identifying whether such data (for example a form filled out manually by a user in one exemplary embodiment shown below) can be corrected or filled in directly.

[0087] Therefore, still in such an embodiment, within the meaning of step S13, text data is required so that verification and possible correction of the data currently being input can be performed, or the data currently being input can be filled in. Verification of the data currently being input and / or filling in the data currently being input can be performed based on reference text data, as described below.

[0088] Reference now Figure 1 Step S2, reference text data TXT2 comes from one or more sources (which are different from, for example, currently being executed applications). For example, these sources can be files at least temporarily stored in the memory MEM of the device according to this specification, or web pages that the user has recently (or frequently) viewed. For example, files (image files containing text, or editable text files, or other files) can be stored in a dedicated memory area for access by the method described herein, so as to identify text data TXT2 that can be used as reference data therein. In the case of, for example, image files, character recognition can help determine whether the text data generated by the conversion of images to text can be used as reference data. Optionally, these files (or links to these files) can be stored in the above-mentioned memory area, where the index is related to the nature of the data that can be used, and the data that can be used are, for example, as follows:

[0089] - Data specific to the user’s identity (data from the user’s identity documents, or social security number, or other data);

[0090] - Data acquired in an occupational context;

[0091] -wait.

[0092] Reference now Figure 1 In step S14, the questions asked during the execution of the application or the text data being written by the user are semantically analyzed so that a semantic graph can be created in step S5. The semantic graph GRAPH1 can be, for example, in the form of a tree in an abstract semantic representation, for example, in the form of a tree in "AMR" ("Abstract Meaning Representation"). Recall that AMR is a computer language for semantic representation. An AMR graph can represent one or more complete sentences. AMR differs from syntactic analysis in that different sentences with similar meanings can have the same AMR graph even if they are not expressed in the same way. The tree graph (particularly the AMR graph) then includes nodes and leaves, the leaves being the answers to the questions represented by the upstream nodes (or "predicates"). For example, "5:30 p.m." can be the leaf of the predicate node "At what time?".

[0093] Thus, in a specific embodiment, the creation of graph GRAPH1 may make it possible to create a structure for the required text data in order to identify text data from a source that may correspond to the required data. Figure 1 After establishing this graph GRAPH1 in step S5, the method also proposes to carry out semantic analysis to reference data TXT2 in step S3 and to establish semantic graph GRAPH2 of the same nature (e.g., AMR) based on these reference data in step S4. If files corresponding to different sources are indexed in the memory according to a specific theme, the selection of the source in step S2 can also depend on the required text data. For example, if the predicate of the graph GRAPH1 of the required data TXT1 corresponds to the theme (e.g., asking the question "social security number?"), then the file related to the theme "user identity" can be selected and (after being convertible into text data) it is semantically analyzed.

[0094] Then, step S6 comprises comparing the two graphs GRAPH1 and GRAPH2 to identify similarities, in particular similarities of predicates. For example, if common nodes are identified between the two graphs, the leaves of the second graph GRAPH2 from these nodes may be answers to the questions formulated in the predicates of the first graph GRAPH1. These answers may therefore correspond to the text data required in step S13. At the end of this step S6, the reference text data TXT2 may therefore allow recommendations to be formulated in step S7, which are intended to answer the questions of the first graph GRAPH1 or may be intended to correct the leaves of the first graph GRAPH1. In at least some embodiments, these recommendations in the form of text data may be fed directly to the executing application (without user intervention) or may be presented to the user in step S15.

[0095] In this step S15, for example, in an application where a user fills in a form (online on a web page, or manually on paper), the human-machine interface may be managed to display a window presenting text corresponding to the recommendation on the screen (transparently overlaid on or next to the form in the web page, or on the lenses of augmented reality smart glasses when the user is writing on paper or a whiteboard). Additionally or alternatively, the human-machine interface may include an audio headset, and a recommendation message (the message indicating, for example, the required text data or a recommendation for correcting the text entered by the user) may be played on the audio headset.

[0096] The reference data delivered by the above-mentioned sources in step S2 therefore make it possible to assist the input of data required by the application or by the user during the activity of inputting such data (this assistance may be provided via the management of the man-machine interface).

[0097] As described above, the sources providing these reference data can be stored in the memory. However, in order to avoid overloading the memory and / or to avoid proposing irrelevant reference data, a test can be performed in step S8 to determine whether the user confirms the recommendation by entering the required data. If this is not the case, the reference to the file as the source of the reference data can be deleted from the memory in step S10. For example, artificial intelligence can be implemented to determine the data sources to be retained and which sources may be relevant by learning from the recommendations confirmed by the user. Additionally or alternatively, in step S9, if the source has not been used within a threshold time period (e.g., several days or more), the reference to the source can be deleted from the memory in step S10.

[0098] Thus, such an embodiment may assist the user in inputting data, in particular by providing data directly to the application requiring the data (the user can verify whether the data filled directly in, for example, an online form is correct) or by providing management of the human-machine interface. Thus, an improvement (e.g., optimization) of the interaction between applications available to the user is proposed, which improves the user's browsing experience by positioning information, for example, "above" the application (along the lines shown below). Figure 6 The Z-axis of the "display screen" shown) or next to the application (along the X, Y axes), or more generally, by overlaying information, combining and linking information useful to the user's activities in the case of augmented reality glasses.

[0099] It is assumed that the actions between the applications are interconnected, for example, according to priority rules set by the user in the above-mentioned computer program (and which can be improved by artificial intelligence learning). The application of these rules can occur, for example, after detecting a word or sentence that can be viewed on the screen (independent of the application currently being used). Therefore, computer modules such as optical character recognition ("OCR") applications, abstract meaning representation ("AMR") computer applications, computer programs for identifying similar predicates in various graphs and searching for response leaves can be implemented.

[0100] An example of a technique for identifying an acquisition context within the meaning of step S1 is shown below, with the aim of proposing, for example, the management of a human-machine interface within the context of the restitution of the data required.

[0101] A multi-source auxiliary module may be provided, requiring input from a context analysis module ("context analyzer"). A first functional block identifies, for example, an application currently in use on one or more screens of the user. If this is implemented in an augmented reality environment, the first functional block identifies the real or virtual area that the user is focusing on. This identification may be performed by eye tracking, and / or by tracking taps on a touch surface (pad), and / or by tracking another input interface such as a mouse (mouse tracking), or more generally by tracking system events.

[0102] This step may, for example, generate such an image or extract an image from the application area that the user is viewing.

[0103] For a user who is frequently observing two separate applications (e.g., where there is continuous focus on the two applications and / or observable movement of the user's gaze is detected by matching the X,Y positions of the applications present on the screen) and within a configurable time period, the two images are therefore copied and associated with the time period T (e.g., according to test S12 shown above).

[0104] Figure 2An example of an application is shown, in which a user of an instant messaging application expresses to a communicator that he wishes to go to a certain city on a certain date. At the same time, he accesses another application to make an online reservation to go to the city, such as Figure 3 Then two images are generated, corresponding to Figure 2 and Figure 3 .

[0105] Of course, the scope of application of the method can be extended to virtual reality or augmented reality scenarios. For example, augmented reality glasses can observe and display a (real) whiteboard and implement a method for recognizing handwritten characters on the board based on acquiring a real image of the board. A TV screen that reproduces information can also be an object of image acquisition. Therefore, a service combination can be realized in a mixed reality world (virtual / real).

[0106] Optical recognition of the characters present in the image thus reproduced, as well as a precise analysis of the characters of these text areas (to distinguish, for example, editable areas from labels) can be performed. Character retrieval can be performed, for example, via the following scenarios: at the request of the operating system of the information processing device used, or in order to recognize editable areas, analysis of the appearance of the mouse pointer when hovering over an area containing characters (for example, no longer a simple arrow when the mouse pointer is over an editable area), or the appearance of the area itself (for example, with or without a border, a different background color, or other distinguishing features).

[0107] Figure 2 and Figure 3 The examples in relate to full text areas, but it is also possible to access text content from more complex components such as drop-down lists, radio buttons with different titles depending on the position of the button, or other possible components.

[0108] The visible texts can then be extracted from the images. Their locations in the images can be stored in memory. A semantic analysis of the viewable texts is performed to build a semantic graph for each image.

[0109] For example, in Figure 2 and Figure 3 In the case of Figure 2 In the corresponding image 1: "I need to go to Rennes next Monday, January 7", the text attribute can be "Location I1: X, Y / non-editable text (=NE)".

[0110] In Image 2 ( Figure 3 ), the text attributes are:

[0111] - Location I2: "My Trip Management Application" at X, Y (NE),

[0112] - Position 13: "Flight / Train" "Hotel" "Car" "Door to Door" at X, Y,

[0113] - Position I4: "round trip" "one way" at X, Y,

[0114] - Position I5: "From Office" at X, Y,

[0115] - Position I6: "to input location" at X, Y;

[0116] - At position I7: "When?" of X, Y (NE),

[0117] - Position I8: "Departure" at X, Y (NE),

[0118] - Location I9: "Thursday, February 10, 2022" at X, Y,

[0119] - Position I10: "12:30" at X,Y,

[0120] etc.

[0121] The semantic graph representation of the extracted text may be performed, for example, using Abstract Meaning Representation (AMR) technology.Other techniques for graphically rendering the meaning of a sentence may also be used.

[0122] like Figure 4 As shown in the example in , the semantic graph shows the topics and parameters in a tree. Figure 4 The tree in corresponds more precisely to Figure 2 For at least one node of the tree (eg, for each node), attributes such as the X / Y position of the text and its characteristics (editable or non-editable text) may also be stored in the memory.

[0123] The text extracted from image 2 may be: "My Travel Management App", "Flight / Train Hotel Car", "Door to Door", "Round Trip / One Way", "From Office", "Location A", "Enter Location", "When? Departure", "Thursday, February 10, 2022" (the last indication is, for example, the default).

[0124] In some contexts, the resulting graphs may be very similar, and a simple comparison of the graphs may indicate a match between the data from images 1 and 2 being displayed on the screen. However, in some cases, the predicates may be different, but still semantically close. For example: "t / travel" and "g / go" are close in their parameters ("startpoint", "endpoint", "traveler", "goer") (where the parameter "goer" represents an entity in motion).

[0125] However, in some embodiments, a pre-programming may be proposed that identifies overall similarities between predicates so that similarities of AMR predicates may be identified between two graphs to perform a match between corresponding information from these graphs.

[0126] Thus, for example, corresponding to an editable area on the screen of the device (eg Figure 3 The graph GRAPH1 of the form in can therefore be composed of graphs from other sources (e.g. Figure 2 One or more other elements of the graph GRAPH2 ("input" elements) of the data of the recent chat in the above step S15 are (at least partially) filled in. Thus, these elements can be proposed in the recommendation. The recommendation can be displayed, for example, in the form of a window that opens on the user's screen (management of the human-machine interface in the above step S15), in a Figure 6 The other windows shown are "above" to propose these elements to the user and, if the user accepts them, use these elements to fill in the data requested in the current application ("Ryan", "Monday, January 7, 2022"). If the user agrees with the recommendations made (i.e., if the device's understanding of the task to be performed seems correct to the user), the text proposed in the semantic context can be entered directly into the application.

[0127] An opened user interface window may be positioned, for example, above other active windows and as close as possible to the application involved, for example in a transparent manner. Figure 6 Additionally or alternatively, a voice interface (eg, using speech synthesis) may also be managed to guide the user to input, for example, numbers or spelling cities.

[0128] Recommendations can also be built by utilizing the learning module. For example, if an action proposed in a created interface is indeed performed (and therefore confirmed by the user), it can then be submitted again in future uses. A recommendation window can also be opened when a semantic link has been confirmed, typically in a previous use.

[0129] In another use case below, corrections are recommended to a user after the user has entered text data.

[0130] In this scenario involving mixed reality, the user has augmented reality glasses. Such glasses are equipped with a camera module capable of capturing the image the user is looking at and capable of analyzing the image, in particular detecting written characters (usually on). For example, the user looks at a health insurance card for a few seconds. The above method can be implemented to convert the captured image data into text data by character recognition. Thus, for example, the following information is extracted:

[0131] - "Health Insurance Card: 290042216205897", and

[0132] - "Card issued on August 9, 2002".

[0133] The semantic graph GRAPH2 of these reference data allows identifying leaf 290042216205897 as corresponding to a predicate of type "n / number".

[0134] Next, the user fills out an online form that asks: "Please enter your health insurance card number: XXXXXX." However, in doing so, the user mistakenly enters: "290042216205889."

[0135] Semantic analysis of the question asked creates a graph GRAPH1 in which the expected response follows a predicate of type "n / number". However, the response entered by the user does not match the response in the reference graph GRAPH2 that follows a predicate of the same type in a similar branch.

[0136] Thus, an error is detected in the entered number and a pop-up window warns the user via a graphical interface (where a recommendation message is superimposed as close as possible to the error message). Additionally or alternatively, the sound interface may sound an error message on an audio headset (e.g., a beep or a voice message providing a recommendation, in this case a valid number) based on the user's preferences.

[0137] Therefore, the above method can be applied in different environments, in particular in a fully digital work environment and / or in virtual reality or augmented reality, and in at least one embodiment, can provide assistance in the use of at least one application. For example, the method can assist users in combining different applications to complete tasks. In particular, as shown in the above use case, the implementation of the method can be performed in real time and provide dynamic, contextual managed data (different from "cold data"). In addition, at least in some embodiments, the source used (step S2) can provide local aspects for the execution of the method (for example, if necessary, without using Internet data), so unlike, for example, the use of a shared knowledge base, these embodiments have advantages in data protection (or privacy), and the source used provides local aspects based on the user experience occurring on a device such as a terminal (usually).

[0138] Figure 7 An example of an embodiment of such a device DEV for implementing the method defined above is shown, which may generally comprise a processing circuit provided with at least:

[0139] a first interface INT1 for receiving reference data (for example data in the form of image files, or data from a connected whiteboard, or other data) transmitted from different sources (SOU1, SOU2, etc.), which data can be stored at least temporarily in a memory MEM,

[0140] a processor PROC and a memory MEM which may in particular also store instruction code data of a computer program within the meaning of the present disclosure, the memory being accessible by the processor PROC in order to read the instruction codes and to execute the above-described method, and

[0141] A second interface INT2 which controls means of a human-machine interface HMI, such as a screen (for opening a recommendation window) and / or audio headphones or others.

Claims

1. A method implemented by an information processing device, comprising: - obtaining text data (S2) based on data delivered by at least one source, and - Based on semantic similarities between at least some of the obtained data and at least some of the data required by at least one currently running computer application, issuing recommendations related to the required data, at least one user interface of the device allowing interaction with the at least one currently running computer application.

2. The method according to claim 1, comprising identifying at least part of the required text data in the obtained text data (S7), and wherein, The recommendations are based on the recognized text data.

3. The method according to claim 1 or 2, wherein: The requested data is the data currently being written by the user of the device.

4. The method according to claims 1 to 3, wherein: The recommendations include suggestions for elements of correcting and / or filling in the required data.

5. The method according to any one of claims 1 to 4, wherein: The semantic similarity considers the similarity between a first semantic graph established for the obtained data and a second semantic graph established for the required data.

6. The method according to claims 1 to 5, comprising: - establishing the first semantic graph based at least on the text data to be verified, - performing a search for at least partial similarity between the first semantic graph and the second semantic graph to identify at least part of the text data to be verified in the obtained text data, and - If the data from the recognized portion is different from the corresponding obtained text data, a correction recommendation is issued.

7. The method according to any one of claims 1 to 6, comprising: - managing a human-machine interface connected to said information processing device (S15), and - sending a message corresponding to the recommendation via the human-machine interface.

8. The method according to claim 7, wherein: The human machine interface comprises a screen (HMI) displaying at least a first window specific to the computer application, and the method comprises: - Controlling the display of the recommendation in a second window on the screen simultaneously with the first window (Fig. 6).

9. The method according to any one of claims 1 to 8, wherein: The recommendation includes recognized text data, and the method includes: - providing the recognized text data to said computer application.

10. The method according to at least one of the preceding claims, wherein: The first semantic graph and the second semantic graph are tree structures and include: * represents a node for a predicate, and * represents text data and is a leaf that is a response to the predicate, The method comprises (Figure 4, Figure 5): - identifying nodes having a common predicate between said first graph and said second graph, and - selecting leaves from said nodes having a common predicate in said second graph, The selected leaf corresponds to the partially required text data among the obtained text data.

11. The method according to at least one of the preceding claims, wherein: The first semantic graph and the second semantic graph are in the form of Abstract Semantic Representation computer language, or "AMR".

12. The method according to at least one of the preceding claims, wherein: The data delivered by at least one source is image data containing text, and the method comprises: - Processing the image data (S2) by character recognition to obtain text data (TXT2).

13. The method according to at least one of the preceding claims, wherein: The data delivered by at least one source is audio data, and the method comprises: - Processing the audio data to detect speech signals and converting the speech signals into text data.

14. The method according to at least one of the preceding claims, wherein: The data delivered by at least one source is data delivered by a computerized device for capturing handwritten characters, and the method comprises: -Performing character recognition on handwritten characters to convert the input characters into text data.

15. The method according to at least one of the preceding claims, wherein: The data transmitted by at least one source is stored in a memory of the information processing device within a first time period after the method was last implemented using the device (S9).

16. The method according to at least one of the preceding claims, comprising implementing an artificial intelligence programmed to learn relevant recommendations based on user feedback (S8, S10) and, after learning, to select relevant recommendations to be issued based on the recognized text data.

Images