Method and apparatus for retrieving smart information from an electronic device

Abstract

Accordingly, embodiments herein disclose a method and apparatus for retrieving intelligent information from an electronic device (100). The method includes receiving input from a user by the electronic device (100). Furthermore, the method includes identifying, by the electronic device (100), at least one data item to generate at least one metadata tag. Furthermore, the method includes automatically generating, by the electronic device (100), the at least one metadata tag associated with the at least one data item based on a plurality of parameters. Furthermore, the method includes providing, by the electronic device (100), at least one priority for the at least one metadata tag. Furthermore, the method includes storing, by the electronic device (100), the at least one metadata tag at the electronic device (100).

Description

Technical Field

[0001] The present disclosure relates to machine learning (ML) and artificial intelligence (AI), and more particularly to a method and apparatus for retrieving intelligent information from an electronic device. Background Art

[0002] Typically, users have a large amount of data (e.g., data items such as captured pictures, screenshots, web browsing, and call logs) in electronic devices (e.g., Figure 1 ). Every time a user wants to access, for example, a specific image / data, the user must manually scan for specific information / relevant data from the large amount of data contained in the electronic device. Existing systems do not have a simple way to intelligently retrieve such data, and existing methods can be cumbersome and hinder the user experience. Because finding relevant data is a significant task, there is a need to address the issues in existing systems. To solve this task, collaboration between data retrieval technology, machine learning (ML), and artificial intelligence (AI) is required.

[0003] It would therefore be desirable to address the above-mentioned drawbacks or other disadvantages, or at least provide useful alternatives. Summary of the Invention

[0004] Technical issues

[0005] A primary object of the embodiments herein is to provide a method and apparatus for retrieving smart information from an electronic device.

[0006] It is another object of an embodiment to receive input from a user and identify at least one data item to generate at least one metadata tag.

[0007] Another object of an embodiment is to automatically generate the at least one metadata tag associated with the at least one data item based on a plurality of parameters, and to provide at least one priority for the at least one metadata tag.

[0008] It is another object of an embodiment to receive at least one candidate metadata tag and compare the at least one candidate metadata tag with the at least one stored metadata tag.

[0009] Another object of an embodiment is to retrieve at least one data item corresponding to the at least one candidate metadata tag in response to determining a match between the at least one candidate metadata tag and the at least one stored metadata tag, and perform at least one action using the retrieved at least one data item.

[0010] Technical Solution

[0011] Accordingly, embodiments herein disclose a method and apparatus for retrieving intelligent information from an electronic device. The method includes receiving, by the electronic device, direct input or indirect input (e.g., taking a screenshot) from a user. Furthermore, the method includes identifying, by the electronic device, at least one data item to generate at least one metadata tag. Furthermore, the method includes automatically generating, by the electronic device, the at least one metadata tag associated with the at least one data item based on multiple parameters. Furthermore, the method includes providing, by the electronic device, at least one priority for the at least one metadata tag. Furthermore, the method includes storing, by the electronic device, the at least one metadata tag at the electronic device.

[0012] In an embodiment, the method further includes receiving, by the electronic device, at least one candidate metadata tag. Furthermore, the method includes comparing, by the electronic device, the at least one candidate metadata tag with the at least one stored metadata tag. Furthermore, the method includes, in response to determining a match between the at least one candidate metadata tag and the at least one stored metadata tag, retrieving, by the electronic device, at least one data item corresponding to the at least one candidate metadata tag. Furthermore, the method includes performing, by the electronic device, at least one action using the retrieved at least one data item.

[0013] In an embodiment, at least one data item corresponding to the at least one candidate metadata tag is retrieved based on the priority associated with the at least one candidate metadata tag.

[0014] In an embodiment, the method includes identifying, by the electronic device, at least one of an image block, a text block, and an audio block available in the data item. Furthermore, the method includes determining, by the electronic device, a plurality of parameters associated with the at least one of the image block, the text block, and the audio block. Furthermore, the method includes generating, by the electronic device, the at least one metadata tag associated with the at least one data item based on the plurality of parameters.

[0015] In an embodiment, the plurality of parameters of the image block include at least one of a scene object block, an expression block, a face block, and an activity block.

[0016] In an embodiment, the plurality of parameters of the text block include at least one of a keyword block, a language identification block, a classification block, a text summary block, and an electronic device content aggregator block.

[0017] In an embodiment, the plurality of parameters of the audio block comprises at least one of an audio summary and a language identification.

[0018] In an embodiment, the electronic device generates the at least one metadata tag locally without interacting with any network device.

[0019] In an embodiment, the at least one metadata tag is user-definable.

[0020] In an embodiment, the at least one data item comprises an image file, a video file, an audio file, and a text document.

[0021] Therefore, embodiments herein provide an electronic device for retrieving intelligent information. The electronic device includes a processor and a memory. The processor is configured to receive input from a user. The processor is further configured to identify at least one data item to generate at least one metadata tag. The processor is further configured to automatically generate the at least one metadata tag associated with the at least one data item based on multiple parameters. The processor is further configured to assign at least one priority to the at least one metadata tag. The processor is further configured to store the at least one metadata tag at the electronic device.

[0022] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and accompanying drawings. However, it should be understood that the following description, which indicates preferred embodiments and many of their specific details, is given by way of illustration and not limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.

[0023] Beneficial effects

[0024] According to various embodiments, the electronic device may provide for retrieving intelligent information from the electronic device. According to various embodiments, input may be received from a user and at least one data item may be identified to generate at least one metadata tag. According to various embodiments, the at least one metadata tag associated with the at least one data item may be automatically generated based on a plurality of parameters, and at least one priority may be provided for the at least one metadata tag. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The method is illustrated in the accompanying drawings, in which the same reference characters indicate corresponding parts in the various figures. The embodiments of the present invention will be better understood from the following description with reference to the accompanying drawings, in which:

[0026] Figure 1 An existing search system for specific data for quick reference or sharing with at least one second user through an electronic device according to the prior art disclosed herein is shown;

[0027] Figure 2 A block diagram illustrating an electronic device for retrieving smart information according to an embodiment as disclosed herein;

[0028] Figure 3A and Figure 3Bis a flow chart illustrating a method of generating metadata tags in an electronic device according to an embodiment as disclosed herein;

[0029] Figure 4A 、 Figure 4B 、 Figure 4C and Figure 4D is an example diagram of a method for generating metadata tags in an electronic device according to an embodiment as disclosed herein;

[0030] Figure 5 is a flow chart illustrating a method for retrieving smart information from an electronic device according to an embodiment as disclosed herein;

[0031] Figure 6A 、 Figure 6B and Figure 6C is an example illustration of retrieving intelligent information using intelligent metadata tags and text content summaries of a virtual assistant of an electronic device according to an embodiment as disclosed herein;

[0032] Figure 7A 、 Figure 7B 、 Figure 7C and Figure 7D is another example illustration of intelligent metadata tags and summaries generated for a browser application of an electronic device according to an embodiment as disclosed herein;

[0033] Figure 8A 、 Figure 8B and Figure 8C is another example diagram of generating metadata tags based on image features according to an embodiment as disclosed herein;

[0034] Figure 9A 、 Figure 9B and Figure 9C is another example illustration of generating metadata tags for a ride service application of an electronic device according to an embodiment as disclosed herein;

[0035] 10A to 10F is another example illustration of learning intelligent attributes of an intelligent summarizer to generate metadata tags for a browser application of an electronic device according to an embodiment as disclosed herein;

[0036] Figure 11A 、 Figure 11B 、 Figure 11C and Figure 11D is another example illustration of generating metadata tags created for an incoming call according to an embodiment as disclosed herein;

[0037] Figure 12A 、 Figure 12B 、 Figure 12C and Figure 12Dis another example illustration of a smart summarizer generated based on a conversation summary of a messaging application for an electronic device according to an embodiment as disclosed herein;

[0038] Figure 13A 、 Figure 13B 、 Figure 13C and Figure 13D is another example illustration of generating metadata tags for a voice recorder application of an electronic device according to an embodiment as disclosed herein;

[0039] Figure 14A 、 Figure 14B and Figure 14C is another example illustration of smart writing generated for a messaging application of an electronic device according to an embodiment as disclosed herein; and

[0040] Figure 15A and Figure 15B is another example illustration of smart reply, smart share, and smart compose generated based on a received uniform resource locator (URL) of a message application for an electronic device according to an embodiment as disclosed herein. DETAILED DESCRIPTION

[0041] The embodiments of this invention and their various features and advantageous details are explained more fully with reference to the non-limiting embodiments shown in the accompanying drawings and described in detail in the following description. Descriptions of well-known components and processing technologies are omitted so as not to unnecessarily obscure the embodiments of this invention. In addition, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments. Unless otherwise indicated, the term "or" as used herein refers to a non-exclusive or. The examples used herein are intended only to facilitate understanding of the manner in which the embodiments of this invention can be practiced, and are also intended to enable those skilled in the art to practice the embodiments of this invention. Therefore, the examples should not be interpreted as limiting the scope of the embodiments of this invention.

[0042] As traditional in the art, can be described and illustrated embodiment according to the block of one or more functions described for execution.These blocks (may be referred to as unit or module etc. in this article) are physically realized by analog circuit or digital circuit (such as logic gate, integrated circuit, microprocessor, microcontroller, memory circuit, passive electronic component, active electronic component, optical component, hard-wired circuit etc.), and can be alternatively driven by firmware and software.For example, circuit can be realized in one or more semiconductor chips, or be realized on the substrate support such as printed circuit board etc. The circuit constituting block can be realized by dedicated hardware, or can be realized by processor (such as, one or more programmed microprocessors and associated circuit), or can be realized by the combination of dedicated hardware of some functions of execution block and processor of other functions of execution block.Without departing from the scope of the present invention, each block of embodiment can be physically divided into two or more interactive and discrete blocks.Similarly, without departing from the scope of the present invention, the block of embodiment can be physically combined into more complicated block.

[0043] The accompanying drawings help to easily understand various technical features, and it should be understood that the embodiments presented herein are not limited by the accompanying drawings. Therefore, the present disclosure should be interpreted as extending to any changes, equivalents and replacements other than those specifically set forth in the accompanying drawings. Although the terms first, second, etc. can be used herein to describe various elements, these elements should not be limited by these terms. These terms are usually only used to distinguish one element from another element.

[0044] Therefore, embodiments herein disclose a method and apparatus for retrieving intelligent information from an electronic device. The method includes receiving, by the electronic device, direct or indirect input from a user. Furthermore, the method includes identifying, by the electronic device, at least one data item to generate at least one metadata tag. Furthermore, the method includes automatically generating, by the electronic device, at least one metadata tag associated with the at least one data item based on a plurality of parameters. Furthermore, the method includes providing, by the electronic device, at least one priority for the at least one metadata tag. Furthermore, the method includes storing, by the electronic device, the at least one metadata tag at the electronic device.

[0045] Referring now to the drawings, and more particularly to the Figure 2 to Figure 1 5, shows a preferred embodiment.

[0046] Figure 2 A block diagram of an electronic device (100) for retrieving smart information according to an embodiment as disclosed herein is shown. The electronic device (100) may be, for example, but not limited to, a smartphone, a laptop computer, a desktop computer, a smart watch, a smart TV, etc. In an embodiment, the electronic device (100) includes a processor (120), a memory (130), a display (140), and a communicator (150).

[0047] The processor (120) communicates with the memory (130), the display (140), and the communicator (150). The processor (120) is configured to execute instructions stored in the memory (130) and perform various processes.

[0048] In an embodiment, the processor (120) is configured to receive input from a user (e.g., via gestures, a touch screen, voice communication). Furthermore, the processor (120) is configured to identify at least one data item (e.g., an image file, a video file, an audio file, and a text document) to generate at least one metadata tag (e.g., a smart summarizer). Furthermore, the processor (120) is configured to automatically generate at least one metadata tag associated with the at least one data item based on a plurality of parameters, wherein the plurality of parameters utilize a cognitive map. Furthermore, the processor (120) is configured to provide at least one priority for the at least one metadata tag. Furthermore, the processor (120) is configured to store the at least one metadata tag at the electronic device (100).

[0049] In an embodiment, the processor (120) is configured to receive at least one candidate metadata tag (e.g., a tag input by a user). Furthermore, the processor (120) is configured to compare the at least one candidate metadata tag with at least one stored metadata tag. Furthermore, the processor (120) is configured to, in response to determining a match between the at least one candidate metadata tag and the at least one stored metadata tag, retrieve at least one data item corresponding to the at least one candidate metadata tag. Furthermore, the processor (120) is configured to perform at least one action (e.g., sharing the at least one data item with at least one second user, storing the at least one data item in the electronic device (100)) using the retrieved at least one data item.

[0050] In an embodiment, at least one data item is retrieved based on a priority associated with at least one candidate metadata tag.

[0051] In an embodiment, the processor (120) is configured to identify at least one of an image block, a text block, and an audio block available in a data item. Furthermore, the processor (120) is configured to determine a plurality of parameters associated with at least one of the image block, the text block, and the audio block. Furthermore, the processor (120) is configured to generate at least one metadata tag associated with the at least one data item based on the plurality of parameters.

[0052] In an embodiment, the plurality of parameters of the image block include at least one of the following: a scene object block (e.g., beach, flower, computer), an expression block (e.g., funny, angry, surprised), a face block (e.g., facial recognition of a user), and an activity block (e.g., running, dancing, sleeping, singing, swimming).

[0053] In an embodiment, the multiple parameters of the text block include at least one of the following items: a keyword block (e.g., Seoul), a language identification block (e.g., Spanish, Hindi, Korean), a classification block, a text summary block (e.g., abstract, refined), and an electronic device content aggregator block (e.g., calendar information, weather information, location information, application category information, application metadata).

[0054] In an embodiment, the plurality of parameters of the plurality of audio blocks include at least one of an audio summary and a language identification (e.g., Spanish, Hindi, Korean). The electronic device (100) generates at least one metadata tag locally without interacting with any network device. The at least one metadata tag is user-definable.

[0055] In an embodiment, the processor (120) includes an input recognizer (121), a data item analysis engine (122), and a metadata tag generator (123).

[0056] An input recognizer (121) receives input from a user. A data item analysis engine (122) recognizes at least one data item (e.g., at least one of an image, text, and audio) to generate at least one metadata tag. A metadata tag generator (123) automatically generates at least one metadata tag associated with the at least one data item based on a plurality of parameters. Furthermore, the metadata tag generator (123) includes an image processing engine (123a), a text locator (123b), a script recognizer (123c), an optical recognizer (123d), a language detector (123e), a keyword extraction engine (123f), a keyword expansion engine (123g), a cognitive map (123h), and a speech engine (123i).

[0057] The image processing engine (123a) performs bilateral filtering on the image. Furthermore, the image processing engine (123a) converts the bilateral image into a grayscale image. Furthermore, the image processing engine (123a) performs custom binarization on the grayscale image. Furthermore, the image processing engine (123a) extracts features from the image to generate feature labels. The text locator (123b) identifies text regions / blocks in the grayscale image. The script identifier (123c) identifies the script in the identified text region of the grayscale image.

[0058] An optical recognizer (123d) based on a deep neural network identifies text areas and extracts text from an image. In addition, a language detector (123e) identifies the language of the text of each script and stores the identified language. A keyword extraction engine (123f) receives the extracted text from the optical recognizer (123d) in a live pipeline, receives language details from the language detector (123e), and loads a language-specific neural model to extract the most important parts of the text. A keyword expansion engine (123g) adds new keywords to the current text, wherein the new keywords are highly relevant, synonymous, or strongly relevant to the text. A cognitive map (123h) determines the ranking of each keyword of the text based on a user-personalized summary. A speech engine (123i) converts the incoming speech signal into text and feeds it to the live pipeline to generate metadata tags on the electronic device (100).

[0059] The memory (130) also stores instructions to be executed by the processor (120). The memory (130) may include a non-volatile storage element. Examples of such non-volatile storage elements may include a magnetic hard disk, an optical disk, a floppy disk, a flash memory, or a form of electrically programmable memory (EPROM) or electrically erasable programmable memory (EEPROM). Furthermore, in some examples, the memory (130) may be considered a non-transitory storage medium. The term "non-transitory" may indicate that the storage medium is not implemented as a carrier wave or propagating signal. However, the term "non-transitory" should not be interpreted as meaning that the memory (130) is non-removable. In some examples, the memory (130) may be configured to store a larger amount of information than the memory. In a specific example, the non-transitory storage medium may store data that may change over time (e.g., in random access memory (RAM) or cache memory). In an embodiment, the memory (130) may be an internal storage unit, or it may be an external storage unit of the electronic device (100), cloud storage, or any other type of external storage.

[0060] The memory (130) includes an application repository (130a) to store metadata tags of different candidate applications (e.g., call application, gallery application, camera application, business application, education application, lifestyle application, entertainment application, tool application, travel application, health and fitness application).

[0061] The communicator (150) is configured to communicate internally between internal hardware components and with external devices via one or more networks.

[0062] although Figure 2Various hardware components of the electronic device (100) are shown, but it will be understood that other embodiments are not limited thereto. In other embodiments, the electronic device (100) may include fewer or greater numbers of components. Furthermore, the labels or names of the components are for illustrative purposes only and do not limit the scope of the present invention. One or more components may be combined to perform the same or substantially similar functions as intelligently retrieving information.

[0063] Figure 3A and Figure 3B is a flow chart illustrating a method of generating metadata tags in an electronic device (100) according to an embodiment as disclosed herein. Operations (302-324) are performed by the electronic device (100).

[0064] Symbol "3A" indicates that at 302, the method includes identifying at least one image on the display (140). At 304, the method includes performing a bilateral filtering process on the image. The bilateral filtering process is used for edge-preserving smoothing. At 306, the method includes converting the bilateral image into a grayscale image. At 308, the method includes performing a custom binarization on the grayscale image. A detailed description of the custom binarization is given in symbol "3B". At 310, the method includes performing text localization (i.e., text localization on the device / not connected to an external network) using Canny edge detection by a text localizer (123b). Canny edge detection is used to measure the horizontal and vertical edges present around the text area / block in the grayscale image. At 312, the method includes script identification (i.e., script identification on the device / not connected to an external network) by a DNN-based script identifier (123c), taking the text block as input and identifying the script of the text (e.g., "Latin", "Cyrillic", "Chinese", "Japanese", "Korean", "Ashokan").

[0065] At 314, the optical character recognition (OCR) of the optical recognizer (123d) takes the custom binarization output and the script recognition output as input. The optical recognizer (123d) is script-dependent. Therefore, the script recognition is optimally loaded into the optical recognizer (123d) for solution on the electronic device (100). In addition, the script recognition loads the necessary script resources for the neural model of the optical character recognition. The optical recognizer (123d) recognizes the text area and compares the text with a database consisting of different types of characters and extracts the text from the image using a DNN. At 316, the method includes automatically identifying the language by a DNN-based language detector (123e) (i.e., a language detector on the device / not connected to an external network). In addition, the identified language is stored in the language detector (123e). For each script, there are multiple languages ​​(e.g., Latin has more than 40 languages ​​and Ashoka has more than 15 languages). Loading all languages ​​will cause memory problems. To overcome the memory problem, the proposed system uses a language detector (123e) to dynamically add languages ​​in the electronic device (100).

[0066] At 318, the keyword extraction engine (123f) takes the language detector (123e) output and the optical recognizer (123d) output as input. The keyword extraction engine (123f) is language-dependent, and each language has its own set of grammars, so detecting the language is important for correctly identifying keywords. The keyword extraction engine (123f) analyzes the text and extracts the most important text from the text. This helps summarize the content of the text and identify the main topics.

[0067] At 304a to 306a, feature extraction performed by the image processing engine (123a) uses a DNN-like convolutional neural network (CNN) architecture to extract features from at least one image on the display (140) and generate labels based on the extracted features.

[0068] At 320, the keyword expansion engine (123g) module takes as input the extracted feature tag output and the keyword extraction engine (123f) output. The keyword expansion engine (123g) is language-dependent. The keyword expansion engine (123g) adds new keywords to the current text, wherein the new keywords are highly relevant, synonymous, or strongly relevant to the text in the text. At 322 to 324, the ranking of each keyword in the text is determined based on the user's personalized summary through the cognitive map (123) ranking.

[0069] In an embodiment, symbol "3B" indicates custom binarization. At 308a, the method includes detecting edges using canny edges on a grayscale image. At 308b, the method includes identifying contours by using the detected edges. Based on these contours, connected components are evaluated. At 308c, the method includes deciding whether further contour processing is necessary. At 308da to 308fa, the method includes processing the contour to successfully add it to a processing list. At 308db to 308eb, the method includes calculating the intensities of all neighboring points and, accordingly, making a decision to convert the pixel to white or black. For optimization, processing is performed in four different threads, which speeds up the processing of the custom binarization.

[0070] Figure 4A 、 Figure 4B 、 Figure 4C and Figure 4D is an example illustration of a method for generating metadata tags in an electronic device (100) according to an embodiment as disclosed herein. Figure 3A and Figure 3B The technical features are explained in .

[0071] A user of the electronic device (100) searches for "men's clothing" in a search browser and takes a screenshot of the displayed web page. The screenshot may trigger a visual prompt to the user to initiate tag extraction, or may directly trigger tag extraction. The method identifies an image of men's clothing on a display (140) and performs various image processing operations at 304-308. At 310, the method detects a text region of the displayed item. At 312, the method is used to identify a script based on the detected text. At 314, the method identifies text regions such as "search", "men's clothing", "latest", "GIF", "HD", "products", "wedding", "casual", "party", "formal", "men's suit fashion", "aaa.com", "bbb.com", "ccc.com", "ddd.com". At 316, the method is used to identify a language based on the text region. In the example, the identified language is Chinese.

[0072] At 318, as shown in symbol "1", the method analyzes the text area and extracts the most important text, such as "search", "men's clothing", "wedding", "casual", "party", "formal", "men", "suit", "fashion", "aaa.com", "bbb.com", "ccc.com", "ddd.com".

[0073] At 304a to 306a, the method extracts features of the displayed items and generates labels, as shown in symbol "2," such as "screenshot," "webpage," "clothing," and "pattern." At 320, the method adds new keywords, such as "clothing," "jacket," "shirt," "ring," "wedding," "celebration," "party," "fun," "screenshot," and "design." At 322, the method ranks each keyword based on the user's personalized summary. Symbol "3" shows the final text output / labels generated based on the displayed items of the search browser.

[0074] Figure 5 is a flow chart (500) illustrating a method for retrieving smart information from an electronic device (100) according to an embodiment as disclosed herein. Operations (502 to 518a) are performed by the electronic device (100).

[0075] At 502, the method includes receiving input from a user. At 504, the method includes identifying at least one data item to generate at least one metadata tag. At 506, the method includes automatically generating at least one metadata tag associated with the at least one data item based on a plurality of parameters. At 508, the method includes providing at least one priority to the at least one metadata tag. At 510, the method includes storing the at least one metadata tag at the electronic device (100). At 512 to 514, the method includes comparing at least one candidate metadata tag with at least one stored metadata tag. At 516b, the method includes not performing an action when the at least one candidate metadata tag does not match the at least one stored metadata tag. At 516a, the method includes retrieving at least one data item corresponding to the at least one candidate metadata tag when the at least one candidate metadata tag matches the at least one stored metadata tag. At 518a, the method includes performing at least one action using the retrieved at least one data item.

[0076] Figure 6A 、 Figure 6B and Figure 6C is an example illustration of retrieving intelligent information using intelligent metadata tags and text content summaries of a virtual assistant of an electronic device (100) according to an embodiment as disclosed herein.

[0077] Symbol "6A" indicates that the virtual assistant analyzes the content of the screen (i.e., display (140)) of the electronic device (100). In the example, the screen content is related to cricket news. Symbol "6B" indicates that the electronic device (100) automatically generates at least one metadata tag (e.g., a topic tag, a highlight card) related to the screen content, and at least one metadata tag is provided based on the priority of the user of the electronic device (100). In the example, the topic tags are "#CSK", "#SRH", "#IPL", "#PlaceNameA", and "#PersonNameA". In addition, the highlight cards are "PersonNameA talks about the match" and "Live from PlaceNameA". The generated topic tags can be edited by the user of the electronic device (100) in an expanded mode (e.g., adding more topic tags), and the highlight cards give the user of the electronic device (100) a detailed summary of the screen content. In addition, the electronic device (100) stores all or at least one of the topic tags and highlight cards selected / created by the user.

[0078] Symbol "6C" indicates that the virtual assistant receives at least one candidate metadata tag. In the example, the candidate metadata tag is "#IPL". The virtual assistant then compares the candidate metadata tag with the stored metadata tags and retrieves at least one data item (e.g., an image, a screenshot from the electronic device (100)) corresponding to the at least one candidate metadata tag.

[0079] Figure 7A 、 Figure 7B 、 Figure 7C and Figure 7D is another example illustration of intelligent metadata tags and summaries generated for a browser application of an electronic device (100) according to an embodiment as disclosed herein.

[0080] Symbol "7A" indicates that the electronic device (100) receives input (e.g., gesture, swipe) from the user of the electronic device (100). The electronic device (100) then analyzes the screen content of the electronic device (100). In the example, the screen content is related to "India vs Australia". Symbol "7B" indicates that the electronic device (100) automatically generates at least one metadata tag (e.g., topic tag, highlight card) related to the screen content, and the at least one metadata tag is provided based on the priority of the user of the electronic device (100). In the example, the topic tags are "#cricket", "#India", "#Australia", "#seriesmatch", and "#personnameA". In addition, the highlight cards are "Victory on Australian soil" and "Special to the bowlers". The generated topic tags and highlight cards have an expanded mode (i.e., by pressing the "More" button).

[0081] Symbol "7C" indicates that the user of the electronic device (100) performs at least one operation of editing a generated topic tag / adding a new topic tag / saving a topic tag in the electronic device (100). In addition, the user of the electronic device (100) shares the topic tag with at least one second user. Symbol "7D" indicates that a highlight card gives a detailed summary of the screen content to the user of the electronic device (100). In addition, the electronic device (100) stores the highlight card in the electronic device (100). In addition, the user of the electronic device (100) shares the highlight card with at least one second user.

[0082] Figure 8A 、 Figure 8B and Figure 8C is another example diagram of generating metadata tags based on image features according to an embodiment as disclosed herein.

[0083] Symbol "8A" indicates that the electronic device (100) receives input from the user of the electronic device (100). The electronic device (100) then analyzes the screen content of the electronic device (100). In the example, the screen content is an image. Symbol "8B" indicates that the electronic device (100) automatically generates at least one metadata tag (e.g., a topic tag) related to the screen content, and the at least one metadata tag is provided based on the priority of the user of the electronic device (100). In the example, the topic tags are "#beach", "#sand", "#sky", "#tibetan antelope", "#evening", "#person", "#horse", "#shadow", "#cloud", and "#water". The generated topic tags have an expansion mode (i.e., by pressing the "more" button).

[0084] Symbol "8C" indicates that the user of the electronic device (100) performs at least one operation of editing a generated hashtag / adding a new hashtag / saving a hashtag in the electronic device (100). In addition, the user of the electronic device (100) shares the hashtag with at least one second user.

[0085] Figure 9A 、 Figure 9B and Figure 9C is another example illustration of generating metadata tags for a ride service application of an electronic device (100) according to an embodiment as disclosed herein.

[0086] Symbol "9A" indicates that the electronic device (100) receives input from the user of the electronic device (100). The electronic device (100) then analyzes the screen content of the electronic device (100). In the example, the screen content is related to ride service information. Symbol "9B" indicates that the electronic device (100) automatically generates at least one metadata tag (e.g., a topic tag) related to the screen content, and the at least one metadata tag is provided based on the priority of the user of the electronic device (100). In the example, the topic tags are "#BuildingA,AMainRoad", "#AShoppingMall", "#i20", "#KA03CF9XXX", "#Mr.A", "#TotalFee70", and "#TTL1Day". The generated topic tags have an expansion mode (i.e., by pressing the "More" button). TTL1Day is a time that survives for one day, and the TTL1Day tag can be modified by the user of the electronic device (100). In addition, the TTL time is the time that the screenshot is available on the electronic device (100), after which the screenshot is automatically deleted, which helps keep the gallery organized and free of clutter.

[0087] Symbol "9C" indicates that the user of the electronic device (100) performs at least one operation of editing a generated hashtag / adding a new hashtag / saving a hashtag in the electronic device (100). In addition, the user of the electronic device (100) shares the hashtag with at least one second user.

[0088] 10A to 10F is another example illustration of learning intelligent attributes of an intelligent summarizer to generate metadata tags for a browser application of an electronic device (100) according to an embodiment as disclosed herein.

[0089] Symbol "10A" indicates that the electronic device (100) receives input from the user of the electronic device (100). Then, the electronic device (100) analyzes the screen content of the electronic device (100). In the example, the screen content is related to an image of furniture. Symbol "10B" indicates that the electronic device (100) automatically generates at least one metadata tag (e.g., a topic tag) related to the screen content, and the at least one metadata tag is provided based on the priority of the user of the electronic device (100). In the example, the topic tags are "#furniture", "#room", "#sofa", "#table", "#chair", "#aaa.com", "#bbb.com", "#ccc.com", and "#ddd.com". The generated topic tags can be edited by the user of the electronic device (100) in an extended mode. In addition, the electronic device (100) stores the topic tags in the electronic device (100).

[0090] Symbol "10C" indicates that input (e.g., zoom) is received from the user of the electronic device (100). The electronic device (100) then analyzes the screen content of the electronic device (100). In the example, the screen content is associated with a specific image. Symbol "10D" indicates that the electronic device (100) automatically generates at least one metadata tag (e.g., a topic tag) associated with the screen content, and the at least one metadata tag is provided based on the priority of the user of the electronic device (100). In the example, the topic tags are "#room", "#sofa", "#table", "#lamp", "#window", "#"pillow", "#bbb.com", and "#zoom". The generated topic tags can be edited by the user of the electronic device (100) in an extended mode. In addition, the electronic device (100) stores the topic tags in the electronic device (100).

[0091] Symbol "10E" indicates that the electronic device (100) receives at least one candidate metadata tag. In the example, the candidate metadata tag is "#sofas". The electronic device (100) then compares the candidate metadata tag with the stored metadata tags and retrieves at least one data item (i.e., a furniture image) corresponding to the at least one candidate metadata tag.

[0092] Symbol "10F" indicates that the electronic device (100) receives at least one candidate metadata tag. In the example, the candidate metadata tags are "#sofas" and "#zoom". The electronic device (100) then compares the candidate metadata tags with the stored metadata tags and retrieves at least one data item corresponding to the at least one candidate metadata tag (i.e., a specific image and a furniture image).

[0093] Figure 11A 、 Figure 11B 、 Figure 11C and Figure 11D is another example illustration of generating metadata tags created for an incoming call according to embodiments as disclosed herein.

[0094] Symbol "11A" indicates that the electronic device (100) receives input from the user of the electronic device (100). Then, the electronic device (100) analyzes the voice content of the incoming voice call from the unknown number. Symbol "11B" indicates that the electronic device (100) automatically generates at least one metadata tag (e.g., a topic tag) related to the incoming voice call session, and the at least one metadata tag is provided based on the priority of the user of the electronic device (100). In the example, the topic tags are "#3bhk", "#room", "#personnameA", "#placenameA", and "#reminder". The generated topic tags can be edited by the user of the electronic device (100) in an extended mode. In addition, the electronic device (100) stores the topic tags.

[0095] Symbols "11C to 11D" indicate that the electronic device (100) receives at least one candidate metadata tag. In the example, the candidate metadata tag is "#3bhk". The electronic device (100) then compares the candidate metadata tag with the stored metadata tags and retrieves at least one data item (e.g., a mobile phone number) corresponding to the at least one candidate metadata tag.

[0096] Figure 12A 、 Figure 12B 、 Figure 12C and Figure 12D is another example illustration of a smart summarizer based on conversation summary generation for a messaging application of an electronic device (100) according to an embodiment as disclosed herein.

[0097] Symbol "12A" indicates that the electronic device (100) receives input from the user of the electronic device (100). The electronic device (100) then analyzes the screen content of the electronic device (100). In the example, the screen content is related to an incoming voice call from Mr. Jack (i.e., the second user). Symbol "12B" indicates that the electronic device (100) automatically generates at least one metadata tag (e.g., a topic tag) related to the incoming voice call session, and the at least one metadata tag is provided based on the priority of the user of the electronic device (100). In the example, the topic tags are "#abc station", "#airport", "#07:30am", and "#passport". The generated topic tags can be edited by the user of the electronic device (100) in an extended mode. In addition, the electronic device (100) stores the topic tags.

[0098] Symbols "12C to 12D" indicate a message conversation between the user of the electronic device (100) and Mr. Jack. The message conversation is related to an incoming voice call. When the user of the electronic device (100) types "discuss on the phone," the electronic device (100) automatically retrieves a generated hashtag with actionable features (e.g., booking a taxi, setting an alarm) and shares it with Mr. A.

[0099] Figure 13A 、 Figure 13B 、 Figure 13C and Figure 13D is another example illustration of generating metadata tags for a voice recorder application of an electronic device (100) according to an embodiment as disclosed herein.

[0100] Symbol "13A" indicates that the electronic device (100) receives input from the user of the electronic device (100). Then, the electronic device (100) analyzes the voice content related to the voice recording. Symbol "13B" indicates that the electronic device (100) automatically generates at least one metadata tag (e.g., a topic tag) related to the voice recording, and the at least one metadata tag is provided based on the priority of the user of the electronic device (100). In the example, the topic tags are "#June20th", "#9amMeeting", "#Room7", "#TjHotel", and "#SaturdayMorning". The generated topic tags can be edited by the user of the electronic device (100) in an extended mode. In addition, the electronic device (100) stores the topic tags.

[0101] Symbols "13C to 13D" indicate that the electronic device (100) receives at least one candidate metadata tag. In the example, the candidate metadata tag is "#TjHotel". The electronic device (100) then compares the candidate metadata tag with the stored metadata tags and retrieves at least one data item (i.e., voice 002) corresponding to the at least one candidate metadata tag.

[0102] Figure 14A 、 Figure 14B and Figure 14C is another example illustration of smart writing generated for a messaging application of an electronic device (100) according to an embodiment as disclosed herein.

[0103] Symbol "14A" indicates a message conversation between the user of the electronic device (100) and Mr. A. The message conversation is related to a lunch party. Symbols "14B to 14C" indicate that when the user of the electronic device (100) types "#Mr. A", the electronic device (100) automatically retrieves suggestions (e.g., pictures, documents, videos, and messages) related to Mr. A. The user of the electronic device (100) selects at least one of the suggestions and shares it with Mr. A. The suggestions are maintained for reference based on a plurality of added metadata tags.

[0104] Figure 15A and Figure 15B is another example illustration of smart reply, smart share, and smart compose generated based on a received uniform resource locator (URL) of a messaging application for an electronic device (100) according to an embodiment as disclosed herein.

[0105] Symbol "15A" indicates a message conversation between the user of the electronic device (100) and Mr. A. The user of the electronic device (100) receives a URL from Mr. A. The electronic device (100) summarizes the URL content and provides summary sentence suggestions when "#" is typed. Symbol "15B" indicates that the electronic device (100) provides intelligent suggestions for sharing URLs.

[0106] In an embodiment, Figures 8A-15B The example of metadata tag generation given in can also be extended to tag summarization.

[0107] The embodiments disclosed herein may be implemented using at least one software program running on at least one hardware device and performing network management functions to control the elements.

[0108] The foregoing description of the specific embodiments will fully disclose the general nature of the embodiments herein so that others can easily modify and / or adapt the various applications of these specific embodiments by applying current knowledge without departing from the general concepts, and therefore, such adjustments and modifications should and are intended to be understood within the meaning and range of equivalents of the disclosed embodiments. It will be understood that the phraseology or terminology employed herein is for the purpose of description and not limitation. Therefore, although the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein may be practiced with modification within the spirit and scope of the embodiments as described herein.

Claims

1. A method for retrieving smart information from an electronic device (100), comprising: The electronic device (100) displays screen content; While the screen content is displayed on a display of the electronic device (100), receiving, by the electronic device (100), an input from a user for initiating extraction of at least one metadata tag associated with the screen content; In response to receiving the input from the user, extracting features from the screen content by the electronic device (100); The electronic device (100) identifies at least one data item of the screen content based on the extracted features to generate the at least one metadata tag; In response to identifying the at least one data item, determining, by the electronic device (100), a plurality of parameters associated with the at least one data item; Based on receiving the input from the user, the electronic device (100) generates the at least one metadata tag associated with the at least one data item based on the plurality of parameters, wherein the at least one metadata tag includes at least one topic tag and at least one highlight card providing a detailed summary of the screen content, and the electronic device (100) controls a display to stop displaying the screen content and display the at least one metadata tag including the at least one topic tag and the at least one highlight card, wherein the at least one topic tag and the at least one highlight card are displayed based on a specific priority of the user; and storing, by the electronic device (100), the at least one metadata tag at a memory (130) of the electronic device (100), The step of generating the at least one metadata tag includes: The electronic device (100) identifies at least one of an available image block, a text block, and an audio block as the at least one data item in the screen content; determining, by the electronic device (100), the plurality of parameters associated with the at least one of the image block, the text block, and the audio block; and The at least one metadata tag associated with the at least one data item is generated by the electronic device (100) based on the plurality of parameters.

2. The method of claim 1, further comprising: The electronic device (100) receives at least one candidate metadata tag input by a user; comparing, by the electronic device (100), the at least one candidate metadata tag with the at least one metadata tag stored in the memory (130); In response to determining a match between the at least one candidate metadata tag and the at least one metadata tag stored in the memory (130), retrieving, by the electronic device (100), at least one data item corresponding to the at least one candidate metadata tag; and At least one action related to at least one operation of storing or sharing the at least one data item is performed by the electronic device (100).

3. The method according to claim 2, wherein: The at least one data item is retrieved based on the particular priority associated with the at least one candidate metadata tag.

4. The method according to claim 1, wherein The plurality of parameters of the image block include at least one of a scene object block, an expression block, a face block, and an activity block, wherein the plurality of parameters of the text block include at least one of a keyword block, a language identification block, a classification block, a text summary block, and an electronic device (100) content aggregator block, and The multiple parameters of the audio block include at least one of audio summary and language identification.

5. The method of claim 1 , further comprising: The at least one metadata tag is generated locally by the electronic device (100) without interacting with any network device.

6. The method of claim 1, wherein: The at least one metadata tag is customizable by at least one of editing, adding, or sharing based on input from a user.

7. The method of claim 1, wherein: The at least one data item includes at least one of an image file, a video file, an audio file, and a text document.

8. An electronic device (100) for retrieving intelligent information, comprising: Display (140); a processor (120) operatively connected to a display (140); as well as The memory (130) stores instructions, wherein when the instructions are executed by the processor (120), the electronic device (100) performs the following operations: Displaying screen content on a display (140); receiving input from a user for initiating extraction of at least one metadata tag associated with the screen content while the screen content is displayed on a display (140); extracting features from the screen content in response to receiving the input from the user; identifying at least one data item of the screen content based on the extracted features to generate the at least one metadata tag; determining a plurality of parameters associated with the at least one data item; generating, based on receiving the input from the user, the at least one metadata tag associated with the at least one data item based on the plurality of parameters, wherein the at least one metadata tag includes at least one topic tag and at least one highlight card providing a detailed summary of the screen content, and controlling a display to stop displaying the screen content and to display the at least one metadata tag including the at least one topic tag and the at least one highlight card, wherein the at least one topic tag and the at least one highlight card are displayed based on a specific priority of the user; and storing the at least one metadata tag at a memory (130), When the instructions are executed by the processor (120), the electronic device (100) further causes the electronic device (100) to perform the following operations: identifying at least one of an available image block, a text block, and an audio block as the at least one data item in the screen content; determining the plurality of parameters associated with the at least one of an image block, a text block, and an audio block; and The at least one metadata tag associated with the at least one data item is generated based on the plurality of parameters.

9. The electronic device (100) according to claim 8, wherein: When the instructions are executed by the processor (120), the electronic device (100) further causes the electronic device (100) to perform the following operations: receiving at least one candidate metadata tag input by a user; comparing the at least one candidate metadata tag with the at least one metadata tag stored in a memory (130); In response to determining a match between the at least one candidate metadata tag and the at least one metadata tag stored in memory (130), retrieving at least one data item corresponding to the at least one candidate metadata tag; and At least one action related to at least one operation in storing or sharing the at least one data item is performed.

10. The electronic device (100) according to claim 9, wherein: The at least one data item is retrieved based on the particular priority associated with the at least one candidate metadata tag.

11. The electronic device (100) according to claim 8, wherein: The plurality of parameters of the image block include at least one of a scene object block, an expression block, a face block, and an activity block, wherein the plurality of parameters of the text block include at least one of a keyword block, a language identification block, a classification block, a text summary block, and an electronic device (100) content aggregator block, and The multiple parameters of the audio block include at least one of audio summary and language identification.

12. The electronic device (100) according to claim 8, wherein: When the instructions are executed by the processor (120), the electronic device (100) further causes the electronic device (100) to perform the following operations: The at least one metadata tag is generated locally without interacting with any network device.

13. The electronic device (100) according to claim 8, wherein: The at least one metadata tag is customizable by at least one of editing, adding, or sharing based on input from a user, and The at least one data item includes at least one of an image file, a video file, an audio file, and a text document.

Images