Electronic device and control method thereof

Abstract

An electronic device is disclosed. The electronic device capable of generating sentences according to the present disclosure comprises: at least one sensor; a memory to store at least one instruction; and at least one processor, where the at least one processor executes at least one instruction to: obtain a plurality of pieces of data related to a daily life of a user based on a sensed value obtained from the at least one sensor; determining a user-related event corresponding to at least one piece of data in the plurality of pieces of data based on the similarity of the plurality of pieces of data; determining a plurality of keywords related to events determined from the plurality of pieces of data; and generating at least one sentence corresponding to an event related to the user based on the plurality of keywords.

Description

Technical Field

[0001] The present disclosure relates to an electronic device that obtains a plurality of data based on a sensing value obtained from at least one sensor to generate a sentence corresponding to an event related to a user. Background Art

[0002] An electronic device that generates sentences corresponding to events related to a user can be applied and used in various ways, for example, on a computer, a smart phone, a tablet PC, a laptop computer, etc. Specifically, the electronic device can control a display to display events related to the user and keywords and sentences corresponding to the events. Summary of the invention

[0003] According to an embodiment of the present disclosure, an electronic device capable of generating a sentence includes: at least one sensor, a memory storing at least one instruction, and at least one processor executing the at least one instruction. The at least one processor can obtain a plurality of data related to a user's daily life based on a sensing value obtained from the at least one sensor; determine an event related to the user corresponding to at least one of the plurality of data based on similarity of the plurality of data; determine a plurality of keywords related to the determined event based on the plurality of data; and generate at least one sentence corresponding to the event related to the user based on the plurality of keywords.

[0004] According to an embodiment of the present disclosure, a method for controlling an electronic device capable of generating sentences includes: obtaining multiple data related to a user's daily life based on sensing values ​​obtained from at least one sensor; determining an event related to the user corresponding to at least one of the multiple data based on similarities of the multiple data; determining multiple keywords related to the determined event based on the multiple data; and generating at least one sentence corresponding to the event related to the user based on the multiple keywords.

[0005] According to an embodiment of the present disclosure, an electronic device may include at least one sensor, a memory storing at least one instruction, and at least one processor executing the at least one instruction, wherein a non-temporary computer-readable medium stores computer instructions, wherein the computer instructions, when executed by the processor of the electronic device, enable the electronic device to: obtain multiple data related to a user's daily life based on sensing values ​​obtained from the at least one sensor; determine an event related to the user corresponding to at least one of the multiple data based on similarities of the multiple data; determine multiple keywords related to the determined event based on the multiple data; and generate at least one sentence corresponding to the event related to the user based on the multiple keywords. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 is a block diagram showing a configuration of an electronic device according to an embodiment; Figure 2 is a flowchart illustrating a process of determining a plurality of data as events related to a user according to an embodiment; Figure 3 is a flowchart illustrating a process of calculating similarity of a plurality of data based on an artificial intelligence (AI) model according to an embodiment; Figure 4 is a view showing a process of calculating similarities of a plurality of data based on position data according to an embodiment, and Figure 5 is a flowchart illustrating a process of generating a sentence and displaying the sentence on a display if a user input is received according to an embodiment of the present disclosure; Figure 6 to Figure 7 is a view showing a process of displaying an event related to a user on a display according to an embodiment; Figure 8 is a flow chart illustrating a process of obtaining keyword statistics according to an embodiment; Fig. 9 is a block diagram showing a detailed configuration of an electronic device according to an embodiment; and Fig.10 is a flowchart illustrating a method of generating a sentence according to an embodiment. DETAILED DESCRIPTION

[0007] The embodiments of the present disclosure may be modified in various forms and may have various embodiments, wherein specific embodiments will be illustrated in the drawings and specifically explained in the detailed description. However, it should be noted that the various embodiments are not intended to limit the scope of the present disclosure to specific embodiments, but should be interpreted as including all modifications, equivalents and / or alternatives of the embodiments of the present disclosure. With respect to the detailed description of the drawings, similar components may be designated by similar reference numerals.

[0008] In the case where it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure in describing the present disclosure, the detailed description thereof will be omitted.

[0009] In addition, the following embodiments can be modified in various forms, and the scope of the technical ideas of the present disclosure is not limited to the following embodiments. On the contrary, these embodiments are provided to make the present disclosure more complete and complete, and to fully convey the technical ideas of the present disclosure to those skilled in the art.

[0010] The terms used in the present disclosure are only used to explain specific embodiments and are not intended to limit the scope of the present disclosure. A singular expression includes a plural expression unless it is clearly defined differently in the context.

[0011] In the present disclosure, expressions such as “have”, “may have”, “include” or “may include” indicate the presence of such characteristics (for example, numerical values, functions, operations, or components such as parts), and the expression does not exclude the presence of additional characteristics.

[0012] In the present disclosure, the expressions "A or B", "at least one of A and / or B", "one or more of A and / or B", etc. may include all possible combinations of the listed items. For example, "A or B", "at least one of A and B", or "at least one of A or B" may refer to all of the following: (1) including at least one A, (2) including at least one B, or (3) including all of at least one A and at least one B.

[0013] The expressions "1st", "2nd", "first", "second", etc. used in the present disclosure may be used to describe various elements regardless of any order and / or importance, wherein the expressions are merely used to distinguish one element from another and are not intended to limit the elements.

[0014] Furthermore, descriptions that one element (e.g., a first element) is “(operably or communicatively) coupled” to / is “(operably or communicatively) coupled to” another element (e.g., the second element) or is “connected to” another element (e.g., the second element) should be interpreted as follows: the one element is directly coupled to the other element, or the one element is coupled to the other element through other elements (e.g., a third element).

[0015] Conversely, description that one element (eg, a first element) is “directly coupled” or “directly connected” to another element (eg, a second element) may be interpreted to mean that other elements (eg, a third element) do not exist between the one element and the other element.

[0016] The expression "configured to (or set to)" used in the present disclosure may be used interchangeably with other expressions (such as "suitable for", "capable of", "designed to", "adapted to", "manufactured to", or "capable of") as appropriate. The term "configured to (or set to)" may not necessarily mean that a device is "specially designed to" in terms of hardware.

[0017] Conversely, in some cases, the expression "a device configured to..." may mean that the device is "capable" of performing an operation together with another device or component. For example, the phrase "at least one processor configured to perform A, B, and C" may mean at least one dedicated processor (e.g., at least one embedded processor) for performing the relevant operations or at least one general-purpose processor (e.g., a CPU or an application processor) capable of performing the relevant operations by executing one or more software programs stored in a memory device.

[0018] In the embodiments of the present disclosure, a "module" or "component" may perform at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. In addition, in addition to a "module" or "component" that needs to be implemented as specific hardware, multiple "modules" or "components" may be integrated into at least one module and implemented as at least one processor.

[0019] In addition, various elements and regions in the drawings are schematically shown. Therefore, the technical concept of the present disclosure is not limited by the relative sizes or intervals shown in the drawings.

[0020] Hereinafter, with reference to the accompanying drawings, embodiments according to the present disclosure are described in detail to be easily implemented by those skilled in the art.

[0021] Nowadays, electronic devices such as smart devices have become popular all over the world, and in particular, the penetration rate of smart phones in South Korea is as high as 95%. Through the smart device, multiple data related to the user's daily life, such as location data, movement data or photo data, can be automatically collected.

[0022] Furthermore, it has become popular to record and share daily life through many social network services (SNS), and to record parts that are not desired to be made public, a simulated diary, a memo application, or a diary application is used in parallel.

[0023] Conventional smart devices automatically collect a plurality of data related to the daily life of the user and simply manage the data. For example, each of weather data, photo data, and location data is stored and displayed only as separate information in each of weather maps, photo galleries, and travel route services by date, wherein the plurality of data are not organized and combined and recorded.

[0024] Regarding the above-mentioned multiple data, users can easily record their daily lives through diary applications or SNS of smart devices, but it is difficult for users to manage records in an organized manner because these records may be volatile. For example, there are many cases where an album of photos printed 40 years ago is retained, while photos taken by a smart phone used 20 years ago are not retained. In addition, due to the trend-sensitive characteristics of SNS, services popular in each era among many social services are selected and used, and therefore, records between services are not connected in an organized manner. As an example, it is difficult to check the records of daily life recorded in each of Cyworld, Facebook, and Instagram at once.

[0025] Therefore, the electronic device 100 according to an embodiment of the present disclosure can obtain a plurality of data related to the daily life of the user, determine an event related to the user, determine a plurality of keywords related to the user to generate a sentence, and automatically record the daily life of the user by using the sentence. In addition, the electronic device 100 can provide events related to the user such as SNS posts in a timeline in the form of cards.

[0026] Figure 1 is a block diagram for illustrating the operation of an electronic device according to an embodiment.

[0027] refer to Figure 1 , the electronic device 100 may include at least one sensor 110 , a memory 120 , and at least one processor 130 .

[0028] At least one sensor 110 may detect an operating state (e.g., power or temperature) or an external environment state (e.g., user state) of the electronic device 100, and may generate an electrical signal or data value corresponding to the detected state. That is, the at least one sensor 110 may obtain various data and multiple data related to the user of the electronic device 100. According to an embodiment, the at least one sensor 110 may include a GPS that can obtain location data of the electronic device 100, at least one accelerometer / gyroscope sensor that can obtain movement data of the electronic device 100, at least one temperature sensor, at least one humidity sensor, at least one image sensor that can take a photo, at least one grip sensor, at least one proximity sensor, at least one color sensor, at least one infrared (IR) sensor, at least one biometric sensor, at least one illumination sensor, at least one gesture sensor, at least one air pressure sensor, or at least one magnetic sensor.

[0029] The memory 120 may temporarily or non-temporarily store various programs or data, and transmit the stored information to the at least one processor 130 according to the call of the at least one processor 130. In addition, the memory 120 may store various information required for the operation, processing, control operation, etc. of the at least one processor 130 in an electronic format.

[0030] The memory 120 may include, for example, at least one of a main memory or an auxiliary memory. The main memory may be implemented by using a semiconductor storage medium such as a ROM and / or a RAM. The ROM may include, for example, a general-purpose ROM, an EPROM, an EEPROM, and / or a mask-ROM. The RAM may include a DRAM and / or an SRAM. The auxiliary memory may be implemented by using at least one storage medium that can store data permanently or semi-permanently, such as a flash memory device, a secure digital (SD) card, a solid-state drive (SSD), a hard disk drive (HDD), a magnetic drum, an optical medium such as a compact disk (CD), a DVD, or a laser disk, a magnetic tape, a magneto-optical disk, and / or a floppy disk.

[0031] The memory 120 may store at least one instruction related to the electronic device 100. In addition, the memory 120 may store an operating system (O / S) for driving the electronic device 100. In addition, the memory 120 may store various software programs or applications so that the electronic device 100 operates according to various embodiments of the present disclosure. In addition, the memory 120 may include a magnetic storage medium, such as a semiconductor memory, such as a volatile memory, such as a frame buffer or a flash memory or a hard disk.

[0032] The memory 120 may store at least one instruction.

[0033] The memory 120 may store a plurality of data related to the daily life of the user. Specifically, the memory 120 may store a plurality of data including at least one of location data, movement data, weather data, temperature data, date data, day of the week data, time data, or photo data. In addition, the memory 120 may store a plurality of keywords determined from the plurality of data and at least one sentence generated from the plurality of keywords. In addition, the memory 120 may store an AI model for determining the similarity of the plurality of data.

[0034] At least one processor 130 according to an embodiment of the present disclosure may obtain a plurality of data related to the daily life of the user based on the sensing value obtained from at least one sensor 110. Here, the plurality of data may include context information related to the daily life of the user. Here, the context information may include location, movement, weather, temperature, date, day of the week, time, photo, etc.

[0035] In addition, the at least one processor 130 may obtain a plurality of data related to the daily life of the user based on the sensing value obtained from the at least one sensor.

[0036] Thereafter, the processor may determine an event related to the user corresponding to at least one of the plurality of data based on the similarities of the plurality of data.

[0037] Subsequently, the processor may determine a plurality of keywords related to the event determined based on the plurality of data, and generate at least one sentence corresponding to the event related to the user based on the determined plurality of keywords.

[0038] Here, the at least one processor 130 may calculate the similarity of the plurality of data based on an AI model including at least one of the plurality of data, machine learning, neural network, gene, deep learning, or classification algorithm.

[0039] In addition, the at least one processor 130 may determine first position data obtained first among a plurality of position data, and may calculate similarities of the plurality of data based on a distance value between the first position data and second position data obtained after the first position data.

[0040] Optionally, the at least one processor 130 may calculate the similarity of the plurality of data based on a time period during which the plurality of movement data continue.

[0041] Here, the plurality of data corresponding to the event may include at least one of position data, movement data, weather data, temperature data, date data, day of the week data, time data, or photo data. In addition, the at least one processor 130 may determine a plurality of keywords including at least one of the following items: a position keyword, a movement keyword, a weather keyword, a temperature keyword, a date keyword, a day of the week keyword, a time keyword, or a photo keyword corresponding to the position data, movement data, weather data, temperature data, date data, day of the week data, time data, or photo data.

[0042] In addition, the at least one processor 130 may receive a user input corresponding to at least one of the multiple keywords from the user, and change at least one of the multiple keywords based on the user input. That is, the at least one processor 130 may generate at least one sentence corresponding to an event related to the user based on the changed multiple keywords.

[0043] In addition, the at least one processor 130 may receive a user input corresponding to at least one sentence from the user and change the at least one sentence based on the user input. Here, if the user input is received, the at least one processor 130 may analyze the components of the at least one sentence corresponding to the user input and generate at least one sentence corresponding to an event related to the user.

[0044] In addition, the electronic device 100 may further include a display 140, and display at least one of the plurality of data corresponding to events related to the user, the plurality of keywords, and the at least one sentence on the display.

[0045] Furthermore, the at least one processor 130 may calculate the number of identical keywords based on the plurality of keywords, and obtain keyword statistics based on the calculated number of keywords.

[0046] As above, the operation of the at least one processor 130 is described in detail hereinafter.

[0047] Figure 2 is a flowchart illustrating a process of determining a plurality of data as events related to a user according to an embodiment.

[0048] refer to Figure 2 , the at least one processor 130 may determine an event related to the user corresponding to at least one data among the plurality of data based on the similarity of the plurality of data.

[0049] Specifically, the at least one processor 130 may determine the plurality of data as events related to the user based on a threshold value, an elapsed time, and a threshold time. Here, the threshold value corresponds to a criterion for determining that the similarity between the plurality of data is low. In addition, the elapsed time corresponds to a time elapsed after obtaining data less than the threshold value. In addition, the threshold time corresponds to a criterion for determining the plurality of data as events related to the user by comparing with the threshold value.

[0050] In order to determine an event related to a user, first, the at least one processor 130 may obtain a plurality of data (S201).

[0051] Subsequently, the at least one processor 130 may postpone the determination of the plurality of data (S202). The reason for postponing the determination of the plurality of data after calculating the similarity is because it is considered that determining an event related to the user based on only one data with low similarity may be incorrect. That is, the event related to the user may be determined considering the noise of the data.

[0052] In addition, the at least one processor 130 may calculate similarities of the plurality of data ( S203 ).

[0053] Here, the similarity of the plurality of data is a numerically calculated degree of an event related to the user determined to correspond to at least one data among the plurality of data obtained by the at least one processor 130. Figure 3 and Figure 4 , describes the process of calculating similarity.

[0054] Figure 3 is a flowchart illustrating a process of calculating similarities of a plurality of data based on an artificial intelligence (AI) model according to an embodiment.

[0055] refer to Figure 3 , at least one processor 130 may calculate the similarity of multiple data based on an AI model including at least one of multiple data, machine learning, neural network, gene, deep learning, or classification algorithms.

[0056] Here, the AI ​​model may include multiple neural network layers. Each layer has multiple weight values, and the operation of the layer is performed by the operation result of the previous layer and the operation of multiple weight values. Examples of neural networks are convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recursive deep neural networks (BRDNNs), and deep Q networks, wherein the neural network is not limited to the aforementioned examples, excluding the case where a neural network is specified in the present disclosure.

[0057] At least one processor 130 may generate a knowledge graph related to the user's daily life based on the AI ​​model. In addition, at least one processor 130 may calculate the similarity of multiple data based on the knowledge graph. The knowledge graph is a graph of the correlation between specific concepts, and corresponds to a graph in which a concept starting from a broad category is connected to a detailed concept. By using this knowledge graph, the similarity between two concepts can also be quantitatively measured. Specifically, the knowledge graph can express concepts related to the user's daily life as vertices and edges connecting each vertex. Here, the similarity between each concept can be calculated as the number of edges connected to the vertex corresponding to each concept. Here, if a weight value is defined on the edge, the similarity between the concepts can be calculated considering the relevant weight value.

[0058] As an example, when the movement data “walking” is identified in a state where the current location data is identified as “park”, the at least one processor 130 may calculate the similarity between “park” and “walking” through the knowledge graph.

[0059] As another example, if "quesadilla 320" is identified among "Mexican food 310" through photo data while "dinner 300" is identified through current location data and time data, at least one processor 130 may calculate the similarity between the location data, time data, and photo data that can be identified as "dinner 300", "Mexican food 310", and "quesadilla 320".

[0060] In addition, the plurality of data may include a plurality of position data. The at least one processor 130 may determine first position data obtained first among the plurality of position data, and may calculate similarity of the plurality of data based on a distance value between the first position data and second position data obtained after the first position data.

[0061] As an example, if it is identified that the first location data is (0m, 0m) and the second location data is (1500m, 1700m), the at least one processor 130 may determine that the distance between the first location data and the second location data is 800m. That is, the at least one processor 130 may calculate that the similarity between the first location data and the second location data is low because an event associated with the user at the first location data and an event associated with the user at the second location are generally determined as separate events.

[0062] Figure 4 is a view illustrating a process of calculating similarities of a plurality of data based on position data according to an embodiment.

[0063] refer to Figure 4 , as the distance between the first position and the second position becomes farther, the at least one processor 130 may calculate the similarity of the plurality of data as linearly low. In addition, as the distance between the first position and the second position becomes farther, the at least one processor 130 may calculate the similarity of the plurality of data as nonlinearly low. In addition, if the distance is determined to be a specific distance T or greater, the at least one processor 130 may calculate the similarity of the plurality of data as 0. As above, the process of calculating the similarity of the plurality of data based on the position data is merely an example and is not limited thereto.

[0064] In addition, the plurality of data may include a plurality of movement data. Here, the movement data may include data related to movement states such as stop, walking movement, running movement, cycling movement, and vehicle movement. The at least one processor 130 may calculate the similarity of the plurality of data based on the time period for which the plurality of motion data lasts. As an example, if the movement data of the "stop" state lasts for 30 minutes in a restaurant, and after the movement data of the "walking movement" state lasts for 5 minutes, the movement data of the "stop" state lasts for 20 minutes, the at least one processor 130 may determine the data as a plurality of data having a high correlation to calculate the similarity as high.

[0065] In addition, the plurality of data may include photo data. The at least one processor 130 may identify an object included in the photo data, and may calculate similarities of the plurality of data based on the identified object. As an example, if the photo data is identified as “fried corn quesadilla” photographed at the location data identified as “restaurant”, the at least one processor 130 may calculate the similarity between the location data and the photo data as high.

[0066] As above, after calculating the similarities of the plurality of data, the at least one processor 130 may determine whether the similarities of the plurality of data are greater than a threshold value ( S204 ).

[0067] If the similarity of the last obtained data is less than the threshold value, the at least one processor 130 may initialize the elapsed time and start calculating the elapsed time (S204). The elapsed time corresponds to the time elapsed after the last plurality of data having low similarity among the plurality of data are obtained. For example, the elapsed time corresponds to the time elapsed for obtaining the plurality of second data when obtaining the plurality of second data having low similarity with the plurality of first data previously obtained.

[0068] If the similarity of the last obtained data among the plurality of data is greater than the threshold, the at least one processor 130 may determine whether to start calculating the elapsed time (S206). Here, if the elapsed time is not started, the at least one processor 130 continues to obtain the plurality of data.

[0069] If the calculation of the elapsed time starts, the at least one processor 130 may determine whether the elapsed time is greater than the threshold time (S207). Here, if the elapsed time is greater than the threshold time, the at least one processor 130 may deem that the last obtained data is not noise, and determine an event related to the user corresponding to at least one of the multiple data (S208). As an example, if after obtaining a plurality of first data having a high similarity to "restaurant", a plurality of second data having a low similarity to "restaurant" and a high similarity to "cafe" are obtained, the at least one processor 130 may initialize the elapsed time and may start calculating the elapsed time, wherein the elapsed time is the time elapsed for obtaining the plurality of second data. As above, if the elapsed time is a specific time (threshold time) or longer, the at least one processor 130 may determine the plurality of first data having a high similarity to "restaurant" as an event related to the user.

[0070] In addition, the at least one processor 130 may end the calculation of the elapsed time ( S209 ).

[0071] As above, the process for determining an event related to the user may be repeatedly performed, and the at least one processor 130 may determine a plurality of events by repeating the process.

[0072] At least one processor 130 may determine a plurality of keywords related to an event determined based on a plurality of data. That is, at least one processor 130 may determine a plurality of keywords including at least one of the following: a location keyword, a weather keyword, a temperature keyword, a date keyword, a day of the week keyword, a time keyword, or a photo keyword corresponding to location data, weather data, temperature data, date data, day of the week data, time data, or photo data.

[0073] As an example, if multiple data corresponding to an event include "Latitude: 37.5581, Longitude: 126.9346, Moving: Stop, Start Time: March 7, 2022 18:27:41, End Time: March 7, 2022 20:10:30, Temperature: 11 degrees, Humidity: 20, Weather: Cloudy", at least one processor 130 may determine keywords such as "Mexican restaurant, quesadilla restaurant, dinner, cloudy night, spring, Monday, weekdays, and 1~2 hours".

[0074] Furthermore, if the plurality of data includes photo data, the at least one processor 130 may recognize an object included in the photo data and determine an additional keyword, such as "quesadilla, Mexican food, beer, or tequila."

[0075] In addition to the plurality of data, the at least one processor 130 may also include a keyword input from a user based on a sensed value obtained from the at least one sensor. For example, if the event associated with the user is determined to be associated with a movie theater, the at least one processor 130 may determine the type of movie, the genre of the movie, etc. input from the user as a keyword.

[0076] As an example, in a case where the event is determined to be "dinner", the at least one processor 130 may determine "quesadilla, beer, tequila, etc." input from the user as a keyword.

[0077] At least one processor 130 may generate at least one sentence corresponding to an event related to the user based on multiple keywords. If multiple keywords are determined, at least one processor 130 may generate at least one sentence based on an AI model. The specific AI model is as mentioned above.

[0078] As an example, if keywords such as "Mexican food, dinner, cloudy, quesadilla, and happy" are determined, at least one processor 130 may generate at least one sentence such as, "I had dinner at a place that sells Mexican food. The weather was cloudy, but I had fun eating quesadillas."

[0079] Figure 5is a flowchart showing a process of generating a sentence and displaying the sentence on a display if a user input is received according to an embodiment. Figure 2 A detailed description of the overlapping parts of the description.

[0080] refer to Figure 5 , the at least one processor 130 may receive a user input corresponding to at least one of the plurality of keywords from the user (S504). That is, the at least one processor 130 may receive at least one input from the user for changing, adding, or deleting at least one of the plurality of keywords. As above, if the input is to change, add, or delete at least one of the plurality of keywords, the at least one processor 130 may change at least one of the plurality of keywords (S505).

[0081] As above, in addition to receiving the user's input to directly change, add or delete keywords, the at least one processor 130 may send a question related to the keyword to the user to change at least one of the multiple keywords. That is, the at least one processor 130 may send a question related to the keyword to the user by means of ordinary exposure such as a push alert or a message within the service, and may receive a user input for the question to change, add or delete a keyword.

[0082] As an example, if "Dongsuwon CGV Cinema" is determined as a location keyword, at least one processor 130 may transmit a question such as "Is that Dongsuwon CGV Cinema?" to the user. For this question, if a user input of "yes" is received, at least one processor 130 determines the location keyword as "cinema". On the contrary, if a user input of "no" is received, at least one processor 130 may determine the location keyword again based on the location data and transmit the question to the user, or the location keyword may be changed by user input. As above, not only keywords that cannot be determined based on sensing values ​​can be added by user input, but also keywords such as the user's mood or the user's vacation status that cannot be determined based on sensing values ​​can be added by user input.

[0083] In addition, if the location keyword is determined to be "cinema" through the above questions, at least one processor 130 may transmit derivative questions related to the location keyword to the user. As an example, if the location keyword is determined to be "cinema" and the time keyword is determined to be "2~4 hours", at least one processor 130 may transmit questions about whether the user watched a movie or what movie the user watched to the user. On the contrary, if the time keyword is determined to be "less than one hour", at least one processor 130 may regard the user's visit as a simple visit to the cinema and may not transmit derivative questions to the user. As another example, if the location keyword is determined to be "restaurant", at least one processor 130 may transmit derivative questions about what food the user ate to the user, or if the location keyword is determined to be "home", at least one processor 130 may transmit derivative questions about what activities the user did at home to the user.

[0084] In addition, the at least one processor 130 may generate at least one sentence corresponding to an event related to the user based on the changed multiple keywords (S506). As an example, if keywords such as "Mexican food, dinner, cloudy, fried corn quesadilla, and happy" are determined, but a user input of changing "cloudy" to "sunny" is received, the at least one processor 130 may generate at least one sentence such as "I had dinner at a place selling Mexican food. I was happier eating fried corn quesadillas on a sunny day."

[0085] In addition, the at least one processor 130 may receive a user input corresponding to at least one sentence from the user (S507). That is, the at least one processor 130 may receive at least one user input of the user input for changing, adding, or deleting the generated sentence.

[0086] As an example, if keywords such as “Mexican food, dinner, cloudy day, quesadilla, and happy” are determined, and subsequently user input is received for changing a generated sentence such as “I had dinner at a place that sells Mexican food. The weather was cloudy, but I was happy to eat quesadillas” to a sentence such as “I was happy to eat Mexican food, quesadillas on a cloudy day,” the at least one processor 130 may change the generated sentence to a sentence such as “I was happy to eat Mexican food, quesadillas on a cloudy day.”

[0087] Here, if a user input is received, at least one processor 130 may analyze the components of at least one sentence corresponding to the user input, and generate at least one sentence corresponding to an event related to the user. That is, at least one processor 130 may learn the sentence components corresponding to the user input with an AI model based on deep learning. As a result, if a plurality of keywords are determined based on a plurality of data, at least one processor 130 may generate a sentence corresponding to the learned sentence components. As an example, if a user input is received for changing a generated sentence such as "I had dinner at a place selling Mexican food" to a sentence "I am very happy to eat Mexican food, fried corn tortillas on a cloudy day", at least one processor 130 may be trained to generate a sentence by first arranging the weather keywords and then changing the location keyword "a place selling Mexican food" to "Mexican food" derived therefrom.

[0088] If the sentence is generated as above, the at least one processor 130 may display at least one of a plurality of data corresponding to an event related to the user, a plurality of keywords, and at least one sentence on the display (S509).

[0089] Figure 6 to Figure 7 is a view illustrating a process of displaying an event related to a user on a display according to an embodiment.

[0090] refer to Figure 6 , at least one processor 130 may display at least one of a plurality of data, a plurality of keywords, and a sentence in a timeline format on the display 140. Specifically, the at least one processor 130 may display the date and day of the week at the upper end of the timeline on the display 140, and may regard a plurality of data related to the determined location keyword as one event and display the data on the display 140. As an example, if the location keyword is determined to be "Beer House A", the at least one processor 130 may display "Beer House A610" as a title, and display the location data and time data below the title. Here, if the photo data 620 is obtained, the at least one processor 130 may display the photo data 620 below the location data. In addition, the at least one processor 130 may display the generated sentence 630 below the photo data 620.

[0091] In addition, the at least one processor 130 may regard a plurality of data related to the determined movement keyword as one event and display the data on the display 140. As an example, the at least one processor 130 may display movement data 640 corresponding to "walking" on the timeline and display time data below the movement data.

[0092] refer to Figure 7, the at least one processor 130 may display a plurality of location data and a plurality of movement data obtained for one day on the display 140. The at least one processor 130 may specify the obtained plurality of location data and a plurality of movement data on the map 710, and may display the map 710 on which the plurality of location data and a plurality of movement data are specified on a time axis.

[0093] As above, after displaying an event related to the user, the at least one processor 130 may receive a user input to change at least one of a plurality of data, a plurality of keywords, and at least one sentence corresponding to the event.

[0094] Figure 8 is a flow chart illustrating a process of obtaining keyword statistics according to an embodiment.

[0095] refer to Figure 8 , at least one processor 130 may calculate the number of identical keywords based on a plurality of keywords (S801). That is, if an event associated with one user is determined, at least one processor 130 may calculate the number of identical keywords based on a plurality of keywords corresponding to an event associated with one user. In addition, if an event associated with one or more users is determined, at least one processor 130 may calculate the number of identical keywords based on a plurality of keywords corresponding to an event associated with one or more users. As an example, if "beer," "soju," and "food" are determined as keywords corresponding to an event at 7 to 8 p.m., and "beer," "vodka," and "tequila" are determined as keywords corresponding to an event at 9 to 11 p.m., at least one processor 130 may calculate the number of "beer" as 2, the number of "soju" as 1, the number of "food" as 1, the number of "vodka" as 1, and the number of "tequila" as 1.

[0096] In addition, the at least one processor 130 may obtain keyword statistics based on the calculated number of keywords (S802). That is, the at least one processor 130 may obtain keyword statistics based on the calculated number of keywords using methods such as a histogram, a bar graph, a mosaic, a tree diagram, a pie chart, a line graph, a Pareto chart, or a cumulative bar graph. In addition, this is only an example, and the method for obtaining keyword statistics is not limited thereto.

[0097] Fig. 9 is a block diagram showing a detailed configuration of an electronic device according to an embodiment.

[0098] refer to Fig. 9, the electronic device 100 may include at least one sensor 110, a memory 120, at least one processor 130, a display 140, a microphone 150, an input interface 160, a speaker 170, and a communication interface 180. Figure 1 A detailed description of the overlapping parts of the description.

[0099] The display 140 may include various types of display panels, such as a liquid crystal display (LCD) panel, an organic light emitting diode (OLED) panel, an active matrix organic light emitting diode (AM-OLED) panel, a liquid crystal on silicon (LcoS) panel, a quantum dot light emitting diode (QLED) panel, and a digital light processing (DLP) panel, a plasma display panel (PDP), an organic LED panel, and a micro LED panel, but is not limited thereto. In addition, the display 140 may be configured with a touch screen together with a touch panel, and may be configured by a flexible panel. Specifically, the display 140 may display a plurality of data corresponding to an event related to a user, a plurality of keywords, and at least one sentence. As an example, the display 140 may display a plurality of data corresponding to an event related to a user, a plurality of keywords, and at least one sentence in a timeline form.

[0100] The microphone 150 may represent a module that obtains sound and converts the sound into an electrical signal, and may be a condenser microphone, a ribbon microphone, a dynamic microphone, a piezoelectric element microphone, a carbon microphone, or a micro-electromechanical system (MEMS) microphone. In addition, the microphone may be implemented in an omnidirectional method, a bidirectional method, a unidirectional method, a sub-cardioid method, a super-cardioid method, or a super-cardioid method. That is, the sound data obtained by the microphone 150 may be included in a plurality of data. In addition, the electronic device 100 may receive a user input that changes a plurality of keywords or at least one sentence through the microphone 150.

[0101] The input interface 160 may include a circuit and receive a user command for setting or selecting various functions supported by the electronic device 100. To this end, the input interface 160 may include a plurality of buttons and be implemented as a touch screen capable of simultaneously performing the function of a display.

[0102] In this case, the at least one processor 130 may control the operation of the electronic device 100 based on a user command input through the input interface 160. For example, the at least one processor 130 may control the electronic device 100 based on an on / off command of the electronic device 100, an on / off command of a function of the electronic device 100, etc. input through the input interface 160.

[0103] The speaker 170 may output audio sounds. Specifically, the at least one processor 130 may output various alarm sounds or voice guidance messages related to the operation of the electronic device 100 through the speaker 170. That is, the at least one processor 130 may output an alarm sound through the speaker 170 for questions related to keywords and derivative questions related thereto from the user, or may directly output a question sentence.

[0104] Fig.10 is a flowchart illustrating a method of generating a sentence according to an embodiment.

[0105] refer to Fig.10 , the method may include obtaining a plurality of data related to the daily life of the user based on a sensing value obtained from at least one sensor (S1001). Here, the plurality of data may include at least one of location data, movement data, weather data, temperature data, date data, day of the week data, time data, or photo data.

[0106] Furthermore, the method may include determining a user-related event corresponding to at least one of the plurality of data based on similarities of the plurality of data ( S1002 ).

[0107] Thereafter, the method may include determining a plurality of keywords related to an event determined based on the plurality of data ( S1003 ).

[0108] Subsequently, the method may include generating at least one sentence corresponding to an event related to the user based on the plurality of keywords ( S1004 ).

[0109] In step S1002, the method may include calculating the similarity of multiple data based on an AI model including at least one of multiple data, machine learning, neural network, gene, deep learning, or classification algorithm.

[0110] In addition, in step S1002, the method may include determining first position data obtained first among multiple position data, and may calculate similarity of the multiple data based on a distance value between the first position data and second position data obtained after the first position data.

[0111] Optionally, in step S1002, the method may include calculating similarities of the plurality of data based on a time period during which the plurality of movement data last.

[0112] In step S1003, the method may include determining multiple keywords including at least one of the following items: location keywords, mobile keywords, weather keywords, temperature keywords, date keywords, day of the week keywords, time keywords or photo keywords corresponding to location data, mobile data, weather data, temperature data, date data, day of the week data, time data or photo data.

[0113] In step S1004, the method may include receiving a user input corresponding to at least one of the plurality of keywords from the user. Here, the method may include changing at least one of the plurality of keywords based on the user input. In addition, the method may include generating at least one sentence corresponding to an event related to the user based on the changed plurality of keywords.

[0114] In step S1004 , the method may include receiving a user input corresponding to at least one sentence from a user, and changing the at least one sentence based on the user input.

[0115] Furthermore, at step S1004, the method may include: if a user input is received, analyzing at least one sentence component corresponding to the user input. Here, the method may include generating at least one sentence corresponding to an event related to the user based on a plurality of components and a plurality of keywords.

[0116] Furthermore, the method may include displaying at least one of a plurality of data corresponding to events related to the user, a plurality of keywords, and at least one sentence.

[0117] Furthermore, the method may include calculating the number of identical keywords based on the plurality of keywords, and obtaining keyword statistics based on the calculated number of keywords.

[0118] The AI-related functions according to the present disclosure are operated through the memory and processor of the electronic device.

[0119] The processor may be configured by one or more processors. Here, the one or more processors may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), or a neural processing unit (NPU), but is not limited to the examples of the aforementioned processors.

[0120] The CPU is a general-purpose processor that can perform not only general operations but also AI operations, and can efficiently execute complex programs through a multi-layer cache structure. The CPU is conducive to a serial processing method in which an organic connection between a previous calculation result and a next calculation result is possible through sequential calculation. The general-purpose processor is not limited to the aforementioned examples, excluding the case where the present disclosure specifies it as the aforementioned CPU.

[0121] A GPU is a processor for large-scale operations (such as floating-point operations for graphics processing), and can integrate cores on a large scale to perform large-scale operations in parallel. In particular, compared with a CPU, a GPU can facilitate parallel processing methods such as convolution operations. In addition, a GPU can be used as a co-processor to supplement the functions of a CPU. The processor for large-scale operations is not limited to the aforementioned examples, excluding the case where the present disclosure specifies it as the aforementioned GPU.

[0122] NPU is a processor dedicated to AI operations using artificial neural networks, where each layer configuring the artificial neural network can be implemented as hardware (e.g., silicon). Here, the NPU is designed to be dedicated to the specifications required by the manufacturer, so its degree of freedom is lower than that of the CPU or GPU, but it can effectively perform the AI ​​operations required by the manufacturer. In addition, as a processor dedicated to AI operations, the NPU can be implemented in various forms, such as a tensor processing unit (TPU), an intelligent processing unit (IPU), or a visual processing unit (VPU). The AI ​​processor is not limited to the examples mentioned above, excluding the case where the present disclosure specifies it as the NPU mentioned above.

[0123] In addition, one or more processors may be implemented as a system on chip (SoC). Here, in addition to one or more processors, the SoC may further include a memory and a network interface, such as a bus for data communication between the processor and the memory.

[0124] If the SoC included in the electronic device includes a plurality of processors, the electronic device may perform AI-related operations (e.g., operations related to learning or inference of an AI model) by using some of the plurality of processors. For example, the electronic device may perform AI-related operations by using at least one of a GPU, an NPU, a VPU, a TPU, or a hardware accelerator dedicated to AI operations (such as convolution operations or matrix product calculations) among the plurality of processors. In addition, this is merely an example, and it is apparent that AI-related operations may be processed by using a general-purpose processor such as a CPU.

[0125] In addition, the electronic device can perform operations for AI-related functions by using multiple cores (e.g., dual cores, quad cores) included in one processor. Specifically, the electronic device can perform AI operations such as convolution operations and matrix product calculations in parallel by using multiple cores included in the processor.

[0126] The one or more processors may control the electronic device to process input data according to predefined operating rules or AI models stored in the memory. The predefined operating rules or AI models are constructed by learning.

[0127] Here, building by learning means building a predefined operating rule or AI model with desired characteristics by applying a learning algorithm to various learning data. The learning can be performed in the device itself that executes the AI ​​according to the present disclosure, and can also be performed by a separate server / system.

[0128] The AI ​​model may include multiple neural network layers. At least one layer has at least one weight value, and the operation of the layer is performed by the operation result of the previous layer and at least one defined operation. Examples of neural networks are convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), deep Q networks, and converters, wherein the neural network is not limited to the aforementioned examples, excluding the case where a neural network is specified in the present disclosure.

[0129] A learning algorithm is a method in which a given target device (e.g., a robot) is trained by using various learning data so that the given target device can make decisions or predict decisions by itself. Examples of learning algorithms are supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, wherein the learning algorithm is not limited to the aforementioned examples, excluding the case where the learning algorithm is specified in the present disclosure.

[0130] In addition, according to an embodiment of the present disclosure, the above-mentioned various examples may be implemented as software including instructions stored in a machine (e.g., computer) readable storage medium. A machine may refer to a device that calls instructions stored in a storage medium and can operate according to the called instructions, wherein the machine may include a device according to an embodiment of the disclosure. If the instruction is executed by at least one processor, the at least one processor may directly execute the function corresponding to the instruction, or execute the function corresponding to the instruction by using other components under the control of at least one processor. The instruction may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" only means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), wherein the term does not distinguish between a case where data is semi-permanently stored in a storage medium and a case where data is temporarily stored in a storage medium. For example, a "non-transitory storage medium" may include a buffer that temporarily stores data.

[0131] According to an embodiment, the method according to various examples disclosed in the present disclosure may be provided to be included in a computer program product. The computer program product may be traded between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or via an application store (e.g., a play store).TM ) online distribution (e.g., downloading or uploading), or directly between two user devices (e.g., smart phones). In the case of online distribution, at least part of the computer program product (e.g., a downloadable application) may be at least temporarily stored or may be temporarily generated in a machine-readable storage medium (e.g., a memory of a manufacturer's server, an application store's server, or a relay server).

[0132] As above, preferred examples of the present disclosure are shown and described. However, it is obvious that the present disclosure is not limited to the specific examples mentioned above, and those skilled in the art may implement various modifications without departing from the subject matter of the present disclosure claimed in the scope of the claims, wherein these modifications should not be understood independently of the technical spirit or prospect of the present disclosure.

Claims

1. An electronic device capable of generating sentences, comprising: at least one sensor; a memory storing at least one instruction; as well as at least one processor executing the at least one instruction, Wherein, the at least one processor is configured to: obtaining a plurality of data related to a user's daily life based on a sensing value obtained from the at least one sensor; determining an event related to the user corresponding to at least one of the plurality of data based on similarities of the plurality of data; determining a plurality of keywords related to the determined event based on the plurality of data; and At least one sentence corresponding to an event related to the user is generated based on the plurality of keywords.

2. The electronic device according to claim 1, wherein: The at least one processor is configured to: Calculating similarities of the plurality of data based on an artificial intelligence (AI) model, wherein the AI ​​model includes at least one of the plurality of data, machine learning, neural network, genetics, deep learning, or classification algorithms; and Events related to the user are determined based on the similarity.

3. The electronic device according to claim 1, wherein: The plurality of data includes a plurality of location data, and Wherein, the at least one processor is configured to: Determine first position data obtained first among the plurality of position data; calculating similarities of the plurality of data based on a distance value between the first position data and second position data obtained after the first position data; and Events related to the user are determined based on the similarity.

4. The electronic device according to claim 1, wherein: The plurality of data includes a plurality of movement data, and Wherein, the at least one processor is configured to: Calculating similarities of the plurality of data based on a time period during which the plurality of movement data lasts; and Events related to the user are determined based on the similarity.

5. The electronic device as claimed in claim 1, wherein: The plurality of data corresponding to the event include at least one of location data, movement data, weather data, temperature data, date data, day of the week data, time data, and photo data, and Wherein, the at least one processor is configured to: Determine a plurality of keywords including at least one of the following items: location keywords, movement keywords, weather keywords, temperature keywords, date keywords, day of the week keywords, time keywords and photo keywords corresponding to the location data, the movement data, the weather data, the temperature data, the date data, the day of the week data, the time data and the photo data.

6. The electronic device as claimed in claim 1, wherein: The at least one processor is configured to: receiving a user input corresponding to at least one of the plurality of keywords from the user; changing at least one of the plurality of keywords based on the user input; as well as At least one sentence corresponding to an event related to the user is generated based on the changed plurality of keywords.

7. The electronic device as claimed in claim 1, wherein: The at least one processor is configured to: receiving a user input corresponding to the at least one sentence from the user; and The at least one sentence is changed based on the user input.

8. The electronic device as claimed in claim 7, wherein: The at least one processor is configured to: based on receiving the user input, analyzing components of the at least one sentence corresponding to the user input; as well as At least one sentence corresponding to an event related to the user is generated based on the analyzed components and the plurality of keywords.

9. The electronic device as claimed in claim 1, further comprising: monitor, Wherein, the at least one processor is configured to: At least one of the plurality of data, the plurality of keywords, or the at least one sentence corresponding to an event related to the user is displayed on the display.

10. The electronic device according to claim 1, wherein: The at least one processor is configured to: calculating the number of identical keywords based on the plurality of keywords; and Keyword statistics are obtained based on the calculated number of keywords.

11. A method for controlling an electronic device capable of generating a sentence, comprising: obtaining a plurality of data related to a user's daily life based on a sensing value obtained from at least one sensor; determining an event related to the user corresponding to at least one of the plurality of data based on similarities of the plurality of data; determining a plurality of keywords related to the determined event based on the plurality of data; as well as At least one sentence corresponding to an event related to the user is generated based on the plurality of keywords.

12. The method of claim 11, wherein: Events related to the user are determined to include: Calculating similarities of the plurality of data based on an AI model, wherein the AI ​​model includes at least one of the plurality of data, machine learning, neural network, genetics, deep learning, or a classification algorithm; and Events related to the user are determined based on the similarity.

13. The method of claim 11, wherein: The plurality of data includes a plurality of location data, and Wherein, determining the events related to the user includes: Determine first position data obtained first among the plurality of position data; calculating similarities of the plurality of data based on a distance value between the first position data and second position data obtained after the first position data; and Events related to the user are determined based on the similarity.

14. The method of claim 11, wherein: The plurality of data includes a plurality of movement data, and Wherein, determining the events related to the user includes: Calculating similarities of the plurality of data based on a time period during which the plurality of movement data lasts; and Events related to the user are determined based on the similarity.

15. The method of claim 11, wherein: The plurality of data corresponding to the event include at least one of location data, movement data, weather data, temperature data, date data, day of the week data, time data, and photo data, and Wherein, determining the multiple keywords includes: Determine a plurality of keywords including at least one of the following items: location keywords, movement keywords, weather keywords, temperature keywords, date keywords, day of the week keywords, time keywords and photo keywords corresponding to the location data, the movement data, the weather data, the temperature data, the date data, the day of the week data, the time data and the photo data.

Images