Lifelogging device and method utilizing audio recognition

Abstract

The present invention relates to a lifelogging device and method utilizing audio recognition, and more particularly to a device capable of recording and classifying audio lifelogs using an artificial intelligence algorithm. To this end, the lifelogging device of the present invention includes an input unit for inputting lifelog data including audio signals, an analysis unit for analyzing the input data, a determination unit for classifying the data based on the analyzed analysis value, and a recording unit for recording the input data and the classified class of the data.

Description

[Technical Field]

[0001] The present invention relates to a lifelogging device and method utilizing audio recognition, and to a device that can record and classify audio lifelogs using artificial intelligence algorithms. [Background technology]

[0002] With the changing times, life log services have become available that record information that users can obtain in their daily lives and search for it as needed. Conventional life log services collect and record information from users' daily lives using cameras, GPS, pulse meters, distance meters, etc., and then make the recorded information available for later use. Health-related life logs have traditionally been used by measuring a user's travel distance, altitude, pulse rate, and other biometric signals using band-type wearable devices. To obtain various user information, conventional life log services require the use of separate sensors attached to the user's body. However, attaching multiple sensors to the user's body not only causes inconvenience in the user's daily life but also may cause user resistance, making them difficult to commercialize. Furthermore, conventional life log services have problems with the large amount of data collected for each piece of information, occupying a large amount of memory space and failing to recognize a user's overall lifestyle patterns, such as their location and situation.

[0003] In order to solve the problems of the prior art, the present invention proposes a lifelog device and method that utilizes audio recognition and can be applied to earphones that users have traditionally used. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Republic of Korea Patent No. 10-1777609 (2017.09.06) Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention seeks to provide a lifelogging device that utilizes only audio recognition.

[0006] The present invention also aims to provide a device that can record a life log solely through an ear device worn by the user, while also collecting information other than audio recognition.

[0007] Another objective is to enable recorded life logs to be searched and collected by tag. [Means for solving the problem]

[0008] In order to achieve the above object, the present invention provides a life log device comprising: an input unit into which life log data including an audio signal is input; an analysis unit that extracts an analysis value of the input data; a judgment unit that classifies the class of the data based on the extracted analysis value; and a recording unit that records the input data and the classified class of the data. [Effects of the Invention]

[0009] According to the present invention, it is possible to perform life logging using audio signals with just one ear device.

[0010] In addition, it is possible to collect and record the user's visiting location, situation, conversation, and various audio information through audio signals.

[0011] In addition, the recorded information is automatically classified by an artificial intelligence algorithm, allowing users to extract desired information through searches. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic diagram showing the overall system of a life log device according to an embodiment of the present invention; [Figure 2] 1 is a diagram illustrating a life log device according to an embodiment of the present invention. [Figure 3] 1 is a block diagram showing the configuration of a life log device according to an embodiment of the present invention. [Figure 4] 1 is a diagram showing a screen of a search unit of a life log device according to a first embodiment of the present invention; [Figure 5] 1 is a diagram showing a screen of a search unit of a life log device according to a first embodiment of the present invention; [Figure 6] 1 is a diagram showing a screen of a search unit of a life log device according to a first embodiment of the present invention; [Figure 7] 1 is a diagram showing a screen of a search unit of the life log device according to the first embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] A life log device comprising: an input unit to which life log data including an audio signal is input; an analysis unit to extract an analysis value of the input data; a judgment unit to classify the class of the data based on the extracted analysis value; and a recording unit to record the input data and the classified class of the data.

[0014] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in various different forms and is not limited to the drawings and embodiments disclosed below. Furthermore, in order to clearly explain the present invention in the drawings, parts that are not related to the present invention are omitted, and the same or similar reference numerals in the drawings indicate the same or similar components.

[0015] The objectives and effects of the present invention will be naturally understood or will become clearer through the following description, and detailed explanations will be omitted if it is determined that such explanations may unnecessarily obscure the gist of the present invention. Therefore, the objectives and effects of the present invention are not limited to the following description.

[0016] The invention will now be described in more detail with reference to the drawings.

[0017] FIG. 1 is a schematic diagram showing the overall system of a life log device according to an embodiment of the present invention.

[0018] Briefly describing one embodiment of the present invention with reference to FIG. 1, the present invention relates to a lifelogging device that receives, records, and stores lifelog data input from a user's daily life. Specifically, the device can determine and record various information based on an audio signal through analysis. More specifically, as shown in FIG. 1, one or more audio signals (Audio Data) can be input to a lifelogging device according to one embodiment of the present invention, and the input audio signals can be classified into classes based on analysis values ​​of each data analyzed by an artificial intelligence algorithm. In this case, the classes can be classified according to a user's pre-classification settings and can be upgraded through learning by the artificial intelligence algorithm. The audio signals classified into classes can be extracted in response to a user's search or request in the future, allowing the user to easily provide desired data.

[0019] Hereinafter, the lifelogging device of the present invention will be described in more detail with reference to Figures 2 and 3. Figure 2 is a diagram showing a lifelogging device according to an embodiment of the present invention, and Figure 3 is a block diagram showing the configuration of a lifelogging device according to an embodiment of the present invention.

[0020] 2 and 3, according to one embodiment, the lifelogging device of the present invention may include an input unit 110, an analysis unit 120, a determination unit 130, a recording unit 140, a time setting unit 150, and a search unit 210. Before describing each component in detail, the lifelogging device may be implemented using conventional earphones, headsets, various wearable devices, wireless terminals, etc. For convenience of explanation, the lifelogging device will be described using wireless earphones, which are the most preferred embodiment, in one embodiment of the present invention.

[0021] The input unit 110 is configured to receive lifelog data including an audio signal. Specifically, it may receive an externally generated audio signal through a sensor such as a microphone. In addition to the audio signal, it may also receive input including at least one of the time at which the data is input, the user's location, the user's pulse rate, and body temperature through various sensors. However, most essentially, the input unit 110 of the lifelog device of the present invention receives an audio signal. Specifically, the input audio signal may include external noise, sounds of people talking nearby, various environmental sounds, etc.

[0022] The analysis unit 120 may extract an analysis value from data input via the input unit 110. The analysis unit 120 is configured to analyze the input data and transmit the analysis value extracted from the audio signal included in the input data to the determination unit 130. Specifically, the analysis value may be a value extracted from at least one of the frequency and amplitude of the audio signal, but more preferably, may be provided as an abstract, non-intuitive specific value that can be determined by an AI algorithm. Specifically, an accurate numerical value may be transmitted, but the value of the audio signal itself may be transmitted and a classification and determination value may be extracted based on the learning results of the AI ​​algorithm. More specifically, when multiple audio signals are input, each audio signal may be analyzed separately, and an analysis value may be extracted for each audio signal and transmitted to the determination unit 130. In this case, the analysis value may include analysis information of the audio signal as well as other information of the measured data. Specifically, when the input unit 110 receives a user's biometric information along with the audio signal, the analysis value of the user's biometric information, such as pulse rate and body temperature, may also be transmitted to the determination unit 130.

[0023] The determination unit 130 classifies data into classes based on the analysis value of each piece of data extracted by the analysis unit 120. Specifically, the data classes classified by the determination unit 130 may be classified according to the user's convenience, such as including one or more of human voice, music, everyday sounds, other environmental sounds, and noise, depending on the user's settings. More specifically, the determination unit 130 may classify data into classes based on the analysis value using a pre-installed artificial intelligence algorithm, and the artificial intelligence algorithm may learn from the final judgment value obtained by classifying the classes through user feedback. Specifically, if the judgment value determined by the determination unit 130 is not the classification set by the user, the user may optionally modify the class of the corresponding data, and the artificial intelligence algorithm may learn the modification result, thereby upgrading the data to provide a judgment value customized for the user. The determination unit 130 may classify data into classes based on the analysis value, but may also determine information related to the location where the corresponding data was input. In detail, if the determination unit 130 recognizes the corresponding audio signal as a subway announcement based on the analysis value of the audio signal and classifies the corresponding data as a subway announcement, the determination value may classify the corresponding data as a subway announcement and may also include location information such as public transportation or subway. In this way, the determination unit 130 may match the determination value of the data with information related to the location where the data is input and transmit it to the recording unit 140.

[0024] The recording unit 140 is configured to record the judgment value of the data judged by the judgment unit 130, and can match and record the data class and location information included in the judgment value judged by the judgment unit 130. In addition, the input time of the corresponding data and the user's physical information at that time can also be matched and recorded. Specifically, the recording unit 140 matches and records the judgment value of the data input, analyzed, and judged by the input unit 110 with various information, but can remove data judged to be in the noise class without recording it for memory efficiency. More specifically, data including audio signals judged to be in the noise class and information matched to the corresponding data can all be removed, and only meaningful data and information can be stored in the recording unit 140.

[0025] The time setting unit 150 is configured to set a time for receiving data input from the input unit 110. The lifelogging device of the present invention can input data only for a predetermined time set via the time setting unit 150 through a user's operation. Specifically, the time setting unit 150 can conserve battery power in the lifelogging device according to an embodiment of the present invention and minimize data input during meaningless situations. The time setting unit 150 can also control the on / off of the input unit 110 of the lifelogging device, or the on / off can be controlled by an artificial intelligence algorithm that recognizes specific words or situations. Specifically, because battery consumption is lower in standby mode than when the input unit 110 is activated, the standby mode can be maintained. However, if a specific word is input or a specific location or situation is recognized, the standby mode can be interrupted and the input unit 110 can be activated. Specifically, if a specific frequency range or words such as "on" or "wake up" are recognized, the standby mode can be interrupted and lifelogging can be initiated.

[0026] The search unit 210 is configured to search for data information recorded in the recording unit 140, and the recording unit 140 can extract and provide recorded data matching keyword information input by the search unit 210. Specifically, the search unit 210 may be provided in an external user terminal or in the life logging device of the present invention. More specifically, a search may be performed by directly inputting keywords through the external user terminal, or by the user inputting keywords by voice through a microphone of the user terminal or the life logging device. The data stored in the recording unit 140 may be matched with information such as classes and keywords and stored, and matching data based on the keyword information input by the search unit 210 may be extracted and provided to the user.

[0027] A more detailed first embodiment of extracting recorded life log data will be described below with reference to Figures 4 to 7. Figures 4 to 7 are diagrams showing screens of a search unit of a life log device according to a first embodiment of the present invention.

[0028] First, as described above, the life log data input through the input unit 110 is classified into classes according to each data type through an artificial intelligence algorithm, and the data can be matched with the classified classes and recorded in the recording unit 140. The user can then extract desired information data based on class or keyword information through the search unit 210. According to the first embodiment, the search unit 210 may be provided in a separate user terminal and, more specifically, provided to the user through a screen on the user terminal. As shown in FIG. 4, the screen provided to the user may be provided based on the classes into which the input data is classified, and the user can select the class they wish to search. Therefore, when a user clicks on CLASS1 or makes a search request through voice to search for data classified into CLASS1 as shown in FIG. 4, a list of data classified into CLASS1 can be extracted as shown in FIG. 5. Thereafter, when the user selects one of the extracted data lists, the audio signal of the selected data can be played and listened to. Specifically, when the user selects Audio Data1 from the data list classified into CLASS1 as shown in FIG. 5, the selected Audio Data1 can be played as shown in FIG. 6. In another embodiment, other information matched with the corresponding Audio Data1 may also be output. In particular, recorded information may be extracted by matching it with the user's biorhythm at the time, place, and circumstances when the Audio Data1 was input. According to the first embodiment, if a user wants to add tag information to Audio Data1, as shown at the bottom of Figure 6, the user selects the Add Tag button, and a tag information input window appears as shown in Figure 7. The user can add tag information by directly entering tag information in the input window. Adding tag information in this way matches the input tag information with the corresponding data, allowing the corresponding data to be found simply by searching for the tag information.

[0029] A second embodiment, which is yet another embodiment, will now be described in detail.

[0030] According to the second embodiment, life log data input through the input unit 110 may be classified into classes for each data type through an AI algorithm, and the data may be matched with the classified classes and recorded in the recording unit 140. The classes may be automatically classified through learning by the AI ​​algorithm, or the classification may be adjusted according to a user's request. More specifically, the classification classes may be classified into one level, or may be further subdivided into sub-categories of higher-level categories. For example, if a class of human conversation sounds is a higher-level category, the sub-categories may be classified by analyzing the voice in detail, including the situation, the number of people being conversed with, and more specifically, who the conversation target is. Thereafter, the search unit 210 may provide one or more of the number of times and the duration of data classified by class according to a user's search. More specifically, when a user requests information classified as CLASS 1, the user may confirm information such as how many times the data classified as CLASS 1 has been performed and the total duration. According to a specific embodiment, the system may provide information such as the number of times a user washed their hands per day by providing the number of times a user brushed their teeth per day and the duration of tooth brushing by providing the number of times a user brushed their teeth per day by providing the number of times a user brushed their teeth per day, thereby helping the user manage their correct lifestyle habits and patterns. Furthermore, by tracking keyboard typing sounds and analyzing the classes classified as typing sounds, the system may provide summaries of typing time and number of typings, thereby enabling a general check of the level of concentration at work, actual work time, workload, etc. Furthermore, if the system classifies the sounds as conversation sounds with others, the AI ​​algorithm may learn frequently interacting speakers to provide information on the type of conversation with each speaker, and may also summarize key parts of the conversation by grasping the accent and pitch of the conversation between speakers.

[0031] In this way, the life log device utilizing the audio recognition of the present invention has the effect of classifying and saving audio signals from the user's daily life, allowing the user to extract a desired audio signal through a keyword search, and providing information such as the number and duration of each class based on the audio signal.

[0032] The present invention relates to one embodiment, which is merely an example, and those skilled in the art may be able to make various modifications and equivalent embodiments. Therefore, the scope of the present invention is not limited by the above example and the accompanying drawings. [Industrial Applicability]

[0033] According to the present invention, it is possible to perform life logging using audio signals with just one ear device.

[0034] In addition, it is possible to collect and record the user's visiting location, situation, conversation, and various audio information through audio signals.

[0035] In addition, the recorded information is automatically classified by an artificial intelligence algorithm, allowing users to extract desired information through searches.

Claims

1. an input unit (110) to which life log data including an audio signal is input; an analysis unit (120) for extracting analytical values ​​from the input data; a determination unit (130) for classifying the data into classes based on the extracted analytical values; and a recording unit (140) for recording the input data and the classified class of the data; A search unit (210) for searching the recorded data information; The analysis unit (120) transmitting an analysis value extracted from the audio signal included in the input data to the determination unit (130); The determination unit (130) The judgment value of the data is matched with information related to the location where the data was input, the time of input of the data, and the physical information of the user at the time of input of the data, and transmitted to the recording unit (140); The recording unit (140) The judgment value of the data judged by the judgment unit (130) is recorded, but the data judged as a noise class and information matched to the data judged as a noise class are removed, The determination unit (130) classifying the data into classes based on the analysis values ​​using a pre-installed artificial intelligence algorithm; The artificial intelligence algorithm learning based on user feedback on the judgment value judged by the judgment unit (130); If the judgment value obtained by classifying the class by the judgment unit is not the classification set by the user, learning the class of the data corrected by the user; The recording unit (140) The search unit (210) extracts recorded data that matches the input keyword information, The search unit (210) A life log device that provides at least one of the number of times and the time of the classified class data.

2. The method further includes a time setting unit (150) that sets a time for receiving the data input from the input unit (110); The life log device according to claim 1, wherein the data is input for a predetermined time period set in advance through the time setting unit (150).

3. The life log data inputted by the input unit (110) is 3. The life log device according to claim 2, wherein the data includes at least one of the time when the data is input, the user's location, the user's pulse rate, and the user's body temperature.

4. The search unit (210) It is provided through the screen of the user's device, When the user selects a classified class of the data, A list of data classified into the relevant class is extracted, Select one of the extracted data lists, 2. The life log device according to claim 1, wherein an audio signal of the selected data is reproduced.

5. 5. The life log device according to claim 4, wherein the user additionally inputs arbitrary tag information to the selected data audio signal, and the input tag information is matched with the corresponding data.

Images