Information processing system and program

Abstract

To save part of acquired voice information as a voice record without specifying a start position and an end position of a desired period to be recorded as the voice record.SOLUTION: A voice information temporary storage part 33 acquires an external voice as voice information by a voice input part 31 and temporarily stores it. If a trigger detection part 38 detects a previously set specific condition being not met, a memo information selection part 34 selects voice information including voice information acquired before the time when the specific condition being met is detected, and stores the selected voice information as voice memo information in the form of a voice record in a memo information storage part 35.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an information processing system and a program. [Background technology]

[0002] Patent document 1 discloses an information processing device that, when a first word is detected from acquired voice data, creates a voice memo by converting the voice data after the first word is detected into text until a second word is detected, and displays the voice memo at a specified position on the screen. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-111191 A Summary of the Invention [Problem to be solved by the invention]

[0004] There are systems that allow users to specify the start and end positions of a period they wish to save for acquired voice data by operating the user or by speaking specific words, and generate voice data for the specified period as a voice memo. If users try to save voice memos as audio recordings during various tasks using such a system, they must consciously speak or operate the system to save the voice memo, which may disrupt the work they are currently doing.

[0005] An object of the present disclosure is to provide an information processing system and program that can save a portion of acquired audio information as an audio recording without specifying the start and end positions of the period that you want to keep as an audio recording. [Means for solving the problem]

[0006] The information processing system of the first aspect includes a processor and a memory, external voice is sequentially acquired as voice information and temporarily stored in said memory; When it is detected that a preset specific condition is satisfied, voice information including voice information acquired before the time when it was detected that the specific condition was satisfied is stored in the memory as a voice record.

[0007] An information processing system according to a second aspect of the present invention is the information processing system according to the first aspect, wherein the processor: Converting the acquired voice information into text information and temporarily storing it in the memory; When it is detected that a preset specific condition is satisfied, voice information including voice information before the time when it was detected that the specific condition was satisfied is stored in the memory as a voice record going back in time to a part that has linguistic connection.

[0008] A third aspect of the information processing system is the information processing system of the second aspect, in which, when it is detected that a predetermined specific condition is satisfied, the processor stores audio information including audio information from the detection point of a predetermined word to the current time in the memory as an audio recording.

[0009] The information processing system of the fourth aspect is the information processing system of the second aspect, A position detection unit that detects the position of the system itself is further provided, When the processor detects that a specific predetermined condition is satisfied, it records and stores in the memory audio information including audio information obtained from the time when the position of the system entered within a range of a predetermined distance from the current position to the current time.

[0010] An information processing system according to a fifth aspect is the information processing system according to the second aspect, Further comprising an image capturing unit for capturing an image, When an image is captured by the imaging unit, the processor identifies an object included in the image captured by the imaging unit, and records and stores in the memory audio information that was obtained before the time the image was captured by the imaging unit and includes audio information of a phrase that includes the name of the identified object.

[0011] A sixth aspect of the information processing system is the information processing system of the second aspect, in which, when it is detected that a predetermined specific condition is satisfied, the processor records and stores in the memory audio information including audio information obtained from a time that is a predetermined time before the current time to the current time.

[0012] In a seventh aspect of the information processing system, in the information processing system of the first aspect, when a user operation is detected, the processor stores audio information in the memory as an audio recording, the audio information including audio information obtained before the time when the user operation was detected.

[0013] An information processing system according to an eighth aspect is the information processing system according to the first aspect, wherein the processor: Converting the acquired voice information into text information and temporarily storing it in the memory; When a predetermined word or group of words is detected in the converted text information, audio information including audio information acquired before the time when the word or group of words was detected is stored in the memory as an audio record.

[0014] An information processing system of a ninth aspect is the information processing system of the first aspect, Further comprising an angular velocity sensor or an acceleration sensor, When a predetermined motion is detected by the angular velocity sensor or acceleration sensor, the processor stores audio information, including audio information acquired before the time the motion was detected, in the memory as an audio recording.

[0015] An information processing system of a tenth aspect is the information processing system of the first aspect, Further comprising an image capturing unit for capturing an image, When an image is captured by the imaging unit, the processor stores audio information, including audio information acquired before the time when the image was captured by the imaging unit, in the memory as an audio record.

[0016] An information processing system of an eleventh aspect is the information processing system of the first aspect, A position detection unit is further provided to detect the position of the system itself, When the processor detects that the system has moved within a specified range from a preset position, the processor stores voice information in the memory as a voice record, including voice information obtained before the time when it was detected that the system has moved within the specified range from the preset position.

[0017] In an information processing system of a 12th aspect, in the information processing system of the 11th aspect, the processor also stores in the memory as an audio record the audio information acquired until the system moves outside a predetermined range from a preset position.

[0018] An information processing system of a thirteenth aspect is the information processing system of the first aspect, A biometric information detection unit that detects biometric information of a user is further provided, When the user's biometric information detected by the biometric information detection unit changes to a predetermined state, the processor stores in the memory as an audio recording audio information including audio information acquired before the time it was detected that the user's biometric information had changed to the predetermined state.

[0019] A program according to a fourteenth aspect includes a step of sequentially acquiring and temporarily storing external sounds as sound information; When it is detected that a predetermined specific condition has been satisfied, the computer is caused to execute a step of saving, as an audio recording, audio information including audio information acquired before the time when it was detected that the specific condition had been satisfied. Effect of the Invention

[0020] According to the information processing system of the first aspect, it is possible to save a part of the acquired voice information as a voice recording without specifying the start and end positions of the period that is to be kept as a voice recording.

[0021] According to the information processing system of the second aspect, it is possible to save a linguistically meaningful portion of the acquired audio information as an audio recording without specifying the start and end positions of the period that is desired to be kept as an audio recording.

[0022] According to the information processing system of the third aspect, a series of voices relating to preset words can be recorded and stored.

[0023] According to the information processing system of the fourth aspect, audio information linked to image capture by the image capture section can be recorded and stored.

[0024] According to the information processing system of the fifth aspect, audio information linked to image capture by the image capture section can be recorded and stored.

[0025] According to the information processing system of the sixth aspect, audio information for a preset period of time can be saved as an audio record.

[0026] According to the information processing system of the seventh aspect, the user can save the voice information as a voice recording by performing only one operation.

[0027] According to the information processing system of the eighth aspect, necessary voice information can be saved as a voice recording simply by setting in advance a word or a group of words according to a situation in which a voice recording is desired to be kept.

[0028] According to the information processing system of the ninth aspect, it is possible to save voice information as a voice recording without requiring any user operation.

[0029] According to the information processing system of the tenth aspect, audio information when the image capturing section captures an image can be saved as an audio recording.

[0030] According to the information processing system of the eleventh aspect, it is possible to store voice information when moving to a preset location as a voice record.

[0031] According to the information processing system of the twelfth aspect, audio information while being located in a preset location can be saved as an audio record.

[0032] According to the information processing system of the thirteenth aspect, voice information when the biometric information of the user changes can be stored as a voice record.

[0033] According to the program of the fourteenth aspect, it is possible to save a part of the acquired voice information as a voice recording without specifying the start and end positions of the period that is to be kept as a voice recording. [Brief description of the drawings]

[0034] [Figure 1] 1 is a diagram illustrating a system configuration of a voice memo storage system according to an embodiment of the present disclosure. [Diagram 2] 1 is a block diagram showing a hardware configuration of a mobile terminal device 10 according to an embodiment of the present disclosure. [Diagram 3] 1 is a block diagram showing a functional configuration of a mobile terminal device 10 according to an embodiment of the present disclosure. [Figure 4] 1 is a flowchart for explaining the overall operation of a voice memo storage system according to an embodiment of the present disclosure. [Diagram 5] 5 is a flowchart for explaining details of the trigger detection process in step S104 shown in the flowchart of FIG. 4. [Figure 6] 13 is a diagram for explaining a specific example of a memo information selection process in the memo information selection unit 34. FIG. [Figure 7]13 is a diagram for explaining another specific example of the memo information selection process in the memo information selection unit 34. FIG. [Figure 8] 13 is a diagram for explaining another specific example of the memo information selection process in the memo information selection unit 34. FIG. [Figure 9] 13 is a diagram showing an example of a display screen of voice memo information in the terminal device 40. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] Next, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0036] FIG. 1 is a diagram showing a system configuration of a voice memo storage system according to an embodiment of the present disclosure.

[0037] As shown in FIG. 1, the voice memo storage system of one embodiment of the present disclosure is composed of a portable terminal device 10 such as a smartphone held by the memo recorder, a microphone 60 and a wearable sensor 70 worn by the memo recorder, a storage server 20, and a terminal device 40 operated by the memo user.

[0038] The voice memo storage system of this embodiment is a system that records the voice of the memo writer or a conversation between the memo writer and other people in the vicinity as audio information, saves it as voice memo information that is part of the recorded audio information, and allows the voice memo information to be checked later on the terminal device 40.

[0039] The microphone 60 is a sound collecting device such as a lapel microphone that is attached to the memo recorder. The wearable sensor 70 is a biometric information detecting unit that detects the biometric information of the memo recorder, who is the user. Specifically, the wearable sensor 70 is attached to the arm of the memo recorder, for example, and acquires various biometric information such as the memo recorder's blood pressure, pulse, brainwaves, and body temperature. The microphone 60 and the wearable sensor 70 are connected to the mobile terminal device 10 by a short-range wireless communication line such as Bluetooth (registered trademark). Therefore, the voice information collected by the microphone 60 and the biometric information acquired by the wearable sensor 70 are transmitted to the mobile terminal device 10. Note that the voice of the memo recorder may be directly acquired as voice information by a microphone provided in the mobile terminal device 10 without using the microphone 60. The microphone 60 may also be a microphone part of a headset or the like that is attached to the memo recorder.

[0040] The mobile terminal device 10 records the voice of the memo writer or a conversation between the memo writer and other people nearby as voice information using the microphone 60, and saves a part of the recorded voice information as voice memo information.

[0041] The mobile terminal device 10 is connected to the Internet 30 via a wireless LAN terminal 50, and is connected to a storage server 20 via the Internet 30. The mobile terminal device 10 stores the acquired voice memo information in the storage server 20 via the Internet 30. The storage server 20 provides a voice memo storage service, and stores the voice memo information transmitted from the mobile terminal device 10.

[0042] A memo user who wishes to view the stored voice memo information can view the stored voice memo information by operating the terminal device 40 to access the storage server 20. Note that the memo recorder and the memo user may be the same person or different people.

[0043] A typical usage pattern of the voice memo storage system of this embodiment is assumed to be a situation in which a person's hands are full while performing a certain task and he or she is unable to write a memo by hand or input a memo into a terminal device.

[0044] A specific example of the work content is, for example, the case where a record of children at a nursery school or kindergarten is saved as a voice memo. In this case, it is used to record the educational content of the children, record things noticed by observing the children's reactions, record conversations with the children, etc.

[0045] Another example of a specific task is the storage of student records as voice memos at an elementary school, which are used to record the contents of the education given to students, record observations of students' reactions, record conversations with students, etc.

[0046] Another specific example of the work content is the recording of a patient or a care recipient at the nursing or care site as a voice memo, which is used to record the nursing or care provided to the patient or the care recipient, record observations of the patient's or the care recipient's reactions, record conversations with the patient or the care recipient, etc.

[0047] Furthermore, another specific example of the type of work that can be done using the notebook is to record things that you notice, ideas that come to mind, or points that you want to check later during presentations, explanations, business negotiations, interviews, visits to exhibitions, work at construction sites or factories, etc.

[0048] In all of the above-mentioned tasks, since the worker is dealing with people or objects, it is often impossible to divert attention from the person or objects, or both hands are occupied by equipment or the like and the worker is unable to perform any operations.

[0049] It is also possible to use a voice recorder or the like to record all audio that is spoken during work in order to keep a record of the audio information, or to use speech-to-text conversion technology to convert the recorded audio information into text information.

[0050] However, such a method of recording all voice information during work requires the user to extract only the necessary information from the vast amount of voice information, which can be a time-consuming process.

[0051] For this reason, there are systems that allow the user to specify the start and end positions of a period of acquired voice data that the user wants to save by operating the system or by speaking specific words, and generate voice data for the specified period as a voice memo. However, when using such a system to save voice memos as audio recordings during various tasks, the user must consciously speak or operate the system to save the voice memo, which may interfere with the task at hand.

[0052] Therefore, in the voice memo storage system of this embodiment, by performing the control described below, it is possible to save a portion of the acquired voice information as an audio recording without specifying the start and end positions of the period that you want to keep as an audio recording.

[0053] Next, the hardware configuration of the mobile terminal device 10 in the voice memo storage system of this embodiment is shown in FIG.

[0054] 2, the mobile terminal device 10 has a CPU 11, a memory 12, a storage device 13 such as a flash memory, a wireless communication interface (abbreviated as IF) 14 that transmits and receives data via a short-distance wireless communication line, a wireless LAN communication line, and a mobile phone communication line, a user interface (abbreviated as UI) device 15 such as a touch panel or a liquid crystal display, a GPS (Global Positioning System) sensor 16, a camera 17, an acceleration sensor 18, and a gyro sensor 19 which is an angular velocity sensor. These components are connected to each other via a control bus.

[0055] The CPU 11 is a processor that executes a predetermined process based on a control program stored in the memory 12 or the storage device 13 to control the operation of the mobile terminal device 10. In the present embodiment, the CPU 11 is described as reading and executing the control program stored in the memory 12 or the storage device 13, but is not limited to this. The control program may be provided in a form recorded on a computer-readable recording medium. For example, the program may be provided in a form recorded on an optical disc such as a CD (Compact Disc)-ROM or a DVD (Digital Versatile Disc)-ROM, or in a semiconductor memory such as a USB (Universal Serial Bus) memory or a memory card. The control program may also be obtained from an external device via a communication line connected to the communication interface 14.

[0056] FIG. 3 is a block diagram showing the functional configuration of the mobile terminal device 10 realized by executing the above control program.

[0057] As shown in FIG. 3, the mobile terminal device 10 of this embodiment includes a voice input unit 31, a voice-to-text conversion unit 32, a voice information temporary storage unit 33, a memo information selection unit 34, a memo information storage unit 35, a memo information display unit 36, a data transmission / reception unit 37, a trigger detection unit 38, an operation reception unit 41, an emotion determination unit 42, a position detection unit 43, a photographing unit 44, and a motion detection unit 45.

[0058] The voice input unit 31 inputs voice information including the voice of the person taking the memo via the microphone 60. The voice text conversion unit 32 analyzes the voice information input by the voice input unit 31 and converts it into text data. The voice information temporary storage unit 33 sequentially acquires external voice as voice information by the voice input unit 31 and temporarily stores it. More specifically, the voice information temporary storage unit 33 temporarily stores text data, which is text information obtained by converting the voice information acquired by the voice input unit 31 in the voice text conversion unit 32.

[0059] The trigger detection unit 38 detects the timing for selecting the voice information to be saved as voice memo information from the voice information in text format stored in the voice information temporary storage unit 33. Specifically, the trigger detection unit 38 detects a trigger indicating the timing for cutting out the temporarily stored voice information as voice memo information based on whether a preset specific condition is satisfied. Then, the trigger detection unit 38 outputs a save instruction to the memo information selection unit 34 to instruct saving of the voice memo information at the timing when the preset specific condition is satisfied. Details of the specific condition based on which the trigger detection unit 38 detects the trigger indicating the timing for cutting out the voice memo information from the voice information will be described later.

[0060] When the trigger detection unit 38 detects that a predetermined specific condition has been satisfied, the memo information selection unit 34 selects audio information including audio information acquired before the time when it was detected that the specific condition was satisfied, and stores the selected audio information in the memo information storage unit 35 as audio memo information, which is an audio recording.

[0061] When the trigger detection unit 38 detects that a preset specific condition has been satisfied, the memo information selection unit 34 stores voice information including voice information from before the time when it was detected that the specific condition was satisfied in the memo information storage unit 35 as voice memo information, going back in time to the part that has linguistic connections.

[0062] In addition, when the trigger detection unit 38 detects that a specific condition set in advance has been satisfied, the memo information selection unit 34 may select only the voice information acquired before the time when it was detected that the specific condition was satisfied, and store the selected voice information in the memo information storage unit 35 as voice memo information, which is a voice recording.

[0063] The memo information storage unit 35 stores the voice memo information selected by the memo information selection unit 34 together with attribute information such as the recording time when the voice information was recorded and the location information of the place where the voice information was obtained.

[0064] The memo information display unit 36 ​​displays to the user the voice memo information stored in the memo information storage unit 35. The data transmission / reception unit 37 transmits the voice memo information stored in the memo information storage unit 35 to the storage server 20.

[0065] In this embodiment, the memo information storage unit 35 is described as being provided in the mobile terminal device 10, but the memo information storage unit 35 may be provided in the storage server 20 instead of in the mobile terminal device 10. In this case, the voice memo information is stored in the storage server 20, so that both the mobile terminal device 10 and the terminal device 40 access the storage server 20 to view the stored voice memo information.

[0066] In the voice memo saving system of this embodiment, the trigger detection unit 38 determines the timing to save the voice memo information, and the memo information selection unit 34 retroactively selects temporarily stored voice information and saves it as voice memo information. Therefore, according to this embodiment, the voice memo information is saved without the user having to take the time to specify the start and end positions of the period of the voice information to be saved as voice memo information.

[0067] The operation reception unit 41 receives operations by a user. In addition, in the case where an operation button is provided on the microphone 60 worn by the memo recorder, the operation reception unit 41 receives that the operation button provided on the microphone 60 has been operated.

[0068] The emotion determination unit 42 determines the emotion of the memo writer based on the biological information acquired by the wearable sensor 70. Specifically, the emotion determination unit 42 determines whether the memo writer is surprised, sad, happy, neutral, or the like.

[0069] The position detection unit 43 detects the current position of the mobile terminal device 10, which is the system itself, based on position information acquired by a GPS positioning system if outdoors, and based on the radio wave intensity of a short-distance wireless line such as Bluetooth (registered trademark) if indoors. The photographing unit 44 photographs an image using a camera in the mobile terminal device 10.

[0070] The motion detection unit 45 detects the physical motion of the memo recorder holding the mobile terminal device 10, based on the detection signals from the acceleration sensor 18 and the gyro sensor 19. For example, the motion detection unit 45 detects the motion state of the memo recorder, such as whether the memo recorder is running, jumping, clapping, or the like.

[0071] [Operation of trigger detection unit 38] Next, a specific description will be given of what specific condition is satisfied when the trigger detection section 38 determines the timing for saving the voice memo information.

[0072] (1) User instructions When the operation reception unit 41 receives that the user has operated the operation button provided on the microphone 60, or when the acceleration sensor 18 and the gyro sensor 19 detect that the user has tapped the mobile terminal device 10, the trigger detection unit 38 determines that it is time to save the voice memo information. Note that the user's operation in this case is an operation method that does not interfere with the ongoing work.

[0073] In this case, when a user operation is detected by the trigger detection unit 38, the memo information selection unit 34 stores voice information including voice information acquired before the time when the user operation was detected in the memo information storage unit 35 as voice memo information.

[0074] (2) Detection of specific voice words When a preset word or group of words is detected in the text information converted by the voice-to-text conversion unit 32, the trigger detection unit 38 determines that it is time to save the voice memo information.

[0075] In this case, when the trigger detection unit 38 detects a specific voice word, which is a predetermined word or group of words, in the text information, the memo information selection unit 34 stores voice information including voice information acquired before the time when the specific voice word was detected in the memo information storage unit 35 as voice memo information.

[0076] For example, when voice memo information is saved in a nursery school or kindergarten, the words "You did a good job" are set as the specific voice words. Also, when voice memo information is saved in a device inspection site, the words "Work completed" are set as the specific voice words.

[0077] Such detection of specific voice words can be performed by comparing the text information stored in the voice information temporary storage unit 33 with the set specific voice words.

[0078] (3) Detection of specific actions When the motion detection unit 45 detects a preset motion based on a detection signal from the gyro sensor 19 or the acceleration sensor 18, the trigger detection unit 38 determines that it is time to save the voice memo information.

[0079] In this case, when a predetermined action is detected by the trigger detection unit 38, the memo information selection unit 34 stores voice information including voice information acquired before the time the action was detected in the memo information storage unit 35 as voice memo information.

[0080] For example, when the memo recorder jumps up and down, claps his / her hands, or suddenly starts running, the trigger detection unit 38 determines that it is time to save the voice memo information. By doing this, in a nursery school or kindergarten, the voice memo information can be saved simply by the memo recorder, a nursery teacher or kindergarten teacher, clapping.

[0081] (4) Detection of photo taking When the photographing section 44 photographs an image, the trigger detection section 38 determines that it is time to save the voice memo information.

[0082] In this case, when the trigger detection unit 38 detects that an image has been captured by the photographing unit 44, the memo information selection unit 34 stores audio information including audio information acquired before the time the image was captured by the photographing unit 44 in the memo information storage unit 35 as voice memo information.

[0083] In this way, when one wishes to record a conversation while taking a photograph, the voice at that time can be saved as voice memo information simply by taking a photograph.

[0084] (5) Detect movement to a specific location When the trigger detection section 38 determines that the position information detected by the position detection section 43 has moved to a specific location, the trigger detection section 38 determines that it is time to save the voice memo information.

[0085] In this case, when the trigger detection unit 38 detects that the mobile terminal device 10 has moved within a predetermined range from a preset position, the memo information selection unit 34 stores, as voice memo information, voice information including voice information acquired before the time when it was detected that the mobile terminal device 10 has moved within the predetermined range from the preset position, in the memo information storage unit 35. Then, the memo information selection unit 34 also stores, as voice memo information, voice information acquired until the mobile terminal device 10 moves out of the predetermined range from the preset position.

[0086] For example, when the trigger detection unit 38 detects that the mobile terminal device 10 has moved to a meeting room, or to a place where a certain machine is operated, the trigger detection unit 38 determines that it is time to save the voice memo information. In this way, it becomes possible to save the voice in a certain place as voice memo information.

[0087] (6) Detecting changes in emotions The trigger detection unit 38 determines that it is time to save the voice memo information when the emotion determination unit 42 detects a change in the emotion of the user (the memo recorder), that is, when it detects that the user's biometric information has changed to a preset state.

[0088] In this case, when the trigger detection unit 38 detects that the user's biometric information has changed to a preset state, the memo information selection unit 34 stores voice information including voice information acquired before the time when it was detected that the user's biometric information had changed to the preset state in the memo information storage unit 35 as voice memo information.

[0089] By configuring in this way, for example, when a nursery teacher at a nursery school is surprised at the progress of a child, the voice recorded at that time can be saved as voice memo information.

[0090] [Operation of memo information selection unit 34] The memo information selection unit 34 generates voice memo information from the temporarily stored voice information at the timing of receiving a save instruction from the trigger detection unit 38. Specifically, the memo information selection unit 34 traces back the information stored in the voice information temporary storage unit 33 and saves voice information from the past to the present or from the past to the future as voice memo information. Note that the voice information selection unit 34 saves voice information from a point in the past to the present as voice memo information, or from a point in the past to a point in the future as voice memo information, is determined depending on the content of the voice information to be saved as voice memo information.

[0091] (1) Selection by time When the trigger detection unit 38 detects that a specific condition set in advance has been satisfied, the memo information selection unit 34 stores, as voice memo information, in the memo information storage unit 35, voice information including voice information acquired from a time that is a preset time before the current time to the current time. For example, when a save instruction is given by the trigger detection unit 38, the memo information selection unit 34 selects and stores, as voice memo information, voice information for 30 seconds before the save instruction and 30 seconds after the save instruction. In other words, when a save instruction is given by the trigger detection unit 38, the memo information selection unit 34 selects and stores, as voice memo information, one minute of voice 30 seconds before the save instruction.

[0092] (2) Selection by location When the memo information selection unit 34 detects that a predetermined specific condition is satisfied, the memo information selection unit 34 stores, in the memo information storage unit 35 as voice memo information, voice information including voice information acquired from the time when the position of the mobile terminal device 10 enters a range within a predetermined distance from the current position to the current time.

[0093] For example, a case will be described where the location range is set to a radius of 10 meters. In this case, the memo information selection unit 34 stores, as voice memo information, the voice information from the time when the mobile terminal device 10 entered within a radius of 10 meters in the past based on the location at the time when the save instruction was received from the trigger detection unit 38, until the mobile terminal device 10 moves out of that range.

[0094] (3) Selection by Semantic Analysis When the memo information selection unit 34 detects that a predetermined specific condition is satisfied, the memo information selection unit 34 stores voice information including voice information from the point where a predetermined word is detected to the current time in the memo information storage unit 35 as voice memo information.

[0095] Specifically, the memo information selection unit 34 searches back through phrases of text information stored before the time when the save instruction was received from the trigger detection unit 38, and selects, as voice memo information, voice information from the detection point of a preset proper noun such as a specific person's name or event name to the present. For example, when a proper noun such as "XX-chan" is set, if voice information such as "XX-chan did a good job drawing" exists in the stored past voice information, the memo information selection unit 34 saves the voice information from that voice information to the present as voice memo information.

[0096] (4) Selection based on the results of analysis of images captured in photographs When photography is performed by the photographing unit 44, the memo information selection unit 34 identifies an object included in the image photographed by the photographing unit 44, and stores audio information, which was obtained before the time photographing was performed by the photographing unit 44 and includes audio information of a phrase including the name of the identified object, in the memo information storage unit 35 as audio memo information.

[0097] Specifically, when voice memo information is saved in response to a photograph being taken, the memo information selection unit 34 identifies people, scenery, events, etc. that appear in the photograph, and saves voice information of phrases that include the names of the identified people, events, or scenery as voice memo information.

[0098] The above-mentioned selection methods (1) to (4) may be combined by a user by setting priorities in advance. For example, the priority of selection by location may be set higher than the priority of selection by semantic analysis, and the priority of selection by time may be set higher than the priority of selection by location. When a save instruction is received from the trigger detection unit 38, the memo information selection unit 34 may select and save the voice memo information based on the selection method with the highest priority among the selection methods applicable at that time.

[0099] Next, the operation of the voice memo storage system of this embodiment will be described in detail with reference to the drawings.

[0100] First, the overall operation of the voice memo storage system of this embodiment will be described with reference to the flowchart of FIG.

[0101] First, in step S101, voice information is input from the microphone 60 worn by the memo recorder by the voice input unit 31. Then, in step S102, the input voice information is converted into text information by the voice text conversion unit 32. Then, in step S103, the converted text information is stored in the voice information temporary storage unit 33.

[0102] Next, in step S104, the trigger detection unit 38 detects the presence or absence of a trigger for determining the timing for saving the stored voice information as voice memo information.

[0103] Then, in step S105, the memo information selection unit 34 determines whether or not a trigger has been detected by the trigger detection unit 38, that is, whether or not a save instruction has been received from the trigger detection unit 38.

[0104] If it is determined in step S105 that the trigger has not been detected, the processes in steps S101 to S104 are repeated.

[0105] Then, if it is determined in step S105 that the trigger has been detected, in step S106, the memo information selection unit 34 selects voice information to be saved from the voice information stored in the voice information temporary storage unit 33. In step S107, the selected voice information is saved in the memo information saving unit 35 by the memo information selection unit 34 as voice memo information.

[0106] Next, details of the trigger detection process in step S104 shown in the flowchart of FIG. 4 will be described with reference to the flowchart of FIG.

[0107] In the trigger detection unit 38, in step S201, it is determined whether or not a user operation has been performed, and in step S202, it is determined whether or not a specific voice word has been detected in the temporarily stored voice information. In addition, in the trigger detection unit 38, in step S203, it is determined whether or not a specific action of the memo recorder has been detected by the action detection unit 45. Furthermore, in the trigger detection unit 38, in step S204, it is determined whether or not a photograph has been taken by the photographing unit 44, and in step S205, it is determined whether or not the mobile terminal device 10 has entered a specific area. Finally, in step S206, the trigger detection unit 38 determines whether or not a specific emotional change has occurred in the memo recorder based on the emotion of the memo recorder determined by the emotion determination unit 42.

[0108] Then, if an affirmative determination is made in any of steps S201 to S206, the trigger detection unit 38 determines in step S208 that a trigger has been generated.

[0109] Furthermore, if negative determinations are made in all of steps S201 to S206, the trigger detection unit 38 determines in step S207 that no trigger has been detected.

[0110] Next, a specific example of the memo information selection process in the memo information selection unit 34 will be described with reference to FIGS.

[0111] In the memo information selection process example 1 shown in FIG. 6, a case is shown in which, using a button operation by a user as a trigger, voice information selected based on the location at the time the trigger was detected is saved as voice memo information.

[0112] 6, it is assumed that the memo recorder operates the operation button of the microphone 60 at time T1. In this case, the voice memo information is stored in the mobile terminal device 10 according to the following procedure.

[0113] (1) First, the memo recorder operates the operation button of the microphone 60 at time T1. (2) Then, the operation acceptance unit 41 accepts the operation of this operation button, and the trigger detection unit detects the operation of this operation button as a trigger, and transmits a save instruction to the memo information selection unit . (3) Upon receiving the save instruction, the memo information selection unit 34 acquires current location information from the location detection unit 43. In the memo information selection process example 1 shown in Fig. 6, it is assumed that the acquired location information indicates the playground. (4) Since the current location information obtained from the location detection unit 43 indicates the playground, the memo information selection unit 34 selects the voice information ranging from the time the memo recorder enters the playground to the time he or she leaves the playground as voice memo information and stores it in the memo information storage unit 35.

[0114] By storing the voice memo information in the above-mentioned procedure, text information such as "You managed to jump the rope 100 times, well done" is stored as the voice memo information.

[0115] Next, a memo information selection process example 2 shown in Fig. 7 will be described. In the memo information selection process example 2 shown in Fig. 7, the detection of a preset specific voice word is used as a trigger to select and save voice information from the detection point of a word determined to be a person's name by semantic analysis to the present as voice memo information.

[0116] 7, it is assumed that the word "Nice job" is registered as a specific voice word in advance, and that this specific voice word is detected in the text information at time T2. In this case, the mobile terminal device 10 stores the voice memo information in the following procedure.

[0117] (1) The trigger detection unit 38 constantly checks whether the text information successively stored in the voice information temporary storage unit 33 contains a specific voice word that has been set in advance. (2) Here, it is assumed that the trigger detection unit 38 detects the specific voice word “good job” at time T2. Then, the trigger detection unit 38 detects the specific voice word as a trigger and transmits a save instruction to the memo information selection unit 34. (3) Then, the memo information selection unit 34, which has received the save instruction, searches back in time through past text information stored in the voice information temporary storage unit 33 to search for words that are determined to be a person's name. Here, it is assumed that the word "XX-chan" has been found as a word indicating a person's name. (4) Then, the memo information selection unit 34 selects the voice information from the point where the word “XX-chan” was detected to the present as voice memo information, and stores it in the memo information storage unit 35.

[0118] By saving the voice memo information in the above-mentioned procedure, text information such as "Hey XX-chan, you jumped the rope 100 times, well done" is saved as the voice memo information.

[0119] Next, a description will be given of a memo information selection process example 3 shown in Fig. 8. In the memo information selection process example 3 shown in Fig. 8, a case is illustrated in which, triggered by the taking of a photograph, voice information related to a person included in the photographed image is selected and saved as voice memo information.

[0120] 8, it is assumed that a photograph is taken by the photographing unit 44 of the mobile terminal device 10 at time T3. In this case, the mobile terminal device 10 stores the voice memo information in the following procedure.

[0121] (1) The trigger detection unit 38 constantly monitors whether an application program for taking photographs is started in the mobile terminal device 10 and whether a photograph is taken by the photographing unit 44. (2) Here, it is assumed that a photograph is taken by the photographing unit 44 at time T3. Then, the trigger detection unit 38 detects the photograph being taken by the photographing unit 44 as a trigger, and transmits a save instruction to the memo information selection unit 34. (3) The memo information selection unit 34, which has received the save instruction, uses the face recognition function to identify a person in the photo image captured by the photographing unit 44. Here, it is assumed that the name of the identified person is "XX-chan." (4) Then, the memo information selection unit 34 searches back in time among past text information stored in the voice information temporary storage unit 33 to search for the word "XX-chan." Then, the memo information selection unit 34 selects voice information of a phrase including the word "XX-chan" that was acquired before the time T3 when the photograph was taken as voice memo information and stores it in the memo information storage unit 35.

[0122] By saving the voice memo information in the above-mentioned procedure, text information such as "Hey XX-chan, you jumped the rope 100 times, well done" is saved as the voice memo information.

[0123] The voice memo information stored in the memo information storage unit 35 by the above-described process is stored in the storage server 20 by the data transmission / reception unit 37. Therefore, the memo user can view the stored voice memo information by operating the terminal device 40 to access the storage server 20.

[0124] An example of the voice memo information display screen thus displayed on the terminal device 40 is shown in Fig. 9. In the example of the voice memo information display screen shown in Fig. 9, it can be seen that a part of the voice information recorded by the memo recorder using the mobile terminal device 10 is saved as voice memo information. Note that Fig. 9 shows a case where a plurality of pieces of voice memo information are displayed in order of recording date and time.

[0125] Although FIG. 9 shows a case where voice memo information is viewed on the terminal device 40, a similar display is also performed when voice memo information saved in the memo information display section 36 of the mobile terminal device 10 is viewed.

[0126] In each of the above embodiments, the term "processor" refers to a processor in a broad sense, including general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0127] In addition, the operations of the processor in each of the above embodiments may be performed not only by one processor, but also by multiple processors located at physically separate locations working together. Furthermore, the order of the operations of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate.

[0128] In this embodiment, the term "system" includes both a system made up of multiple devices and a system made up of a single device.

[0129] [Variations] In the above embodiment, the present invention is described as being applied to a mobile terminal device 10 such as a smartphone, but the present disclosure is not limited to this, and the present disclosure can also be similarly applied to information processing devices or information processing systems that have a function of recording audio other than a mobile terminal device.

[0130] [Note] Preferred embodiments of the present disclosure will be described below.

[0131] (((1))) It has a processor and memory, The processor, external voice is sequentially acquired as voice information and temporarily stored in said memory; when it is detected that a preset specific condition is satisfied, storing in the memory as a voice record voice information including voice information acquired before the time when it is detected that the preset specific condition is satisfied; Information processing system.

[0132] (((2))) The processor converts the acquired voice information into text information and temporarily stores the text information in the memory; When it is detected that a predetermined specific condition has been satisfied, audio information including audio information from before the time when it was detected that the specific condition was satisfied is stored in the memory as an audio recording, going back in time to the part that has a linguistic connection (((1))).

[0133] (((3))) and storing, in the memory, speech information including speech information from a point where the predetermined word was detected to a current time as a speech record, when the processor detects that a predetermined specific condition is satisfied. The information processing system according to (((2))).

[0134] (((4))) A position detection unit that detects the position of the system itself is further provided, When it is detected that a predetermined specific condition is satisfied, the processor records voice information including voice information acquired from the time when the position of the system itself entered within a range of a predetermined distance from the current position to the current time, and stores the voice information in the memory. The information processing system according to (((2))).

[0135] (((5))) Further comprising an image capturing unit for capturing an image, When the image is captured by the image capturing unit, the processor identifies an object included in the image captured by the image capturing unit, and records and stores in the memory audio information that was acquired before the image was captured by the image capturing unit and includes audio information of a phrase that includes the name of the identified object. The information processing system according to (((2))).

[0136] (((6))) When it is detected that a predetermined specific condition is satisfied, the processor records voice information including voice information acquired from a time that is a predetermined time before the current time to the current time, and stores the voice information in the memory. The information processing system according to (((2))).

[0137] (((7))) When a user operation is detected, the processor stores, in the memory, voice information including voice information acquired before the time when the user operation was detected as a voice record. 1. An information processing system according to any one of ((1))) to (((6))).

[0138] (((8))) The processor converts the acquired voice information into text information and temporarily stores the text information in the memory; when a predetermined word or group of words is detected in the converted text information, storing in said memory as a voice record voice information including voice information acquired before the time when the word or group of words was detected; 1. An information processing system according to any one of ((1))) to (((6))).

[0139] (((9))) Further comprising an angular velocity sensor or an acceleration sensor, When a preset motion is detected by the angular velocity sensor or the acceleration sensor, the processor stores, in the memory as a voice record, voice information including voice information acquired before the time when the preset motion was detected. 1. An information processing system according to any one of ((1))) to (((6))).

[0140] (((10))) Further comprising an image capturing unit for capturing an image, When the image capturing unit captures an image, the processor stores, in the memory as an audio record, audio information including audio information acquired before the image capturing unit captures an image. 1. An information processing system according to any one of ((1))) to (((6))).

[0141] (((11))) A position detection unit is further provided to detect the position of the system itself, When it is detected that the system has moved within a predetermined range from a preset position, the processor stores, in the memory as a voice record, voice information including voice information acquired before the time when it was detected that the system has moved within the predetermined range from the preset position. 1. An information processing system according to any one of ((1))) to (((6))).

[0142] (((12))) The processor also stores in the memory as a voice record voice information acquired by the system from a preset position until the system moves out of a predetermined range. The information processing system according to (((11))).

[0143] (((13))) A biometric information detection unit that detects biometric information of a user is further provided, When the biometric information of the user detected by the biometric information detection unit changes to a preset state, the processor stores, in the memory as a voice record, voice information including voice information acquired before the time when it was detected that the biometric information of the user changed to the preset state. 1. An information processing system according to any one of ((1))) to (((6))).

[0144] (((14))) A step of sequentially acquiring and temporarily storing external voice as voice information; When it is detected that a preset specific condition is satisfied, storing, as a voice record, voice information including voice information acquired before the time when it is detected that the preset specific condition is satisfied; A program for causing a computer to execute the following.

[0145] The effects of the above configuration are described below.

[0146] According to the information processing system of (((1))), it is possible to save a portion of the acquired voice information as a voice recording without specifying the start and end positions of the period that is to be kept as a voice recording.

[0147] According to the information processing system (((2))), it is possible to save a linguistically meaningful portion of the acquired audio information as an audio recording without specifying the start and end positions of the period that you want to keep as an audio recording.

[0148] The information processing system of (((3))) allows a series of sounds relating to predefined words to be recorded and stored.

[0149] According to the information processing system of (((4))), audio information linked to imaging by the imaging unit can be recorded and stored.

[0150] According to the information processing system of (((5))), audio information linked to imaging by the imaging unit can be recorded and stored.

[0151] According to the information processing system of (((6))), audio information for a preset period of time can be stored as an audio record.

[0152] According to the information processing system of (((7))), a user can save voice information as a voice recording by performing a single operation.

[0153] According to the information processing system of (((8))), it is possible to save necessary voice information as a voice recording simply by setting in advance a word or a group of words according to a situation for which a voice recording is desired to be kept.

[0154] According to the information processing system of (((9))), voice information can be saved as a voice recording without requiring user operation.

[0155] According to the information processing system of (((10))), audio information when imaging is performed by the imaging unit can be saved as an audio recording.

[0156] According to the information processing system of (((11))), voice information generated when moving to a preset location can be saved as a voice record.

[0157] The information processing system of (((12))) can store voice information while located in a preset location as a voice record.

[0158] According to the information processing system of (((13))), voice information when a user's biometric information changes can be stored as a voice record.

[0159] The program (((14))) allows a portion of the acquired audio information to be saved as an audio recording without specifying the start and end positions of the period that is to be retained as an audio recording. [Explanation of symbols]

[0160] 10. Portable terminal device 11 CPU 12. Memory 13 Storage device 14 Unattached Communication Interface 15 User Interface Device 16 GPS Sensor 17 Camera 18 Acceleration Sensor 19 Gyro sensor 20 Storage Server 30 Internet 31 Audio input section 32 Speech to text conversion unit 33 Temporary voice information storage unit 34 Memo information selection section 35 Memo information storage section 36 Memo information display section 37 Data transmission and reception unit 38 Trigger detection section 40 Terminal Equipment 41 Operation reception section 42 Emotion Judgment Department 43 Position detection unit 44 Photography Department 45 Motion detection unit 50 Wireless LAN Terminal 60. Mike 70 Wearable Sensors

Claims

1. It has a processor and memory, The processor, external voice is sequentially acquired as voice information and temporarily stored in said memory; when it is detected that a preset specific condition is satisfied, storing in the memory as a voice record voice information including voice information acquired before the time when it is detected that the preset specific condition is satisfied; Information processing system.

2. The processor converts the acquired voice information into text information and temporarily stores the text information in the memory; The information processing system of claim 1, wherein when it is detected that a predetermined specific condition has been satisfied, audio information including audio information prior to the time when it was detected that the specific condition was satisfied is stored in the memory as an audio recording, going back in time to the part that has linguistic connections.

3. and storing, in the memory, speech information including speech information from a point where the predetermined word was detected to a current time as a speech record, when the processor detects that a predetermined specific condition is satisfied.

3. The information processing system according to claim 2.

4. A position detection unit is further provided to detect the position of the system itself, When it is detected that a predetermined specific condition is satisfied, the processor records voice information including voice information acquired from the time when the position of the system itself entered within a range of a predetermined distance from the current position to the current time, and stores the voice information in the memory.

3. The information processing system according to claim 2.

5. Further comprising an image capturing unit for capturing an image, When the image is captured by the image capturing unit, the processor identifies an object included in the image captured by the image capturing unit, and records and stores in the memory audio information that was acquired before the image was captured by the image capturing unit and includes audio information of a phrase that includes the name of the identified object.

3. The information processing system according to claim 2.

6. When it is detected that a predetermined specific condition is satisfied, the processor records voice information including voice information acquired from a time that is a predetermined time before the current time to the current time, and stores the voice information in the memory.

3. The information processing system according to claim 2.

7. When a user operation is detected, the processor stores, in the memory, voice information including voice information acquired before the time when the user operation was detected as a voice record.

2. The information processing system according to claim 1.

8. The processor converts the acquired voice information into text information and temporarily stores the text information in the memory; when a predetermined word or group of words is detected in the converted text information, storing in said memory as a voice record voice information including voice information acquired before the time when the word or group of words was detected; 2. The information processing system according to claim 1.

9. Further comprising an angular velocity sensor or an acceleration sensor, When a preset motion is detected by the angular velocity sensor or the acceleration sensor, the processor stores, in the memory as a voice record, voice information including voice information acquired before the time when the preset motion was detected.

2. The information processing system according to claim 1.

10. Further comprising an image capturing unit for capturing an image, When the image capturing unit captures an image, the processor stores, in the memory as an audio record, audio information including audio information acquired before the image capturing unit captures an image.

2. The information processing system according to claim 1.

11. A position detection unit is further provided to detect the position of the system itself, When it is detected that the system has moved within a predetermined range from a preset position, the processor stores, in the memory as a voice record, voice information including voice information acquired before the time when it was detected that the system has moved within the predetermined range from the preset position.

2. The information processing system according to claim 1.

12. The processor also stores in the memory as a voice record voice information acquired by the system from a preset position until the system moves out of a predetermined range.

12. The information processing system according to claim 11.

13. A biometric information detection unit that detects biometric information of a user is further provided, When the biometric information of the user detected by the biometric information detection unit changes to a preset state, the processor stores, in the memory as a voice record, voice information including voice information acquired before the time when it was detected that the biometric information of the user changed to the preset state.

2. The information processing system according to claim 1.

14. A step of sequentially acquiring and temporarily storing external voice as voice information; When it is detected that a preset specific condition is satisfied, storing, as a voice record, voice information including voice information acquired before the time when it is detected that the preset specific condition is satisfied; A program for causing a computer to execute the following.

Images