Terminals, systems, methods, and programs

Abstract

One of the objectives of the present invention is to enable a caregiver to monitor a user terminal held by the person being monitored in a simpler manner. [Solution] A terminal for a person being monitored, carried by the person being monitored, which, upon receiving an operation on the terminal, starts recording sound and acquires the location information of the terminal being monitored, stops recording after a predetermined period of time has elapsed, and transmits the sound data recorded over the predetermined period and the location information of the terminal being monitored to a server.

Description

Technical Field

[0001] The present invention relates to a terminal, a system, a method, and a program.

Background Art

[0002] In recent years, various systems using the location information of user terminals have been proposed. Such systems include, as in Patent Document 1, a system that detects an emergency of a user terminal using an application and automatically transmits information such as the location information and voice of the user terminal.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One object of the present invention is to enable a user terminal possessed by a person under guardianship to be more easily monitored by a guardian.

Means for Solving the Problems

[0005] 〔Configuration 1〕 In order to solve the above-described problems, one aspect of the present disclosure provides a terminal for a person under guardianship carried by a person under guardianship, which, when receiving an operation on the terminal for the person under guardianship, starts recording sound and acquires the location information of the terminal for the person under guardianship, stops the recording after a predetermined period has elapsed, and transmits the sound data recorded over the predetermined period and the location information of the terminal for the person under guardianship to a server.

[0006] 〔Configuration 2〕 Furthermore, in one aspect of this disclosure, the monitored terminal may be the monitored terminal described in [Configuration 1], which is configured to, upon receiving an operation to the monitored terminal, start acquiring location information of the monitored terminal for a predetermined period of time, and after the predetermined period has elapsed, stop acquiring location information of the monitored terminal and transmit the location information acquired during the predetermined period to the server.

[0007] [Structure 3] Furthermore, in one aspect of this disclosure, the monitored terminal may be the monitored terminal described in [Configuration 1] or [Configuration 2], wherein the operation of the monitored terminal is a single press operation on the notification device of the monitored terminal.

[0008] [Structure 4] Furthermore, in one aspect of this disclosure, the location information of the monitored person's terminal acquired during the predetermined period is acquired at a shorter interval than the location information acquired outside the predetermined period, and the monitored person's terminal may be one of the monitored person's terminals described in any of [Configuration 1] to [Configuration 3].

[0009] [Composition 5] Furthermore, one aspect of the present disclosure may be a monitored terminal according to any one of [Configuration 1] to [Configuration 4], which is further configured to cause the display device of the monitored terminal to emit light over the predetermined period of time.

[0010] [Composition 6] Furthermore, in one aspect of this disclosure, the predetermined period may be set in a monitor terminal carried by the monitor of the person being monitored, and the monitor terminal may be one of the monitor terminals described in any of [Configuration 1] to [Configuration 5].

[0011] [Composition 7] To solve the above-mentioned problems, one aspect of the present disclosure provides a system comprising a terminal for a person being monitored, carried by the person being monitored, and a server, wherein the terminal for the person being monitored is configured to: start recording sound and acquire location information of the terminal when an operation is received on the terminal; stop recording after a predetermined period of time has elapsed; and transmit the sound data recorded over the predetermined period and the location information of the terminal to the server.

[0012] [Structure 8] Furthermore, one aspect of this disclosure may be the system described in [Configuration 7], in which the server is configured to notify a monitoring terminal carried by the monitor of a link to the received sound data and location information of the monitoring terminal.

[0013] [Composition 9] Furthermore, one aspect of this disclosure is a system described in [Configuration 7] in which, upon receiving the notification, the monitoring terminal displays a link to the sound data and address information corresponding to the location information of the monitored person's terminal on the display unit of the monitoring terminal.

[0014] [Composition 10] To solve the above-mentioned problems, one aspect of the present disclosure provides a method executed by the processor of a terminal for persons being monitored, which, upon receiving an operation on the terminal for persons being monitored, includes: starting sound recording and acquiring location information of the terminal for persons being monitored; stopping the recording after a predetermined period of time has elapsed; and transmitting the sound data recorded over the predetermined period and the location information of the terminal for persons being monitored to a server.

[0015] [Composition 11] To solve the above-mentioned problems, one aspect of the present disclosure provides a program which, when executed by a terminal processor for a person being monitored carried by the person being monitored, receives an operation on the terminal for a person being monitored carried by the person being monitored, starts recording sound, acquires location information of the terminal for a person being monitored, stops the recording after a predetermined period of time has elapsed, and transmits the sound data recorded over the predetermined period and the location information of the terminal for a person being monitored to a server.

[0016] [Composition 12] To solve the above-mentioned problems, one aspect of the present disclosure provides a caregiver terminal carried by a caregiver, configured to acquire a first input action by the caregiver, extract first touch stroke information from the first movement of the first input action, including the starting point, ending point, and trajectory of the first movement, and set the caregiver's care range based on the first touch stroke information.

[0017] [Composition 13] Furthermore, in one aspect of the present disclosure, the monitoring terminal described in [Configuration 12] is configured such that when the distance between the starting point and the ending point of the first movement is greater than or equal to a first predetermined distance, the monitoring range is set as a band-shaped area with an offset width from the trajectory, and when the distance between the starting point and the ending point of the first movement is less than the first predetermined distance, the monitoring range is set as a closed area defined by the trajectory and a straight line connecting the starting point and the ending point.

[0018] [Composition 14] Furthermore, in one aspect of this disclosure, the strip-shaped area may be generated by moving a circle with a radius of a second predetermined distance along the trajectory, and the monitoring terminal may be as described in [Configuration 12] or [Configuration 13].

[0019] [Composition 15] In addition, one aspect of the present disclosure may be the caregiver terminal according to any one of [Configuration 12] to [Configuration 14], wherein the second predetermined distance can be set by the caregiver.

[0020] 〔Configuration 16〕 In addition, one aspect of the present disclosure may be the caregiver terminal according to any one of [Configuration 12] to [Configuration 15], further configured to reduce the number of plots constituting the trajectory and display the set monitoring range on the caregiver terminal.

[0021] 〔Configuration 17〕 In addition, one aspect of the present disclosure may be the caregiver terminal according to any one of [Configuration 12] to [Configuration 16], further configured to acquire a second input operation by the caregiver, extract second touch stroke information including a starting point, an end point, and a trajectory from a second movement of the second input operation, and modify the set monitoring range of the person under care based on the first touch stroke information and the second touch stroke information when the trajectory of the second movement overlaps at least partially with the set monitoring range.

[0022] 〔Configuration 18〕 In addition, one aspect of the present disclosure may be the caregiver terminal according to [Configuration 17], wherein the set monitoring range is corrected by combining an area defined by the second touch stroke information with the monitoring range when a first predetermined operation is received.

[0023] 〔Configuration 19〕 In addition, one aspect of the present disclosure may be the caregiver terminal according to [Configuration 17] or [Configuration 18], wherein the set monitoring range is corrected by deleting an area defined by the second touch stroke information from the monitoring range when a second predetermined operation is received.

[0024] 〔Configuration 20〕 Furthermore, in one aspect of this disclosure, the monitoring terminal may be one of the configurations described in [Configuration 17] to [Configuration 19], wherein the modification of the set monitoring range by the second input operation can be repeatedly performed until a third predetermined operation is accepted.

[0025] [Composition 21] To solve the above-mentioned problems, one aspect of the present disclosure provides a method performed by a processor of a monitor terminal carried by a monitor, the monitor terminal comprising: acquiring a first input action by the monitor; extracting touch stroke information from a first movement of the first input action, including the starting point, ending point, and trajectory of the first movement; and setting a monitoring range for the person being monitored based on the touch stroke information.

[0026] [Composition 22] To solve the above-mentioned problems, one aspect of the present disclosure provides a program which, when executed by the processor of a monitor terminal carried by the monitor, acquires a first input action by the monitor, extracts touch stroke information from the first movement of the first input action, including the starting point, ending point, and trajectory of the first movement, and sets the monitoring range of the person being monitored based on the touch stroke information.

[0027] [Composition 23] To solve the above-mentioned problems, one aspect of the present disclosure provides a system comprising a terminal for the person being monitored, a terminal for the monitor, carried by the monitor, and a server, wherein the server obtains from the monitor terminal text entered by the monitor and an identification number assigned to the monitor, selects one of a plurality of synthesized voice models according to the profile corresponding to the monitor's identification number, converts the text into synthesized voice using the selected synthesized voice model, and outputs the synthesized voice to the terminal for the person being monitored.

[0028] [Composition 24] Furthermore, in one aspect of this disclosure, the system may be as described in [Configuration 23], wherein the profile includes at least one of the gender and age of the caregiver, and the synthesized voice model is selected according to at least one of the gender and age of the caregiver.

[0029] [Composition 25] Furthermore, one aspect of this disclosure may be the system described in [Configuration 23] or [Configuration 24], wherein, in the profile, if the gender of the caregiver is not registered, a synthesized voice model synthesized with a neutral voice is selected.

[0030] [Composition 26] Furthermore, in one aspect of the present disclosure, the system may be one of the configurations described in any of [Configuration 23] to [Configuration 25], wherein when the monitored terminal acquires the synthesized voice, it notifies the monitored person of the input of text by at least one of the following: lighting up the display device of the monitored terminal, vibrating the monitored terminal, or outputting a sound from the monitored terminal.

[0031] [Composition 27] To solve the above-mentioned problems, one aspect of the present disclosure provides a method that is performed on a system comprising a terminal for the person being monitored, a terminal for the monitor, and a server, the method comprising: obtaining text entered by the monitor and an identification number assigned to the monitor from the monitor terminal; selecting one of a plurality of synthesized speech models according to the profile corresponding to the monitor's identification number; and converting the text into synthesized speech using the selected synthesized speech model and outputting the synthesized speech to the terminal for the person being monitored.

[0032] [Composition 28] To solve the above-mentioned problems, one aspect of the present disclosure provides a program which, when executed by a server processor, obtains text entered by a caregiver and an identification number assigned to the caregiver from a caregiver terminal carried by the caregiver, selects one of a plurality of synthesized voice models according to the profile corresponding to the caregiver's identification number, converts the text into synthesized voice using the selected synthesized voice model, and outputs the synthesized voice to a caregiver terminal carried by the person being cared for. [Brief explanation of the drawing]

[0033] [Figure 1] This is a schematic diagram of a system according to one embodiment of the present invention. [Figure 2] This figure shows a schematic appearance of a first user terminal (terminal for the person being monitored) according to one embodiment of the present invention. [Figure 3] This is a functional block diagram of a system relating to one embodiment of the present invention. [Figure 4] This shows an example of a screen displayed on the display unit of a second user terminal (monitoring terminal) according to the first embodiment of the present invention. [Figure 5] This is an example of a screen displayed on the display unit of a second user terminal according to the first embodiment of the present invention. [Figure 6] This is an example of an activity history confirmation screen displayed on the display unit of a second user terminal according to the first embodiment of the present invention. [Figure 7] This is a flowchart illustrating the procedure for processing performed between a first user terminal, a second user terminal, and a server according to the first embodiment of the present invention. [Figure 8] This is an example of a monitoring range setting screen displayed on the display unit of a second user terminal according to a second embodiment of the present invention. [Figure 9] This is an example of a monitoring range setting screen displayed on the display unit of a second user terminal according to a second embodiment of the present invention. [Figure 10]This is an example of a monitoring range setting screen displayed on the display unit of a second user terminal according to a second embodiment of the present invention. [Figure 11] This is an example of a monitoring range setting screen displayed on the display unit of a second user terminal according to a second embodiment of the present invention. [Figure 12] An example screen for modifying the monitoring range according to the second embodiment of the present invention is shown. [Figure 13] An example screen for modifying the monitoring range according to the second embodiment of the present invention is shown. [Figure 14] An example screen for modifying the monitoring range according to the second embodiment of the present invention is shown. [Figure 15] An example screen for modifying the monitoring range according to the second embodiment of the present invention is shown. [Figure 16] An example screen for modifying the monitoring range according to the second embodiment of the present invention is shown. [Figure 17] An example screen for modifying the monitoring range according to the second embodiment of the present invention is shown. [Figure 18] This flowchart shows the procedure for setting the initial monitoring range in a second user terminal according to a second embodiment of the present invention. [Figure 19] This flowchart shows the procedure for modifying and changing the monitoring range set in the second user terminal according to the second embodiment of the present invention. [Figure 20] This flowchart shows the procedure for voice quality conversion processing performed by a system according to the third embodiment of the present invention. [Figure 21] An example of the data structure for information of a second user related to one embodiment of the present invention is shown. [Figure 22] An example of a data structure for sound quality information related to one embodiment of the present invention is shown. [Modes for carrying out the invention]

[0034] [Details of the embodiments of this disclosure] Embodiments of the present disclosure will be described below with reference to the drawings. Note that the following description is merely an example and is not intended to limit the technical scope of the present invention to these embodiments. In the drawings, identical or similar elements are denoted by identical or similar reference numerals, and redundant descriptions of identical or similar elements may be omitted in the description of each embodiment. Furthermore, features shown in each embodiment are applicable to other embodiments as long as they do not contradict each other. However, embodiments of the present disclosure are not necessarily limited to these forms. It will be apparent to those skilled in the art that embodiments of the present disclosure can take various forms that fall within the scope defined in the claims.

[0035] <System 1 Hardware Configuration> Figure 1 is a schematic diagram of System 1 according to one embodiment of the present invention. System 1 comprises a first user terminal 10 (terminal for the person being monitored) carried by the person being monitored (e.g., a child, an elderly person), a second user terminal 30 (terminal for the caregiver) carried by the caregiver (e.g., a parent), and a server 50 configured to communicate with the first user terminal 10 and the second user terminal 30 via Network 2. In the example shown in Figure 1, System 1 is configured to have one first user terminal 10 and one second user terminal 30, but the number of first user terminals 10, second user terminals 30, and servers 50 in System 1 is arbitrary. In the following description, the person being monitored will also be referred to as the first user, and the caregiver will also be referred to as the second user.

[0036] Network 2 may consist of, for example, the Internet, a mobile communication system (e.g., 3G, 4G, 5G, LTE (Long Term Evolution), etc.), WiFi (Wireless Fidelity), Bluetooth (registered trademark), other communication lines, or a combination thereof.

[0037] (Server 50) The server 50 (in other words, the information processing device) may be, for example, a general-purpose computer such as a workstation or personal computer, a cloud server, or a combination thereof. The server 50 includes a processor 51 for executing programs, a storage device 52 for storing programs, and a communication interface (IF) 53 for communicating with external devices such as a first user terminal 10 and a second user terminal 30. Although not shown in Figure 1, the server 50 may also include an input / output interface and a display device.

[0038] The processor 51 controls the operation of the entire server 50. The processor 51 may include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit), etc. The processor 51 reads and executes programs from the storage device 52. The storage device 52 is, for example, an HDD (Hard Disk Drive) or a semiconductor storage device (SSD (Solid State Drive)). Various data and information processing programs are stored in the storage device 52.

[0039] Communication IF53 is an interface for communicating with other devices (for example, the first user terminal 10, the second user terminal 30, etc.).

[0040] (First user terminal 10) The first user terminal 10 (in other words, an information processing device, a terminal for the person being monitored) includes a processor 11 for executing a program, a storage device 12 for storing the program, a communication interface (IF) 13 for communicating with an external device such as a server 50, an input / output interface (IF) 14, a microphone 15, a speaker 16, a display device (LED) 17, a notification device 18, a location acquisition device 19, and an input device 20. The first user terminal 10 may be configured as an IoT (Internet of Things) terminal and can be configured to communicate with a base station (not shown) via LTE-M (Long-Term Evolution for Machines), which is one of the LPWA (Low-Power Wide-Area) standards.

[0041] The processor 11 controls the overall operation of the first user terminal 10. The processor 11 reads and executes programs from the storage device 12. The storage device 12 stores various data and information processing programs. The storage device 12 can store identification numbers for identifying the first user terminal 10, information processing programs, voice data transmitted from the server 50, and various notifications. By assigning the terminal identification number to the data transmitted from the first user terminal 10, the server 50 can determine which first user terminal 10 the received data originated from. The processor 11 can also perform processing such as compression and decompression of voice data. These processes may be performed by a dedicated voice conversion device (not shown) instead of the processor 11, or in cooperation with the processor 11.

[0042] The microphone 15 is a device that converts sound vibrations into electrical signals. The user of the first user terminal 10 can input voice and ambient sounds using the microphone 15. The microphone 15 can be controlled by the processor 11, the notification device 18 (described later), and the input device 20. The sound data of the input sound is transmitted to the server 50 via the communication interface 13.

[0043] Speaker 16 is an audio device that converts electrical signals into sound. Speaker 16 can play sound data transmitted from the second user terminal 30 via the server 50. Speaker 16 is controlled by the processor 11 and the input device 20 (described later), and can play sound data when, for example, the input device 20 is operated.

[0044] The display device 17 is, for example, an LED. The display device 17 emits light (on or blinks) in a set pattern or a predetermined pattern or color to indicate that a message has been received, that sound is being collected by the microphone 15, or that sound is being recorded.

[0045] The notification device 18 is a device that uses a location acquisition device 19, such as a GPS sensor, to notify the second user terminal 30 of the current location information of the first user terminal 10 via the server 50. When the first user presses the notification device 18, the location information of the first user terminal 10 at the time of operation can be notified to the second user terminal 30. The notification device 18 may also light up (turn on or blink) when operated and turn off when the transmission of location information is complete.

[0046] Furthermore, upon receiving a single press operation from the first user, the notification device 18 can identify the current location information of the first user terminal 10, collect and record ambient sounds using the microphone 15 for a predetermined period, and transmit the recorded sound along with the identified location information to the server 50. The location information identification and recording functions can be enabled by settings on the first user terminal 10, the second user terminal 30, and the server 50. Enabling the recording function is performed, for example, on the condition that a recording period is set. In one example, this function is enabled by the first user of the first user terminal 10 setting the recording period on the application settings screen (screen 400 in Figure 4, described later). In another example, this function can be enabled on the condition that the recording period is set in advance in the firmware of the second user terminal 30. In yet another example, this function is enabled on the condition that the recording period is set by a value obtained from the server 50 by the second user terminal 30. The recording period may be set in advance before receiving a single press operation to the notification device 18, or it may be set when the single press operation is received. The notification device 18 may be configured to start sound collection and recording with the first short press operation (e.g., less than 0.5 seconds) and to cancel sound collection and recording with a second short press operation immediately following (e.g., less than a few seconds). In this way, the notification device 18 can cause the first user terminal 10 to perform different processes depending on the number of times it is pressed, or the interval between presses if it is pressed multiple times.

[0047] The position acquisition device 19 periodically, or when the notification device 18 is operated, receives at least one of the following signals: for example, signals transmitted from multiple GPS satellites, signals from a wireless LAN access point (AP), signals from a mobile phone base station, or Bluetooth ("Bluetooth" is a registered trademark; the registered trademark is omitted hereinafter) beacon signals, and transmits them to the server 50.

[0048] When the input device 20 receives input of a first operation from the first user, it can play sound data transmitted from the second user terminal 30 via the speaker 16. The first operation by the first user to the input device 20 is not particularly limited, but is a single, short press operation on the input device 20 (for example, less than 0.5 seconds).

[0049] Furthermore, a few seconds after receiving the second operation input from the first user (for example, 2 seconds later), the input device 20 begins collecting and recording sound using the microphone 15. While the second operation is being received, the microphone 15 converts ambient sounds and voices into electrical signals. These electrical signals are converted into digital data (sound data) by the microphone 15's A / D converter (not shown) and transmitted to the server 50. Once the second operation is completed, the input device 20 transmits the sound data to the server 50.

[0050] The second operation by the first user on the input device 20 is a press operation for a different duration than the first operation, and is not particularly limited, but is a long press (hold) operation in which the input device 20 is kept pressed. In this case, the input device 20 records while the long press operation is continued, and when the long press operation is released, it sends the sound data to the server 50.

[0051] (Second user terminal 30) The second user terminal 30 (in other words, the information processing device) may be, for example, a smartphone, feature phone, PDA (Personal Digital Assistant), tablet computer, personal computer, wearable device, etc.

[0052] The second user terminal 30 comprises a processor 31, a storage device 32, a communication interface (IF) 33, an input / output IF 34, a microphone 35, a speaker 36, and a touchscreen 37. These components of the second user terminal 30 are connected to each other by a communication bus.

[0053] The processor 31 controls the overall operation of the second user terminal 30. The processor 31 reads and executes programs from the storage device 32. The storage device 32 stores various data and information processing programs. The storage device 32 stores an identification number for identifying the second user terminal 30, information processing programs, location information and sound data transmitted from the first user terminal 10, etc. By assigning the terminal identification number to the data transmitted from the second user terminal 30, the server 50 can determine which second user terminal 30 the received data was transmitted from.

[0054] The microphone 35 is a device that converts sound vibrations into electrical signals. Users of the second user terminal 30 can input voice using the microphone 35. The input voice data can be transmitted to the server 50 via the communication IF 33.

[0055] Speaker 36 is an audio device that converts electrical signals into sound. Speaker 36 can play sound data transmitted from the first user terminal 10.

[0056] The touchscreen 37 includes an input unit 38 that accepts input from a second user and a display unit 39 that outputs an image. The input unit 38 has the function of detecting the position input to the input surface by user operation (e.g., touch operation, tap operation, slide operation, swipe operation, stroke operation, and flick operation, etc.) and transmitting information indicating the detected position as an input signal. The display unit 39 may be, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display.

[0057] <Appearance of the first user terminal 10> Figure 2 shows a schematic appearance of a first user terminal 10 according to one embodiment of the present invention. The first user terminal 10 has a microphone 15, a speaker 16, a display unit (e.g., an LED display unit) 17, and a notification device 18 arranged on its front, and an input device 20 arranged on the side of the first user terminal 10. In this embodiment, the display device 17 is shown as being provided as one on the front of the first user terminal 10, but additional display devices 17 may be provided on the side. Furthermore, the arrangement of the microphone 15, speaker 16, display device 17, notification device 18, and input device 20 is not limited to the illustrated example and may be arranged at any position on the first user terminal 10, for example, on the side. In addition, the notification device 18 and the input device 20 may be configured to have different shapes, sizes, and colors. This allows even a first user (e.g., a child) to easily identify each device.

[0058] <System Functional Configuration> Figure 3 is a functional block diagram of System 1 according to one embodiment of the present invention. In this embodiment, the server 50 identifies each user and the first user terminal 10 and second user terminal 30 using user accounts that have been registered in advance. The method of registering accounts is not particularly limited, but for example, the second user terminal 30 may transmit information necessary for user account registration (information of the first user who is being monitored, and information of the second user who is monitoring) to the server 50 based on the operation of the second user, and the server 50 may create and save accounts for each user based on the received information.

[0059] The server 50 functions as a control unit 510 and a storage unit 520 through the cooperation of the processor 51, storage device 52, communication interface 53, etc. The storage unit 520 stores various data used by the control unit 510. The various data are not particularly limited, but include a management program 521 for managing the first user terminal 10 and the second user terminal 30, user information 522, behavior history information 523, and voice quality information 524.

[0060] The management program 521 is a program for realizing a monitoring service in which the second user terminal 30 monitors the first user terminal 10. User information 522, behavior history information 523, and voice quality information 524 are data that the control unit 510 refers to when executing the management program 521.

[0061] Furthermore, the management program 521 may include, in addition to the program executed on the server 50, the functions of a program sent to the first user terminal 10 and executed on the first user terminal 10 (the monitored program 121 described later) or a program executed on the second user terminal 30 (the monitoring program 321 described later).

[0062] User information 522 is information managed for each user account. User information 522 includes first user information 522A and second user information 522B.

[0063] The first user information 522A includes, but is not limited to, an identifier for the first user, an identifier for a related second user, and monitoring range information for the first user as data items.

[0064] The monitoring range information includes data defining the monitoring range of the first user, including the position coordinates (latitude and longitude) of each vertex within that range. The monitoring range is the range for which notifications are sent to the second user terminal 30 according to the location information of the first user terminal 10. The monitoring range may be set to correspond to the first user's normal living area or to correspond to an area that is dangerous to the first user. When System 1 detects that the first user terminal 10 has left the living area, it sends an alarm notification to the second user terminal 30. Also, when System 1 detects that the first user terminal 10 has entered the dangerous area, it sends an alarm notification to the second user terminal 30.

[0065] The second user information 522B includes, but is not limited to, the second user's identifier, gender, address, language, age, voice quality, and the identifier of the first user being monitored, as data items. Multiple second users can be associated with one first user. An example of the data structure of the second user information 522B will be described later with reference to Figure 21.

[0066] The activity history information 523 is information managed for each first user account. The activity history information 523 is not particularly limited, but includes information about the activity history of the first user over a predetermined period of time since the notification device 18 was operated (for example, the history of the location information of the first user terminal 10, the date and time each location information was acquired, the history of sound data acquired by the first user terminal 10, text data converted from sound data, the date and time the notification device 18 of the first user terminal 10 was operated, and information on whether the first user terminal 10 left or entered the set monitoring range).

[0067] The voice quality information 524 stores synthesized speech models with different voice qualities depending on gender, age, and generation. These synthesized speech models can be pre-stored in the server 50. An example of the data structure of the voice quality information 524 will be described later using Figure 22.

[0068] The control unit 510 controls various processes related to monitoring the first user by executing a management program 521 stored in the storage unit 520. The control unit 510 includes, for example, a transmitting / receiving unit 511, a server processing unit 512, and a data management unit 513.

[0069] The transmitting / receiving unit 511 transmits or receives various types of data. For example, the transmitting / receiving unit 511 receives requests to transmit various types of data and programs, location information and input sound data from the first user terminal 10, and text data and the like that input to the second user terminal 30, and passes them to the server processing unit 512. The transmitting / receiving unit 511 also transmits various types of data and programs, including instructions, to the first user terminal 10 and the second user terminal 30 in accordance with the control of the server processing unit 512.

[0070] The server processing unit 512 provides the monitoring service for the first user terminal 10 by the second user terminal 30 by executing calculation processes described in the management program 521 in response to requests from the first user terminal 10. For example, when the server processing unit 512 receives a request for processing for the monitoring service or data to be processed from the first user terminal 10 or the second user terminal 30, it executes the processing for the monitoring service.

[0071] The server processing unit 512 can estimate the location of the first user terminal 10 by integrating data obtained from multiple different positioning means. Specifically, the server processing unit 512 acquires location information from the first user terminal 10's location acquisition device 19 by combining data obtained from multiple positioning means, such as signals from a GPS receiver, signals from a wireless LAN access point, and signals from a mobile phone base station. By sequentially processing this positioning data using a Kalman filter or the like, the server processing unit 512 can determine the location of the first user terminal 10 more accurately. As a result, even if the first user terminal 10 has limited resources such as battery consumption and communication speed, the server 50's resources can be used to determine the accurate location of the first user terminal 10.

[0072] The data management unit 513 manages the management program 521, user information 522, behavioral history information 523, and voice quality information 524, etc.

[0073] The first user terminal 10 functions as a control unit 110 and a storage unit 120 through the cooperation of a processor 11, a storage device 12, a communication IF 13, an input / output IF 14, a microphone 15, a speaker 16, a display device 17, a notification device 18, a location acquisition device 19, an input device 20 (Figure 1), etc. The storage unit 120 stores various data used by the control unit 110. Examples of such data include a monitoring program 121, user information 122, and sound data 123.

[0074] The monitoring program 121 is a program for enabling a second user to monitor the first user terminal 10. User information 122 and sound data 123 are data that the control unit 110 refers to when executing the monitoring program 121.

[0075] User information 122 is data relating to the account of a user using the first user terminal 10, and may include the identifier of the first user and monitoring range information.

[0076] Furthermore, the user information 122 may include location information indicating the current location of the first user terminal 10 detected by the location acquisition device 19, and the date and time corresponding to said location information. This location information and the corresponding date and time may be transmitted to the server 50 via the network 2 and stored as the server 50's activity history information 523.

[0077] The sound data 123 may include sound data acquired by the first user terminal 10, sound data based on operations of the second user terminal 30 acquired from the server 50, and preset voice data having specific acoustic characteristics, such as voice quality and intonation. The preset voice data is not particularly limited, but may include voice data of voice actors, actors, idols, etc.

[0078] The control unit 110 controls various monitoring-related processes performed on the first user terminal 10 by executing the monitoring program 121 stored in the memory unit 120. The control unit 110 includes, for example, an operation reception unit 111, a transmission / reception unit 112, and an operation control unit 113.

[0079] The operation reception unit 111 receives operations (hereinafter also referred to as "input operations") input by the user via the input IF 14. Specifically, the operation reception unit 111 detects input operations to the notification device 18 and the input device 20, as well as the type of input operation. Examples of input operation types include a short, single press operation to the notification device 18, a second press operation immediately following a short, single press operation, a short press operation to the input device 20, and a long press (hold) operation that maintains a pressed state.

[0080] The transmitting / receiving unit 112 transmits and receives various types of data. The transmitting / receiving unit 112 transmits sound data recorded by the first user terminal 10 and location information to the server 50. The transmitting / receiving unit 112 can receive monitoring range information set by the second user terminal 30, sound data based on operations of the second user terminal 30, and various programs from the server 50.

[0081] The operation control unit 113 performs various processes for monitoring the first user terminal 10. The operation control unit 113 is not particularly limited, but it performs processes for setting the recording period on the first user terminal 10, processes for starting and ending the recording of sound data, and processes for notifying when the first user terminal 10 deviates from the monitoring range. Note that the first user terminal 10 does not need to store monitoring range information. The system may be configured to receive a notification from the server 50 when the first user terminal 10 deviates from the monitoring range.

[0082] The second user terminal 30 functions as a control unit 310 and a storage unit 320 through the cooperation of a processor 31, storage device 32, communication interface 33, input / output interface 34, microphone 35, speaker 36, touchscreen 37 (Figure 1), etc. The storage unit 320 stores various data used by the control unit 310. Examples of such data include a monitoring program 321, user information 322, and behavioral history information 323.

[0083] The monitoring program 321 is a program for enabling a second user to monitor the first user terminal 10. User information 322 and behavioral history information 323 are data that the control unit 310 refers to when executing the monitoring program 321.

[0084] User information 322 is data relating to the second user's account and may include information similar to that of the server 50's user information 522 described above. User information 322 may include information about the second user (e.g., the second user's identifier, gender, address, language, and age) and information about the first user being monitored (e.g., the first user's identifier and monitoring range information).

[0085] Furthermore, the user information 322 may include location information indicating the current location of the first user terminal 10 detected by the location acquisition device 19, and the date and time corresponding to said location information. This location information and the corresponding date and time can be received from the server 50 via the network 2.

[0086] The activity history information 323 is information relating to the activity history of the first user terminal 10 over a predetermined period of time from the time the notification device 18 of the first user terminal 10 was operated, and may include information similar to the activity history information 523 of the server 50 described above.

[0087] The control unit 310 controls various monitoring-related processes performed on the first user terminal 10 by executing a monitoring program 321 stored in the memory unit 320. The control unit 310 includes, for example, an operation reception unit 311, a transmission / reception unit 312, an operation control unit 313, and a display control unit 314.

[0088] The operation reception unit 311 receives operations (hereinafter also referred to as "input operations") input by the user via the input unit 38. Specifically, when an input operation is performed on the input unit 38, the operation reception unit 311 detects the screen coordinates of the input location and the type of input operation. Examples of input operations include various operations performed with fingers, such as touch operations, tap operations, slide operations, swipe operations, flick operations, touch stroke operations where a finger or other object is in contact with the screen and a trajectory is drawn, pinch-in operations, and pinch-out operations. These can be used for various functions such as setting and modifying the monitoring range. Input operations are not limited to operations that physically contact the input unit 38 (for example, the touchscreen 37), but can also include non-contact operations. Operations that terminate an input operation that was previously performed, such as a touch-off operation that ends contact with the touchscreen 37, can also be considered a form of input operation.

[0089] The transmitting / receiving unit 312 can send and receive various types of data to and from the server 50.

[0090] The transmitting / receiving unit 312 can transmit various data and operation information entered by the second user to the server 50. The data entered by the second user includes voice data, text data, monitoring range information, etc. The transmitting / receiving unit 312 can receive various data such as operations and sound data entered at the first user terminal 10, programs, etc., from the server 50.

[0091] The operation control unit 313 performs various processes for monitoring the first user terminal 10 by the second user terminal 30. The operation control unit 313, although not particularly limited, performs processes for setting the recording period on the first user terminal 10, processes for setting the monitoring range by the second user, and processes for receiving voice and text input by the second user.

[0092] The display control unit 314 causes the display unit 39 to display various images, such as a map image that includes the monitoring area.

[0093] Note that the functions of the first user terminal 10, the second user terminal 30, and the server 50 shown in Figure 3 are merely examples. Each of the devices, the first user terminal 10, the second user terminal 30, and the server 50, may also have at least some of the functions of other devices. In other words, in this embodiment, the server 50 may have some of the functional blocks of the first user terminal 10 and the second user terminal 30, and the first user terminal 10 or the second user terminal 30 may have some of the functional blocks of the server 50.

[0094] <First Embodiment> Next, with reference to Figures 4 to 7, the information processing performed in System 1 according to the first embodiment of the present invention will be described.

[0095] In this embodiment, the first user terminal 10 starts acquiring and recording sound data of the surrounding sounds in response to a single input operation, and stops acquiring and recording sound data after a predetermined period of time. After the predetermined period of time has elapsed, the sound data is automatically stopped, and the sound data is transmitted to the server 50 along with the location information of the first user terminal 10.

[0096] According to this disclosure, a single one-touch operation (a single press of the notification device 18 in Figure 2) initiates the acquisition and recording of ambient sound data using the microphone 15, and the acquisition and recording of sound data automatically stops after a predetermined period. Therefore, unlike the PTT (Push-to-Talk) method, the first user does not need to continuously press a specific button to speak. This disclosure prevents unintended cessation of sound data acquisition and recording due to releasing the operation, such as releasing the hand during recording. As a result, even a first user who is not accustomed to long-pressing buttons (for example, a child) can operate the first user terminal 10 with a single touch using one hand. Furthermore, the automatic cessation of sound data acquisition and recording suppresses unnecessary battery consumption in the first user terminal 10. In addition, because ambient sound recording and stopping can be done with a single touch, it can be easily operated even in situations where voice input using the microphone 15 is difficult (for example, in an emergency or when hands are occupied).

[0097] Furthermore, according to this disclosure, with a single touch operation, the location information of the first user terminal 10 is transmitted to the second user terminal 30 along with the sound data, allowing the second user to simultaneously understand the first user's voice and location.

[0098] Figure 4 shows an example of a screen displayed on the display unit 39 of a second user terminal 30 according to the first embodiment of the present invention. Figure 4 shows a recording period setting screen 400 for setting a predetermined recording period (hereinafter sometimes referred to as the recording period) on the second user terminal 30.

[0099] In the example shown in Figure 4, the name of the second user 401 (which can also be a handle name; in the example shown, it is "Hiroko") is displayed at the top of screen 400. Also at the top of screen 400, the update interval priority mode 402 and eco mode 403, which can be selected by the second user, are displayed, and in the example in Figure 4, the update interval priority mode 402 is selected.

[0100] In update interval priority mode 402, the second user terminal 30 can acquire location information and battery level information of the first user terminal 10 from the first user terminal 10 at the set update interval. In eco mode 403, location information and battery level information of the first user terminal 10 can be acquired from the first user terminal 10 at a longer interval than when update interval priority mode 402 is set.

[0101] At the bottom of screen 400, along with the text "Please select the recording time" 404, several options for the recording period 405 ("Do not record", "5 seconds", "10 seconds", ... "30 seconds", "Cancel") are displayed. In the example shown, the recording period can be selected in 5-second increments up to a maximum of 30 seconds.

[0102] Figure 5 shows an example of a screen 500 displayed on the display unit 39 of a second user terminal 30 according to a first embodiment of the present invention. When the notification device 18 (Figure 1) is operated on the first user terminal 10, a push notification indicating that the notification device 18 has been operated is displayed on the display unit 39 of the second user terminal 30. When the monitoring program 321 is activated on the second user terminal 30 in response to the push notification, a notification screen 500 indicating that the notification device 18 has been operated is displayed on the display unit 39.

[0103] In the example shown in Figure 5, a pop-up display 501 containing the notification "Notification button pressed" is displayed in the center of the notification screen 500.

[0104] The pop-up display 501 includes, but is not limited to, information for identifying the first user (the first user's name 502, a facial photograph or image 505 for visually identifying the first user), the time of the event 503, the location information of the first user terminal 10 504, a playback icon 506 for playing sound data recorded on the first user terminal 10, and an activity history confirmation icon 507 for displaying the location history of the first user terminal 10 over a predetermined period on the display unit 39.

[0105] The event occurrence time 503 indicates the date and time when a predetermined event occurred on the first user terminal 10. The event occurrence time 503 is not particularly limited, but is at least the time when the notification device 18 on the first user terminal 10 was pressed and location information was acquired, or the time when sound data recorded on the first user terminal 10 was transmitted to the server 50.

[0106] Location information 504 is address information corresponding to the location of the first user terminal 10, as determined by the server 50 based on the location information of the first user terminal 10 when the notification device 18 is pressed on the first user terminal 10. The address information may be at a detailed level (e.g., down to the street number, block number, and house number), a general level (e.g., down to the name of the city or town), or the name of a building, etc. The address information may be changed depending on the accuracy of the location determination.

[0107] The playback icon 506 is a playback link icon for accessing the location where the audio data is stored (for example, the server 50 or the storage unit of the second user terminal 30). When the second user operates the playback icon 506, the audio data is played on the second user terminal 30. The playback time of the recorded audio data is displayed near the playback icon 506.

[0108] When the operation reception unit 311 receives an operation (e.g., a tap) on the activity history confirmation icon 507 shown in Figure 5, it transitions to an activity history confirmation screen 600 (see Figure 6) which includes the location information history of the first user terminal 10 over a predetermined period.

[0109] Figure 6 shows an example of an activity history confirmation screen 600 displayed on the display unit 39 of a second user terminal 30 according to the first embodiment of the present invention. The activity history confirmation screen 600 can display at least a portion of the first user's activity history information on a map. The displayed map screen may be a three-dimensional map or a two-dimensional map. In this embodiment, an example in which the map screen is a two-dimensional map will be described.

[0110] The activity history confirmation screen 600 includes two modes: an activity history display mode and a current location display mode.

[0111] In the activity history display mode (the mode in which the activity history icon 608 is tapped), all travel routes 607 during the recording period can be displayed overlaid on a map so that the current location of the first user terminal 10 or the location at the end of the recording period is centered on the screen. The travel routes 607 are generated by overlaying the location history of the first user terminal 10 onto the map.

[0112] In current location display mode (the mode in which the current location icon 609 is tapped), the travel route 607 is displayed on a map zoomed in to a narrower area than in the activity history display mode, so that the current location of the first user's terminal 10 is centered on the screen. On the narrower area of ​​the map, a portion of the travel route 607 traveled by the first user can be displayed. In current location display mode, the first user's current location can be confirmed in the center of the screen, and the surrounding area can be displayed in more detail. By switching between these modes, the second user can switch between zoomed-in and zoomed-out views of the map centered on the first user.

[0113] At the top of the activity history confirmation screen 600, a window 611 is displayed. This window 611 displays the playback time 601 of the audio data, a text display 602 showing the audio data converted to text, and a playback icon 603 for playing the audio data. The text display 602 is obtained when the server 50 converts the audio data received from the first user terminal 10 into text using speech-to-text (STT) technology. If the audio data contains sounds other than speech, the server 50 can extract the speech from the audio data and convert the extracted speech into text. If speech cannot be extracted from the audio data, speech-to-text conversion is not performed, and the text display 602 is not displayed.

[0114] The activity history confirmation screen 600 can also display a photograph or image 605 (for example, a photograph of the first user) to identify the first user, the time the event occurred 604, the first user's current location pin 606 on the map, and the first user's name 610.

[0115] Next, with reference to Figure 7, a flowchart 700 of the process performed by System 1 according to one embodiment of the present disclosure will be described.

[0116] Figure 7 is a flowchart 700 showing the procedure of processing performed between the first user terminal 10, the second user terminal 30, and the server 50 according to the first embodiment of the present invention.

[0117] First, in step S1 of Figure 7, the display control unit 214 of the second user terminal 30 displays the recording period setting screen 400 on the display unit 39 and accepts the second user's selection of the recording period. Upon accepting the selection of the recording period, it sends the second user's identification number along with the recording period to the server 50.

[0118] Next, in step S2, the server processing unit 512 of the server 50 identifies the first user terminal 10 corresponding to the second user's identification number as the first user terminal 10 to be monitored. The server 50 sets the recording period in association with the first user's identification number and sends the recording period to the first user terminal 10.

[0119] In step S3, the first user terminal 10 sets the acquired recording period.

[0120] Next, in step S4, when the operation reception unit 111 of the first user terminal 10 detects a single press operation on the notification device 18, it uses this press as a trigger to proceed to step S5.

[0121] In step S5, the operation control unit 113 starts acquiring sound data from the microphone 15 and recording the sound data, and also starts acquiring location information of the first user terminal 10 using the location acquisition device 19. The location acquisition device 19 can change the interval at which location information is acquired, triggered by the pressing of the notification device 18. Specifically, it acquires location information of the first user terminal 10 at a shorter interval (e.g., once every 5 seconds) than the normal interval (e.g., once every hour) when the notification device 18 is not pressed. This makes it possible to acquire location information at a high frequency in certain situations such as emergencies. The location information acquisition interval can be set by the second user terminal 30.

[0122] Furthermore, in step S5, the operation control unit 113 can cause the display device (LED) 17 of the first user terminal 10 to emit light (flashing or lighting). In other words, the first user terminal 10 can simultaneously start three different processes—acquisition of sound data, acquisition of location information, and illumination of the display device 17—with a single operation input to the notification device 18.

[0123] Next, in step S6, the operation control unit 113 determines whether the recording period set in step S3 has elapsed. If the recording period has not elapsed (no in step S6), it returns to step S5. On the other hand, if the operation control unit 113 determines that the recording period set in step S3 has elapsed (yes in step S6), it proceeds to step S7.

[0124] Next, in step S7, the operation control unit 113 stops acquiring sound data and waits until the acquisition of location information is complete. Once the acquisition of location information is complete, the display device 17 stops emitting light. The transmitting / receiving unit 112 transmits the event occurrence time 503, the recorded sound data for the recording period, and the location information history, along with the identification number of the first user, to the server 50.

[0125] Next, in step S8, the transmitting / receiving unit 511 of the server 50 receives the event occurrence time 503, sound data during the recording period, location information, and the identification number of the first user. It then associates these with the identification number of the first user and stores them in the storage unit 520 as activity history information 523. The server processing unit 512 can also reverse geocode the latitude and longitude of the location information to identify the address, place name, building name, etc., of the first user terminal 10. Furthermore, the server processing unit 512 can extract speech from the acquired sound data and convert the extracted speech into text using speech-to-text conversion technology.

[0126] Furthermore, in step S8, the server 50's transmitting / receiving unit 511 sends a notification to the second user terminal 30 indicating that it has received an operation from the notification device 18, along with the identified address and the sound data. Alternatively, instead of sending the sound data itself, a playback link to access the location where the sound data is stored may be sent.

[0127] Furthermore, in step S8, the transmitting / receiving unit 511 of the server 50 may generate push notification data including the identification information of the first user terminal 10 and a string of characters indicating that an operation to the notification device 18 has been received, and send it to an external push notification service server (not shown) (for example, Apple's APNs (Apple Push Notification service) or Google's FCM (Firebase Cloud Messaging)).

[0128] Next, in step S9, when the transmitting / receiving unit 312 of the second user terminal 30 receives an urgent notification that it has received an operation to the notification device 18, along with the address and the playback link, the display control unit 314 notifies that an operation has been performed on the notification device 18, generates a pop-up notification area prompting the second user to take a specific action, and displays the pop-up notification area on the screen of the second user terminal 30.

[0129] Next, in step S10, when the operation reception unit 311 of the second user terminal 30 receives an application launch operation from the second user who has confirmed the contents of the pop-up notification area, the display control unit 314 displays the application screen 500 (Figure 5) for monitoring the status, location information, etc. of the first user terminal 10, which is the target of monitoring, on the display unit 39.

[0130] Next, in step S11, when the operation reception unit 311 of the second user terminal 30 receives a tap operation on the activity history confirmation icon 507 on the screen 500 (Figure 5), the transmission / reception unit 312 sends an activity history confirmation instruction to the server 50 requesting confirmation of the activity history.

[0131] Furthermore, when the server 50's transmitting / receiving unit 511 sends push notification data to an external push notification service server (not shown) in step S8, the second user terminal 30 can obtain the push notification data via the push notification service server (step S14). More specifically, the external push notification service server sends push notification data to the second user terminal 30, which is pre-registered in association with the identification information of the first user terminal 10. The second user terminal 30 receives the push notification data from the external push notification service server (step S14). When the application is launched by the second user who has confirmed the push notification (step S15), it can receive an action history confirmation instruction requesting to check the action history. When the second user terminal 30 receives the action history confirmation instruction, the process proceeds to step S12.

[0132] Next, in step S12, the server 50 sends at least a portion of the behavior history information 523 and a playback link to the second user terminal 30 in response to the behavior history confirmation instruction. Note that if the server 50 was unable to perform speech-to-text conversion in step S8, it does not send the text.

[0133] Next, in step S13, the second user terminal 30 stores the received activity history information 523 in the storage unit 320 as activity history information 323. The activity history information 323 may include information covering a predetermined period from the time the notification device 18 of the first user terminal 10 was operated (location information history of the first user terminal 10, the date and time each location information was acquired, and text obtained by converting at least a portion of the voice acquired by the first user terminal 10 into text). Also in step S13, the display control unit 314 of the second user terminal 30 displays the activity history confirmation screen 600 (Figure 6) on the display unit 39. The second user can refer to the activity history confirmation screen 600 to confirm the first user's travel route 607, etc.

[0134] As described above, according to this embodiment, the first user terminal 10 can acquire and record sound data with a single touch operation using one hand, and automatically stop acquiring and recording sound data after a predetermined period of time has elapsed. In addition, the first user terminal 10 can acquire location information at short intervals simultaneously with the acquisition of sound data.

[0135] <Second Embodiment> Next, with reference to Figures 8 to 19, the information processing of System 1 according to a second embodiment of the present invention will be described. System 1 of this disclosure can set monitoring range information for the first user terminal 10, which is the target of monitoring, based on information input from the second user terminal 30.

[0136] In this embodiment, the complex monitoring range of the first user terminal 10 can be set on the map according to the touch stroke operation performed by the second user on the touchscreen 37. Furthermore, the monitoring range can be changed or modified according to new touch stroke operations applied to the already set monitoring range.

[0137] For example, when setting a monitoring area for narrow and complex shapes such as rivers and roads, or complex and intricate shapes such as building sites, users previously had to add and move tens to hundreds of vertices. According to this disclosure, the creation and editing of complex monitoring areas can be performed with a few touch strokes. Therefore, even users unfamiliar with terminal operations can create polygons of complex shapes.

[0138] Figures 8 to 11 show examples of monitoring range setting screens displayed on the display unit 39 of the second user terminal 30 according to the second embodiment of the present invention.

[0139] When the operation reception unit 311 receives a selection operation (tap operation) for the drawing mode icon 807, it can accept a touch stroke operation by the user, such as tracing with a finger on the screen 800. In the example in Figure 8, a trajectory 811 from the starting point 809 to the ending point 810 is drawn on the map on the screen 800 according to the touch stroke operation. When a selection operation for the fill icon 808 is received, the area of ​​lines or circles drawn in drawing mode is converted into polygons (Figure 9). The trajectory of the input operation by the second user is composed of multiple plots. The display control unit 314 may reduce the number of plots that make up the trajectory without significantly distorting the shape of the trajectory, and display the reduced number of plots on the display unit of the second user terminal 30.

[0140] Furthermore, the operation reception unit 311 can delete a polygon from the screen when it receives a selection operation on the trash can icon 801. It can also undo the previous operation (polygon conversion operation, trajectory drawing operation, etc.) when it receives an operation on the undo icon 802. In addition, the operation reception unit 311 can deform a polygon by receiving a new touch stroke operation that overlaps at least part with the periphery or region of the converted polygon. The operation reception unit 311 can accept deletion, undo, and new touch stroke operations of drawn lines until an operation on the save icon 812 is received. As a result, even users unfamiliar with terminal operation can generate polygons with complex shapes.

[0141] When the monitoring area specified by the polygon reaches the desired position and shape for the second user, the second user can save the deformed polygon as the monitoring area by selecting the save icon 812.

[0142] Furthermore, the operation reception unit 311 can receive a display position change operation to set the monitoring range to a location on the map that is the target location for the second user. The display position change operation includes operations on the address search icon 804, the current location icon 805, and the movement mode icon 806.

[0143] When the user selects the address search icon 804, the display control unit 314 pops up a window (not shown) on the display unit 39 to accept input from the second user for the address and facility, and accepts text input for the address and facility name. The second user terminal 30 can use a geocoding service to determine the latitude and longitude from the address and facility name, and move the display to a map centered on the determined latitude and longitude.

[0144] Upon receiving a selection for the current location icon 805, the system can move to a map centered on the current location of the second user terminal 30.

[0145] When a selection is made for the movement mode icon 806, the currently displayed map can be moved according to the slide operation performed by the second user.

[0146] Next, we will explain the details of the process of converting a line or circle region drawn in drawing mode into a polygon using Figures 8 to 11.

[0147] As shown in Figure 8, when the distance between the starting point 809 and the ending point 810 of the touch stroke trajectory is less than a predetermined distance q on the screen, the monitoring range is set as a closed polygon region 901 defined by a straight line or curve connecting the starting point 809 and the ending point 810, and the trajectory 811 (Figure 9). In other words, if the starting point 809 and the ending point 810 of the user's stroke operation do not coincide, and the distance between the starting point 809 and the ending point 810 is less than a predetermined distance q, a closed polygon region is set by automatically interpolating a straight line connecting them.

[0148] On the other hand, as illustrated in Figure 10, when the starting point 1002 and ending point 1003 of the touch stroke trajectory are greater than or equal to a predetermined distance q, the monitoring range is set as a band-shaped polygon region 1101 with an offset width in the trajectory (Figure 11). The distance q may be either the distance on the screen coordinate system of the first user terminal or the actual distance on the map.

[0149] The band-shaped region 1101 shown in Figure 11 can be generated by moving the center of a circle with a predetermined radius r (for example, 0m to 50m on the map) along the touch stroke trajectory in the direction indicated by arrow P, from the starting point 1002 to the ending point 1003. The outer circumference of the trajectory traced by the moving circle forms the shape shown by lines L1 and L2, which forms the band-shaped region 1101. The distance r can be arbitrarily set by the supervisor. When the predetermined distance r is 0, the touch stroke trajectory itself becomes the polygon boundary. The distance r may be either the distance on the screen coordinate system of the first user terminal or the actual distance on the map.

[0150] According to this disclosure, a second user can easily set an elongated monitoring range, including straight lines and curves, using only touch stroke operations.

[0151] Next, we will explain the details of the process for modifying the configured monitoring range, referring to Figures 12 to 17.

[0152] Figures 12 to 17 show example screens for modifying the monitoring range according to a second embodiment of the present invention. System 1 provides means for modifying the polygon by new touch stroke operations by the user until the set monitoring range is confirmed and saved.

[0153] The operation reception unit 311 acquires a second input action (touch stroke operation) performed by a second user on the map displayed on the screen, and extracts second touch stroke information, including the start point, end point, and trajectory of the second movement, from the second movement of the second input action. It determines whether the trajectory of the second movement overlaps with the first monitoring range at least in part, and if an overlap exists, it modifies the first monitoring range based on the first touch stroke information and the second touch stroke information. The modified second monitoring range is set based on the positional relationship between the new input action (second touch stroke operation) by the user and the already drawn polygon (first monitoring range). Specifically, the polygon is modified in the following manner.

[0154] (Expansion and modification through merging) The expansion of the monitoring area is set by performing a Boolean union (union) between the first monitoring area (polygon region 901) and the region defined by the second touch stroke information, when an operation is received on the fill icon 808.

[0155] For example, as shown in Figure 12, if the trajectory of the second movement 902 intersects the drawn monitoring area 901 at multiple points, the additional area defined by the trajectory of the second movement 902 is merged with the drawn first monitoring area 901. This sets up a modified new monitoring area (e.g., polygon area 903) (Figure 13).

[0156] In another embodiment, as shown in Figure 14, when the trajectory of the second movement 904 intersects with the drawn first monitoring area 901 at one point, a band-shaped region with an offset width defined by the second movement trajectory 904 is joined to the drawn monitoring area 901. This sets up a new monitoring area (polygon area 905) (Figure 15).

[0157] (Deletion and correction based on differences) Deleting or modifying the monitoring area is set by performing a Boolean difference operation on the eraser icon 803, which is done by comparing the first monitoring area with the area defined by the second touch stroke information.

[0158] For example, as shown in Figure 16, if the trajectory 906 of the second movement intersects with the drawn monitoring area 903 at multiple points, the area defined by the trajectory 906 of the second movement is removed from the drawn monitoring area 903. In other words, the new monitoring area (polygon area 907) is set by subtracting the area defined by the trajectory 906 of the second movement and the monitoring area 903 at multiple intersections from the drawn monitoring area (Boolean subtraction). This sets the new monitoring area (polygon area 907) (Figure 17).

[0159] Furthermore, if the entire trajectory of the second movement does not intersect with the displayed monitoring range, the first monitoring range will not be modified.

[0160] The operation reception unit 311 accepts modifications to the monitoring range via continuous touch stroke operations until the save icon 812 is operated, and the operation control unit 313 temporarily holds the modified monitoring range. When the operation reception unit 311 receives an operation on the save icon 812, it finalizes and saves the shape of the monitoring range at that point. The user can modify the monitoring range by repeatedly performing multiple touch stroke operations until the final shape of the monitoring range is determined.

[0161] Figure 18 is a flowchart showing the procedure for setting the initial monitoring range in the second user terminal 30 according to the second embodiment of the present invention.

[0162] First, in step S51, the operation reception unit 311 acquires a first input operation to the input unit 38 by the second user. The first input operation includes touch stroke information, which includes the starting point, ending point, and trajectory of the touch stroke movement.

[0163] Next, in step S52, the motion control unit 313 extracts touch stroke information from the movement of the first input motion, including the starting point, ending point, and trajectory of the touch stroke movement.

[0164] Next, in step S53, the motion control unit 313 determines whether the distance between the starting point and the ending point of the touch stroke movement is less than a first predetermined distance q.

[0165] In step S53, if the distance between the starting point and the ending point of the touch stroke is less than the first predetermined distance q, proceed to step S54.

[0166] In step S54, when the motion control unit 313 receives an operation on the fill icon 808, it interpolates the distance between the start point and the end point with a straight line or curve, and sets the monitoring range as a closed polygon region 901 (Figure 9) defined by the movement trajectory and the line connecting the start point and the end point.

[0167] On the other hand, if, in step S53, the distance between the starting point and the ending point of the touch stroke is not less than the first predetermined distance q, the process proceeds to step S55.

[0168] In step S55, when the operation control unit 313 receives an operation on the fill icon 808, it sets the monitoring range to a band-shaped area 1101 (Figure 11) with an offset width in the trajectory.

[0169] Next, in step S56, when the operation control unit 313 receives an operation on the save icon 812, it proceeds to step S57.

[0170] In step S57, when the operation control unit 313 receives an operation on the save icon 812, it saves the set monitoring range, and the transmitting / receiving unit 312 transmits the monitoring range information to the server 50. The monitoring range information includes the coordinates of each vertex of the set monitoring range. These coordinates may be screen coordinates on the display unit of the first user terminal 10, or they may be actual position coordinates (latitude, longitude, etc.). The server 50 can store the latitude and longitude of each received vertex, along with the identifier of the first user terminal 10 associated with the second user terminal 30, in the user information 522.

[0171] On the other hand, in step S56, if the operation control unit 313 does not accept an operation on the save icon 812, it proceeds to the process shown in Figure 19.

[0172] Next, referring to Figure 19, we will explain the workflow for modifying and changing the configured monitoring range.

[0173] Figure 19 is a flowchart showing the procedure for process 1900, which modifies the monitoring range set in the second user terminal 30 according to a second embodiment of the present invention. Process 1900 describes a modification process performed in the second user terminal 30 after the initial monitoring range has been set in the second user terminal 30. The initial monitoring range is assumed to be displayed on the display unit 39.

[0174] First, in step S71, the operation reception unit 311 acquires a second input operation to the input unit 38 by the second user. The second input operation includes second touch stroke information, which includes the starting point, ending point, and trajectory of the touch stroke movement.

[0175] Next, in step S72, the motion control unit 313 extracts second touch stroke information from the movement of the second input motion, including the starting point, ending point, and trajectory of the second touch stroke movement.

[0176] Next, in step S73, the motion control unit 313 determines whether the trajectory of the second movement overlaps with the set monitoring range in at least part of it.

[0177] In step S73, if the trajectory of the second movement overlaps with the initially set monitoring range in at least part of it, the process proceeds to step S74. In the subsequent steps S75 and S76, the set monitoring range is modified and changed based on the first touch stroke information and the second touch stroke information.

[0178] In step S75, when the operation reception unit 311 receives a first predetermined operation (selection operation for the fill icon 808), it modifies and changes the set monitoring range by combining the area defined by the second touch stroke information with the first monitoring range.

[0179] On the other hand, in step S76, when the operation reception unit 311 receives a second predetermined operation (selection operation for the eraser icon 803), it modifies and changes the set monitoring range by deleting the area defined by the second touch stroke information from the first monitoring range.

[0180] Next, in step S77, it is determined whether an operation on the save icon 812 has been accepted.

[0181] If it is determined in step S77 that an operation on the save icon 812 has been accepted ("Yes" in step S77), the process proceeds to step S78.

[0182] In step S78, the operation control unit 313 finalizes and saves the shape of the monitoring area at that point. The transmitting / receiving unit 312 also transmits the finalized monitoring area information to the server 50. The monitoring area information includes the coordinates of the vertices that define the determined monitoring area. These coordinates may be screen coordinates on the display unit of the first user terminal 10, or they may be position coordinates (latitude, longitude, etc.). When the server 50 receives the monitoring area information, it can store the latitude and longitude of each vertex, along with the identifier of the first user terminal 10 associated with the second user terminal 30, in the user information 522.

[0183] On the other hand, if it is determined in step S77 that no operation has been accepted for the save icon 812, the process proceeds to step S79.

[0184] In step S79, the operation control unit 313 visually highlights (e.g., blinks) the save icon 812 on the screen to indicate that it has not been saved, and determines whether or not there has been no operation on the editing screen for a predetermined period of time (e.g., 30 seconds).

[0185] If no operation is performed for a predetermined period of time, the process proceeds to step S78, where the operation control unit 313 saves the monitoring range, and the transmission / reception unit 312 transmits the monitoring range information to the server 50.

[0186] On the other hand, if it is determined in step S79 that an operation on the editing screen has occurred within a predetermined time, the process returns to step S71 and accepts a new touch stroke operation.

[0187] Note that the processing procedure shown in Figure 19 is illustrative and does not limit the content of this disclosure, and some parts of the processing procedure may be omitted or modified. For example, if it is determined in step S77 that the operation on the save icon 812 was not accepted, the process may return to step S71 without going through step S79 and accept a new touch stroke operation.

[0188] In this embodiment, the monitoring range can be set or modified by the user through multiple touch stroke operations. The user can repeatedly perform multiple touch stroke operations until the final shape of the monitoring range is determined. This allows the user to easily set even complex-shaped monitoring ranges.

[0189] The settings and modifications of the monitoring range of the second user terminal 30 were explained in Figures 18 and 19 above.

[0190] Next, we will describe the processes executed on the first user terminal 10 and the server 50 after the monitoring range of the second user terminal 30 has been set.

[0191] The first user terminal 10 can periodically transmit its location coordinates to the server 50 at regular intervals. The server 50 notifies the second user terminal 30 based on the relationship between the acquired location coordinates of the first user terminal 10 and the stored monitoring range.

[0192] When the monitoring area is set to a dangerous area, the server 50 can send a push notification to the second user terminal 30 when the location coordinates of the first user terminal 10 are within or entering the dangerous area. The server 50 also sends a trigger signal to the first user terminal 10 for the lamp to light up and a sound to play. Upon receiving the trigger signal, the first user terminal 10 can flash the display device 17 or play voice data (pre-configured voice of the second user or warning sound) sent from the server 50 through the speaker 16.

[0193] When the monitoring area is set to the living area, the server 50 can send a push notification to the second user terminal 30 when the location coordinates of the first user terminal 10 move outside the living area. The server 50 also sends a trigger signal to the first user terminal 10 for the lamp to light up and a sound to play. Upon receiving the trigger signal, the first user terminal 10 can make the display device 17 blink or play sound data (pre-configured voice or warning sound of the second user) sent from the server 50 through the speaker 16.

[0194] <Third Embodiment> Next, with reference to Figures 20 to 22, the information processing of System 1 according to the third embodiment of the present invention will be described. System 1 of this disclosure converts a message from the second user terminal 30 to the first user terminal 10, which is the target of monitoring, into synthesized speech according to the profile of the second user and sends it.

[0195] When multiple second users (e.g., father, mother, grandparents) are associated with one first user (e.g., a child), if the audio data corresponding to a message is played with the same voice quality, the first user cannot identify who the message is from. By varying the voice quality based on the text entered in the second user terminal 30 on the first user terminal 10 according to the second user's profile, the first user can easily identify who the message is from. Furthermore, when sending a text to the first user, it is cumbersome for the second user to manually select the voice quality of the text-to-speech conversion. According to this disclosure, the voice quality of the text-based speech is automatically selected according to the message sender's profile.

[0196] Figure 20 is a flowchart 2000 showing the procedure for voice quality conversion processing performed in system 1 according to the third embodiment of the present invention.

[0197] First, in step S91, the operation control unit 313 obtains the text entered by the second user from the second user terminal 30 and the identification number assigned to the second user, and the transmission / reception unit 312 transmits the obtained text and identification number to the server 50.

[0198] Next, in step S92, the server processing unit 512 extracts a profile corresponding to the identification number of the second user from the second user information 522B. The profile is not particularly limited, but includes at least one of the second user's gender, age, and address.

[0199] Figure 21 shows an example of the data structure 2100 of the second user information 522B according to a third embodiment of the present invention. The second user information 522B includes, but is not limited to, the identification number of the first user, the identification number of the second user, the gender, age, and address of the second user as data items. Multiple second users are associated with the first user.

[0200] Furthermore, in step S92, the server processing unit 512 reads the voice quality information 524 and selects a synthesized voice model corresponding to the extracted second user profile. The selected synthesized voice model is chosen according to at least one of the second user's gender and age included in the profile.

[0201] Figure 22 shows an example of the data structure 2200 of voice quality information 524 according to one embodiment of the present invention. Voice quality information 524 includes gender and age as data items, and stores different voice quality data associated with each of these items. If the gender of the second user is not registered, a synthesized voice model synthesized with a neutral voice may be selected. Voice quality information 524 may also include region as a data item.

[0202] Next, in step S93, the server processing unit 512 converts the text acquired in step S72 into synthesized speech using the selected synthesized speech model, and outputs the converted synthesized speech to the first user terminal 10.

[0203] Next, in step S94, when the transmitting / receiving unit 112 acquires the converted synthesized voice, the operation control unit 113 notifies the first user terminal 10 of the message reception. This notification includes at least one of the following: lighting up the display device 17 of the first user terminal 10, vibrating the first user terminal 10, or outputting sound from the speaker 16 of the first user terminal 10.

[0204] Next, in step S95, the first user terminal 10 plays synthesized speech from the speaker 16. The synthesized speech may be played immediately after the first user terminal 10 receives the synthesized speech, or it may be played in response to a press operation on the input device 20 of the first user terminal 10, such as a short press operation.

[0205] While embodiments of the present invention have been described above, the embodiments of the invention described above are for the purpose of facilitating understanding of the present invention and do not limit it. The present invention can be modified and improved without departing from its spirit, and of course, the present invention includes equivalents thereof. Furthermore, any combination of embodiments and modifications is possible to the extent that at least some of the above-mentioned problems can be solved or at least some of the effects can be achieved, and any combination or omission of each component described in the claims and specification is possible. [Explanation of Symbols]

[0206] 1: System 2: Network 10: First user terminal 11, 31, 51: Processor 12, 32, 52: Storage device 13, 33, 53: Communication IF 14, 34: Input / Output Interface 15, 35: Microphone 16, 36: Speakers 17:Display device 18: Notification device 19: Location acquisition device 20: Input device 30: Second user terminal 37: Touchscreen 38: Input section 39: Display section 50: Server 110: Control Unit 111: Operation Reception Section 112: Transceiver Unit 113: Operation Control Unit 120: Storage section 121: Program 122: User Information 123: Audio data 212: Server Processing Unit 214: Display Control Unit 310: Control Unit 311: Operation Reception Section 312: Transceiver Unit 313: Operation Control Unit 314: Display Control Unit 320: Storage section 321: Program 322: User Information 323: Activity history information 510: Control Unit 511: Transceiver Unit 512: Server Processing Unit 513: Data Management Department 520: Storage section 521: Management Program 522: User Information 522A: First User Information 522B: Second User Information 523: Activity history information 524: Voice quality information 601: Playback time 602: Text display 603: Play icon 604: Event occurrence time 606: Current location pin 607: Travel Route 608: Activity history icon 609: Current location icon 801: Trash can icon 802: Cancel icon 803: Eraser icon 804: Address search icon 805: Current location icon 806: Movement mode icon 807: Drawing mode icon 808: Fill icon 809, 1002: Origin 810, 1003: Terminal 811 :Trajectory 812: Save icon 901: Polygon region 1101 :Area q: distance r: radius

Claims

1. A terminal for the person being monitored, carried by the person being monitored. Upon receiving an operation on the monitoring terminal, the system starts recording sound and acquires location information of the monitoring terminal for a predetermined period of time. After the predetermined period has elapsed, the recording will be stopped and the acquisition of location information of the monitored person's terminal will be stopped. The audio data recorded over the predetermined period and the location information of the monitored person's terminal acquired over the predetermined period are transmitted to the server. A terminal for the person being monitored, configured to perform the following actions.

2. The monitored terminal according to claim 1, wherein the operation of the monitored terminal is a single press operation on the notification device of the monitored terminal.

3. The location information of the monitored person's terminal acquired during the predetermined period is acquired at a shorter interval than the location information acquired outside the predetermined period, according to claim 1.

4. The monitoring terminal according to claim 2, further configured to cause the display device of the monitoring terminal to emit light over the predetermined period of time.

5. The aforementioned predetermined period is set in a monitor terminal carried by the monitor of the person being monitored, as described in claim 1.

6. A system comprising a terminal for the person being monitored, carried by the person being monitored, and a server, The aforementioned terminal for the person being monitored is When an operation is received on the monitoring terminal, sound recording begins and location information of the monitoring terminal is acquired over a predetermined period of time. After the predetermined period has elapsed, the recording will be stopped and the acquisition of location information from the monitored person's terminal will be stopped. A system configured to transmit to the server sound data recorded over the predetermined period and location information of the monitored person's terminal acquired over the predetermined period.

7. The server notifies the monitor's terminal, carried by the monitor, of a link to the received sound data and the location information of the monitor's terminal. The system according to claim 6, configured to perform the following:

8. The system according to claim 7, wherein when the monitoring terminal receives the notification, it displays a link to the sound data and address information corresponding to the location information of the monitored terminal on the display unit of the monitoring terminal.

9. A method executed by the processor of a device carried by the person being monitored, When an operation is received on the monitoring device carried by the person being monitored, sound recording is started and location information of the monitoring device is acquired for a predetermined period of time. After the predetermined period has elapsed, the recording will be stopped and the acquisition of location information from the monitored person's terminal will be stopped. The audio data recorded over the predetermined period and the location information of the monitored person's terminal acquired over the predetermined period are transmitted to the server. Methods that include...

10. A program, which is executed by a terminal processor for the person being monitored, carried by the person being monitored. When an operation is received on the monitoring device carried by the person being monitored, sound recording begins and location information of the monitoring device is acquired for a predetermined period of time. After the predetermined period has elapsed, the recording will be stopped and the acquisition of location information from the monitored person's terminal will be stopped. A program that transmits to a server sound data recorded over the predetermined period and location information of the monitored person's terminal acquired over the predetermined period.

Images