Necklace-type terminal and data processing system

Abstract

This invention provides a necklace-type terminal and data processing system that improves user convenience while traveling. [Solution] The necklace-type terminal includes a camera that photographs the area around the wearer, a sensor that detects the wearer's biometric data, and a collection unit that collects at least one output from a microphone, and an execution unit that uses the collected output to perform convenience processing to improve the convenience of the wearer when they are traveling.

Description

Technical Field

[0001] The technology of the present disclosure relates to a necklace-type terminal and a data processing system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, there is room for improvement regarding the improvement of convenience during the user's travel.

Means for Solving the Problems

[0005] A first aspect according to the technology of the present disclosure is a necklace-type terminal including a collection unit that collects at least one output of a camera that photographs the surroundings of a wearer, a sensor that detects biometric data of the wearer, and a microphone, and an execution unit that executes a convenience improvement process for improving convenience when the wearer is traveling, using the collected output.

[0006] A second aspect of the technology of the present disclosure is a necklace-type terminal of the first aspect, wherein the convenience processing is a process that performs at least one of the following functions: a local information presentation function which, when the collection unit collects the output of the camera, identifies a travel destination using the captured image obtained by the camera and acquires and presents local information of the identified travel destination; a translation and playback function which, when the collection unit collects the output of the microphone, translates the language of the speech of a local person at the travel destination picked up by the microphone into the language spoken by the wearer and plays it back; and a notification function which, when the collection unit collects the output of the sensor, notifies an emergency contact when the biometric data detected by the sensor indicates an abnormality in the wearer.

[0007] A third aspect of the technology of this disclosure is a necklace-type terminal of the second aspect, wherein when the execution unit performs the local information presentation function as the convenience processing, the collection unit further collects the output of the location detection unit that detects the wearer's location, and the local information presentation function is a function that identifies the travel destination using the captured image and the location detected by the location detection unit.

[0008] A fourth aspect of the technology of this disclosure is a necklace-type terminal of the second or third aspect, wherein the translation and playback function is at least one of the following functions: a function that translates and plays back the language spoken by a local person at the travel destination into the language spoken by the wearer, and a function that translates and plays back the language spoken by the wearer, as picked up by the microphone, into the language of the local person.

[0009] A fifth aspect of the technology of this disclosure is a necklace-type terminal according to the first or second aspect, further comprising a communication unit that transmits the output collected by the collection unit to a data processing unit, wherein the execution unit performs the convenience processing using the response from the data processing unit corresponding to the transmitted output.

[0010] A sixth aspect of the technology of the present disclosure is a data processing system including a necklace-type terminal of the fifth aspect and a data processing device, the data processing device including an input unit that receives the output transmitted by the communication unit, a processing unit that inputs a prompt including information corresponding to the received output to a data generation model and obtains a response for executing the convenience processing using the output of the data generation model, and an output unit that outputs the obtained response to the necklace-type terminal.

[0011] A seventh aspect of the technology of this disclosure is a data processing system of the sixth aspect, wherein the processing unit directly notifies an emergency contact as a response when the input unit receives the biometric data. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system. [Figure 2] This is a conceptual diagram showing an example of the main functions of a data processing device and a necklace-type terminal. [Figure 3] This is a side view showing the configuration of a necklace-type terminal. [Figure 4] This is a top view showing the configuration of a necklace-type terminal. [Figure 5] The functional configuration of the control unit of the necklace-type terminal according to the first embodiment is schematically shown. [Figure 6] The functional configuration of a specific processing unit of a data processing device is shown in general terms. [Figure 7] An example of the operation flow of a specific process by the data processing device according to the first embodiment is schematically shown. [Figure 8] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 9] The functional configuration of the control unit of the necklace-type terminal according to the second embodiment is schematically shown. [Figure 10] This is a sequence diagram showing the overall processing flow by the data processing system according to the second embodiment. [Figure 11] An example of the operation flow of the processing by the necklace-type terminal according to the second embodiment is schematically shown. [Figure 12] An example of the operation flow of the specific processing by the data processing device according to the second embodiment is schematically shown.

Mode for Carrying Out the Invention

[0013] Hereinafter, an example of an embodiment of a data processing device, a data processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be described.

[0015] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of the arithmetic unit include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), or an APU (Accelerated Processing Unit).

[0016] In the following embodiments, the signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the tagged storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0018] In the following embodiments, the tagged communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0020] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0021] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a necklace-type terminal 14. An example of the data processing device 12 is a server. In this embodiment, the data processing device 12 is an example of the "data processing device" according to the technology of the present disclosure, and the necklace-type terminal 14 is an example of the "necklace-type terminal" according to the technology of the present disclosure.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The necklace-type terminal 14 includes a computer 36, a microphone 38, a sensor 39, a speaker 40, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 38, sensor 39, speaker 40, camera 42, and communication interface 44 are also connected to the bus 52.

[0024] The user 20 wearing the necklace-type terminal 14 may be, for example, a patient whose health condition is being diagnosed, or a regular user.

[0025] The microphone 38 picks up the voice emitted by the user 20, who is wearing the necklace-type terminal 14, as well as sounds around the user 20. The microphone 38 also receives instructions from the user 20 by receiving the voice emitted by the user 20. The microphone 38 captures the voice emitted by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 40 outputs audio according to the instructions from the processor 46. The speaker 40 is, for example, a directional speaker and outputs audio towards the user 20's ears.

[0026] Sensor 39 is a sensor that detects biometric data of the user 20, who is wearing the necklace-type terminal 14. For example, sensor 39 may be a heart rate sensor or a blood oxygen sensor.

[0027] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the necklace-type terminal 14.

[0030] As shown in Figure 2, in the data processing device 12, specific processing is performed by the processor 28. The storage 32 stores a specific processing program 56. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58. The data generation model 58 is used by the specific processing unit 290. The storage 32 also includes a data storage unit 54.

[0032] In the necklace-type terminal 14, data acquisition processing is performed by the processor 46. The storage 50 stores the data acquisition program 60. The processor 46 reads the data acquisition program 60 from the storage 50 and executes the read data acquisition program 60 on the RAM 48. The data acquisition processing is realized by the processor 46 operating as a control unit 46A according to the data acquisition program 60 executed on the RAM 48.

[0033] As shown in Figures 3 and 4, the necklace-type terminal 14 includes multiple microphones 38, multiple sensors 39, multiple speakers 40, and multiple cameras 42. Figures 3 and 4 show an example where two microphones 38 are positioned in front of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two sensors 39 are positioned to the right and left of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two speakers 40 are positioned to the right rear and left rear of the user 20 when the user 20 wears the necklace-type terminal 14. They also show an example where two cameras 42 are positioned to the right front and left front of the user 20 when the user 20 wears the necklace-type terminal 14. Finally, they show an example where two sensors 39 are positioned inside the necklace-type terminal 14 so as to contact the user 20's neck when the user 20 wears the necklace-type terminal 14.

[0034] Next, we will explain the processing of the control unit 46A when the necklace-type terminal 14 performs data collection processing to collect data.

[0035] In this embodiment, the data collection process collects the user's biometric data in real time. Furthermore, it collects not only biometric data but also all surrounding environment data. This makes it possible to detect early signs of conditions such as Alzheimer's disease and dementia. It also allows for monitoring of the user's health status (e.g., heart disease).

[0036] As shown in Figure 5, the control unit 46A includes a data acquisition unit 100 and a communication unit 102.

[0037] The data acquisition unit 100 collects the outputs of the microphone 38, sensor 39, and camera 42, respectively.

[0038] The communication unit 102 transmits the outputs of the microphone 38, sensor 39, and camera 42, which are collected by the data acquisition unit 100, to the data processing unit 12.

[0039] Next, we will describe the processing of the specific processing unit 290 when the data processing device 12 performs specific processing to obtain a response corresponding to a user utterance.

[0040] In the specific processing in this embodiment, a response corresponding to the user utterance picked up by the microphone 38 of the necklace-type terminal 14 is obtained using the data generation model 58.

[0041] As shown in Figure 6, the specific processing unit 290 includes an input unit 292, a processing unit 294, and an output unit 296.

[0042] The input unit 292 stores the outputs of the microphone 38, sensor 39, and camera 42, respectively, received from the necklace-type terminal 14, in the data storage unit 54.

[0043] The input unit 292 acquires user utterances received by the necklace-type terminal 14. Specifically, it acquires user utterances picked up by the microphone 38 of the necklace-type terminal 14.

[0044] The processing unit 294 performs specific processing using the data generation model 58. Specifically, it inputs a prompt including user utterances to the data generation model 58 and obtains a generation result. At this time, the outputs of the sensor 39 and camera 42 collected by the data acquisition unit 100 may also be included in the prompt.

[0045] The output unit 296 transmits the result of the specific processing to the necklace-type terminal 14. In the necklace-type terminal 14, the control unit 46A causes the speaker 40 to output the result of the specific processing. In this way, a response corresponding to the user utterance picked up by the microphone 38 is output to the user 20 by the speaker 40. The microphone 38 further acquires the user utterance in response to the result of the specific processing. The control unit 46A transmits the audio data indicating the user utterance acquired by the microphone 38 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the user utterance.

[0046] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0047] The outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 54 are used, for example, to diagnose the health status of user 20. In this case, the outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 54 may be transmitted to a terminal on the medical institution's side. Alternatively, the data processing device 12 may analyze the outputs of the microphone 38, sensor 39, and camera 42 stored in the data storage unit 54 to diagnose the health status of user 20.

[0048] Next, the operation of the data processing system 10 will be explained.

[0049] First, let's explain an example of the data collection process flow.

[0050] When user 20 is wearing the necklace-type terminal 14, the data acquisition unit 100 sequentially collects the outputs of the microphone 38, sensor 39, and camera 42. The communication unit 102 sequentially transmits the outputs of the microphone 38, sensor 39, and camera 42 collected by the data acquisition unit 100 to the data processing unit 12.

[0051] Next, an example of the flow of a specific process will be explained with reference to Figure 7. Here, the input unit 292 of the data processing device 12 sequentially acquires the outputs of the microphone 38, sensor 39, and camera 42 received from the necklace-type terminal 14 and stores them in the data storage unit 54.

[0052] In step S300, the processing unit 294 determines whether a predetermined trigger condition is met. Specifically, the trigger condition may be that the user utterance picked up by the microphone 38 contains a specific word (for example, the name of the agent installed in the necklace-type terminal 14) or a phrase (for example, "Hi! XX" (where XX is the name of the agent)).

[0053] If the trigger condition is met in step S300 (step S300; Yes), the data processing system 10 proceeds to step S301. On the other hand, if the trigger condition is not met in step S300 (step S300; No), the data processing system 10 terminates the specific processing.

[0054] In step S301, the processing unit 294 generates a prompt by adding an instruction to obtain the result of a specific process to the text representing the user utterance picked up by the microphone 38.

[0055] For example, a prompt such as "The user is saying the following: XXX. Please respond as the agent." (XXX is the user's utterance.) can be generated. Alternatively, the outputs of sensor 39 and camera 42 can be added to the prompt to generate a prompt such as "This is biometric data representing the user's heart rate and image data representing the user's surroundings. The user is also saying the following: XXX. Please respond as the agent." (XXX is the user's utterance.)

[0056] In step S303, the processing unit 294 inputs the generated prompt to the data generation model 58 and obtains the result of a specific process based on the output of the data generation model 58.

[0057] In step S304, the output unit 296 outputs the result of the specific processing to the necklace-type terminal 14 and terminates the specific processing.

[0058] [Second Embodiment] In this embodiment, we will describe an example of an embodiment in which the user 20, who is wearing the necklace-type terminal 14, is a traveler who is traveling abroad.

[0059] Modern travel demands convenience and safety, while also requiring solutions to the risks and unforeseen circumstances travelers may face. In particular, there is a need to mitigate the stress caused by language barriers and a lack of local information that independent travelers often encounter in new places.

[0060] Traditional travel support systems often provide local information using devices such as smartphones and tablets. However, these devices are cumbersome to operate, and translation apps are necessary to overcome language barriers. Furthermore, emergency response is limited, making it difficult for travelers to cope with unforeseen circumstances.

[0061] Therefore, the data processing system 10 according to this embodiment aims to solve these problems, overcome the language barrier faced by travelers in real time, easily acquire local information, and enable rapid response in emergencies.

[0062] Figure 8 shows an example of the configuration of the data processing system 10 according to the second embodiment. As shown in Figure 8, the data processing system 10 according to this embodiment differs from the data processing system 10 according to the first embodiment in that an emergency contact device 16 is added.

[0063] The emergency contact device 16 according to this embodiment is a device owned by the person to be contacted (in this embodiment, the user's family) if the user 20 encounters an unforeseen situation while traveling (in this embodiment, the user 20 is in an abnormal state (feeling unwell)).

[0064] In this embodiment, the above-mentioned abnormality is defined as the user 20's stress level, obtained from the biological data output from the sensor 39, exceeding a predetermined level. However, the embodiment is not limited to this form. For example, the above-mentioned abnormality may be defined as the user 20's blood oxygen concentration, obtained from the biological data, falling below a predetermined standard value (e.g., 95%).

[0065] Furthermore, in this embodiment, a portable device such as a smartphone or tablet is used as the emergency contact device 16, but a stationary device may be used instead of the portable device. Also, the above-mentioned unforeseen circumstances are not limited to abnormalities in the user 20, but may also include unexpected accidents or troubles such as the user 20 losing their money. In addition, the persons to be contacted are not limited to the user 20's family, but may also include medical institutions such as hospitals and medical centers, government offices such as police stations and fire stations, and public facilities such as embassies.

[0066] The emergency contact device 16 is connected to the network 54, and the data processing device 12 can send various notifications to the emergency contact device 16 via the network 54.

[0067] As shown in Figure 9, the control unit 46A according to this embodiment includes a data acquisition unit 100, a communication unit 102, and an execution unit 104.

[0068] The data acquisition unit 100 collects the outputs of the microphone 38, sensor 39, and camera 42, respectively.

[0069] The execution unit 104 uses the output collected by the data collection unit 100 to perform convenience processing (hereinafter simply referred to as "convenience processing") to improve convenience when the user 20 is traveling.

[0070] In the data processing system 10 according to this embodiment, as a convenience processing step, it performs a local information presentation function that identifies a travel destination using images captured by the camera 42, and acquires and presents local information such as tourist attractions and facilities at the identified travel destination. For example, by inputting a prompt into the data generation model 58 instructing it to identify a travel destination using images captured by the camera 42 and generate local information such as tourist attractions and facilities at the identified travel destination, the local information is acquired and presented to the user.

[0071] Furthermore, the data processing system 10 according to this embodiment performs a translation and playback function as a convenience process, which translates the language of the speech of the locals at the travel destination, picked up by the microphone 38, into the language spoken by the user 20, and plays it back. For example, by inputting a prompt to the data generation model 58 instructing it to translate the language of the speech of the locals at the travel destination, picked up by the microphone 38, into the language spoken by the user 20, and play it back, audio data representing the translation result is obtained and presented to the user.

[0072] Furthermore, in the data processing system 10 according to this embodiment, as a convenience process, if the biometric data detected by the sensor 39 indicates an abnormality in the user 20, a notification function is executed to notify the emergency contact of the user 20's current location and status (in this embodiment, biometric data), etc. For example, by inputting a prompt to the data generation model 58 instructing it to determine whether or not the biometric data detected by the sensor 39 indicates an abnormality in the user 20, the determination result of whether or not the user 20 is abnormal is obtained, and if it is determined that the user 20 is abnormal, the emergency contact device 16 is notified that the user 20 is abnormal.

[0073] As described above, in the data processing system 10 according to this embodiment, the convenience processing is defined as a process that executes three types of functions: a local information presentation function, a translation and playback function, and a notification function. However, the system is not limited to this configuration. For example, the convenience processing may be defined as a process that executes one or a combination of two of these three types of functions. In this case, the output required for the functions that are not applied does not need to be collected by the data collection unit 100.

[0074] In particular, the data processing system 10 according to this embodiment has, as a translation and playback function, a function that translates and plays back the language spoken by locals at the travel destination into the language spoken by the user 20 (hereinafter referred to as the "first translation function"), as well as a function that translates and plays back the language spoken by the user 20, picked up by the microphone 38, into the language spoken by locals at the travel destination (hereinafter referred to as the "second translation function"). However, the system is not limited to this configuration, and may have a configuration in which only one of the first translation function and the second translation function is used as the translation and playback function.

[0075] In this embodiment of the data processing system 10, similar to the first embodiment, the communication unit 102 transmits the outputs of the microphone 38, sensor 39, and camera 42, collected by the data acquisition unit 100, to the data processing unit 12. The execution unit 104 in this embodiment then performs convenience processing using the response from the data processing unit 12 corresponding to the output transmitted by the communication unit 102. That is, in this embodiment of the data processing system 10, the data processing unit 12 acquires the information necessary for convenience processing and transmits it to the necklace-type terminal 14 as the response.

[0076] Next, we will describe the processing of the specific processing unit 290 when the data processing device 12 performs specific processing to obtain a response corresponding to the above output.

[0077] In the specific processing in this embodiment, the response corresponding to the above output is obtained using the data generation model 58.

[0078] Therefore, the specific processing unit 290 according to this embodiment also includes an input unit 292, a processing unit 294, and an output unit 296, as shown in Figure 6.

[0079] The input unit 292 stores the outputs of the microphone 38, sensor 39, and camera 42, respectively, received from the necklace-type terminal 14, in the data storage unit 54.

[0080] The input unit 292 acquires the above output received by the necklace-type terminal 14. Specifically, it acquires the output from the microphone 38, sensor 39, and camera 42 of the necklace-type terminal 14.

[0081] The processing unit 294 performs specific processing using the data generation model 58. Specifically, it inputs a prompt containing information corresponding to the received output to the data generation model 58 and obtains a generation result, i.e., a response to execute the convenience processing described above.

[0082] The output unit 296 transmits the result of the specific processing as the above response to the necklace-type terminal 14. The execution unit 104 in the necklace-type terminal 14 uses the result of the specific processing to perform convenience processing.

[0083] In this embodiment, when the input unit 292 receives biometric data, the processing unit 294 directly notifies the emergency contact as a response. That is, in the data processing system 10 according to this embodiment, instead of the necklace-type terminal 14 notifying the emergency contact via a notification function, the reception of biometric data from the necklace-type terminal 14 to the data processing unit 12 triggers a notification from the data processing unit 12 directly to the emergency contact if an abnormality occurs in the user 20. However, the system is not limited to this form, and the necklace-type terminal 14 may also use the response received from the data processing unit 12 to notify the emergency contact.

[0084] Figure 10 shows a sequence diagram illustrating the overall processing flow by the data processing system 10 according to this embodiment.

[0085] As shown in Figure 10, the necklace-type terminal 14 transmits the above output to the data processing device 12 when the user 20 is traveling. In response, the data processing device 12 generates the above response by inputting a prompt containing the received output to the data generation model 58 and transmits it to the accessing necklace-type terminal 14. At this time, if the data processing device 12 infers that the user 20 is abnormal from the biometric data contained in the received output, it notifies the emergency contact device 16 that the user 20 is abnormal.

[0086] The necklace-type terminal 14 uses the received response to perform the local information presentation function and translation playback function described above via the speaker 40.

[0087] Next, the operation of the data processing system 10 will be explained.

[0088] First, with reference to Figure 11, an example of the processing flow performed in the necklace-type terminal 14 according to this embodiment will be described. Note that the processing shown in Figure 11 starts when the user 20 puts on the necklace-type terminal 14, but it is not limited to this form. For example, the processing shown in Figure 11 may start when the user 20 gives a command to start execution by speaking, or when the power switch (not shown) of the necklace-type terminal 14 is turned on.

[0089] In step S400, the data collection unit 100 determines whether or not user 20 is traveling at this point. If the determination is negative (step S400; No), the process ends; however, if the determination is positive (step S400; Yes), the process proceeds to step S402. In this embodiment, the determination of whether or not user 20 is traveling is made by inputting a prompt instructing the data generation model 58 to determine whether or not the image shown in the image data obtained by the camera 42 shows an image of a foreign landscape, along with the image data itself. However, the system is not limited to this configuration. For example, the necklace-type terminal 14 may be equipped with a location detection unit (e.g., a GPS (Global Positioning System) chip) to identify its own location, and the location detected by the location detection unit may also be used to determine whether or not user 20 is traveling. Alternatively, the determination of whether or not user 20 is traveling may be made using only the location detected by the location detection unit.

[0090] In step S402, the data acquisition unit 100 collects the above outputs. Specifically, it collects audio data from the microphone 38, biometric data from the sensor 39, and image data from the camera 42.

[0091] In step S404, the communication unit 102 transmits the collected output to the data processing unit 12. Upon receiving the output, the data processing unit 12 transmits the response to the accessing necklace-type terminal 14 as described above.

[0092] Therefore, in step S406, the communication unit 102 waits until it receives the above response, and in step S408, the execution unit 104 uses the received response to perform the output of the above-described local information presentation function and translation playback function as sound from the speaker 40. Thus, in this embodiment, the local information presentation function and translation playback function are performed using the speaker 40, but the system is not limited to this configuration. For example, the output of the local information presentation function and translation playback function may be performed by displaying it on the display unit of a mobile terminal held by the user 20.

[0093] In step S410, the data acquisition unit 100 determines whether a predetermined termination timing has arrived. If the determination is negative (step S410; No), the process returns to step S402; however, if the determination is positive (step S410; Yes), the process terminates. In this embodiment, the termination timing is set to the timing when the user 20 removes the necklace-type terminal 14, but the system is not limited to this configuration. For example, the termination timing could be set to the timing when the user 20 gives a verbal instruction to terminate execution, or when the power switch (not shown) of the necklace-type terminal 14 is opened.

[0094] Next, an example of the flow of a specific process performed in the data processing device 12 according to this embodiment will be described with reference to Figure 12. Here, the input unit 292 of the data processing device 12 sequentially acquires the outputs of the microphone 38, sensor 39, and camera 42 received from the necklace-type terminal 14 and stores them in the data storage unit 54.

[0095] In step S500, the processing unit 294 determines whether a predetermined trigger condition is met. Specifically, the trigger condition may be that the user utterance picked up by the microphone 38 contains a specific word (for example, the name of the agent installed in the necklace-type terminal 14) or a phrase (for example, "Hi! XX" (where XX is the name of the agent)).

[0096] If the trigger condition is met in step S500 (step S500; Yes), proceed to step S502. On the other hand, if the trigger condition is not met in step S500 (step S500; No), terminate the specific process.

[0097] In step S502, the processing unit 294 generates a prompt by adding an instruction to obtain the result of a specific process to the text representing the speech indicated by the audio data picked up by the microphone 38, the biometric data detected by the sensor 39, and the image data captured by the camera 42.

[0098] For example, it can generate a prompt such as: "This is biometric data representing the user's heart rate and image data representing the user's surroundings. The user or a local person is speaking as follows: XXX. If the speech is from the user, translate it into the local language; if the speech is from a local person, translate it into the language the user uses; collect local information from the image data; and if the biometric data suggests a user abnormality, prompt the system to notify emergency contacts." (XXX represents speech from the user or a local person.)

[0099] As described above, this embodiment applies all of biometric data, image data, and speech, but it is not limited to this, and it is also possible to apply only one type or a combination of two of these types.

[0100] In step S504, the processing unit 294 inputs the generated prompt to the data generation model 58 and obtains the result of a specific process based on the output of the data generation model 58. Thus, in this embodiment, responses for all functions of the local information presentation function, translation playback function, and notification function are generated by a single data generation model 58, but the system is not limited to this configuration. For example, responses for each of these functions may be generated using separately prepared data generation models.

[0101] In step S506, the processing unit 294 determines whether the acquired result of the specific processing includes information prompting notification to emergency contacts. If the determination is positive (step S506; Yes), the unit proceeds to step S508.

[0102] In step S508, the processing unit 294 notifies the emergency contact device 16 corresponding to the user 20 that the user 20 is abnormal.

[0103] In step S510, the output unit 296 outputs (transmits) the translation result obtained from the data generation model 58 and the local information, along with instruction information indicating that the translated text and local information indicated by the translation result should be reproduced, to the accessing necklace-type terminal 14 as a response, and terminates the specific processing.

[0104] On the other hand, if the result in step S506 is negative (step S506; No), the process proceeds to step S512, where the output unit 296 outputs (transmits) the result of the specific processing to the necklace-type terminal 14, and the specific processing is terminated.

[0105] In this embodiment, the travel destination is identified using only the captured image in the local information display function, but this is not the only possible configuration. For example, the travel destination may be identified using both the captured image and the location detected by the location detection unit described above. Alternatively, the travel destination may be identified using only the location detected by the location detection unit.

[0106] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0107] In the embodiments described above, examples were given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, involving multiple computers, including computer 22.

[0108] In the embodiments described above, examples were given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0109] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0110] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0111] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0112] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0113] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0114] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0115] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0116] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference. [Explanation of symbols]

[0117] 10 Data Processing Systems 12 Data Processing Devices 14 Necklace-type terminal 16. Emergency Contact Device 38 Microphones 39 Sensors 40 speakers 42 cameras 46A Control Unit 100 Data Acquisition Unit 102 Communications Department 104 Execution Unit 290 Specific Processing Unit 292 Input section 294 Processing Unit 296 Output section< / url:>

Claims

1. A camera that photographs the area around the wearer, a sensor that detects the wearer's biometric data, and a collection unit that collects at least one output from a microphone, An execution unit that uses the collected output to perform convenience processing to improve convenience when the wearer is traveling, A necklace-type terminal that includes this device.

2. The aforementioned convenience processing is, When the collection unit collects the output of the camera, it uses the captured image obtained by the camera to identify the travel destination, and acquires and presents local information about the identified travel destination. When the collection unit collects the output of the microphone, it includes a translation and playback function that translates the language of the speech of the locals at the travel destination picked up by the microphone into the language spoken by the wearer and plays it back. When the collection unit collects the output of the sensor, a notification function is provided to notify an emergency contact when the biometric data detected by the sensor indicates an abnormality in the wearer. This is a process that performs at least one of the following functions: The necklace-type terminal according to claim 1.

3. When the execution unit performs the local information presentation function as the convenience processing, The collection unit further collects the output of the position detection unit that detects the position of the wearer, The aforementioned local information presentation function is a function that identifies the travel destination using the captured image and the location detected by the location detection unit. The necklace-type terminal according to claim 2.

4. The aforementioned translation and playback function is at least one of the following functions: a function that translates and plays back the language spoken by a local person at the travel destination into the language spoken by the wearer; and a function that translates and plays back the language spoken by the wearer, as picked up by the microphone, into the language of the local person. The necklace-type terminal according to claim 2 or claim 3.

5. The system further includes a communication unit that transmits the output collected by the collection unit to a data processing unit, The execution unit executes the convenience processing using the response from the data processing device corresponding to the transmitted output. The necklace-type terminal according to claim 1 or claim 2.

6. The necklace-type terminal described in claim 5, A data processing system including a data processing device, The aforementioned data processing device is An input unit that receives the output transmitted by the communication unit, A processing unit that inputs a prompt containing information corresponding to the received output to a data generation model, and uses the output of the data generation model to obtain a response for executing the convenience processing, An output unit that outputs the acquired response to the necklace-type terminal, A data processing system that includes this.

7. When the input unit receives the biometric data, the processing unit directly notifies the emergency contact as a response. The data processing system according to claim 6.

Images