System

Abstract

A system is provided.SOLUTION: This system is provided with a server means for acquiring data from a database according to contents selected by a user, and for providing the data to a terminal, a terminal means for allowing the user to select the contents, and for transmitting a request to the server, and a terminal means for reproducing and displaying the data provided from the server.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a need for systems that can brighten the lives of the silver generation and reduce the burden on caregivers. Conventional systems lack a centralized location for content aimed at seniors, making them difficult for seniors to use and also making it difficult to reduce the burden on caregivers. Specifically, a single system cannot provide a wide variety of content, such as viewing old videos and songs, health exercises, restoring photos and videos, visiting past locations in the metaverse, writing wills, and proposing new purposes in life. This means that each system fails to meet their needs. Furthermore, the system is difficult to operate, making it difficult for seniors to operate it themselves, increasing the burden on caregivers. [Means for solving the problem]

[0005] To solve the above problems, this invention provides a system including a server means for retrieving data from a database according to content selected by a user and providing it to a terminal, a terminal means for allowing the user to select content and send a request to the server, and a terminal means for playing and displaying the data provided by the server. This system allows elderly people to easily enjoy old videos and songs, practice healthy exercises, restore old photos and videos, and visit virtual spaces in the metaverse. Furthermore, this system also allows users to write wills and propose new purposes in life. This invention allows elderly people to have fun and reduces the burden on caregivers.

[0006] "Users" refer to the elderly people and their caregivers who use the system.

[0007] A "server" is a computer system that retrieves data from a database according to content selected by a user and provides it to a terminal.

[0008] A "terminal" is a device that allows a user to select content, send a request to a server, and then play and display the data provided by the server.

[0009] "Content" refers to information and media such as old videos, songs, health exercises, restored photos and videos, metaverse spaces, tools for writing wills, and suggestions for finding meaning in life.

[0010] A "database" is an information accumulation system that stores various content and user information and allows a server to access and retrieve the necessary data.

[0011] "Restoration processing" is an image processing technique performed by the server to restore old photos and videos uploaded by users to a quality close to their original.

[0012] The "metaverse" is an environment that uses 3D virtual reality technology to allow users to visit places from the past in a virtual space.

[0013] "Video playback" is a function that visually displays video data received by the terminal from the server to the user. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention is a multi-functional content system for use by the elderly of the silver generation. This system consists of three main components: a server, a terminal, and a user.

[0036] server

[0037] The server provides the following functionality:

[0038] 1. Content management: Various content (videos, exercise videos, photo restoration, metaverse, will creation tool, and life purpose suggestions) is stored in a database and provided upon appropriate request.

[0039] 2. User authentication: When a user logs in through a terminal, the user is authenticated by checking against the authentication database.

[0040] 3. Data recovery: Photos and videos uploaded by users from their devices are recovered using image processing technology.

[0041] 4. Metaverse generation: Generate a virtual reality space and allow users to virtually visit places from the past.

[0042] Terminal

[0043] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[0044] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, metaverse, will creation, and purpose in life suggestions.

[0045] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[0046] 3. Data playback and display: Receives data sent from the server, plays back videos and exercise videos, and displays restored photos and virtual spaces.

[0047] 4. Interface provision: Provide users with the interface necessary for the will creation tool and life purpose suggestions, and send the input information to the server.

[0048] User

[0049] Users are seniors and their caregivers who:

[0050] 1. Login: Enter your account information into the terminal interface to log in to the system.

[0051] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[0052] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[0053] 4. Experience: Watch the playback video, try the exercise video, check the restored photos, visit the Metaverse space, write a will, and accept suggestions for a new purpose in life.

[0054] As a concrete example, let us consider a scenario in which a user watches old videos. First, the user logs in to the device and selects the "Watch Videos" menu from the home screen. The device sends a content request to the server. The server retrieves a video list from the database and sends it to the device. The device displays the list, and the user selects a video of interest. The server reads the selected video data from the database and sends it to the device. The device plays the video, and the user watches it.

[0055] In this way, this system provides an interface that is easy for users to operate, and the server manages and provides a variety of content, thereby enriching the lives of the elderly and reducing the burden on caregivers.

[0056] The processing flow will be explained below.

[0057] Video viewing process flow

[0058] Step 1:

[0059] The user enters account information into the terminal interface and presses the login button.

[0060] Step 2:

[0061] The terminal transmits the user's input information to the server.

[0062] Step 3:

[0063] The server compares the received information with the database and returns the authentication result to the terminal.

[0064] Step 4:

[0065] The device displays the authentication result to the user, and if successful, transitions to the home screen.

[0066] Step 5:

[0067] The user selects the "Watch Video" menu from the home screen.

[0068] Step 6:

[0069] The terminal sends a content request to the server.

[0070] Step 7:

[0071] The server retrieves a list of video content from the database and transmits it to the terminal.

[0072] Step 8:

[0073] The device displays a list of videos.

[0074] Step 9:

[0075] The user selects the video they want to watch from the list.

[0076] Step 10:

[0077] The terminal sends a request for details of the selected video to the server.

[0078] Step 11:

[0079] The server reads the video data from the database and sends it to the terminal.

[0080] Step 12:

[0081] The terminal receives the video data and starts playing it.

[0082] Step 13:

[0083] The user watches the video.

[0084] Photo and video restoration process flow

[0085] Step 1:

[0086] The user selects the "Photo / Video Recovery" menu from the home screen.

[0087] Step 2:

[0088] The terminal displays a file selection screen to the user.

[0089] Step 3:

[0090] The user selects the photo or video files they want to restore.

[0091] Step 4:

[0092] The device uploads the selected file to the server.

[0093] Step 5:

[0094] The server stores the received file in the database and executes the restoration process.

[0095] Step 6:

[0096] The server sends the results of the restoration process to the terminal.

[0097] Step 7:

[0098] The device will display the restored photos and videos.

[0099] Step 8:

[0100] The user reviews the restored content.

[0101] Metaverse Processing Flow

[0102] Step 1:

[0103] The user selects the "Metaverse" menu from the home screen.

[0104] Step 2:

[0105] The device sends a metaverse request to the server.

[0106] Step 3:

[0107] The server generates metaverse spatial data and sends it to the terminal.

[0108] Step 4:

[0109] The device displays the metaverse space and provides the user with a virtual reality experience.

[0110] Step 5:

[0111] A user visits past locations in the metaverse.

[0112] Process flow for writing a will

[0113] Step 1:

[0114] The user selects the "Write a Will" menu from the home screen.

[0115] Step 2:

[0116] The device displays a will creation tool.

[0117] Step 3:

[0118] The user inputs the contents of the will.

[0119] Step 4:

[0120] The terminal transmits the entered will data to the server.

[0121] Step 5:

[0122] The server stores the will data in a database and sends the confirmation results to the terminal.

[0123] Step 6:

[0124] The terminal displays the confirmation result to the user.

[0125] Purpose of Life Proposal Processing Flow

[0126] Step 1:

[0127] The user selects the "Life Purpose Suggestions" menu from the home screen.

[0128] Step 2:

[0129] The device displays a questionnaire about the user's interests and preferences.

[0130] Step 3:

[0131] The user answers the question and submits it.

[0132] Step 4:

[0133] The terminal transmits the user's response data to the server.

[0134] Step 5:

[0135] The server analyzes the data and suggests appropriate hobbies and activities.

[0136] Step 6:

[0137] The server sends the proposal results to the terminal.

[0138] Step 7:

[0139] The device displays the suggestions to the user.

[0140] These processing flows will realize a system that is easy for elderly people to use, allows them to enjoy a variety of content, and reduces the burden on caregivers.

[0141] Example 1

[0142] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0143] Conventional content systems for seniors were difficult for seniors to use because they were complicated to operate. Furthermore, there was no system that offered a variety of functions, such as image and video restoration, virtual reality space experiences, and tools for writing wills, all in one place. Furthermore, it was necessary to use separate systems and devices to access specific content, which caused confusion for seniors.

[0144] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0145] In this invention, the server includes an information processing device that retrieves data from a database according to content selected by the user and provides it to the terminal, means for the user to input login information and the information processing device compares the information with the authentication database for authentication, means for the user to upload photos or videos that the user wants to restore from the display device and the information processing device restores them using image processing technology, means for generating a virtual reality space that allows the user to visit past locations in a virtual reality space, and means for the information processing device to generate the virtual reality space and provide it to the display device. This makes it possible to provide a system that provides multifunctional content all at once and is easy for seniors to use.

[0146] An "information processing device" is a device that retrieves data from a database based on a request from a user, performs the specified operation or processing, and provides the results to a terminal.

[0147] The "display device" is a device that allows a user to input data and perform operations via an interface, and that reproduces and displays data transmitted from an information processing device.

[0148] "User authentication" is the process of authenticating a user when they log in to a system by checking the entered login information against an authentication database.

[0149] "Content" refers to digital information and data such as videos, exercise videos, photographs, virtual reality spaces, will-writing tools, and suggestions for purpose in life.

[0150] A "database" is a data storage system for managing and storing various content and user authentication information within the system.

[0151] "Image processing techniques" are techniques used to restore uploaded photos and videos, including noise reduction, resolution enhancement, and color correction.

[0152] "Virtual reality space" refers to a three-dimensional computer-simulated space created to allow users to virtually visit places from the past.

[0153] A "will creation tool" is an application that allows users to create and save their wishes and messages in digital form.

[0154] "Purpose in Life Suggestion" is a system or function that suggests new hobbies and activities to users.

[0155] MODE FOR CARRYING OUT THE INVENTION

[0156] This invention is a multi-functional content system for the elderly, consisting of three main components: a user, a terminal, and a server. This system allows elderly people to easily access various content and enrich their daily lives. Each component of this invention and its operation will be described in detail below.

[0157] 1. Server

[0158] The server plays a central role in the system and provides the following functions:

[0159] 1.1 Content Management

[0160] The server stores various content (videos, exercise videos, photo restoration, virtual reality space, will writing tool, and purpose-in-life suggestions) in a database and provides them in response to user requests. Specifically, the server uses a database management system (DBMS) to efficiently manage content data.

[0161] 1.2 User Authentication

[0162] When a user logs in via a terminal, the server authenticates them against an authentication database. This is achieved using security protocols (e.g., SSL / TLS) and multi-factor authentication technology. Specifically, a mechanism is adopted in which an authentication code is sent after the user enters their username and password.

[0163] 1.3 Data Recovery

[0164] The server uses image processing technology to restore photos and videos uploaded by users from their devices. Specific techniques used include image restoration models using deep learning (e.g., Generative Adversarial Networks, GANs). Using a high-performance GPU enables efficient data processing.

[0165] 1.4 Virtual reality space generation

[0166] The server generates a virtual reality space, allowing users to virtually visit places from the past. This is done using VR development platforms such as Unity and Unreal Engine. The generated virtual reality space data is sent to the device in real time.

[0167] 2. Terminal

[0168] The terminal provides an interface for the user to operate the system and has the following functions:

[0169] 2.1 Content Selection

[0170] It provides an interface for users to select the content they want from the menus of video, exercise videos, restoration requests, virtual reality space, will writing, and life purpose suggestions. Operation becomes even easier if the device is equipped with a touch screen or voice recognition function.

[0171] 2.2 Sending a Request

[0172] Based on the selected content, the device sends an appropriate request to the server. The request is sent using the HTTP / HTTPS protocol. For example, the request might say "The user has selected to watch a video."

[0173] 2.3 Data playback and display

[0174] The device receives the data (video, exercise video, restored photos, virtual reality space) sent from the server and plays and displays it. A device with a GPU is recommended for video playback and virtual reality display.

[0175] 2.4 Interface Provision

[0176] The system provides users with an interface for writing wills and suggesting purpose in life, and sends this to the server. The content entered by the user (e.g., "I want to find a new hobby") is sent to the server, and appropriate feedback is returned to the device.

[0177] 3. Users

[0178] The users are elderly people or their caregivers, and perform the following operations:

[0179] 3.1 Login

[0180] A user logs in to the system by entering account information (user name, password, etc.) on the login screen of the terminal. For example, the user enters the user name "tanaka123" and the password "password123."

[0181] 3.2 Content Use

[0182] Users can select the desired content from each menu and operate it through the device interface. Specifically, they can select the "Video Viewing" menu to watch old travel videos.

[0183] 3.3 Upload

[0184] Upload the photos or videos you want to restore from your device to the server. For example, upload "Photos from a family trip." After the server completes the restoration process, the restored data will be displayed on your device.

[0185] 3.4 Experience

[0186] Users can watch playback videos, practice exercise videos, view restored photos, and visit virtual reality spaces. They can also write wills and accept suggestions for new purposes in life. When using the virtual reality space, users put on a VR headset and explore virtual locations in real time.

[0187] Specific examples

[0188] Login: When a user attempts to log in by entering "tanaka123" and "password123", the server authenticates them and returns a success message.

[0189] Video viewing: When Tanaka selects "Video viewing," the device sends a request to the server, which returns a list of videos. Tanaka selects "Video of family trip" and watches it.

[0190] Photo restoration: When Tanaka tries to restore an old family photo by uploading it from his device, the server restores it and sends the photo back to his device, where Tanaka can view it.

[0191] Virtual reality experience: When Tanaka selects "Virtual reality space of his hometown," the server generates a virtual space, sends it to his device, and Tanaka experiences it.

[0192] Writing a Will: When Tanaka selects the "Will Writing" tool and enters the content, it is sent to the server, where it is processed and feedback is returned.

[0193] In this way, this system provides an interface that users can operate intuitively, and by efficiently managing and providing multifunctional content, it enriches the lives of the elderly and reduces the burden on caregivers.

[0194] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0195] Step 1: The user types the request into the terminal

[0196] A user attempts to log in by entering account information (username and password) into the terminal interface. The input data is provided by the user and sent to the terminal. Based on this, an authentication request is generated and sent to the server.

[0197] Step 2: The device sends an authentication request to the server

[0198] The terminal converts the authentication information entered by the user into a packet format and sends it to the server using a communication protocol (e.g., HTTPS). The input data is the user's account information, and communication data is generated based on this.

[0199] Step 3: Server authenticates

[0200] The server checks the received authentication information against an authentication database. It queries the authentication database to see if the corresponding user information exists. The input data is the authentication request, and the output data is the authentication result (success or failure).

[0201] Step 4: The server returns the authentication result to the device.

[0202] The server generates an authentication result and returns it to the terminal. If successful, it generates and adds a session ID, etc., which allows the user to gain permission to access the system. The output data is the authentication result and the session ID.

[0203] Step 5: The device displays the authentication result

[0204] The terminal displays the authentication result received from the server to the user. If successful, the home screen is displayed and the user is guided to the next operation. The input data is the authentication result, and the output data is an interface that the user can operate.

[0205] Step 6: User selects the content menu

[0206] The user selects the desired content menu (e.g., video viewing, virtual reality experience) from the home screen. The user's operation is input to the terminal, and the next request is generated. The input data is the user's selection.

[0207] Step 7: The device sends a content request to the server

[0208] The terminal generates a content request based on the content menu selected by the user and sends it to the server, which then provides the specified content. The input data is the user's content selection.

[0209] Step 8: The server provides the content list

[0210] The server retrieves the relevant content list from the database based on the request, executes a query against the database, and sends the retrieved list to the terminal. The input data is the content request, and the output data is the content list.

[0211] Step 9: The device displays the content list to the user.

[0212] The terminal displays the content list received from the server to the user. The user selects more specific content from the displayed list. The input data is the content list, and the output data is an interface that the user can operate.

[0213] Step 10: User selects specific content

[0214] The user selects a specific item (e.g., a video of a family trip) from the displayed content list. The selection is input to the terminal, and the next request is generated. The input data is the user's specific content selection.

[0215] Step 11: The device sends a content data request to the server

[0216] The terminal generates and sends a content data request to the server based on the specific content selected by the user, with the input data being the specific content selection.

[0217] Step 12: The server retrieves and sends the content data

[0218] The server retrieves specific content data from the database based on the request, converts the retrieved data into a packet format, and sends it to the terminal. The input data is the content data request, and the output data is the content data.

[0219] Step 13: Your device plays and displays the content

[0220] The terminal plays and displays the content data received from the server to the user. In the case of video, for example, continuous playback is performed using a playback engine. The input data is the content data, and the output data is the content to be played and displayed.

[0221] This process flow allows users to intuitively operate the system and use a wide variety of content.

[0222] (Application example 1)

[0223] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0224] Today's elderly people need access to a variety of content and services to maintain their physical and mental health. However, they often find it difficult to understand and operate digital technology, which creates psychological barriers. Furthermore, they have limited means of reminiscing on past memories, creating a need for systems that allow them to easily access these services. However, existing systems are limited to solving individual issues, and few offer comprehensive, multifunctional services. Therefore, a multifunctional content system is needed that allows elderly people to easily access multiple services and improve their quality of life.

[0225] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0226] In this invention, the server includes server means for retrieving data from a database according to content selected by a user and providing it to a terminal, terminal means for the user to select content and send a request to the server, terminal means for playing and displaying the data provided from the server, means for making suggestions based on the user's activity data and interests using a generative AI model, and means for generating prompt sentences. This allows elderly people to easily access a variety of content and services and receive suggestions tailored to their individual needs.

[0227] "User-selected content" refers to multimedia materials or services that a user wishes to view or execute by operating a terminal.

[0228] A "database" is a digital system that stores data in a structured format that allows it to be efficiently managed and retrieved.

[0229] A "server" is a computer system that provides data and services to terminals over a network.

[0230] A "terminal" is an electronic device that provides an interface for users to operate. Examples include smartphones and head-mounted displays.

[0231] "Server means" refers to the methods or devices used by the server to implement a particular function or role.

[0232] "Terminal means" refers to the method or device used by the terminal to realize a specific function or role.

[0233] A "request" is a request or instruction sent from a terminal to a server in response to a user operation.

[0234] "Playback" means to re-display and output video and audio based on digital data.

[0235] "Display" means to visually represent data or information on a terminal screen or display.

[0236] A "generative AI model" is an artificial intelligence system that uses machine learning or deep learning algorithms to generate new data and suggestions.

[0237] A "prompt" is text that is used to design instructions or questions in a format that is easy for a generative AI model to understand.

[0238] "Activity data" refers to recorded data that indicates a user's past and present behavior and interests.

[0239] "Suggestions" are recommendations or suggested actions provided by a generative AI model based on a user's interests.

[0240] "User interests" refer to the particular areas or activities that interest a user.

[0241] This invention is a multifunctional content system for elderly people. This system consists of three main components: a server, a terminal, and a user. This system allows elderly people to watch past videos, maintain their health by viewing exercise videos, restore photos, and experience the metaverse.

[0242] server

[0243] The server provides the following functionality:

[0244] 1. Content Management: Manage content such as videos, images, and exercise videos from the database and provide them upon appropriate requests.

[0245] 2. User authentication: Authentication is performed based on the user's login information.

[0246] 3. Data restoration: Uploaded photos and videos are restored using image processing technology (e.g., OpenCV).

[0247] 4. Metaverse generation: A virtual reality space is generated using Unity or Unreal Engine, allowing users to experience visiting places from the past.

[0248] 5. Suggestions using generative AI models: Specific suggestions are made using generative AI models (e.g., GPT-4) based on user activity data and interests.

[0249] 6. Prompt Generation: Generate prompts for the generative AI model to provide advanced suggestions and support.

[0250] Terminal

[0251] A terminal is a device that provides an interface for users to operate the system. Its specific functions are as follows:

[0252] 1. Content selection: The user selects the desired content from each menu, such as video, exercise video, restoration request, metaverse, will writing, and life purpose suggestions.

[0253] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[0254] 3. Data playback and display: Receives data sent from the server and plays back videos and exercises. Displays restored photos and embodies the virtual space.

[0255] 4. Interface provision: Provide users with the interface necessary for the will creation tool and life purpose suggestions, and send the input information to the server.

[0256] User

[0257] Users are seniors and their caregivers who:

[0258] 1. Login: Enter your account information into the terminal interface to log in to the system.

[0259] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[0260] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[0261] 4. Experience: Watch the playback video, practice the exercise video, check the restored photos, visit the metaverse, write a will, and accept suggestions for new meanings in life through generative AI models.

[0262] Specific examples

[0263] As a concrete example, let's consider a scenario where the user selects "gymnastics videos":

[0264] 1. The user launches the app and enters their credentials on the login screen.

[0265] 2. The server authenticates using OAuth 2.0 and approves the login.

[0266] 3. Select the "Exercise Video" menu on the home screen.

[0267] 4. Send a request to the server to get a list of exercise videos.

[0268] 5. Select the desired exercise video from the list and play the video via streaming from the server.

[0269] 6. The user performs the exercises while watching the video on a smartphone or head-mounted display.

[0270] Example prompts for generative AI models

[0271] Please suggest some exercise videos suitable for seniors to maintain their health. The user's name is Tanaka, a 70-year-old man who has had difficulty walking in the past. Light stretching to improve flexibility and chair exercises would be appropriate.

[0272] Through this system, seniors can not only enjoy a variety of content from the comfort of their own homes, but also easily use digital technology to help maintain their health.

[0273] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0274] Step 1:

[0275] User login

[0276] Input: User authentication information (username, password)

[0277] Output: Authentication token

[0278] Specific behavior:

[0279] The user starts up the device and enters authentication information (username, password) on the login screen. The device sends this authentication information to the server. The server uses OAuth 2.0 to check against the authentication database, and if authentication is successful, generates an authentication token and sends it back to the device.

[0280] Step 2:

[0281] Content selection and request submission

[0282] Input: User content selection information

[0283] Output: Content List

[0284] Specific behavior:

[0285] After logging in, the user selects a menu item such as "Exercise Videos" from the device's home screen. The device then sends a request to the server based on the selected menu item. The server then retrieves a list of related content from the database and returns it to the device.

[0286] Step 3:

[0287] Content playback and display

[0288] Input: Content selected from the content list

[0289] Output: The video being played or the data being displayed

[0290] Specific behavior:

[0291] The user selects the desired video from the content list displayed on the device. The device then transmits the selected content information to the server, which then streams the corresponding content data in real time. The device then plays or displays the received data.

[0292] Step 4:

[0293] Upload and restore photos and videos

[0294] Input: Photo and video files uploaded by users

[0295] Output: Recovered photo and video files

[0296] Specific behavior:

[0297] The user selects the photos and video files they want to restore and uploads them to the server via their device. The server then restores the data using image processing techniques such as OpenCV and sends the restored files back to the device. The device then displays the restored data.

[0298] Step 5:

[0299] Creating and experiencing the metaverse space

[0300] Input: Information about the user's past locations and memories

[0301] Output: Metaverse space

[0302] Specific behavior:

[0303] The user selects the Metaverse menu through the device and inputs the information about past locations and memories they need. The device then sends this information to the server. The server uses Unity or Unreal Engine to generate a virtual reality space and sends the data back to the device. The user then uses a device (e.g., a head-mounted display) to visit and experience the virtual space.

[0304] Step 6:

[0305] Individualized proposals using generative AI

[0306] Input: Prompts based on user activity data and interests

[0307] Output: Proposals from the generative AI model

[0308] Specific behavior:

[0309] The server generates prompts based on the user's activity data and interests and sends them to a generative AI model. The AI ​​model analyzes the prompts and generates exercise videos, activity suggestions, and other content that are optimal for the user. This information is then sent to the device and displayed to the user.

[0310] Examples of prompt statements

[0311] Please suggest some exercise videos suitable for seniors to maintain their health. The user's name is Tanaka, a 70-year-old man who has had difficulty walking in the past. Light stretching to improve flexibility and chair exercises would be appropriate.

[0312] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0313] This invention is a multi-functional content system for use by the elderly of the silver generation, and by combining it with an emotion engine, it realizes a system that provides optimal content to users. This system consists of four main components: a server, a terminal, a user, and an emotion engine.

[0314] server

[0315] The server provides the following functionality:

[0316] 1. Content management: Various content (videos, exercise videos, photo restoration, metaverse, will creation tool, and life purpose suggestions) is stored in a database and provided upon appropriate request.

[0317] 2. User authentication: When a user logs in through a terminal, the user is authenticated by checking against the authentication database.

[0318] 3. Data recovery: Photos and videos uploaded by users from their devices are recovered using image processing technology.

[0319] 4. Metaverse generation: Generate a virtual reality space and allow users to virtually visit places from the past.

[0320] 5. Emotional Data Management: Emotional data recognized by the emotion engine is stored in a database to provide personalized content.

[0321] Terminal

[0322] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[0323] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, metaverse, will creation, and purpose in life suggestions.

[0324] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[0325] 3. Data playback and display: Receives data sent from the server, plays back videos and exercise videos, and displays restored photos and virtual spaces.

[0326] 4. Interface provision: Provide users with the interface necessary for the will creation tool and life purpose suggestions, and send the input information to the server.

[0327] 5. Emotion data transmission: The emotion data recognized by the emotion engine is transmitted to the server.

[0328] User

[0329] Users are seniors and their caregivers who:

[0330] 1. Login: Enter your account information into the terminal interface to log in to the system.

[0331] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[0332] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[0333] 4. Experience: Watch the playback video, try the exercise video, check the restored photos, visit the Metaverse space, write a will, and accept suggestions for a new purpose in life.

[0334] 5. Emotion input: Emotion data is provided in real time through the device's built-in camera and microphone.

[0335] Emotion Engine

[0336] The Emotion Engine provides the following features:

[0337] 1. Emotion recognition: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time.

[0338] 2. Emotion data generation: Generates recognized emotion data and sends it to the server via the terminal.

[0339] 3. Content recommendation: Based on the recognized sentiment, the server recommends the most suitable content.

[0340] As a concrete example, let us consider a scenario in which a user watches a video. First, the user logs in to the device and selects the "Watch Video" menu from the home screen. The device sends a content request to the server. The server retrieves a video list from the database and sends it to the device. The device displays the list and the user selects a video that interests them. The emotion engine recognizes the user's emotions in real time and sends that data to the server. Based on the recognized emotion data, the server selects a video that best suits the user's mood and sends it to the device. The device plays the video, and the user watches it.

[0341] In this way, this system provides an interface that is easy for users to operate, and the emotion engine recognizes the user's emotions in real time to provide optimal content, thereby improving the quality of life for the elderly and reducing the burden on caregivers.

[0342] The processing flow will be explained below.

[0343] Video viewing process flow

[0344] Step 1:

[0345] The user enters account information into the terminal interface and presses the login button.

[0346] Step 2:

[0347] The terminal transmits the user's input information to the server.

[0348] Step 3:

[0349] The server compares the received information with the database and returns the authentication result to the terminal.

[0350] Step 4:

[0351] The device displays the authentication result to the user, and if successful, transitions to the home screen.

[0352] Step 5:

[0353] The user selects the "Watch Video" menu from the home screen.

[0354] Step 6:

[0355] The terminal sends a content request to the server.

[0356] Step 7:

[0357] The server retrieves a list of video content from the database and transmits it to the terminal.

[0358] Step 8:

[0359] The device displays a list of videos.

[0360] Step 9:

[0361] The user selects the video they want to watch from the list.

[0362] Step 10:

[0363] The terminal sends a request for details of the selected video to the server.

[0364] Step 11:

[0365] The server reads the video data from the database and sends it to the terminal.

[0366] Step 12:

[0367] The emotion engine captures the user's facial expressions and voice data and analyzes emotions in real time.

[0368] Step 13:

[0369] The emotion engine sends the analysis results to the server.

[0370] Step 14:

[0371] The server receives the emotion data and selects the most appropriate video according to the user's current emotional state.

[0372] Step 15:

[0373] The server transmits the selected optimal video data to the terminal.

[0374] Step 16:

[0375] The terminal receives the video data and starts playing it.

[0376] Step 17:

[0377] The user watches the video.

[0378] Photo and video restoration process flow

[0379] Step 1:

[0380] The user selects the "Photo / Video Recovery" menu from the home screen.

[0381] Step 2:

[0382] The terminal displays a file selection screen to the user.

[0383] Step 3:

[0384] The user selects the photo or video files they want to restore.

[0385] Step 4:

[0386] The device uploads the selected file to the server.

[0387] Step 5:

[0388] The server stores the received file in the database and executes the restoration process.

[0389] Step 6:

[0390] The server sends the results of the restoration process to the terminal.

[0391] Step 7:

[0392] The emotion engine captures the user's facial expressions and reaction data and analyzes their emotions.

[0393] Step 8:

[0394] The emotion engine sends the analysis results to the server.

[0395] Step 9:

[0396] The device will display the restored photos and videos.

[0397] Step 10:

[0398] The user reviews the restored content.

[0399] Metaverse Processing Flow

[0400] Step 1:

[0401] The user selects the "Metaverse" menu from the home screen.

[0402] Step 2:

[0403] The device sends a metaverse request to the server.

[0404] Step 3:

[0405] The server generates metaverse spatial data and sends it to the terminal.

[0406] Step 4:

[0407] The device displays the metaverse space and provides the user with a virtual reality experience.

[0408] Step 5:

[0409] The emotion engine analyzes the user's voice and facial expressions and sends the emotion data to the server.

[0410] Step 6:

[0411] The server receives the emotion data and applies experience content appropriate to the user's emotional state.

[0412] Step 7:

[0413] A user visits past locations in the metaverse.

[0414] Process flow for writing a will

[0415] Step 1:

[0416] The user selects the "Write a Will" menu from the home screen.

[0417] Step 2:

[0418] The device displays a will creation tool.

[0419] Step 3:

[0420] The user inputs the contents of the will.

[0421] Step 4:

[0422] The emotion engine analyzes the user's facial expressions and voice and sends the emotion data to the server.

[0423] Step 5:

[0424] The terminal transmits the entered will data to the server.

[0425] Step 6:

[0426] The server stores the will data in a database and sends the confirmation results to the terminal.

[0427] Step 7:

[0428] The terminal displays the confirmation result to the user.

[0429] Purpose of Life Proposal Processing Flow

[0430] Step 1:

[0431] The user selects the "Life Purpose Suggestions" menu from the home screen.

[0432] Step 2:

[0433] The device displays a questionnaire about the user's interests and preferences.

[0434] Step 3:

[0435] The user answers the question and submits it.

[0436] Step 4:

[0437] The emotion engine analyzes the user's responses and real-time emotional state and sends the emotional data to the server.

[0438] Step 5:

[0439] The terminal transmits the user's response data to the server.

[0440] Step 6:

[0441] The server analyzes the data and suggests appropriate hobbies and activities.

[0442] Step 7:

[0443] The server sends the proposal results to the terminal.

[0444] Step 8:

[0445] The device displays the suggestions to the user.

[0446] Step 9:

[0447] The user confirms and engages in suggested activities and hobbies.

[0448] These processing flows create a system that is easy for elderly people to use, allows them to enjoy a variety of content, and, by using an emotion engine, provides a personalized experience, reducing the burden on caregivers.

[0449] Example 2

[0450] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0451] Current content systems for the elderly do not provide content that takes into account the user's emotions, resulting in a poor user experience. Other issues include the difficulty of restoring past photos and videos, and the lack of a virtual reality space that is easy for the elderly to use.

[0452] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0453] In this invention, the server includes an information processing device that retrieves data from a database according to content selected by the user and provides it to the device being used, an emotion analysis device that analyzes the user's voice, facial expression, and movement data to recognize emotions in real time, and a device that recommends optimal content based on the emotion data. This makes it possible to provide optimal content according to the user's emotions, improving the quality of the user experience. It also makes it possible to realize an environment that makes it easy to restore photos and videos and provide virtual reality spaces.

[0454] The term "information processing device means" refers to a device that retrieves data from a database according to content selected by a user and provides the data to the device being used.

[0455] The "device means" refers to a device that allows a user to select content and send a request to an information processing device. It also includes a function to play and display data provided by the information processing device.

[0456] "Emotion analysis means" refers to a device or program that analyzes the user's voice, facial expression, and movement data and recognizes emotions in real time.

[0457] "Means" generally refers to a method, apparatus, or program designed to achieve a specific purpose.

[0458] "Data" refers to information or content acquired and used via information processing device means, and includes various formats such as video, images, audio, and text.

[0459] "Virtual space providing means" refers to a device or program that enables a user to visit past locations in a virtual reality space.

[0460] This invention is a multi-functional content system for use by the elderly of the silver generation, and by combining it with an emotion analysis function, it realizes a system that provides optimal content to users. This system consists of four main components: an information processing device, a user device, a user, and an emotion analysis function.

[0461] Information processing device

[0462] The information processing device provides the following functions:

[0463] 1. Content management: The information processing device stores various content (videos, exercise videos, image restoration, virtual space, will writing tools, and life purpose suggestions) in a database and provides this content in response to user requests. This process uses a database such as MySQL.

[0464] 2. User authentication: When a user logs in through their device, the user is authenticated by checking against an authentication database. Authentication is performed using LDAP or OAuth2.

[0465] 3. Image restoration: The information processing device restores the images and videos uploaded by the user from their device using image processing technology (e.g., OpenCV).

[0466] 4. Virtual space generation: The information processing device generates a virtual reality space, allowing users to virtually visit places from the past. This process uses Unreal Engine and Unity.

[0467] 5. Emotional data management: The information processing device stores the emotional data recognized by the emotion analysis function in a database and provides personalized content.

[0468] Equipment used

[0469] A user device is a device that provides an interface for users to operate the system. It has the following functions:

[0470] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, virtual space, will writing, and purpose in life suggestions.

[0471] 2. Request transmission: Based on the selected content, an appropriate request is sent to the information processing device. An HTTP request is used.

[0472] 3. Data playback and display: Receives data sent from the information processing device, plays back images and exercise videos, and displays restored images and virtual spaces.

[0473] 4. Interface provision: Provides the user with the interface necessary for the will writing tool and life purpose suggestions, and sends the input information to the information processing device.

[0474] 5. Emotion data transmission: Emotion data recognized by the emotion analysis function is transmitted to the information processing device.

[0475] User

[0476] Users are seniors and their caregivers who:

[0477] 1. Login: Enter your account information into the interface of your device to log in to the system.

[0478] 2. Content usage: Select the desired content from each menu and operate it through the interface of the device you are using.

[0479] 3. Upload: Upload the images or videos you want to restore from your device to the information processing device.

[0480] 4. Experience: Watch the video, try out the exercises, check the restored images, and visit the virtual space. You can also write a will and accept suggestions for a new purpose in life.

[0481] 5. Emotion input: Emotion data is provided in real time through the camera and microphone built into the device being used.

[0482] Sentiment analysis function

[0483] The sentiment analysis feature provides the following features:

[0484] 1. Emotion recognition: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time, using technologies such as OpenFace and DeepFace.

[0485] 2. Emotion data generation: Recognized emotion data is generated and sent to the information processing device via the device being used.

[0486] 3. Content recommendation: Based on the recognized emotions, the information processing device recommends the most suitable content.

[0487] Example scenario

[0488] As a specific example, a scenario in which a user watches a video will be described. First, the user logs in to the device being used and selects the "Watch Video" menu from the home screen. The device being used sends a content request to the information processing device. The information processing device retrieves a video list from a database and sends it to the device being used. The device being used displays the list, and the user selects a video of interest. The emotion analysis function recognizes the user's emotions in real time and sends that data to the information processing device. Based on the recognized emotion data, the information processing device selects a video that best suits the user's mood and sends it to the device being used. The device being used plays the video, and the user watches it.

[0489] Example prompt:

[0490] "Please explain the process of watching videos using a multi-functional content system for seniors in the silver generation. Please explain how a user logs in, selects a video, and how the emotion analysis function works."

[0491] In this way, this system provides an interface that is easy for users to operate, and the emotion analysis function recognizes the user's emotions in real time to provide optimal content, thereby improving the quality of life for the elderly and reducing the burden on caregivers.

[0492] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0493] The program processing flow of this system and specific steps

[0494] Processing Steps

[0495] Step 1: User Login

[0496] Step 2: Menu selection

[0497] Step 3: Submit a content request

[0498] Step 4: Providing and displaying content

[0499] Step 5: Collect and send emotion data

[0500] Step 6: Optimize and reselect content

[0501] Specific explanation of each processing step

[0502] Step 1: User Login

[0503] Input: Account information (ID and password)

[0504] Output: Authentication success or failure response

[0505] The user enters their account information on the login screen of their device and presses the login button. The device sends the entered information to the server as an HTTP POST request. The server verifies the authentication information using an LDAP or MySQL database, and if authentication is successful, returns a successful authentication response to the device. If authentication is unsuccessful, it returns an error message.

[0506] Specific operation: The user enters "ID: user123" and "Password: pass456" and clicks the login button. The device sends this information to the server, which queries the LDAP database for authentication.

[0507] Step 2: Menu selection

[0508] Input: User-selected menu item

[0509] Output: Selected menu information

[0510] The user selects the content they want to use (e.g., watching a video) from the menu displayed on the home screen. The device sends the selected menu information to the server via an HTTP GET request.

[0511] Specific operation: The user selects the "Watch Video" menu from the home screen and clicks the "Watch Video" button on the screen. The device sends a request including the selection information to the server.

[0512] Step 3: Submit a content request

[0513] Input: Selected menu information

[0514] Output: A list of matching content

[0515] The terminal transmits the selected menu information to the server, and the server retrieves the corresponding content list from the database and transmits the list to the terminal.

[0516] Specific operation: The device sends a "video viewing" request to the server. The server retrieves the video list from the MySQL database and sends it to the device in JSON format.

[0517] Step 4: Providing and displaying content

[0518] Input: Content list received from the server

[0519] Output: User selects and retrieves content

[0520] The device displays the content list received from the server to the user. The user selects the content of interest from the list, and the device sends the selection information to the server. The server retrieves the relevant content from the database and sends it to the device. The device plays and displays the content.

[0521] Specific operation: The user selects "Video A" from the displayed list and clicks the play button. The device sends the ID of Video A to the server, and the server retrieves the video data and sends the playback data to the device.

[0522] Step 5: Collect and send emotion data

[0523] Input: User's voice, facial expression, and movement data

[0524] Output: Generated emotion data

[0525] While the device is playing video, the camera and microphone collect the user's voice, facial expressions, and movement data. The emotion analysis function analyzes this data and generates emotion data in real time. The emotion data is then sent to the server via an HTTP POST request.

[0526] Specific operation: The device detects the user's facial expressions and voice, and the emotion analysis function recognizes that the user is "having fun." Emotional data is generated and sent to the server.

[0527] Step 6: Optimize and reselect content

[0528] Input: Received emotion data

[0529] Output: Recommend and serve new content

[0530] The server recommends new content based on the received emotion data. The device displays the recommended content to the user, and the user selects it again. The reselected content is then retrieved from the server and displayed.

[0531] Specific operation: The server analyzes the emotional data of "enjoyed" and recommends the more suitable "Video B." The device presents "Video B" to the user, who selects and plays it.

[0532] This will enable optimal content to be provided according to the user's emotions, improving the quality of life for the elderly.

[0533] (Application example 2)

[0534] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0535] When elderly people use content, it is often difficult to operate, and the content provided is often not suited to their individual emotions or state. This leads to problems that reduce the quality of life of the elderly and increase the burden on caregivers. In addition, there is a lack of appropriate exercise guidance to support psychological and physical health, and functions to recreate past memories in virtual space.

[0536] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes information collection device means for acquiring data from the data collection management device in accordance with information selected by the user and providing the data to the information device, information device means for the user to select information and send a request to the information collection management device, information device means for playing and displaying the data provided from the information collection management device, emotion recognition means for recognizing the user's emotion in real time and sending the data to the information collection management device, and information recommendation means for recommending optimal information based on the recognized emotion data. This provides an interface that is easy for the user to operate, and makes it possible to provide optimal content by having the emotion engine recognize the user's emotion in real time.

[0537] "User" refers to an individual or senior citizen who uses the system.

[0538] An "information accumulation and management device" is a device that stores and manages data and provides it as needed.

[0539] "Information equipment" refers to devices that are operated by users, such as smart glasses and smartphones.

[0540] The "information accumulation device means" is a function or system for acquiring data from the information accumulation management device and providing it to the information device.

[0541] The "information device means" is a function or system that allows a user to select information and send a request to the information accumulation management device.

[0542] The "information device display means" is a function or system for reproducing data provided from the information accumulation management device and displaying it on an information device.

[0543] The "emotion recognition means" is a function or system for recognizing the user's emotions in real time and transmitting the data to the information collection and management device.

[0544] The "information recommendation means" is a function or system for recommending optimal information to a user based on the recognized emotion data.

[0545] "Image restoration means" refers to a function or system for restoring images and videos uploaded by users.

[0546] The "virtual space generation means" is a function or system for generating a virtual space in which the user can visit places from the past.

[0547] This invention is a multi-functional content system for use by elderly people in the silver generation, which provides optimal content according to the user's emotions by combining it with an emotion recognition engine. This system consists of four main components: an information accumulation and management device (server), information equipment (terminal), users, and the emotion recognition engine.

[0548] Information accumulation and management device (server)

[0549] The server provides the following functionality:

[0550] 1. Information accumulation device means: Acquires data from a database according to information selected by the user and provides it to the information device.

[0551] 2. Image restoration tool: It has the function to restore images and videos uploaded by users.

[0552] 3. Virtual space generation means: A past location is generated as a virtual reality space and provided to the user.

[0553] 4. Emotion data management: Emotion data recognized by the emotion recognition engine is stored in a database and personalized content is provided.

[0554] Information equipment (terminals)

[0555] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[0556] 1. Information device means: The user selects information and sends a request to the server.

[0557] 2. Information device display means: Receives data provided by the server, plays videos and exercise videos, and displays restored photos and virtual spaces.

[0558] 3. Input interface: Provides users with the interface necessary for the will-writing tool and emotion-based purpose-in-life suggestions, and sends the input content to the server.

[0559] 4. Emotion recognition means: Using the device's built-in camera and microphone, the user's emotional data is acquired in real time and sent to the server.

[0560] User

[0561] Users are seniors and their caregivers who:

[0562] 1. Login: Enter your account information into the terminal interface to log in to the system.

[0563] 2. Content usage: Select the desired information from each menu and operate it through the device interface.

[0564] 3. Upload: Upload the images or videos you want to restore from your device to the server.

[0565] 4. Experience: Watch the video, try out the exercises, check the restored photos, visit the virtual space, and even write a will and accept suggestions for a new purpose in life.

[0566] 5. Emotion input: Provide emotion data in real time via camera or microphone.

[0567] Emotion Recognition Engine

[0568] The emotion recognition engine provides the following features:

[0569] 1. Emotion recognition means: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time.

[0570] 2. Emotion data transmission: Generate recognized emotion data and transmit it to the server via the terminal.

[0571] 3. Information recommendation method: Based on the recognized emotions, the server recommends the most appropriate information.

[0572] Specific examples

[0573] For example, suppose a user is wearing smart glasses and sitting in their living room in a relaxed mood. The emotion recognition engine recognizes the user's relaxed state in real time. Based on this, the server will recommend a landscape video that is most suitable for the relaxed mood, and a beautiful natural scene will be displayed on the smart glasses' display. The user can enjoy this visual experience while remaining relaxed.

[0574] Prompt Sentence Examples

[0575] "How can we recommend beautiful landscape videos to elderly people who are in a relaxed state?"

[0576] "Design a system that uses an emotion recognition engine to recognize emotions in real time and display content based on that in smart glasses."

[0577] The above is a specific embodiment for carrying out the present invention.

[0578] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0579] Step 1:

[0580] A user logs in to an information device (e.g., smart glasses). As input, the user enters account information, and the server authenticates the user based on this information. As output, if authentication is successful, the user can access the home screen.

[0581] Step 2:

[0582] The user selects the desired content menu (e.g., watching a video, watching a gymnastics video, visiting a virtual space) from the home screen. The user's selection is received as input, and this data is sent from the device to the server. As output, the server provides the device with a list of the corresponding content.

[0583] Step 3:

[0584] The server uses an emotion recognition engine to obtain the user's emotion data in real time. It receives data from the camera and microphone as input, analyzes the emotion using an emotion recognition model, and generates the recognized emotion data as output, which is then sent to the server.

[0585] Step 4:

[0586] The server recommends optimal content based on the received emotional data. It receives the emotional data and content request as input, selects optimal content from a database based on the received emotional data, and sends the selected content to the device as output.

[0587] Step 5:

[0588] The terminal plays the content sent from the server and displays it to the user. As input, it takes the received content data and converts it into a format for display. As output, the user can visually confirm the played content on the display of the smart glasses.

[0589] Step 6:

[0590] When a user uploads an image or video they want to repair, their device sends these files to the server. As input, it takes the image or video file and uploads it to the server. As output, the server starts the repair process.

[0591] Step 7:

[0592] The server sends the repaired image or video file to the device. It receives the file to be repaired as input, processes it using an image repair algorithm, and provides the repaired file to the device as output.

[0593] Step 8:

[0594] A user sends a request to visit a virtual space from a terminal. The terminal receives the request data for visiting the virtual space as input and sends it to the server. As output, the server generates a virtual space and provides it to the terminal.

[0595] Step 9:

[0596] The terminal displays the virtual space provided by the server and provides the user with an experience. It receives virtual space data as input and converts it into a display format. As an output, the user can enjoy the experience in the virtual space.

[0597] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0598] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0599] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0600] [Second embodiment]

[0601] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0602] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0603] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0604] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0605] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0606] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0607] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0608] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0609] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0610] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0611] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0612] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0613] This invention is a multi-functional content system for use by the elderly of the silver generation. This system consists of three main components: a server, a terminal, and a user.

[0614] server

[0615] The server provides the following functionality:

[0616] 1. Content management: Various content (videos, exercise videos, photo restoration, metaverse, will creation tool, and life purpose suggestions) is stored in a database and provided upon appropriate request.

[0617] 2. User authentication: When a user logs in through a terminal, the user is authenticated by checking against the authentication database.

[0618] 3. Data recovery: Photos and videos uploaded by users from their devices are recovered using image processing technology.

[0619] 4. Metaverse generation: Generate a virtual reality space and allow users to virtually visit places from the past.

[0620] Terminal

[0621] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[0622] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, metaverse, will creation, and purpose in life suggestions.

[0623] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[0624] 3. Data playback and display: Receives data sent from the server, plays back videos and exercise videos, and displays restored photos and virtual spaces.

[0625] 4. Interface provision: Provide users with the interface necessary for will-writing tools and life purpose suggestions, and send the input information to the server.

[0626] User

[0627] Users are seniors and their caregivers who:

[0628] 1. Login: Enter your account information into the terminal interface to log in to the system.

[0629] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[0630] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[0631] 4. Experience: Watch the playback video, try the exercise video, check the restored photos, visit the Metaverse space, write a will, and accept suggestions for a new purpose in life.

[0632] As a concrete example, let us consider a scenario in which a user watches old videos. First, the user logs in to the device and selects the "Watch Videos" menu from the home screen. The device sends a content request to the server. The server retrieves a video list from the database and sends it to the device. The device displays the list, and the user selects a video of interest. The server reads the selected video data from the database and sends it to the device. The device plays the video, and the user watches it.

[0633] In this way, this system provides an interface that is easy for users to operate, and the server manages and provides a variety of content, thereby enriching the lives of the elderly and reducing the burden on caregivers.

[0634] The processing flow will be explained below.

[0635] Video viewing process flow

[0636] Step 1:

[0637] The user enters account information into the terminal interface and presses the login button.

[0638] Step 2:

[0639] The terminal transmits the user's input information to the server.

[0640] Step 3:

[0641] The server compares the received information with the database and returns the authentication result to the terminal.

[0642] Step 4:

[0643] The device displays the authentication result to the user, and if successful, transitions to the home screen.

[0644] Step 5:

[0645] The user selects the "Watch Video" menu from the home screen.

[0646] Step 6:

[0647] The terminal sends a content request to the server.

[0648] Step 7:

[0649] The server retrieves a list of video content from the database and transmits it to the terminal.

[0650] Step 8:

[0651] The device displays a list of videos.

[0652] Step 9:

[0653] The user selects the video they want to watch from the list.

[0654] Step 10:

[0655] The terminal sends a request for details of the selected video to the server.

[0656] Step 11:

[0657] The server reads the video data from the database and sends it to the terminal.

[0658] Step 12:

[0659] The terminal receives the video data and starts playing it.

[0660] Step 13:

[0661] The user watches the video.

[0662] Photo and video restoration process flow

[0663] Step 1:

[0664] The user selects the "Photo / Video Recovery" menu from the home screen.

[0665] Step 2:

[0666] The terminal displays a file selection screen to the user.

[0667] Step 3:

[0668] The user selects the photo or video files they want to restore.

[0669] Step 4:

[0670] The device uploads the selected file to the server.

[0671] Step 5:

[0672] The server stores the received file in the database and executes the restoration process.

[0673] Step 6:

[0674] The server sends the results of the restoration process to the terminal.

[0675] Step 7:

[0676] The device will display the restored photos and videos.

[0677] Step 8:

[0678] The user reviews the restored content.

[0679] Metaverse Processing Flow

[0680] Step 1:

[0681] The user selects the "Metaverse" menu from the home screen.

[0682] Step 2:

[0683] The device sends a metaverse request to the server.

[0684] Step 3:

[0685] The server generates metaverse spatial data and sends it to the terminal.

[0686] Step 4:

[0687] The device displays the metaverse space and provides the user with a virtual reality experience.

[0688] Step 5:

[0689] A user visits past locations in the metaverse.

[0690] Process flow for writing a will

[0691] Step 1:

[0692] The user selects the "Write a Will" menu from the home screen.

[0693] Step 2:

[0694] The device displays a will creation tool.

[0695] Step 3:

[0696] The user inputs the contents of the will.

[0697] Step 4:

[0698] The terminal transmits the entered will data to the server.

[0699] Step 5:

[0700] The server stores the will data in a database and sends the confirmation results to the terminal.

[0701] Step 6:

[0702] The terminal displays the confirmation result to the user.

[0703] Purpose of Life Proposal Processing Flow

[0704] Step 1:

[0705] The user selects the "Life Purpose Suggestions" menu from the home screen.

[0706] Step 2:

[0707] The device displays a questionnaire about the user's interests and preferences.

[0708] Step 3:

[0709] The user answers the question and submits it.

[0710] Step 4:

[0711] The terminal transmits the user's response data to the server.

[0712] Step 5:

[0713] The server analyzes the data and suggests appropriate hobbies and activities.

[0714] Step 6:

[0715] The server sends the proposal results to the terminal.

[0716] Step 7:

[0717] The device displays the suggestions to the user.

[0718] These processing flows will realize a system that is easy for elderly people to use, allows them to enjoy a variety of content, and reduces the burden on caregivers.

[0719] Example 1

[0720] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0721] Conventional content systems for seniors were difficult for seniors to use because they were complicated to operate. Furthermore, there was no system that offered a variety of functions, such as image and video restoration, virtual reality space experiences, and tools for writing wills, all in one place. Furthermore, it was necessary to use separate systems and devices to access specific content, which caused confusion for seniors.

[0722] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0723] In this invention, the server includes an information processing device that retrieves data from a database according to content selected by the user and provides it to the terminal, means for the user to input login information and the information processing device compares the information with the authentication database for authentication, means for the user to upload photos or videos that the user wants to restore from the display device and the information processing device restores them using image processing technology, means for generating a virtual reality space that allows the user to visit past locations in a virtual reality space, and means for the information processing device to generate the virtual reality space and provide it to the display device. This makes it possible to provide a system that provides multifunctional content all at once and is easy for seniors to use.

[0724] An "information processing device" is a device that retrieves data from a database based on a request from a user, performs the specified operation or processing, and provides the results to a terminal.

[0725] The "display device" is a device that allows a user to input data and perform operations via an interface, and that reproduces and displays data transmitted from an information processing device.

[0726] "User authentication" is the process of authenticating a user when they log in to a system by checking the entered login information against an authentication database.

[0727] "Content" refers to digital information and data such as videos, exercise videos, photographs, virtual reality spaces, will-writing tools, and suggestions for purpose in life.

[0728] A "database" is a data storage system for managing and storing various content and user authentication information within the system.

[0729] "Image processing techniques" are techniques used to restore uploaded photos and videos, including noise reduction, resolution enhancement, and color correction.

[0730] "Virtual reality space" refers to a three-dimensional computer-simulated space created to allow users to virtually visit places from the past.

[0731] A "will creation tool" is an application that allows users to create and save their wishes and messages in digital form.

[0732] "Purpose in Life Suggestion" is a system or function that suggests new hobbies and activities to users.

[0733] MODE FOR CARRYING OUT THE INVENTION

[0734] This invention is a multi-functional content system for the elderly, consisting of three main components: a user, a terminal, and a server. This system allows elderly people to easily access various content and enrich their daily lives. Each component of this invention and its operation will be described in detail below.

[0735] 1. Server

[0736] The server plays a central role in the system and provides the following functions:

[0737] 1.1 Content Management

[0738] The server stores various content (videos, exercise videos, photo restoration, virtual reality space, will writing tool, and purpose-in-life suggestions) in a database and provides them in response to user requests. Specifically, the server uses a database management system (DBMS) to efficiently manage content data.

[0739] 1.2 User Authentication

[0740] When a user logs in via a terminal, the server authenticates them against an authentication database. This is achieved using security protocols (e.g., SSL / TLS) and multi-factor authentication technology. Specifically, a mechanism is adopted in which an authentication code is sent after the user enters their username and password.

[0741] 1.3 Data Recovery

[0742] The server uses image processing technology to restore photos and videos uploaded by users from their devices. Specific techniques used include image restoration models using deep learning (e.g., Generative Adversarial Networks, GANs). Using a high-performance GPU enables efficient data processing.

[0743] 1.4 Virtual reality space generation

[0744] The server generates a virtual reality space, allowing users to virtually visit places from the past. This is done using VR development platforms such as Unity and Unreal Engine. The generated virtual reality space data is sent to the device in real time.

[0745] 2. Terminal

[0746] The terminal provides an interface for users to operate the system and has the following functions:

[0747] 2.1 Content Selection

[0748] It provides an interface for users to select the content they want from the menus of video, exercise videos, restoration requests, virtual reality space, will writing, and life purpose suggestions. Operation becomes even easier if the device is equipped with a touch screen or voice recognition function.

[0749] 2.2 Request Submission

[0750] Based on the selected content, the device sends an appropriate request to the server. The request is sent using the HTTP / HTTPS protocol. For example, the request might say "The user has selected to watch a video."

[0751] 2.3 Data playback and display

[0752] The device receives the data (video, exercise video, restored photos, virtual reality space) sent from the server and plays and displays it. A device with a GPU is recommended for video playback and virtual reality display.

[0753] 2.4 Interface Provision

[0754] The system provides users with an interface for writing wills and suggesting purpose in life, and sends this to the server. The content entered by the user (e.g., "I want to find a new hobby") is sent to the server, and appropriate feedback is returned to the device.

[0755] 3. Users

[0756] The users are elderly people and their caregivers, and perform the following operations:

[0757] 3.1 Login

[0758] A user logs in to the system by entering account information (user name, password, etc.) on the login screen of the terminal. For example, the user enters the user name "tanaka123" and the password "password123."

[0759] 3.2 Content Use

[0760] Users can select the desired content from each menu and operate it through the device interface. Specifically, they can select the "Video Viewing" menu to watch old travel videos.

[0761] 3.3 Upload

[0762] Upload the photos or videos you want to restore from your device to the server. For example, upload "Photos from a family trip." After the server completes the restoration process, the restored data will be displayed on your device.

[0763] 3.4 Experience

[0764] Users can watch playback videos, practice exercise videos, view restored photos, and visit virtual reality spaces. They can also write wills and accept suggestions for new purposes in life. When using the virtual reality space, users put on a VR headset and explore virtual locations in real time.

[0765] Specific examples

[0766] Login: When a user attempts to log in by entering "tanaka123" and "password123", the server authenticates them and returns a success message.

[0767] Video viewing: When Tanaka selects "Video viewing," the device sends a request to the server, which returns a list of videos. Tanaka selects "Video of family trip" and watches it.

[0768] Photo restoration: When Tanaka tries to restore an old family photo by uploading it from his device, the server restores it and sends the photo back to his device, where Tanaka can view it.

[0769] Virtual reality experience: When Tanaka selects "Virtual reality space of his hometown," the server generates a virtual space, sends it to his device, and Tanaka experiences it.

[0770] Writing a Will: When Tanaka selects the "Will Writing" tool and enters the content, it is sent to the server, where it is processed and feedback is returned.

[0771] In this way, this system provides an interface that users can operate intuitively, and by efficiently managing and providing multifunctional content, it enriches the lives of the elderly and reduces the burden on caregivers.

[0772] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0773] Step 1: The user types the request into the terminal

[0774] A user attempts to log in by entering account information (username and password) into the terminal interface. The input data is provided by the user and sent to the terminal. Based on this, an authentication request is generated and sent to the server.

[0775] Step 2: The device sends an authentication request to the server

[0776] The terminal converts the authentication information entered by the user into a packet format and sends it to the server using a communication protocol (e.g., HTTPS). The input data is the user's account information, and communication data is generated based on this.

[0777] Step 3: Server authenticates

[0778] The server checks the received authentication information against an authentication database. It queries the authentication database to see if the corresponding user information exists. The input data is the authentication request, and the output data is the authentication result (success or failure).

[0779] Step 4: The server returns the authentication result to the device.

[0780] The server generates an authentication result and returns it to the terminal. If successful, it generates and adds a session ID, etc., which gives the user permission to access the system. The output data is the authentication result and the session ID.

[0781] Step 5: The device displays the authentication result

[0782] The terminal displays the authentication result received from the server to the user. If successful, the home screen is displayed and the user is guided to the next operation. The input data is the authentication result, and the output data is an interface that the user can operate.

[0783] Step 6: User selects the content menu

[0784] The user selects the desired content menu (e.g., video viewing, virtual reality experience) from the home screen. The user's operation is input to the terminal, and the next request is generated. The input data is the user's selection.

[0785] Step 7: The device sends a content request to the server

[0786] The terminal generates a content request based on the content menu selected by the user and sends it to the server, which then provides the specified content. The input data is the user's content selection.

[0787] Step 8: The server provides the content list

[0788] The server retrieves the relevant content list from the database based on the request, executes a query against the database, and sends the retrieved list to the terminal. The input data is the content request, and the output data is the content list.

[0789] Step 9: The device displays the content list to the user.

[0790] The terminal displays the content list received from the server to the user. The user selects more specific content from the displayed list. The input data is the content list, and the output data is an interface that the user can operate.

[0791] Step 10: User Selects Specific Content

[0792] The user selects a specific item (e.g., a video of a family trip) from the displayed content list. The selection is input to the terminal, and the next request is generated. The input data is the user's specific content selection.

[0793] Step 11: The device sends a content data request to the server

[0794] The terminal generates and sends a content data request to the server based on the specific content selected by the user, with the input data being the specific content selection.

[0795] Step 12: The server retrieves and sends the content data

[0796] The server retrieves specific content data from the database based on the request, converts the retrieved data into a packet format, and sends it to the terminal. The input data is the content data request, and the output data is the content data.

[0797] Step 13: Your device plays and displays the content

[0798] The terminal plays and displays the content data received from the server to the user. In the case of video, for example, continuous playback is performed using a playback engine. The input data is the content data, and the output data is the content to be played and displayed.

[0799] This process flow allows users to intuitively operate the system and use a wide variety of content.

[0800] (Application example 1)

[0801] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0802] Today's elderly people need access to a variety of content and services to maintain their physical and mental health. However, they often find it difficult to understand and operate digital technology, which creates psychological barriers. Furthermore, they have limited means of reminiscing on past memories, creating a need for systems that allow them to easily access these services. However, existing systems are limited to solving individual issues, and few offer comprehensive, multifunctional services. Therefore, a multifunctional content system is needed that allows elderly people to easily access multiple services and improve their quality of life.

[0803] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0804] In this invention, the server includes server means for retrieving data from a database according to content selected by a user and providing it to a terminal, terminal means for the user to select content and send a request to the server, terminal means for playing and displaying the data provided from the server, means for making suggestions based on the user's activity data and interests using a generative AI model, and means for generating prompt sentences. This allows elderly people to easily access a variety of content and services and receive suggestions tailored to their individual needs.

[0805] "User-selected content" refers to multimedia materials or services that a user wishes to view or execute by operating a terminal.

[0806] A "database" is a digital system that stores data in a structured format that allows it to be efficiently managed and retrieved.

[0807] A "server" is a computer system that provides data and services to terminals over a network.

[0808] A "terminal" is an electronic device that provides an interface for users to operate. Examples include smartphones and head-mounted displays.

[0809] "Server means" refers to the methods or devices used by the server to implement a particular function or role.

[0810] "Terminal means" refers to the method or device used by the terminal to realize a specific function or role.

[0811] A "request" is a request or instruction sent from a terminal to a server in response to a user operation.

[0812] "Playback" means to re-display and output video and audio based on digital data.

[0813] "Display" means to visually represent data or information on a terminal screen or display.

[0814] A "generative AI model" is an artificial intelligence system that uses machine learning or deep learning algorithms to generate new data and suggestions.

[0815] A "prompt" is text that is used to design instructions or questions in a format that is easy for a generative AI model to understand.

[0816] "Activity data" refers to recorded data that indicates a user's past and present behavior and interests.

[0817] "Suggestions" are recommendations or suggested actions provided by a generative AI model based on a user's interests.

[0818] "User interests" refer to the particular areas or activities that interest a user.

[0819] This invention is a multifunctional content system for elderly people. This system consists of three main components: a server, a terminal, and a user. This system allows elderly people to watch past videos, maintain their health by viewing exercise videos, restore photos, and experience the metaverse.

[0820] server

[0821] The server provides the following functionality:

[0822] 1. Content Management: Manage content such as videos, images, and exercise videos from the database and provide them upon appropriate requests.

[0823] 2. User authentication: Authentication is performed based on the user's login information.

[0824] 3. Data restoration: Uploaded photos and videos are restored using image processing technology (e.g., OpenCV).

[0825] 4. Metaverse generation: A virtual reality space is generated using Unity or Unreal Engine, allowing users to experience visiting places from the past.

[0826] 5. Suggestions using generative AI models: Specific suggestions are made using generative AI models (e.g., GPT-4) based on user activity data and interests.

[0827] 6. Prompt Generation: Generate prompts for the generative AI model to provide advanced suggestions and support.

[0828] Terminal

[0829] A terminal is a device that provides an interface for users to operate the system. Its specific functions are as follows:

[0830] 1. Content selection: The user selects the desired content from each menu, such as video, exercise video, restoration request, metaverse, will writing, and life purpose suggestions.

[0831] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[0832] 3. Data playback and display: Receives data sent from the server and plays back videos and exercises. Displays restored photos and embodies the virtual space.

[0833] 4. Interface provision: Provide users with the interface necessary for will-writing tools and life purpose suggestions, and send the input information to the server.

[0834] User

[0835] Users are seniors and their caregivers who:

[0836] 1. Login: Enter your account information into the terminal interface to log in to the system.

[0837] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[0838] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[0839] 4. Experience: Watch the playback video, practice the exercise video, check the restored photos, visit the metaverse, write a will, and accept suggestions for new meanings in life through generative AI models.

[0840] Specific examples

[0841] As a concrete example, let's consider a scenario where the user selects "gymnastics videos":

[0842] 1. The user launches the app and enters their credentials on the login screen.

[0843] 2. The server authenticates using OAuth 2.0 and approves the login.

[0844] 3. Select the "Exercise Video" menu on the home screen.

[0845] 4. Send a request to the server to get a list of exercise videos.

[0846] 5. Select the desired exercise video from the list and play the video via streaming from the server.

[0847] 6. The user performs the exercises while watching the video on a smartphone or head-mounted display.

[0848] Example prompts for generative AI models

[0849] Please suggest some exercise videos suitable for seniors to maintain their health. The user's name is Tanaka, a 70-year-old man who has had difficulty walking in the past. Light stretching to improve flexibility and chair exercises would be appropriate.

[0850] Through this system, seniors can not only enjoy a variety of content from the comfort of their own homes, but also easily use digital technology to help maintain their health.

[0851] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0852] Step 1:

[0853] User login

[0854] Input: User authentication information (username, password)

[0855] Output: Authentication token

[0856] Specific behavior:

[0857] The user starts up the device and enters authentication information (username, password) on the login screen. The device sends this authentication information to the server. The server uses OAuth 2.0 to check against the authentication database, and if authentication is successful, generates an authentication token and sends it back to the device.

[0858] Step 2:

[0859] Content selection and request submission

[0860] Input: User content selection information

[0861] Output: Content List

[0862] Specific behavior:

[0863] After logging in, the user selects a menu item such as "Exercise Videos" from the device's home screen. The device then sends a request to the server based on the selected menu item. The server then retrieves a list of related content from the database and returns it to the device.

[0864] Step 3:

[0865] Content playback and display

[0866] Input: Content selected from the content list

[0867] Output: The video being played or the data being displayed

[0868] Specific behavior:

[0869] The user selects the desired video from the content list displayed on the device. The device then transmits the selected content information to the server, which then streams the corresponding content data in real time. The device then plays or displays the received data.

[0870] Step 4:

[0871] Upload and restore photos and videos

[0872] Input: Photo and video files uploaded by users

[0873] Output: Recovered photo and video files

[0874] Specific behavior:

[0875] The user selects the photos and video files they want to restore and uploads them to the server via their device. The server then uses image processing techniques such as OpenCV to restore the data and sends the restored files back to the device. The device then displays the restored data.

[0876] Step 5:

[0877] Creating and experiencing metaverse spaces

[0878] Input: Information about the user's past locations and memories

[0879] Output: Metaverse space

[0880] Specific behavior:

[0881] The user selects the Metaverse menu through the device and inputs the information about past locations and memories they need. The device then sends this information to the server. The server generates a virtual reality space using Unity or Unreal Engine and sends the data back to the device. The user then visits and experiences the virtual space using a device (e.g., a head-mounted display).

[0882] Step 6:

[0883] Individualized proposals using generative AI

[0884] Input: Prompts based on user activity data and interests

[0885] Output: Proposals from the generative AI model

[0886] Specific behavior:

[0887] The server generates prompts based on the user's activity data and interests and sends them to a generative AI model. The AI ​​model analyzes the prompts and generates exercise videos, activity suggestions, and other content that are optimal for the user. This information is then sent to the device and displayed to the user.

[0888] Examples of prompt statements

[0889] Please suggest some exercise videos suitable for seniors to maintain their health. The user's name is Tanaka, a 70-year-old man who has had difficulty walking in the past. Light stretching to improve flexibility and chair exercises would be appropriate.

[0890] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0891] This invention is a multi-functional content system for use by the elderly of the silver generation, and by combining it with an emotion engine, it realizes a system that provides optimal content to users. This system consists of four main components: a server, a terminal, a user, and an emotion engine.

[0892] server

[0893] The server provides the following functionality:

[0894] 1. Content management: Various content (videos, exercise videos, photo restoration, metaverse, will creation tool, and life purpose suggestions) is stored in a database and provided upon appropriate request.

[0895] 2. User authentication: When a user logs in through a terminal, the user is authenticated by checking against the authentication database.

[0896] 3. Data recovery: Photos and videos uploaded by users from their devices are recovered using image processing technology.

[0897] 4. Metaverse generation: Generate a virtual reality space and allow users to virtually visit places from the past.

[0898] 5. Emotional Data Management: Emotional data recognized by the emotion engine is stored in a database to provide personalized content.

[0899] Terminal

[0900] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[0901] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, metaverse, will creation, and purpose in life suggestions.

[0902] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[0903] 3. Data playback and display: Receives data sent from the server, plays back videos and exercise videos, and displays restored photos and virtual spaces.

[0904] 4. Interface provision: Provide users with the interface necessary for will-writing tools and life purpose suggestions, and send the input information to the server.

[0905] 5. Emotion data transmission: The emotion data recognized by the emotion engine is transmitted to the server.

[0906] User

[0907] Users are seniors and their caregivers who:

[0908] 1. Login: Enter your account information into the terminal interface to log in to the system.

[0909] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[0910] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[0911] 4. Experience: Watch the playback video, try the exercise video, check the restored photos, visit the Metaverse space, write a will, and accept suggestions for a new purpose in life.

[0912] 5. Emotion input: Emotion data is provided in real time through the device's built-in camera and microphone.

[0913] Emotion Engine

[0914] The Emotion Engine provides the following features:

[0915] 1. Emotion recognition: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time.

[0916] 2. Emotion data generation: Generates recognized emotion data and sends it to the server via the terminal.

[0917] 3. Content recommendation: Based on the recognized sentiment, the server recommends the most suitable content.

[0918] As a concrete example, let us consider a scenario in which a user watches a video. First, the user logs in to the device and selects the "Watch Video" menu from the home screen. The device sends a content request to the server. The server retrieves a video list from the database and sends it to the device. The device displays the list and the user selects a video that interests them. The emotion engine recognizes the user's emotions in real time and sends that data to the server. Based on the recognized emotion data, the server selects a video that best suits the user's mood and sends it to the device. The device plays the video, and the user watches it.

[0919] In this way, this system provides an interface that is easy for users to operate, and the emotion engine recognizes the user's emotions in real time to provide optimal content, thereby improving the quality of life for the elderly and reducing the burden on caregivers.

[0920] The processing flow will be explained below.

[0921] Video viewing process flow

[0922] Step 1:

[0923] The user enters account information into the terminal interface and presses the login button.

[0924] Step 2:

[0925] The terminal transmits the user's input information to the server.

[0926] Step 3:

[0927] The server compares the received information with the database and returns the authentication result to the terminal.

[0928] Step 4:

[0929] The device displays the authentication result to the user, and if successful, transitions to the home screen.

[0930] Step 5:

[0931] The user selects the "Watch Video" menu from the home screen.

[0932] Step 6:

[0933] The terminal sends a content request to the server.

[0934] Step 7:

[0935] The server retrieves a list of video content from the database and transmits it to the terminal.

[0936] Step 8:

[0937] The device displays a list of videos.

[0938] Step 9:

[0939] The user selects the video they want to watch from the list.

[0940] Step 10:

[0941] The terminal sends a request for details of the selected video to the server.

[0942] Step 11:

[0943] The server reads the video data from the database and sends it to the terminal.

[0944] Step 12:

[0945] The emotion engine captures the user's facial expressions and voice data and analyzes emotions in real time.

[0946] Step 13:

[0947] The emotion engine sends the analysis results to the server.

[0948] Step 14:

[0949] The server receives the emotion data and selects the most appropriate video according to the user's current emotional state.

[0950] Step 15:

[0951] The server transmits the selected optimal video data to the terminal.

[0952] Step 16:

[0953] The terminal receives the video data and starts playing it.

[0954] Step 17:

[0955] The user watches the video.

[0956] Photo and video restoration process flow

[0957] Step 1:

[0958] The user selects the "Photo / Video Recovery" menu from the home screen.

[0959] Step 2:

[0960] The terminal displays a file selection screen to the user.

[0961] Step 3:

[0962] The user selects the photo or video files they want to restore.

[0963] Step 4:

[0964] The device uploads the selected file to the server.

[0965] Step 5:

[0966] The server stores the received file in the database and executes the restoration process.

[0967] Step 6:

[0968] The server sends the results of the restoration process to the terminal.

[0969] Step 7:

[0970] The emotion engine captures the user's facial expressions and reaction data and analyzes their emotions.

[0971] Step 8:

[0972] The emotion engine sends the analysis results to the server.

[0973] Step 9:

[0974] The device will display the restored photos and videos.

[0975] Step 10:

[0976] The user reviews the restored content.

[0977] Metaverse Processing Flow

[0978] Step 1:

[0979] The user selects the "Metaverse" menu from the home screen.

[0980] Step 2:

[0981] The device sends a metaverse request to the server.

[0982] Step 3:

[0983] The server generates metaverse spatial data and sends it to the terminal.

[0984] Step 4:

[0985] The device displays the metaverse space and provides the user with a virtual reality experience.

[0986] Step 5:

[0987] The emotion engine analyzes the user's voice and facial expressions and sends the emotion data to the server.

[0988] Step 6:

[0989] The server receives the emotion data and applies experience content appropriate to the user's emotional state.

[0990] Step 7:

[0991] A user visits past locations in the metaverse.

[0992] Process flow for writing a will

[0993] Step 1:

[0994] The user selects the "Write a Will" menu from the home screen.

[0995] Step 2:

[0996] The device displays a will creation tool.

[0997] Step 3:

[0998] The user inputs the contents of the will.

[0999] Step 4:

[1000] The emotion engine analyzes the user's facial expressions and voice and sends the emotion data to the server.

[1001] Step 5:

[1002] The terminal transmits the entered will data to the server.

[1003] Step 6:

[1004] The server stores the will data in a database and sends the confirmation results to the terminal.

[1005] Step 7:

[1006] The terminal displays the confirmation result to the user.

[1007] Purpose of Life Proposal Processing Flow

[1008] Step 1:

[1009] The user selects the "Life Purpose Suggestions" menu from the home screen.

[1010] Step 2:

[1011] The device displays a questionnaire about the user's interests and preferences.

[1012] Step 3:

[1013] The user answers the question and submits it.

[1014] Step 4:

[1015] The emotion engine analyzes the user's responses and real-time emotional state and sends the emotional data to the server.

[1016] Step 5:

[1017] The terminal transmits the user's response data to the server.

[1018] Step 6:

[1019] The server analyzes the data and suggests appropriate hobbies and activities.

[1020] Step 7:

[1021] The server sends the proposal results to the terminal.

[1022] Step 8:

[1023] The device displays the suggestions to the user.

[1024] Step 9:

[1025] The user confirms and engages in suggested activities and hobbies.

[1026] These processing flows create a system that is easy for elderly people to use, allows them to enjoy a variety of content, and, by using an emotion engine, provides a personalized experience, reducing the burden on caregivers.

[1027] Example 2

[1028] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1029] Current content systems for the elderly do not provide content that takes into account the user's emotions, resulting in a poor user experience. Other issues include the difficulty of restoring past photos and videos, and the lack of a virtual reality space that is easy for the elderly to use.

[1030] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1031] In this invention, the server includes an information processing device that retrieves data from a database according to content selected by the user and provides it to the device being used, an emotion analysis device that analyzes the user's voice, facial expression, and movement data to recognize emotions in real time, and a device that recommends optimal content based on the emotion data. This makes it possible to provide optimal content according to the user's emotions, improving the quality of the user experience. It also makes it possible to realize an environment that makes it easy to restore photos and videos and provide virtual reality spaces.

[1032] The term "information processing device means" refers to a device that retrieves data from a database according to content selected by a user and provides the data to the device being used.

[1033] The "device means" refers to a device that allows a user to select content and send a request to an information processing device. It also includes a function to play and display data provided by the information processing device.

[1034] "Emotion analysis means" refers to a device or program that analyzes the user's voice, facial expression, and movement data and recognizes emotions in real time.

[1035] "Means" generally refers to a method, apparatus, or program designed to achieve a specific purpose.

[1036] "Data" refers to information or content acquired and used via information processing device means, and includes various formats such as video, images, audio, and text.

[1037] "Virtual space providing means" refers to a device or program that enables a user to visit past locations in a virtual reality space.

[1038] This invention is a multi-functional content system for use by the elderly of the silver generation, and by combining it with an emotion analysis function, it realizes a system that provides optimal content to users. This system consists of four main components: an information processing device, a user device, a user, and an emotion analysis function.

[1039] Information processing device

[1040] The information processing device provides the following functions:

[1041] 1. Content management: The information processing device stores various content (videos, exercise videos, image restoration, virtual space, will writing tools, and life purpose suggestions) in a database and provides this content in response to user requests. This process uses a database such as MySQL.

[1042] 2. User authentication: When a user logs in through their device, the user is authenticated by checking against an authentication database. Authentication is performed using LDAP or OAuth2.

[1043] 3. Image restoration: The information processing device restores the images and videos uploaded by the user from their device using image processing technology (e.g., OpenCV).

[1044] 4. Virtual space generation: The information processing device generates a virtual reality space, allowing users to virtually visit places from the past. This process uses Unreal Engine and Unity.

[1045] 5. Emotional data management: The information processing device stores the emotional data recognized by the emotion analysis function in a database and provides personalized content.

[1046] Equipment used

[1047] A user device is a device that provides an interface for users to operate the system. It has the following functions:

[1048] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, virtual space, will writing, and purpose in life suggestions.

[1049] 2. Request transmission: Based on the selected content, an appropriate request is sent to the information processing device. An HTTP request is used.

[1050] 3. Data playback and display: Receives data sent from the information processing device, plays back images and exercise videos, and displays restored images and virtual spaces.

[1051] 4. Interface provision: Provides the user with the interface necessary for the will writing tool and life purpose suggestions, and sends the input information to the information processing device.

[1052] 5. Emotion data transmission: Emotion data recognized by the emotion analysis function is transmitted to the information processing device.

[1053] User

[1054] Users are seniors and their caregivers who:

[1055] 1. Login: Enter your account information into the interface of your device to log in to the system.

[1056] 2. Content usage: Select the desired content from each menu and operate it through the interface of the device you are using.

[1057] 3. Upload: Upload the images or videos you want to restore from your device to the information processing device.

[1058] 4. Experience: Watch the video, try out the exercises, check the restored images, and visit the virtual space. You can also write a will and accept suggestions for a new purpose in life.

[1059] 5. Emotion input: Emotion data is provided in real time through the camera and microphone built into the device being used.

[1060] Sentiment analysis function

[1061] The sentiment analysis feature provides the following features:

[1062] 1. Emotion recognition: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time, using technologies such as OpenFace and DeepFace.

[1063] 2. Emotion data generation: Recognized emotion data is generated and sent to the information processing device via the device being used.

[1064] 3. Content recommendation: Based on the recognized emotions, the information processing device recommends the most suitable content.

[1065] Example scenario

[1066] As a specific example, a scenario in which a user watches a video will be described. First, the user logs in to the device being used and selects the "Watch Video" menu from the home screen. The device being used sends a content request to the information processing device. The information processing device retrieves a video list from a database and sends it to the device being used. The device being used displays the list, and the user selects a video of interest. The emotion analysis function recognizes the user's emotions in real time and sends that data to the information processing device. Based on the recognized emotion data, the information processing device selects a video that best suits the user's mood and sends it to the device being used. The device being used plays the video, and the user watches it.

[1067] Example prompt:

[1068] "Please explain the process of watching videos using a multi-functional content system for seniors in the silver generation. Please explain how a user logs in, selects a video, and how the emotion analysis function works."

[1069] In this way, this system provides an interface that is easy for users to operate, and the emotion analysis function recognizes the user's emotions in real time to provide optimal content, thereby improving the quality of life for the elderly and reducing the burden on caregivers.

[1070] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1071] The program processing flow of this system and specific steps

[1072] Processing Steps

[1073] Step 1: User Login

[1074] Step 2: Menu selection

[1075] Step 3: Submit a content request

[1076] Step 4: Providing and displaying content

[1077] Step 5: Collect and send emotion data

[1078] Step 6: Optimize and reselect content

[1079] Specific explanation of each processing step

[1080] Step 1: User Login

[1081] Input: Account information (ID and password)

[1082] Output: Authentication success or failure response

[1083] The user enters their account information on the login screen of their device and presses the login button. The device sends the entered information to the server as an HTTP POST request. The server verifies the authentication information using an LDAP or MySQL database, and if authentication is successful, returns a successful authentication response to the device. If authentication is unsuccessful, it returns an error message.

[1084] Specific operation: The user enters "ID: user123" and "Password: pass456" and clicks the login button. The device sends this information to the server, which queries the LDAP database for authentication.

[1085] Step 2: Menu selection

[1086] Input: User-selected menu item

[1087] Output: Selected menu information

[1088] The user selects the content they want to use (e.g., watching a video) from the menu displayed on the home screen. The device sends the selected menu information to the server via an HTTP GET request.

[1089] Specific operation: The user selects the "Watch Video" menu from the home screen and clicks the "Watch Video" button on the screen. The device sends a request including the selection information to the server.

[1090] Step 3: Submit a content request

[1091] Input: Selected menu information

[1092] Output: A list of matching content

[1093] The terminal transmits the selected menu information to the server, and the server retrieves the corresponding content list from the database and transmits the list to the terminal.

[1094] Specific operation: The device sends a "video viewing" request to the server. The server retrieves the video list from the MySQL database and sends it to the device in JSON format.

[1095] Step 4: Providing and displaying content

[1096] Input: Content list received from the server

[1097] Output: User selects and retrieves content

[1098] The device displays the content list received from the server to the user. The user selects the content of interest from the list, and the device sends the selection information to the server. The server retrieves the relevant content from the database and sends it to the device. The device plays and displays the content.

[1099] Specific operation: The user selects "Video A" from the displayed list and clicks the play button. The device sends the ID of Video A to the server, and the server retrieves the video data and sends the playback data to the device.

[1100] Step 5: Collect and send emotion data

[1101] Input: User's voice, facial expression, and movement data

[1102] Output: Generated emotion data

[1103] While the device is playing video, the camera and microphone collect the user's voice, facial expressions, and movement data. The emotion analysis function analyzes this data and generates emotion data in real time. The emotion data is then sent to the server via an HTTP POST request.

[1104] Specific operation: The device detects the user's facial expressions and voice, and the emotion analysis function recognizes that the user is "having fun." Emotional data is generated and sent to the server.

[1105] Step 6: Optimize and reselect content

[1106] Input: Received emotion data

[1107] Output: Recommend and serve new content

[1108] The server recommends new content based on the received emotion data. The device displays the recommended content to the user, and the user selects it again. The reselected content is then retrieved from the server and displayed.

[1109] Specific operation: The server analyzes the emotional data of "enjoyed" and recommends the more suitable "Video B." The device presents "Video B" to the user, who selects and plays it.

[1110] This will enable optimal content to be provided according to the user's emotions, improving the quality of life for the elderly.

[1111] (Application example 2)

[1112] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1113] When elderly people use content, it is often difficult to operate, and the content provided is often not suited to their individual emotions or state. This leads to problems that reduce the quality of life of the elderly and increase the burden on caregivers. In addition, there is a lack of appropriate exercise guidance to support psychological and physical health, and functions to recreate past memories in virtual space.

[1114] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes information collection device means for acquiring data from the data collection management device in accordance with information selected by the user and providing the data to the information device, information device means for the user to select information and send a request to the information collection management device, information device means for playing and displaying the data provided from the information collection management device, emotion recognition means for recognizing the user's emotion in real time and sending the data to the information collection management device, and information recommendation means for recommending optimal information based on the recognized emotion data. This provides an interface that is easy for the user to operate, and makes it possible to provide optimal content by having the emotion engine recognize the user's emotion in real time.

[1115] "User" refers to an individual or senior citizen who uses the system.

[1116] An "information accumulation and management device" is a device that stores and manages data and provides it as needed.

[1117] "Information equipment" refers to devices that are operated by users, such as smart glasses and smartphones.

[1118] The "information accumulation device means" is a function or system for acquiring data from the information accumulation management device and providing it to the information device.

[1119] The "information device means" is a function or system that allows a user to select information and send a request to the information accumulation management device.

[1120] The "information device display means" is a function or system for reproducing data provided from the information accumulation management device and displaying it on an information device.

[1121] The "emotion recognition means" is a function or system for recognizing the user's emotions in real time and transmitting the data to the information collection and management device.

[1122] The "information recommendation means" is a function or system for recommending optimal information to a user based on the recognized emotion data.

[1123] "Image restoration means" refers to a function or system for restoring images and videos uploaded by users.

[1124] The "virtual space generation means" is a function or system for generating a virtual space in which the user can visit places from the past.

[1125] This invention is a multi-functional content system for use by elderly people in the silver generation, which provides optimal content according to the user's emotions by combining it with an emotion recognition engine. This system consists of four main components: an information accumulation and management device (server), information equipment (terminal), users, and the emotion recognition engine.

[1126] Information accumulation and management device (server)

[1127] The server provides the following functionality:

[1128] 1. Information accumulation device means: Acquires data from a database according to information selected by the user and provides it to the information device.

[1129] 2. Image restoration tool: It has the function to restore images and videos uploaded by users.

[1130] 3. Virtual space generation means: A past location is generated as a virtual reality space and provided to the user.

[1131] 4. Emotion data management: Emotion data recognized by the emotion recognition engine is stored in a database and personalized content is provided.

[1132] Information equipment (terminals)

[1133] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[1134] 1. Information device means: The user selects information and sends a request to the server.

[1135] 2. Information device display means: Receives data provided by the server, plays videos and exercise videos, and displays restored photos and virtual spaces.

[1136] 3. Input interface: Provides users with the interface necessary for the will-writing tool and emotion-based purpose-in-life suggestions, and sends the input content to the server.

[1137] 4. Emotion recognition means: Using the device's built-in camera and microphone, the user's emotional data is acquired in real time and sent to the server.

[1138] User

[1139] Users are seniors and their caregivers who:

[1140] 1. Login: Enter your account information into the terminal interface to log in to the system.

[1141] 2. Content usage: Select the desired information from each menu and operate it through the device interface.

[1142] 3. Upload: Upload the images or videos you want to restore from your device to the server.

[1143] 4. Experience: Watch the video, try out the exercises, check the restored photos, visit the virtual space, and even write a will and accept suggestions for a new purpose in life.

[1144] 5. Emotion input: Provide emotion data in real time via camera or microphone.

[1145] Emotion Recognition Engine

[1146] The emotion recognition engine provides the following features:

[1147] 1. Emotion recognition means: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time.

[1148] 2. Emotion data transmission: Generate recognized emotion data and transmit it to the server via the terminal.

[1149] 3. Information recommendation method: Based on the recognized emotions, the server recommends the most appropriate information.

[1150] Specific examples

[1151] For example, suppose a user is wearing smart glasses and sitting in their living room in a relaxed mood. The emotion recognition engine recognizes the user's relaxed state in real time. Based on this, the server will recommend a landscape video that is most suitable for the relaxed mood, and a beautiful natural scene will be displayed on the smart glasses' display. The user can enjoy this visual experience while remaining relaxed.

[1152] Prompt Sentence Examples

[1153] "How can we recommend beautiful landscape videos to elderly people who are in a relaxed state?"

[1154] "Design a system that uses an emotion recognition engine to recognize emotions in real time and display content based on that in smart glasses."

[1155] The above is a specific embodiment for carrying out the present invention.

[1156] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1157] Step 1:

[1158] A user logs in to an information device (e.g., smart glasses). As input, the user enters account information, and the server authenticates the user based on this information. As output, if authentication is successful, the user can access the home screen.

[1159] Step 2:

[1160] The user selects the desired content menu (e.g., watching a video, watching a gymnastics video, visiting a virtual space) from the home screen. The user's selection is received as input, and this data is sent from the device to the server. As output, the server provides the device with a list of the corresponding content.

[1161] Step 3:

[1162] The server uses an emotion recognition engine to obtain the user's emotion data in real time. It receives data from the camera and microphone as input, analyzes the emotion using an emotion recognition model, and generates the recognized emotion data as output, which is then sent to the server.

[1163] Step 4:

[1164] The server recommends optimal content based on the received emotional data. It receives the emotional data and content request as input, selects optimal content from a database based on the received emotional data, and sends the selected content to the device as output.

[1165] Step 5:

[1166] The terminal plays the content sent from the server and displays it to the user. As input, it takes the received content data and converts it into a format for display. As output, the user can visually confirm the played content on the display of the smart glasses.

[1167] Step 6:

[1168] When a user uploads an image or video they want to repair, their device sends these files to the server. As input, it takes the image or video file and uploads it to the server. As output, the server starts the repair process.

[1169] Step 7:

[1170] The server sends the repaired image or video file to the device. It receives the file to be repaired as input, processes it using an image repair algorithm, and provides the repaired file to the device as output.

[1171] Step 8:

[1172] A user sends a request to visit a virtual space from a terminal. The terminal receives the request data for visiting the virtual space as input and sends it to the server. As output, the server generates a virtual space and provides it to the terminal.

[1173] Step 9:

[1174] The terminal displays the virtual space provided by the server and provides the user with an experience. It receives virtual space data as input and converts it into a display format. As an output, the user can enjoy the experience in the virtual space.

[1175] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1176] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1177] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1178] [Third embodiment]

[1179] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1180] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1181] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1182] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1183] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1184] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1185] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1186] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1187] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1188] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1189] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1190] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1191] This invention is a multi-functional content system for use by the elderly of the silver generation. This system consists of three main components: a server, a terminal, and a user.

[1192] server

[1193] The server provides the following functionality:

[1194] 1. Content management: Various content (videos, exercise videos, photo restoration, metaverse, will creation tool, and life purpose suggestions) is stored in a database and provided upon appropriate request.

[1195] 2. User authentication: When a user logs in through a terminal, the user is authenticated by checking against the authentication database.

[1196] 3. Data recovery: Photos and videos uploaded by users from their devices are recovered using image processing technology.

[1197] 4. Metaverse generation: Generate a virtual reality space and allow users to virtually visit places from the past.

[1198] Terminal

[1199] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[1200] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, metaverse, will creation, and purpose in life suggestions.

[1201] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[1202] 3. Data playback and display: Receives data sent from the server, plays back videos and exercise videos, and displays restored photos and virtual spaces.

[1203] 4. Interface provision: Provide users with the interface necessary for will-writing tools and life purpose suggestions, and send the input information to the server.

[1204] User

[1205] Users are seniors and their caregivers who:

[1206] 1. Login: Enter your account information into the terminal interface to log in to the system.

[1207] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[1208] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[1209] 4. Experience: Watch the playback video, try the exercise video, check the restored photos, visit the Metaverse space, write a will, and accept suggestions for a new purpose in life.

[1210] As a concrete example, let us consider a scenario in which a user watches old videos. First, the user logs in to the device and selects the "Watch Videos" menu from the home screen. The device sends a content request to the server. The server retrieves a video list from the database and sends it to the device. The device displays the list, and the user selects a video of interest. The server reads the selected video data from the database and sends it to the device. The device plays the video, and the user watches it.

[1211] In this way, this system provides an interface that is easy for users to operate, and the server manages and provides a variety of content, thereby enriching the lives of the elderly and reducing the burden on caregivers.

[1212] The processing flow will be explained below.

[1213] Video viewing process flow

[1214] Step 1:

[1215] The user enters account information into the terminal interface and presses the login button.

[1216] Step 2:

[1217] The terminal transmits the user's input information to the server.

[1218] Step 3:

[1219] The server compares the received information with the database and returns the authentication result to the terminal.

[1220] Step 4:

[1221] The device displays the authentication result to the user, and if successful, transitions to the home screen.

[1222] Step 5:

[1223] The user selects the "Watch Video" menu from the home screen.

[1224] Step 6:

[1225] The terminal sends a content request to the server.

[1226] Step 7:

[1227] The server retrieves a list of video content from the database and transmits it to the terminal.

[1228] Step 8:

[1229] The device displays a list of videos.

[1230] Step 9:

[1231] The user selects the video they want to watch from the list.

[1232] Step 10:

[1233] The terminal sends a request for details of the selected video to the server.

[1234] Step 11:

[1235] The server reads the video data from the database and sends it to the terminal.

[1236] Step 12:

[1237] The terminal receives the video data and starts playing it.

[1238] Step 13:

[1239] The user watches the video.

[1240] Photo and video restoration process flow

[1241] Step 1:

[1242] The user selects the "Photo / Video Recovery" menu from the home screen.

[1243] Step 2:

[1244] The terminal displays a file selection screen to the user.

[1245] Step 3:

[1246] The user selects the photo or video files they want to restore.

[1247] Step 4:

[1248] The device uploads the selected file to the server.

[1249] Step 5:

[1250] The server stores the received file in the database and executes the restoration process.

[1251] Step 6:

[1252] The server sends the results of the restoration process to the terminal.

[1253] Step 7:

[1254] The device will display the restored photos and videos.

[1255] Step 8:

[1256] The user reviews the restored content.

[1257] Metaverse Processing Flow

[1258] Step 1:

[1259] The user selects the "Metaverse" menu from the home screen.

[1260] Step 2:

[1261] The device sends a metaverse request to the server.

[1262] Step 3:

[1263] The server generates metaverse spatial data and sends it to the terminal.

[1264] Step 4:

[1265] The device displays the metaverse space and provides the user with a virtual reality experience.

[1266] Step 5:

[1267] A user visits past locations in the metaverse.

[1268] Process flow for writing a will

[1269] Step 1:

[1270] The user selects the "Write a Will" menu from the home screen.

[1271] Step 2:

[1272] The device displays a will creation tool.

[1273] Step 3:

[1274] The user inputs the contents of the will.

[1275] Step 4:

[1276] The terminal transmits the entered will data to the server.

[1277] Step 5:

[1278] The server stores the will data in a database and sends the confirmation results to the terminal.

[1279] Step 6:

[1280] The terminal displays the confirmation result to the user.

[1281] Purpose of Life Proposal Processing Flow

[1282] Step 1:

[1283] The user selects the "Life Purpose Suggestions" menu from the home screen.

[1284] Step 2:

[1285] The device displays a questionnaire about the user's interests and preferences.

[1286] Step 3:

[1287] The user answers the question and submits it.

[1288] Step 4:

[1289] The terminal transmits the user's response data to the server.

[1290] Step 5:

[1291] The server analyzes the data and suggests appropriate hobbies and activities.

[1292] Step 6:

[1293] The server sends the proposal results to the terminal.

[1294] Step 7:

[1295] The device displays the suggestions to the user.

[1296] These processing flows will realize a system that is easy for elderly people to use, allows them to enjoy a variety of content, and reduces the burden on caregivers.

[1297] Example 1

[1298] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1299] Conventional content systems for seniors were difficult for seniors to use because they were complicated to operate. Furthermore, there was no system that offered a variety of functions, such as image and video restoration, virtual reality space experiences, and tools for writing wills, all in one place. Furthermore, it was necessary to use separate systems and devices to access specific content, which caused confusion for seniors.

[1300] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1301] In this invention, the server includes an information processing device that retrieves data from a database according to content selected by the user and provides it to the terminal, means for the user to input login information and the information processing device compares the information with the authentication database for authentication, means for the user to upload photos or videos that the user wants to restore from the display device and the information processing device restores them using image processing technology, means for generating a virtual reality space that allows the user to visit past locations in a virtual reality space, and means for the information processing device to generate the virtual reality space and provide it to the display device. This makes it possible to provide a system that provides multifunctional content all at once and is easy for seniors to use.

[1302] An "information processing device" is a device that retrieves data from a database based on a request from a user, performs the specified operation or processing, and provides the results to a terminal.

[1303] The "display device" is a device that allows a user to input data and perform operations via an interface, and that reproduces and displays data transmitted from an information processing device.

[1304] "User authentication" is the process of authenticating a user when they log in to a system by checking the entered login information against an authentication database.

[1305] "Content" refers to digital information and data such as videos, exercise videos, photographs, virtual reality spaces, will-writing tools, and suggestions for purpose in life.

[1306] A "database" is a data storage system for managing and storing various content and user authentication information within the system.

[1307] "Image processing techniques" are techniques used to restore uploaded photos and videos, including noise reduction, resolution enhancement, and color correction.

[1308] "Virtual reality space" refers to a three-dimensional computer-simulated space created to allow users to virtually visit places from the past.

[1309] A "will creation tool" is an application that allows users to create and save their wishes and messages in digital form.

[1310] "Purpose in Life Suggestion" is a system or function that suggests new hobbies and activities to users.

[1311] MODE FOR CARRYING OUT THE INVENTION

[1312] This invention is a multi-functional content system for the elderly, consisting of three main components: a user, a terminal, and a server. This system allows elderly people to easily access various content and enrich their daily lives. Each component of this invention and its operation will be described in detail below.

[1313] 1. Server

[1314] The server plays a central role in the system and provides the following functions:

[1315] 1.1 Content Management

[1316] The server stores various content (videos, exercise videos, photo restoration, virtual reality space, will writing tool, and purpose-in-life suggestions) in a database and provides them in response to user requests. Specifically, the server uses a database management system (DBMS) to efficiently manage content data.

[1317] 1.2 User Authentication

[1318] When a user logs in via a terminal, the server authenticates them against an authentication database. This is achieved using security protocols (e.g., SSL / TLS) and multi-factor authentication technology. Specifically, a mechanism is adopted in which an authentication code is sent after the user enters their username and password.

[1319] 1.3 Data Recovery

[1320] The server uses image processing technology to restore photos and videos uploaded by users from their devices. Specific techniques used include image restoration models using deep learning (e.g., Generative Adversarial Networks, GANs). Using a high-performance GPU enables efficient data processing.

[1321] 1.4 Virtual reality space generation

[1322] The server generates a virtual reality space, allowing users to virtually visit places from the past. This is done using VR development platforms such as Unity and Unreal Engine. The generated virtual reality space data is sent to the device in real time.

[1323] 2. Terminal

[1324] The terminal provides an interface for users to operate the system and has the following functions:

[1325] 2.1 Content Selection

[1326] It provides an interface for users to select the content they want from the menus of video, exercise videos, restoration requests, virtual reality space, will writing, and life purpose suggestions. Operation becomes even easier if the device is equipped with a touch screen or voice recognition function.

[1327] 2.2 Request Submission

[1328] Based on the selected content, the device sends an appropriate request to the server. The request is sent using the HTTP / HTTPS protocol. For example, the request might say "The user has selected to watch a video."

[1329] 2.3 Data playback and display

[1330] The device receives the data (video, exercise video, restored photos, virtual reality space) sent from the server and plays and displays it. A device with a GPU is recommended for video playback and virtual reality display.

[1331] 2.4 Interface Provision

[1332] The system provides users with an interface for writing wills and suggesting purpose in life, and sends this to the server. The content entered by the user (e.g., "I want to find a new hobby") is sent to the server, and appropriate feedback is returned to the device.

[1333] 3. Users

[1334] The users are elderly people and their caregivers, and perform the following operations:

[1335] 3.1 Login

[1336] A user logs in to the system by entering account information (user name, password, etc.) on the login screen of the terminal. For example, the user enters the user name "tanaka123" and the password "password123."

[1337] 3.2 Content Use

[1338] Users can select the desired content from each menu and operate it through the device interface. Specifically, they can select the "Video Viewing" menu to watch old travel videos.

[1339] 3.3 Upload

[1340] Upload the photos or videos you want to restore from your device to the server. For example, upload "Photos from a family trip." After the server completes the restoration process, the restored data will be displayed on your device.

[1341] 3.4 Experience

[1342] Users can watch playback videos, practice exercise videos, view restored photos, and visit virtual reality spaces. They can also write wills and accept suggestions for new purposes in life. When using the virtual reality space, users put on a VR headset and explore virtual locations in real time.

[1343] Specific examples

[1344] Login: When a user attempts to log in by entering "tanaka123" and "password123", the server authenticates them and returns a success message.

[1345] Video viewing: When Tanaka selects "Video viewing," the device sends a request to the server, which returns a list of videos. Tanaka selects "Video of family trip" and watches it.

[1346] Photo restoration: When Tanaka tries to restore an old family photo by uploading it from his device, the server restores it and sends the photo back to his device, where Tanaka can view it.

[1347] Virtual reality experience: When Tanaka selects "Virtual reality space of his hometown," the server generates a virtual space, sends it to his device, and Tanaka experiences it.

[1348] Writing a Will: When Tanaka selects the "Will Writing" tool and enters the content, it is sent to the server, where it is processed and feedback is returned.

[1349] In this way, this system provides an interface that users can operate intuitively, and by efficiently managing and providing multifunctional content, it enriches the lives of the elderly and reduces the burden on caregivers.

[1350] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1351] Step 1: The user types the request into the terminal

[1352] A user attempts to log in by entering account information (username and password) into the terminal interface. The input data is provided by the user and sent to the terminal. Based on this, an authentication request is generated and sent to the server.

[1353] Step 2: The device sends an authentication request to the server

[1354] The terminal converts the authentication information entered by the user into a packet format and sends it to the server using a communication protocol (e.g., HTTPS). The input data is the user's account information, and communication data is generated based on this.

[1355] Step 3: Server authenticates

[1356] The server checks the received authentication information against an authentication database. It queries the authentication database to see if the corresponding user information exists. The input data is the authentication request, and the output data is the authentication result (success or failure).

[1357] Step 4: The server returns the authentication result to the device.

[1358] The server generates an authentication result and returns it to the terminal. If successful, it generates and adds a session ID, etc., which gives the user permission to access the system. The output data is the authentication result and the session ID.

[1359] Step 5: The device displays the authentication result

[1360] The terminal displays the authentication result received from the server to the user. If successful, the home screen is displayed and the user is guided to the next operation. The input data is the authentication result, and the output data is an interface that the user can operate.

[1361] Step 6: User selects the content menu

[1362] The user selects the desired content menu (e.g., video viewing, virtual reality experience) from the home screen. The user's operation is input to the terminal, and the next request is generated. The input data is the user's selection.

[1363] Step 7: The device sends a content request to the server

[1364] The terminal generates a content request based on the content menu selected by the user and sends it to the server, which then provides the specified content. The input data is the user's content selection.

[1365] Step 8: The server provides the content list

[1366] The server retrieves the relevant content list from the database based on the request, executes a query against the database, and sends the retrieved list to the terminal. The input data is the content request, and the output data is the content list.

[1367] Step 9: The device displays the content list to the user.

[1368] The terminal displays the content list received from the server to the user. The user selects more specific content from the displayed list. The input data is the content list, and the output data is an interface that the user can operate.

[1369] Step 10: User Selects Specific Content

[1370] The user selects a specific item (e.g., a video of a family trip) from the displayed content list. The selection is input to the terminal, and the next request is generated. The input data is the user's specific content selection.

[1371] Step 11: The device sends a content data request to the server

[1372] The terminal generates and sends a content data request to the server based on the specific content selected by the user, with the input data being the specific content selection.

[1373] Step 12: The server retrieves and sends the content data

[1374] The server retrieves specific content data from the database based on the request, converts the retrieved data into a packet format, and sends it to the terminal. The input data is the content data request, and the output data is the content data.

[1375] Step 13: Your device plays and displays the content

[1376] The terminal plays and displays the content data received from the server to the user. In the case of video, for example, continuous playback is performed using a playback engine. The input data is the content data, and the output data is the content to be played and displayed.

[1377] This process flow allows users to intuitively operate the system and use a wide variety of content.

[1378] (Application example 1)

[1379] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1380] Today's elderly people need access to a variety of content and services to maintain their physical and mental health. However, they often find it difficult to understand and operate digital technology, which creates psychological barriers. Furthermore, they have limited means of reminiscing on past memories, creating a need for systems that allow them to easily access these services. However, existing systems are limited to solving individual issues, and few offer comprehensive, multifunctional services. Therefore, a multifunctional content system is needed that allows elderly people to easily access multiple services and improve their quality of life.

[1381] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1382] In this invention, the server includes server means for retrieving data from a database according to content selected by a user and providing it to a terminal, terminal means for the user to select content and send a request to the server, terminal means for playing and displaying the data provided from the server, means for making suggestions based on the user's activity data and interests using a generative AI model, and means for generating prompt sentences. This allows elderly people to easily access a variety of content and services and receive suggestions tailored to their individual needs.

[1383] "User-selected content" refers to multimedia materials or services that a user wishes to view or execute by operating a terminal.

[1384] A "database" is a digital system that stores data in a structured format that allows it to be efficiently managed and retrieved.

[1385] A "server" is a computer system that provides data and services to terminals over a network.

[1386] A "terminal" is an electronic device that provides an interface for users to operate. Examples include smartphones and head-mounted displays.

[1387] "Server means" refers to the methods or devices used by the server to implement a particular function or role.

[1388] "Terminal means" refers to the method or device used by the terminal to realize a specific function or role.

[1389] A "request" is a request or instruction sent from a terminal to a server in response to a user operation.

[1390] "Playback" means to re-display and output video and audio based on digital data.

[1391] "Display" means to visually represent data or information on a terminal screen or display.

[1392] A "generative AI model" is an artificial intelligence system that uses machine learning or deep learning algorithms to generate new data and suggestions.

[1393] A "prompt" is text that is used to design instructions or questions in a format that is easy for a generative AI model to understand.

[1394] "Activity data" refers to recorded data that indicates a user's past and present behavior and interests.

[1395] "Suggestions" are recommendations or suggested actions provided by a generative AI model based on a user's interests.

[1396] "User interests" refer to the particular areas or activities that interest a user.

[1397] This invention is a multifunctional content system for elderly people. This system consists of three main components: a server, a terminal, and a user. This system allows elderly people to watch past videos, maintain their health by viewing exercise videos, restore photos, and experience the metaverse.

[1398] server

[1399] The server provides the following functionality:

[1400] 1. Content Management: Manage content such as videos, images, and exercise videos from the database and provide them upon appropriate requests.

[1401] 2. User authentication: Authentication is performed based on the user's login information.

[1402] 3. Data restoration: Uploaded photos and videos are restored using image processing technology (e.g., OpenCV).

[1403] 4. Metaverse generation: A virtual reality space is generated using Unity or Unreal Engine, allowing users to experience visiting places from the past.

[1404] 5. Suggestions using generative AI models: Specific suggestions are made using generative AI models (e.g., GPT-4) based on user activity data and interests.

[1405] 6. Prompt Generation: Generate prompts for the generative AI model to provide advanced suggestions and support.

[1406] Terminal

[1407] A terminal is a device that provides an interface for users to operate the system. Its specific functions are as follows:

[1408] 1. Content selection: The user selects the desired content from each menu, such as video, exercise video, restoration request, metaverse, will writing, and life purpose suggestions.

[1409] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[1410] 3. Data playback and display: Receives data sent from the server and plays back videos and exercises. Displays restored photos and embodies the virtual space.

[1411] 4. Interface provision: Provide users with the interface necessary for will-writing tools and life purpose suggestions, and send the input information to the server.

[1412] User

[1413] Users are seniors and their caregivers who:

[1414] 1. Login: Enter your account information into the terminal interface to log in to the system.

[1415] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[1416] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[1417] 4. Experience: Watch the playback video, practice the exercise video, check the restored photos, visit the metaverse, write a will, and accept suggestions for new meanings in life through generative AI models.

[1418] Specific examples

[1419] As a concrete example, let's consider a scenario where the user selects "gymnastics videos":

[1420] 1. The user launches the app and enters their credentials on the login screen.

[1421] 2. The server authenticates using OAuth 2.0 and approves the login.

[1422] 3. Select the "Exercise Video" menu on the home screen.

[1423] 4. Send a request to the server to get a list of exercise videos.

[1424] 5. Select the desired exercise video from the list and play the video via streaming from the server.

[1425] 6. The user performs the exercises while watching the video on a smartphone or head-mounted display.

[1426] Example prompts for generative AI models

[1427] Please suggest some exercise videos suitable for seniors to maintain their health. The user's name is Tanaka, a 70-year-old man who has had difficulty walking in the past. Light stretching to improve flexibility and chair exercises would be appropriate.

[1428] Through this system, seniors can not only enjoy a variety of content from the comfort of their own homes, but also easily use digital technology to help maintain their health.

[1429] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1430] Step 1:

[1431] User login

[1432] Input: User authentication information (username, password)

[1433] Output: Authentication token

[1434] Specific behavior:

[1435] The user starts up the device and enters authentication information (username, password) on the login screen. The device sends this authentication information to the server. The server uses OAuth 2.0 to check against the authentication database, and if authentication is successful, generates an authentication token and sends it back to the device.

[1436] Step 2:

[1437] Content selection and request submission

[1438] Input: User content selection information

[1439] Output: Content List

[1440] Specific behavior:

[1441] After logging in, the user selects a menu item such as "Exercise Videos" from the device's home screen. The device then sends a request to the server based on the selected menu item. The server then retrieves a list of related content from the database and returns it to the device.

[1442] Step 3:

[1443] Content playback and display

[1444] Input: Content selected from the content list

[1445] Output: The video being played or the data being displayed

[1446] Specific behavior:

[1447] The user selects the desired video from the content list displayed on the device. The device then transmits the selected content information to the server, which then streams the corresponding content data in real time. The device then plays or displays the received data.

[1448] Step 4:

[1449] Upload and restore photos and videos

[1450] Input: Photo and video files uploaded by users

[1451] Output: Recovered photo and video files

[1452] Specific behavior:

[1453] The user selects the photos and video files they want to restore and uploads them to the server via their device. The server then uses image processing techniques such as OpenCV to restore the data and sends the restored files back to the device. The device then displays the restored data.

[1454] Step 5:

[1455] Creating and experiencing metaverse spaces

[1456] Input: Information about the user's past locations and memories

[1457] Output: Metaverse space

[1458] Specific behavior:

[1459] The user selects the Metaverse menu through the device and inputs the information about past locations and memories they need. The device then sends this information to the server. The server generates a virtual reality space using Unity or Unreal Engine and sends the data back to the device. The user then visits and experiences the virtual space using a device (e.g., a head-mounted display).

[1460] Step 6:

[1461] Individualized proposals using generative AI

[1462] Input: Prompts based on user activity data and interests

[1463] Output: Proposals from the generative AI model

[1464] Specific behavior:

[1465] The server generates prompts based on the user's activity data and interests and sends them to a generative AI model. The AI ​​model analyzes the prompts and generates exercise videos, activity suggestions, and other content that are optimal for the user. This information is then sent to the device and displayed to the user.

[1466] Examples of prompt statements

[1467] Please suggest some exercise videos suitable for seniors to maintain their health. The user's name is Tanaka, a 70-year-old man who has had difficulty walking in the past. Light stretching to improve flexibility and chair exercises would be appropriate.

[1468] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1469] This invention is a multi-functional content system for use by the elderly of the silver generation, and by combining it with an emotion engine, it realizes a system that provides optimal content to users. This system consists of four main components: a server, a terminal, a user, and an emotion engine.

[1470] server

[1471] The server provides the following functionality:

[1472] 1. Content management: Various content (videos, exercise videos, photo restoration, metaverse, will creation tool, and life purpose suggestions) is stored in a database and provided upon appropriate request.

[1473] 2. User authentication: When a user logs in through a terminal, the user is authenticated by checking against the authentication database.

[1474] 3. Data recovery: Photos and videos uploaded by users from their devices are recovered using image processing technology.

[1475] 4. Metaverse generation: Generate a virtual reality space and allow users to virtually visit places from the past.

[1476] 5. Emotional Data Management: Emotional data recognized by the emotion engine is stored in a database to provide personalized content.

[1477] Terminal

[1478] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[1479] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, metaverse, will creation, and purpose in life suggestions.

[1480] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[1481] 3. Data playback and display: Receives data sent from the server, plays back videos and exercise videos, and displays restored photos and virtual spaces.

[1482] 4. Interface provision: Provide users with the interface necessary for will-writing tools and life purpose suggestions, and send the input information to the server.

[1483] 5. Emotion data transmission: The emotion data recognized by the emotion engine is transmitted to the server.

[1484] User

[1485] Users are seniors and their caregivers who:

[1486] 1. Login: Enter your account information into the terminal interface to log in to the system.

[1487] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[1488] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[1489] 4. Experience: Watch the playback video, try the exercise video, check the restored photos, visit the Metaverse space, write a will, and accept suggestions for a new purpose in life.

[1490] 5. Emotion input: Emotion data is provided in real time through the device's built-in camera and microphone.

[1491] Emotion Engine

[1492] The Emotion Engine provides the following features:

[1493] 1. Emotion recognition: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time.

[1494] 2. Emotion data generation: Generates recognized emotion data and sends it to the server via the terminal.

[1495] 3. Content recommendation: Based on the recognized sentiment, the server recommends the most suitable content.

[1496] As a concrete example, let us consider a scenario in which a user watches a video. First, the user logs in to the device and selects the "Watch Video" menu from the home screen. The device sends a content request to the server. The server retrieves a video list from the database and sends it to the device. The device displays the list and the user selects a video that interests them. The emotion engine recognizes the user's emotions in real time and sends that data to the server. Based on the recognized emotion data, the server selects a video that best suits the user's mood and sends it to the device. The device plays the video, and the user watches it.

[1497] In this way, this system provides an interface that is easy for users to operate, and the emotion engine recognizes the user's emotions in real time to provide optimal content, thereby improving the quality of life for the elderly and reducing the burden on caregivers.

[1498] The processing flow will be explained below.

[1499] Video viewing process flow

[1500] Step 1:

[1501] The user enters account information into the terminal interface and presses the login button.

[1502] Step 2:

[1503] The terminal transmits the user's input information to the server.

[1504] Step 3:

[1505] The server compares the received information with the database and returns the authentication result to the terminal.

[1506] Step 4:

[1507] The device displays the authentication result to the user, and if successful, transitions to the home screen.

[1508] Step 5:

[1509] The user selects the "Watch Video" menu from the home screen.

[1510] Step 6:

[1511] The terminal sends a content request to the server.

[1512] Step 7:

[1513] The server retrieves a list of video content from the database and transmits it to the terminal.

[1514] Step 8:

[1515] The device displays a list of videos.

[1516] Step 9:

[1517] The user selects the video they want to watch from the list.

[1518] Step 10:

[1519] The terminal sends a request for details of the selected video to the server.

[1520] Step 11:

[1521] The server reads the video data from the database and sends it to the terminal.

[1522] Step 12:

[1523] The emotion engine captures the user's facial expressions and voice data and analyzes emotions in real time.

[1524] Step 13:

[1525] The emotion engine sends the analysis results to the server.

[1526] Step 14:

[1527] The server receives the emotion data and selects the most appropriate video according to the user's current emotional state.

[1528] Step 15:

[1529] The server transmits the selected optimal video data to the terminal.

[1530] Step 16:

[1531] The terminal receives the video data and starts playing it.

[1532] Step 17:

[1533] The user watches the video.

[1534] Photo and video restoration process flow

[1535] Step 1:

[1536] The user selects the "Photo / Video Recovery" menu from the home screen.

[1537] Step 2:

[1538] The terminal displays a file selection screen to the user.

[1539] Step 3:

[1540] The user selects the photo or video files they want to restore.

[1541] Step 4:

[1542] The device uploads the selected file to the server.

[1543] Step 5:

[1544] The server stores the received file in the database and executes the restoration process.

[1545] Step 6:

[1546] The server sends the results of the restoration process to the terminal.

[1547] Step 7:

[1548] The emotion engine captures the user's facial expressions and reaction data and analyzes their emotions.

[1549] Step 8:

[1550] The emotion engine sends the analysis results to the server.

[1551] Step 9:

[1552] The device will display the restored photos and videos.

[1553] Step 10:

[1554] The user reviews the restored content.

[1555] Metaverse Processing Flow

[1556] Step 1:

[1557] The user selects the "Metaverse" menu from the home screen.

[1558] Step 2:

[1559] The device sends a metaverse request to the server.

[1560] Step 3:

[1561] The server generates metaverse spatial data and sends it to the terminal.

[1562] Step 4:

[1563] The device displays the metaverse space and provides the user with a virtual reality experience.

[1564] Step 5:

[1565] The emotion engine analyzes the user's voice and facial expressions and sends the emotion data to the server.

[1566] Step 6:

[1567] The server receives the emotion data and applies experience content appropriate to the user's emotional state.

[1568] Step 7:

[1569] A user visits past locations in the metaverse.

[1570] Process flow for writing a will

[1571] Step 1:

[1572] The user selects the "Write a Will" menu from the home screen.

[1573] Step 2:

[1574] The device displays a will creation tool.

[1575] Step 3:

[1576] The user inputs the contents of the will.

[1577] Step 4:

[1578] The emotion engine analyzes the user's facial expressions and voice and sends the emotion data to the server.

[1579] Step 5:

[1580] The terminal transmits the entered will data to the server.

[1581] Step 6:

[1582] The server stores the will data in a database and sends the confirmation results to the terminal.

[1583] Step 7:

[1584] The terminal displays the confirmation result to the user.

[1585] Purpose of Life Proposal Processing Flow

[1586] Step 1:

[1587] The user selects the "Life Purpose Suggestions" menu from the home screen.

[1588] Step 2:

[1589] The device displays a questionnaire about the user's interests and preferences.

[1590] Step 3:

[1591] The user answers the question and submits it.

[1592] Step 4:

[1593] The emotion engine analyzes the user's responses and real-time emotional state and sends the emotional data to the server.

[1594] Step 5:

[1595] The terminal transmits the user's response data to the server.

[1596] Step 6:

[1597] The server analyzes the data and suggests appropriate hobbies and activities.

[1598] Step 7:

[1599] The server sends the proposal results to the terminal.

[1600] Step 8:

[1601] The device displays the suggestions to the user.

[1602] Step 9:

[1603] The user confirms and engages in suggested activities and hobbies.

[1604] These processing flows create a system that is easy for elderly people to use, allows them to enjoy a variety of content, and, by using an emotion engine, provides a personalized experience, reducing the burden on caregivers.

[1605] Example 2

[1606] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1607] Current content systems for the elderly do not provide content that takes into account the user's emotions, resulting in a poor user experience. Other issues include the difficulty of restoring past photos and videos, and the lack of a virtual reality space that is easy for the elderly to use.

[1608] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1609] In this invention, the server includes an information processing device that retrieves data from a database according to content selected by the user and provides it to the device being used, an emotion analysis device that analyzes the user's voice, facial expression, and movement data to recognize emotions in real time, and a device that recommends optimal content based on the emotion data. This makes it possible to provide optimal content according to the user's emotions, improving the quality of the user experience. It also makes it possible to realize an environment that makes it easy to restore photos and videos and provide virtual reality spaces.

[1610] The term "information processing device means" refers to a device that retrieves data from a database according to content selected by a user and provides the data to the device being used.

[1611] The "device means" refers to a device that allows a user to select content and send a request to an information processing device. It also includes a function to play and display data provided by the information processing device.

[1612] "Emotion analysis means" refers to a device or program that analyzes the user's voice, facial expression, and movement data and recognizes emotions in real time.

[1613] "Means" generally refers to a method, apparatus, or program designed to achieve a specific purpose.

[1614] "Data" refers to information or content acquired and used via information processing device means, and includes various formats such as video, images, audio, and text.

[1615] "Virtual space providing means" refers to a device or program that enables a user to visit past locations in a virtual reality space.

[1616] This invention is a multi-functional content system for use by the elderly of the silver generation, and by combining it with an emotion analysis function, it realizes a system that provides optimal content to users. This system consists of four main components: an information processing device, a user device, a user, and an emotion analysis function.

[1617] Information processing device

[1618] The information processing device provides the following functions:

[1619] 1. Content management: The information processing device stores various content (videos, exercise videos, image restoration, virtual space, will writing tools, and life purpose suggestions) in a database and provides this content in response to user requests. This process uses a database such as MySQL.

[1620] 2. User authentication: When a user logs in through their device, the user is authenticated by checking against an authentication database. Authentication is performed using LDAP or OAuth2.

[1621] 3. Image restoration: The information processing device restores the images and videos uploaded by the user from their device using image processing technology (e.g., OpenCV).

[1622] 4. Virtual space generation: The information processing device generates a virtual reality space, allowing users to virtually visit places from the past. This process uses Unreal Engine and Unity.

[1623] 5. Emotional data management: The information processing device stores the emotional data recognized by the emotion analysis function in a database and provides personalized content.

[1624] Equipment used

[1625] A user device is a device that provides an interface for users to operate the system. It has the following functions:

[1626] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, virtual space, will writing, and purpose in life suggestions.

[1627] 2. Request transmission: Based on the selected content, an appropriate request is sent to the information processing device. An HTTP request is used.

[1628] 3. Data playback and display: Receives data sent from the information processing device, plays back images and exercise videos, and displays restored images and virtual spaces.

[1629] 4. Interface provision: Provides the user with the interface necessary for the will writing tool and life purpose suggestions, and sends the input information to the information processing device.

[1630] 5. Emotion data transmission: Emotion data recognized by the emotion analysis function is transmitted to the information processing device.

[1631] User

[1632] Users are seniors and their caregivers who:

[1633] 1. Login: Enter your account information into the interface of your device to log in to the system.

[1634] 2. Content usage: Select the desired content from each menu and operate it through the interface of the device you are using.

[1635] 3. Upload: Upload the images or videos you want to restore from your device to the information processing device.

[1636] 4. Experience: Watch the video, try out the exercises, check the restored images, and visit the virtual space. You can also write a will and accept suggestions for a new purpose in life.

[1637] 5. Emotion input: Emotion data is provided in real time through the camera and microphone built into the device being used.

[1638] Sentiment analysis function

[1639] The sentiment analysis feature provides the following features:

[1640] 1. Emotion recognition: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time, using technologies such as OpenFace and DeepFace.

[1641] 2. Emotion data generation: Recognized emotion data is generated and sent to the information processing device via the device being used.

[1642] 3. Content recommendation: Based on the recognized emotions, the information processing device recommends the most suitable content.

[1643] Example scenario

[1644] As a specific example, a scenario in which a user watches a video will be described. First, the user logs in to the device being used and selects the "Watch Video" menu from the home screen. The device being used sends a content request to the information processing device. The information processing device retrieves a video list from a database and sends it to the device being used. The device being used displays the list, and the user selects a video of interest. The emotion analysis function recognizes the user's emotions in real time and sends that data to the information processing device. Based on the recognized emotion data, the information processing device selects a video that best suits the user's mood and sends it to the device being used. The device being used plays the video, and the user watches it.

[1645] Example prompt:

[1646] "Please explain the process of watching videos using a multi-functional content system for seniors in the silver generation. Please explain how a user logs in, selects a video, and how the emotion analysis function works."

[1647] In this way, this system provides an interface that is easy for users to operate, and the emotion analysis function recognizes the user's emotions in real time to provide optimal content, thereby improving the quality of life for the elderly and reducing the burden on caregivers.

[1648] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1649] The program processing flow of this system and specific steps

[1650] Processing Steps

[1651] Step 1: User Login

[1652] Step 2: Menu selection

[1653] Step 3: Submit a content request

[1654] Step 4: Providing and displaying content

[1655] Step 5: Collect and send emotion data

[1656] Step 6: Optimize and reselect content

[1657] Specific explanation of each processing step

[1658] Step 1: User Login

[1659] Input: Account information (ID and password)

[1660] Output: Authentication success or failure response

[1661] The user enters their account information on the login screen of their device and presses the login button. The device sends the entered information to the server as an HTTP POST request. The server verifies the authentication information using an LDAP or MySQL database, and if authentication is successful, returns a successful authentication response to the device. If authentication is unsuccessful, it returns an error message.

[1662] Specific operation: The user enters "ID: user123" and "Password: pass456" and clicks the login button. The device sends this information to the server, which queries the LDAP database for authentication.

[1663] Step 2: Menu selection

[1664] Input: User-selected menu item

[1665] Output: Selected menu information

[1666] The user selects the content they want to use (e.g., watching a video) from the menu displayed on the home screen. The device sends the selected menu information to the server via an HTTP GET request.

[1667] Specific operation: The user selects the "Watch Video" menu from the home screen and clicks the "Watch Video" button on the screen. The device sends a request including the selection information to the server.

[1668] Step 3: Submit a content request

[1669] Input: Selected menu information

[1670] Output: A list of matching content

[1671] The terminal transmits the selected menu information to the server, and the server retrieves the corresponding content list from the database and transmits the list to the terminal.

[1672] Specific operation: The device sends a "video viewing" request to the server. The server retrieves the video list from the MySQL database and sends it to the device in JSON format.

[1673] Step 4: Providing and displaying content

[1674] Input: Content list received from the server

[1675] Output: User selects and retrieves content

[1676] The device displays the content list received from the server to the user. The user selects the content of interest from the list, and the device sends the selection information to the server. The server retrieves the relevant content from the database and sends it to the device. The device plays and displays the content.

[1677] Specific operation: The user selects "Video A" from the displayed list and clicks the play button. The device sends the ID of Video A to the server, and the server retrieves the video data and sends the playback data to the device.

[1678] Step 5: Collect and send emotion data

[1679] Input: User's voice, facial expression, and movement data

[1680] Output: Generated emotion data

[1681] While the device is playing video, the camera and microphone collect the user's voice, facial expressions, and movement data. The emotion analysis function analyzes this data and generates emotion data in real time. The emotion data is then sent to the server via an HTTP POST request.

[1682] Specific operation: The device detects the user's facial expressions and voice, and the emotion analysis function recognizes that the user is "having fun." Emotional data is generated and sent to the server.

[1683] Step 6: Optimize and reselect content

[1684] Input: Received emotion data

[1685] Output: Recommend and serve new content

[1686] The server recommends new content based on the received emotion data. The device displays the recommended content to the user, and the user selects it again. The reselected content is then retrieved from the server and displayed.

[1687] Specific operation: The server analyzes the emotional data of "enjoyed" and recommends the more suitable "Video B." The device presents "Video B" to the user, who selects and plays it.

[1688] This will enable optimal content to be provided according to the user's emotions, improving the quality of life for the elderly.

[1689] (Application example 2)

[1690] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1691] When elderly people use content, it is often difficult to operate, and the content provided is often not suited to their individual emotions or state. This leads to problems that reduce the quality of life of the elderly and increase the burden on caregivers. In addition, there is a lack of appropriate exercise guidance to support psychological and physical health, and functions to recreate past memories in virtual space.

[1692] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes information collection device means for acquiring data from the data collection management device in accordance with information selected by the user and providing the data to the information device, information device means for the user to select information and send a request to the information collection management device, information device means for playing and displaying the data provided from the information collection management device, emotion recognition means for recognizing the user's emotion in real time and sending the data to the information collection management device, and information recommendation means for recommending optimal information based on the recognized emotion data. This provides an interface that is easy for the user to operate, and makes it possible to provide optimal content by having the emotion engine recognize the user's emotion in real time.

[1693] "User" refers to an individual or senior citizen who uses the system.

[1694] An "information accumulation and management device" is a device that stores and manages data and provides it as needed.

[1695] "Information equipment" refers to devices that are operated by users, such as smart glasses and smartphones.

[1696] The "information accumulation device means" is a function or system for acquiring data from the information accumulation management device and providing it to the information device.

[1697] The "information device means" is a function or system that allows a user to select information and send a request to the information accumulation management device.

[1698] The "information device display means" is a function or system for reproducing data provided from the information accumulation management device and displaying it on an information device.

[1699] The "emotion recognition means" is a function or system for recognizing the user's emotions in real time and transmitting the data to the information collection and management device.

[1700] The "information recommendation means" is a function or system for recommending optimal information to a user based on the recognized emotion data.

[1701] "Image restoration means" refers to a function or system for restoring images and videos uploaded by users.

[1702] The "virtual space generation means" is a function or system for generating a virtual space in which the user can visit places from the past.

[1703] This invention is a multi-functional content system for use by elderly people in the silver generation, which provides optimal content according to the user's emotions by combining it with an emotion recognition engine. This system consists of four main components: an information accumulation and management device (server), information equipment (terminal), users, and the emotion recognition engine.

[1704] Information accumulation and management device (server)

[1705] The server provides the following functionality:

[1706] 1. Information accumulation device means: Acquires data from a database according to information selected by the user and provides it to the information device.

[1707] 2. Image restoration tool: It has the function to restore images and videos uploaded by users.

[1708] 3. Virtual space generation means: A past location is generated as a virtual reality space and provided to the user.

[1709] 4. Emotion data management: Emotion data recognized by the emotion recognition engine is stored in a database and personalized content is provided.

[1710] Information equipment (terminals)

[1711] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[1712] 1. Information device means: The user selects information and sends a request to the server.

[1713] 2. Information device display means: Receives data provided by the server, plays videos and exercise videos, and displays restored photos and virtual spaces.

[1714] 3. Input interface: Provides users with the interface necessary for the will-writing tool and emotion-based purpose-in-life suggestions, and sends the input content to the server.

[1715] 4. Emotion recognition means: Using the device's built-in camera and microphone, the user's emotional data is acquired in real time and sent to the server.

[1716] User

[1717] Users are seniors and their caregivers who:

[1718] 1. Login: Enter your account information into the terminal interface to log in to the system.

[1719] 2. Content usage: Select the desired information from each menu and operate it through the device interface.

[1720] 3. Upload: Upload the images or videos you want to restore from your device to the server.

[1721] 4. Experience: Watch the video, try out the exercises, check the restored photos, visit the virtual space, and even write a will and accept suggestions for a new purpose in life.

[1722] 5. Emotion input: Provide emotion data in real time via camera or microphone.

[1723] Emotion Recognition Engine

[1724] The emotion recognition engine provides the following features:

[1725] 1. Emotion recognition means: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time.

[1726] 2. Emotion data transmission: Generate recognized emotion data and transmit it to the server via the terminal.

[1727] 3. Information recommendation method: Based on the recognized emotions, the server recommends the most appropriate information.

[1728] Specific examples

[1729] For example, suppose a user is wearing smart glasses and sitting in their living room in a relaxed mood. The emotion recognition engine recognizes the user's relaxed state in real time. Based on this, the server will recommend a landscape video that is most suitable for the relaxed mood, and a beautiful natural scene will be displayed on the smart glasses' display. The user can enjoy this visual experience while remaining relaxed.

[1730] Prompt Sentence Examples

[1731] "How can we recommend beautiful landscape videos to elderly people who are in a relaxed state?"

[1732] "Design a system that uses an emotion recognition engine to recognize emotions in real time and display content based on that in smart glasses."

[1733] The above is a specific embodiment for carrying out the present invention.

[1734] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1735] Step 1:

[1736] A user logs in to an information device (e.g., smart glasses). As input, the user enters account information, and the server authenticates the user based on this information. As output, if authentication is successful, the user can access the home screen.

[1737] Step 2:

[1738] The user selects the desired content menu (e.g., watching a video, watching a gymnastics video, visiting a virtual space) from the home screen. The user's selection is received as input, and this data is sent from the device to the server. As output, the server provides the device with a list of the corresponding content.

[1739] Step 3:

[1740] The server uses an emotion recognition engine to obtain the user's emotion data in real time. It receives data from the camera and microphone as input, analyzes the emotion using an emotion recognition model, and generates the recognized emotion data as output, which is then sent to the server.

[1741] Step 4:

[1742] The server recommends optimal content based on the received emotional data. It receives the emotional data and content request as input, selects optimal content from a database based on the received emotional data, and sends the selected content to the device as output.

[1743] Step 5:

[1744] The terminal plays the content sent from the server and displays it to the user. As input, it takes the received content data and converts it into a format for display. As output, the user can visually confirm the played content on the display of the smart glasses.

[1745] Step 6:

[1746] When a user uploads an image or video they want to repair, their device sends these files to the server. As input, it takes the image or video file and uploads it to the server. As output, the server starts the repair process.

[1747] Step 7:

[1748] The server sends the repaired image or video file to the device. It receives the file to be repaired as input, processes it using an image repair algorithm, and provides the repaired file to the device as output.

[1749] Step 8:

[1750] A user sends a request to visit a virtual space from a terminal. The terminal receives the request data for visiting the virtual space as input and sends it to the server. As output, the server generates a virtual space and provides it to the terminal.

[1751] Step 9:

[1752] The terminal displays the virtual space provided by the server and provides the user with an experience. It receives virtual space data as input and converts it into a display format. As an output, the user can enjoy the experience in the virtual space.

[1753] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1754] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1755] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1756] [Fourth embodiment]

[1757] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1758] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1759] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1760] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1761] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1762] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1763] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1764] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1765] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1766] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1767] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1768] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1769] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1770] This invention is a multi-functional content system for use by the elderly of the silver generation. This system consists of three main components: a server, a terminal, and a user.

[1771] server

[1772] The server provides the following functionality:

[1773] 1. Content management: Various content (videos, exercise videos, photo restoration, metaverse, will creation tool, and life purpose suggestions) is stored in a database and provided upon appropriate request.

[1774] 2. User authentication: When a user logs in through a terminal, the user is authenticated by checking against the authentication database.

[1775] 3. Data recovery: Photos and videos uploaded by users from their devices are recovered using image processing technology.

[1776] 4. Metaverse generation: Generate a virtual reality space and allow users to virtually visit places from the past.

[1777] Terminal

[1778] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[1779] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, metaverse, will creation, and purpose in life suggestions.

[1780] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[1781] 3. Data playback and display: Receives data sent from the server, plays back videos and exercise videos, and displays restored photos and virtual spaces.

[1782] 4. Interface provision: Provide users with the interface necessary for will-writing tools and life purpose suggestions, and send the input information to the server.

[1783] User

[1784] Users are seniors and their caregivers who:

[1785] 1. Login: Enter your account information into the terminal interface to log in to the system.

[1786] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[1787] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[1788] 4. Experience: Watch the playback video, try the exercise video, check the restored photos, visit the Metaverse space, write a will, and accept suggestions for a new purpose in life.

[1789] As a concrete example, let us consider a scenario in which a user watches old videos. First, the user logs in to the device and selects the "Watch Videos" menu from the home screen. The device sends a content request to the server. The server retrieves a video list from the database and sends it to the device. The device displays the list, and the user selects a video of interest. The server reads the selected video data from the database and sends it to the device. The device plays the video, and the user watches it.

[1790] In this way, this system provides an interface that is easy for users to operate, and the server manages and provides a variety of content, thereby enriching the lives of the elderly and reducing the burden on caregivers.

[1791] The processing flow will be explained below.

[1792] Video viewing process flow

[1793] Step 1:

[1794] The user enters account information into the terminal interface and presses the login button.

[1795] Step 2:

[1796] The terminal transmits the user's input information to the server.

[1797] Step 3:

[1798] The server compares the received information with the database and returns the authentication result to the terminal.

[1799] Step 4:

[1800] The device displays the authentication result to the user, and if successful, transitions to the home screen.

[1801] Step 5:

[1802] The user selects the "Watch Video" menu from the home screen.

[1803] Step 6:

[1804] The terminal sends a content request to the server.

[1805] Step 7:

[1806] The server retrieves a list of video content from the database and transmits it to the terminal.

[1807] Step 8:

[1808] The device displays a list of videos.

[1809] Step 9:

[1810] The user selects the video they want to watch from the list.

[1811] Step 10:

[1812] The terminal sends a request for details of the selected video to the server.

[1813] Step 11:

[1814] The server reads the video data from the database and sends it to the terminal.

[1815] Step 12:

[1816] The terminal receives the video data and starts playing it.

[1817] Step 13:

[1818] The user watches the video.

[1819] Photo and video restoration process flow

[1820] Step 1:

[1821] The user selects the "Photo / Video Recovery" menu from the home screen.

[1822] Step 2:

[1823] The terminal displays a file selection screen to the user.

[1824] Step 3:

[1825] The user selects the photo or video files they want to restore.

[1826] Step 4:

[1827] The device uploads the selected file to the server.

[1828] Step 5:

[1829] The server stores the received file in the database and executes the restoration process.

[1830] Step 6:

[1831] The server sends the results of the restoration process to the terminal.

[1832] Step 7:

[1833] The device will display the restored photos and videos.

[1834] Step 8:

[1835] The user reviews the restored content.

[1836] Metaverse Processing Flow

[1837] Step 1:

[1838] The user selects the "Metaverse" menu from the home screen.

[1839] Step 2:

[1840] The device sends a metaverse request to the server.

[1841] Step 3:

[1842] The server generates metaverse spatial data and sends it to the terminal.

[1843] Step 4:

[1844] The device displays the metaverse space and provides the user with a virtual reality experience.

[1845] Step 5:

[1846] A user visits past locations in the metaverse.

[1847] Process flow for writing a will

[1848] Step 1:

[1849] The user selects the "Write a Will" menu from the home screen.

[1850] Step 2:

[1851] The device displays a will creation tool.

[1852] Step 3:

[1853] The user inputs the contents of the will.

[1854] Step 4:

[1855] The terminal transmits the entered will data to the server.

[1856] Step 5:

[1857] The server stores the will data in a database and sends the confirmation results to the terminal.

[1858] Step 6:

[1859] The terminal displays the confirmation result to the user.

[1860] Purpose of Life Proposal Processing Flow

[1861] Step 1:

[1862] The user selects the "Life Purpose Suggestions" menu from the home screen.

[1863] Step 2:

[1864] The device displays a questionnaire about the user's interests and preferences.

[1865] Step 3:

[1866] The user answers the question and submits it.

[1867] Step 4:

[1868] The terminal transmits the user's response data to the server.

[1869] Step 5:

[1870] The server analyzes the data and suggests appropriate hobbies and activities.

[1871] Step 6:

[1872] The server sends the proposal results to the terminal.

[1873] Step 7:

[1874] The device displays the suggestions to the user.

[1875] These processing flows will realize a system that is easy for elderly people to use, allows them to enjoy a variety of content, and reduces the burden on caregivers.

[1876] Example 1

[1877] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1878] Conventional content systems for seniors were difficult for seniors to use because they were complicated to operate. Furthermore, there was no system that offered a variety of functions, such as image and video restoration, virtual reality space experiences, and tools for writing wills, all in one place. Furthermore, it was necessary to use separate systems and devices to access specific content, which caused confusion for seniors.

[1879] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1880] In this invention, the server includes an information processing device that retrieves data from a database according to content selected by the user and provides it to the terminal, means for the user to input login information and the information processing device compares the information with the authentication database for authentication, means for the user to upload photos or videos that the user wants to restore from the display device and the information processing device restores them using image processing technology, means for generating a virtual reality space that allows the user to visit past locations in a virtual reality space, and means for the information processing device to generate the virtual reality space and provide it to the display device. This makes it possible to provide a system that provides multifunctional content all at once and is easy for seniors to use.

[1881] An "information processing device" is a device that retrieves data from a database based on a request from a user, performs the specified operation or processing, and provides the results to a terminal.

[1882] The "display device" is a device that allows a user to input data and perform operations via an interface, and that reproduces and displays data transmitted from an information processing device.

[1883] "User authentication" is the process of authenticating a user when they log in to a system by checking the entered login information against an authentication database.

[1884] "Content" refers to digital information and data such as videos, exercise videos, photographs, virtual reality spaces, will-writing tools, and suggestions for purpose in life.

[1885] A "database" is a data storage system for managing and storing various content and user authentication information within the system.

[1886] "Image processing techniques" are techniques used to restore uploaded photos and videos, including noise reduction, resolution enhancement, and color correction.

[1887] "Virtual reality space" refers to a three-dimensional computer-simulated space created to allow users to virtually visit places from the past.

[1888] A "will creation tool" is an application that allows users to create and save their wishes and messages in digital form.

[1889] "Purpose in Life Suggestion" is a system or function that suggests new hobbies and activities to users.

[1890] MODE FOR CARRYING OUT THE INVENTION

[1891] This invention is a multi-functional content system for the elderly, consisting of three main components: a user, a terminal, and a server. This system allows elderly people to easily access various content and enrich their daily lives. Each component of this invention and its operation will be described in detail below.

[1892] 1. Server

[1893] The server plays a central role in the system and provides the following functions:

[1894] 1.1 Content Management

[1895] The server stores various content (videos, exercise videos, photo restoration, virtual reality space, will writing tool, and purpose-in-life suggestions) in a database and provides them in response to user requests. Specifically, the server uses a database management system (DBMS) to efficiently manage content data.

[1896] 1.2 User Authentication

[1897] When a user logs in via a terminal, the server authenticates them against an authentication database. This is achieved using security protocols (e.g., SSL / TLS) and multi-factor authentication technology. Specifically, a mechanism is adopted in which an authentication code is sent after the user enters their username and password.

[1898] 1.3 Data Recovery

[1899] The server uses image processing technology to restore photos and videos uploaded by users from their devices. Specific techniques used include image restoration models using deep learning (e.g., Generative Adversarial Networks, GANs). Using a high-performance GPU enables efficient data processing.

[1900] 1.4 Virtual reality space generation

[1901] The server generates a virtual reality space, allowing users to virtually visit places from the past. This is done using VR development platforms such as Unity and Unreal Engine. The generated virtual reality space data is sent to the device in real time.

[1902] 2. Terminal

[1903] The terminal provides an interface for users to operate the system and has the following functions:

[1904] 2.1 Content Selection

[1905] It provides an interface for users to select the content they want from the menus of video, exercise videos, restoration requests, virtual reality space, will writing, and life purpose suggestions. Operation becomes even easier if the device is equipped with a touch screen or voice recognition function.

[1906] 2.2 Request Submission

[1907] Based on the selected content, the device sends an appropriate request to the server. The request is sent using the HTTP / HTTPS protocol. For example, the request might say "The user has selected to watch a video."

[1908] 2.3 Data playback and display

[1909] The device receives the data (video, exercise video, restored photos, virtual reality space) sent from the server and plays and displays it. A device with a GPU is recommended for video playback and virtual reality display.

[1910] 2.4 Interface Provision

[1911] The system provides users with an interface for writing wills and suggesting purpose in life, and sends this to the server. The content entered by the user (e.g., "I want to find a new hobby") is sent to the server, and appropriate feedback is returned to the device.

[1912] 3. Users

[1913] The users are elderly people and their caregivers, and perform the following operations:

[1914] 3.1 Login

[1915] A user logs in to the system by entering account information (user name, password, etc.) on the login screen of the terminal. For example, the user enters the user name "tanaka123" and the password "password123."

[1916] 3.2 Content Use

[1917] Users can select the desired content from each menu and operate it through the device interface. Specifically, they can select the "Video Viewing" menu to watch old travel videos.

[1918] 3.3 Upload

[1919] Upload the photos or videos you want to restore from your device to the server. For example, upload "Photos from a family trip." After the server completes the restoration process, the restored data will be displayed on your device.

[1920] 3.4 Experience

[1921] Users can watch playback videos, practice exercise videos, view restored photos, and visit virtual reality spaces. They can also write wills and accept suggestions for new purposes in life. When using the virtual reality space, users put on a VR headset and explore virtual locations in real time.

[1922] Specific examples

[1923] Login: When a user attempts to log in by entering "tanaka123" and "password123", the server authenticates them and returns a success message.

[1924] Video viewing: When Tanaka selects "Video viewing," the device sends a request to the server, which returns a list of videos. Tanaka selects "Video of family trip" and watches it.

[1925] Photo restoration: When Tanaka tries to restore an old family photo by uploading it from his device, the server restores it and sends the photo back to his device, where Tanaka can view it.

[1926] Virtual reality experience: When Tanaka selects "Virtual reality space of his hometown," the server generates a virtual space, sends it to his device, and Tanaka experiences it.

[1927] Writing a Will: When Tanaka selects the "Will Writing" tool and enters the content, it is sent to the server, where it is processed and feedback is returned.

[1928] In this way, this system provides an interface that users can operate intuitively, and by efficiently managing and providing multifunctional content, it enriches the lives of the elderly and reduces the burden on caregivers.

[1929] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1930] Step 1: The user types the request into the terminal

[1931] A user attempts to log in by entering account information (username and password) into the terminal interface. The input data is provided by the user and sent to the terminal. Based on this, an authentication request is generated and sent to the server.

[1932] Step 2: The device sends an authentication request to the server

[1933] The terminal converts the authentication information entered by the user into a packet format and sends it to the server using a communication protocol (e.g., HTTPS). The input data is the user's account information, and communication data is generated based on this.

[1934] Step 3: Server authenticates

[1935] The server checks the received authentication information against an authentication database. It queries the authentication database to see if the corresponding user information exists. The input data is the authentication request, and the output data is the authentication result (success or failure).

[1936] Step 4: The server returns the authentication result to the device.

[1937] The server generates an authentication result and returns it to the terminal. If successful, it generates and adds a session ID, etc., which gives the user permission to access the system. The output data is the authentication result and the session ID.

[1938] Step 5: The device displays the authentication result

[1939] The terminal displays the authentication result received from the server to the user. If successful, the home screen is displayed and the user is guided to the next operation. The input data is the authentication result, and the output data is an interface that the user can operate.

[1940] Step 6: User selects the content menu

[1941] The user selects the desired content menu (e.g., video viewing, virtual reality experience) from the home screen. The user's operation is input to the terminal, and the next request is generated. The input data is the user's selection.

[1942] Step 7: The device sends a content request to the server

[1943] The terminal generates a content request based on the content menu selected by the user and sends it to the server, which then provides the specified content. The input data is the user's content selection.

[1944] Step 8: The server provides the content list

[1945] The server retrieves the relevant content list from the database based on the request, executes a query against the database, and sends the retrieved list to the terminal. The input data is the content request, and the output data is the content list.

[1946] Step 9: The device displays the content list to the user.

[1947] The terminal displays the content list received from the server to the user. The user selects more specific content from the displayed list. The input data is the content list, and the output data is an interface that the user can operate.

[1948] Step 10: User Selects Specific Content

[1949] The user selects a specific item (e.g., a video of a family trip) from the displayed content list. The selection is input to the terminal, and the next request is generated. The input data is the user's specific content selection.

[1950] Step 11: The device sends a content data request to the server

[1951] The terminal generates and sends a content data request to the server based on the specific content selected by the user, with the input data being the specific content selection.

[1952] Step 12: The server retrieves and sends the content data

[1953] The server retrieves specific content data from the database based on the request, converts the retrieved data into a packet format, and sends it to the terminal. The input data is the content data request, and the output data is the content data.

[1954] Step 13: Your device plays and displays the content

[1955] The terminal plays and displays the content data received from the server to the user. In the case of video, for example, continuous playback is performed using a playback engine. The input data is the content data, and the output data is the content to be played and displayed.

[1956] This process flow allows users to intuitively operate the system and use a wide variety of content.

[1957] (Application example 1)

[1958] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1959] Today's elderly people need access to a variety of content and services to maintain their physical and mental health. However, they often find it difficult to understand and operate digital technology, which creates psychological barriers. Furthermore, they have limited means of reminiscing on past memories, creating a need for systems that allow them to easily access these services. However, existing systems are limited to solving individual issues, and few offer comprehensive, multifunctional services. Therefore, a multifunctional content system is needed that allows elderly people to easily access multiple services and improve their quality of life.

[1960] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1961] In this invention, the server includes server means for retrieving data from a database according to content selected by a user and providing it to a terminal, terminal means for the user to select content and send a request to the server, terminal means for playing and displaying the data provided from the server, means for making suggestions based on the user's activity data and interests using a generative AI model, and means for generating prompt sentences. This allows elderly people to easily access a variety of content and services and receive suggestions tailored to their individual needs.

[1962] "User-selected content" refers to multimedia materials or services that a user wishes to view or execute by operating a terminal.

[1963] A "database" is a digital system that stores data in a structured format that allows it to be efficiently managed and retrieved.

[1964] A "server" is a computer system that provides data and services to terminals over a network.

[1965] A "terminal" is an electronic device that provides an interface for users to operate. Examples include smartphones and head-mounted displays.

[1966] "Server means" refers to the methods or devices used by the server to implement a particular function or role.

[1967] "Terminal means" refers to the method or device used by the terminal to realize a specific function or role.

[1968] A "request" is a request or instruction sent from a terminal to a server in response to a user operation.

[1969] "Playback" means to re-display and output video and audio based on digital data.

[1970] "Display" means to visually represent data or information on a terminal screen or display.

[1971] A "generative AI model" is an artificial intelligence system that uses machine learning or deep learning algorithms to generate new data and suggestions.

[1972] A "prompt" is text that is used to design instructions or questions in a format that is easy for a generative AI model to understand.

[1973] "Activity data" refers to recorded data that indicates a user's past and present behavior and interests.

[1974] "Suggestions" are recommendations or suggested actions provided by a generative AI model based on a user's interests.

[1975] "User interests" refer to the particular areas or activities that interest a user.

[1976] This invention is a multifunctional content system for elderly people. This system consists of three main components: a server, a terminal, and a user. This system allows elderly people to watch past videos, maintain their health by viewing exercise videos, restore photos, and experience the metaverse.

[1977] server

[1978] The server provides the following functionality:

[1979] 1. Content Management: Manage content such as videos, images, and exercise videos from the database and provide them upon appropriate requests.

[1980] 2. User authentication: Authentication is performed based on the user's login information.

[1981] 3. Data restoration: Uploaded photos and videos are restored using image processing technology (e.g., OpenCV).

[1982] 4. Metaverse generation: A virtual reality space is generated using Unity or Unreal Engine, allowing users to experience visiting places from the past.

[1983] 5. Suggestions using generative AI models: Specific suggestions are made using generative AI models (e.g., GPT-4) based on user activity data and interests.

[1984] 6. Prompt Generation: Generate prompts for the generative AI model to provide advanced suggestions and support.

[1985] Terminal

[1986] A terminal is a device that provides an interface for users to operate the system. Its specific functions are as follows:

[1987] 1. Content selection: The user selects the desired content from each menu, such as video, exercise video, restoration request, metaverse, will writing, and life purpose suggestions.

[1988] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[1989] 3. Data playback and display: Receives data sent from the server and plays back videos and exercises. Displays restored photos and embodies the virtual space.

[1990] 4. Interface provision: Provide users with the interface necessary for will-writing tools and life purpose suggestions, and send the input information to the server.

[1991] User

[1992] Users are seniors and their caregivers who:

[1993] 1. Login: Enter your account information into the terminal interface to log in to the system.

[1994] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[1995] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[1996] 4. Experience: Watch the playback video, practice the exercise video, check the restored photos, visit the metaverse, write a will, and accept suggestions for new meanings in life through generative AI models.

[1997] Specific examples

[1998] As a concrete example, let's consider a scenario where the user selects "gymnastics videos":

[1999] 1. The user launches the app and enters their credentials on the login screen.

[2000] 2. The server authenticates using OAuth 2.0 and approves the login.

[2001] 3. Select the "Exercise Video" menu on the home screen.

[2002] 4. Send a request to the server to get a list of exercise videos.

[2003] 5. Select the desired exercise video from the list and play the video via streaming from the server.

[2004] 6. The user performs the exercises while watching the video on a smartphone or head-mounted display.

[2005] Example prompts for generative AI models

[2006] Please suggest some exercise videos suitable for seniors to maintain their health. The user's name is Tanaka, a 70-year-old man who has had difficulty walking in the past. Light stretching to improve flexibility and chair exercises would be appropriate.

[2007] Through this system, seniors can not only enjoy a variety of content from the comfort of their own homes, but also easily use digital technology to help maintain their health.

[2008] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2009] Step 1:

[2010] User login

[2011] Input: User authentication information (username, password)

[2012] Output: Authentication token

[2013] Specific behavior:

[2014] The user starts up the device and enters authentication information (username, password) on the login screen. The device sends this authentication information to the server. The server uses OAuth 2.0 to check against the authentication database, and if authentication is successful, generates an authentication token and sends it back to the device.

[2015] Step 2:

[2016] Content selection and request submission

[2017] Input: User content selection information

[2018] Output: Content List

[2019] Specific behavior:

[2020] After logging in, the user selects a menu item such as "Exercise Videos" from the device's home screen. The device then sends a request to the server based on the selected menu item. The server then retrieves a list of related content from the database and returns it to the device.

[2021] Step 3:

[2022] Content playback and display

[2023] Input: Content selected from the content list

[2024] Output: The video being played or the data being displayed

[2025] Specific behavior:

[2026] The user selects the desired video from the content list displayed on the device. The device then transmits the selected content information to the server, which then streams the corresponding content data in real time. The device then plays or displays the received data.

[2027] Step 4:

[2028] Upload and restore photos and videos

[2029] Input: Photo and video files uploaded by users

[2030] Output: Recovered photo and video files

[2031] Specific behavior:

[2032] The user selects the photos and video files they want to restore and uploads them to the server via their device. The server then uses image processing techniques such as OpenCV to restore the data and sends the restored files back to the device. The device then displays the restored data.

[2033] Step 5:

[2034] Creating and experiencing metaverse spaces

[2035] Input: Information about the user's past locations and memories

[2036] Output: Metaverse space

[2037] Specific behavior:

[2038] The user selects the Metaverse menu through the device and inputs the information about past locations and memories they need. The device then sends this information to the server. The server generates a virtual reality space using Unity or Unreal Engine and sends the data back to the device. The user then visits and experiences the virtual space using a device (e.g., a head-mounted display).

[2039] Step 6:

[2040] Individualized proposals using generative AI

[2041] Input: Prompts based on user activity data and interests

[2042] Output: Proposals from the generative AI model

[2043] Specific behavior:

[2044] The server generates prompts based on the user's activity data and interests and sends them to a generative AI model. The AI ​​model analyzes the prompts and generates exercise videos, activity suggestions, and other content that are optimal for the user. This information is then sent to the device and displayed to the user.

[2045] Examples of prompt statements

[2046] Please suggest some exercise videos suitable for seniors to maintain their health. The user's name is Tanaka, a 70-year-old man who has had difficulty walking in the past. Light stretching to improve flexibility and chair exercises would be appropriate.

[2047] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2048] This invention is a multi-functional content system for use by the elderly of the silver generation, and by combining it with an emotion engine, it realizes a system that provides optimal content to users. This system consists of four main components: a server, a terminal, a user, and an emotion engine.

[2049] server

[2050] The server provides the following functionality:

[2051] 1. Content management: Various content (videos, exercise videos, photo restoration, metaverse, will creation tool, and life purpose suggestions) is stored in a database and provided upon appropriate request.

[2052] 2. User authentication: When a user logs in through a terminal, the user is authenticated by checking against the authentication database.

[2053] 3. Data recovery: Photos and videos uploaded by users from their devices are recovered using image processing technology.

[2054] 4. Metaverse generation: Generate a virtual reality space and allow users to virtually visit places from the past.

[2055] 5. Emotional Data Management: Emotional data recognized by the emotion engine is stored in a database to provide personalized content.

[2056] Terminal

[2057] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[2058] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, metaverse, will creation, and purpose in life suggestions.

[2059] 2. Send Request: Based on the selected content, send the appropriate request to the server.

[2060] 3. Data playback and display: Receives data sent from the server, plays back videos and exercise videos, and displays restored photos and virtual spaces.

[2061] 4. Interface provision: Provide users with the interface necessary for will-writing tools and life purpose suggestions, and send the input information to the server.

[2062] 5. Emotion data transmission: The emotion data recognized by the emotion engine is transmitted to the server.

[2063] User

[2064] Users are seniors and their caregivers who:

[2065] 1. Login: Enter your account information into the terminal interface to log in to the system.

[2066] 2. Content usage: Select the desired content from each menu and operate it through the device interface.

[2067] 3. Upload: Upload the photos and videos you want to restore from your device to the server.

[2068] 4. Experience: Watch the playback video, try the exercise video, check the restored photos, visit the Metaverse space, write a will, and accept suggestions for a new purpose in life.

[2069] 5. Emotion input: Emotion data is provided in real time through the device's built-in camera and microphone.

[2070] Emotion Engine

[2071] The Emotion Engine provides the following features:

[2072] 1. Emotion recognition: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time.

[2073] 2. Emotion data generation: Generates recognized emotion data and sends it to the server via the terminal.

[2074] 3. Content recommendation: Based on the recognized sentiment, the server recommends the most suitable content.

[2075] As a concrete example, let us consider a scenario in which a user watches a video. First, the user logs in to the device and selects the "Watch Video" menu from the home screen. The device sends a content request to the server. The server retrieves a video list from the database and sends it to the device. The device displays the list and the user selects a video that interests them. The emotion engine recognizes the user's emotions in real time and sends that data to the server. Based on the recognized emotion data, the server selects a video that best suits the user's mood and sends it to the device. The device plays the video, and the user watches it.

[2076] In this way, this system provides an interface that is easy for users to operate, and the emotion engine recognizes the user's emotions in real time to provide optimal content, thereby improving the quality of life for the elderly and reducing the burden on caregivers.

[2077] The processing flow will be explained below.

[2078] Video viewing process flow

[2079] Step 1:

[2080] The user enters account information into the terminal interface and presses the login button.

[2081] Step 2:

[2082] The terminal transmits the user's input information to the server.

[2083] Step 3:

[2084] The server compares the received information with the database and returns the authentication result to the terminal.

[2085] Step 4:

[2086] The device displays the authentication result to the user, and if successful, transitions to the home screen.

[2087] Step 5:

[2088] The user selects the "Watch Video" menu from the home screen.

[2089] Step 6:

[2090] The terminal sends a content request to the server.

[2091] Step 7:

[2092] The server retrieves a list of video content from the database and transmits it to the terminal.

[2093] Step 8:

[2094] The device displays a list of videos.

[2095] Step 9:

[2096] The user selects the video they want to watch from the list.

[2097] Step 10:

[2098] The terminal sends a request for details of the selected video to the server.

[2099] Step 11:

[2100] The server reads the video data from the database and sends it to the terminal.

[2101] Step 12:

[2102] The emotion engine captures the user's facial expressions and voice data and analyzes emotions in real time.

[2103] Step 13:

[2104] The emotion engine sends the analysis results to the server.

[2105] Step 14:

[2106] The server receives the emotion data and selects the most appropriate video according to the user's current emotional state.

[2107] Step 15:

[2108] The server transmits the selected optimal video data to the terminal.

[2109] Step 16:

[2110] The terminal receives the video data and starts playing it.

[2111] Step 17:

[2112] The user watches the video.

[2113] Photo and video restoration process flow

[2114] Step 1:

[2115] The user selects the "Photo / Video Recovery" menu from the home screen.

[2116] Step 2:

[2117] The terminal displays a file selection screen to the user.

[2118] Step 3:

[2119] The user selects the photo or video files they want to restore.

[2120] Step 4:

[2121] The device uploads the selected file to the server.

[2122] Step 5:

[2123] The server stores the received file in the database and executes the restoration process.

[2124] Step 6:

[2125] The server sends the results of the restoration process to the terminal.

[2126] Step 7:

[2127] The emotion engine captures the user's facial expressions and reaction data and analyzes their emotions.

[2128] Step 8:

[2129] The emotion engine sends the analysis results to the server.

[2130] Step 9:

[2131] The device will display the restored photos and videos.

[2132] Step 10:

[2133] The user reviews the restored content.

[2134] Metaverse Processing Flow

[2135] Step 1:

[2136] The user selects the "Metaverse" menu from the home screen.

[2137] Step 2:

[2138] The device sends a metaverse request to the server.

[2139] Step 3:

[2140] The server generates metaverse spatial data and sends it to the terminal.

[2141] Step 4:

[2142] The device displays the metaverse space and provides the user with a virtual reality experience.

[2143] Step 5:

[2144] The emotion engine analyzes the user's voice and facial expressions and sends the emotion data to the server.

[2145] Step 6:

[2146] The server receives the emotion data and applies experience content appropriate to the user's emotional state.

[2147] Step 7:

[2148] A user visits past locations in the metaverse.

[2149] Process flow for writing a will

[2150] Step 1:

[2151] The user selects the "Write a Will" menu from the home screen.

[2152] Step 2:

[2153] The device displays a will creation tool.

[2154] Step 3:

[2155] The user inputs the contents of the will.

[2156] Step 4:

[2157] The emotion engine analyzes the user's facial expressions and voice and sends the emotion data to the server.

[2158] Step 5:

[2159] The terminal transmits the entered will data to the server.

[2160] Step 6:

[2161] The server stores the will data in a database and sends the confirmation results to the terminal.

[2162] Step 7:

[2163] The terminal displays the confirmation result to the user.

[2164] Purpose of Life Proposal Processing Flow

[2165] Step 1:

[2166] The user selects the "Life Purpose Suggestions" menu from the home screen.

[2167] Step 2:

[2168] The device displays a questionnaire about the user's interests and preferences.

[2169] Step 3:

[2170] The user answers the question and submits it.

[2171] Step 4:

[2172] The emotion engine analyzes the user's responses and real-time emotional state and sends the emotional data to the server.

[2173] Step 5:

[2174] The terminal transmits the user's response data to the server.

[2175] Step 6:

[2176] The server analyzes the data and suggests appropriate hobbies and activities.

[2177] Step 7:

[2178] The server sends the proposal results to the terminal.

[2179] Step 8:

[2180] The device displays the suggestions to the user.

[2181] Step 9:

[2182] The user confirms and engages in suggested activities and hobbies.

[2183] These processing flows create a system that is easy for elderly people to use, allows them to enjoy a variety of content, and, by using an emotion engine, provides a personalized experience, reducing the burden on caregivers.

[2184] Example 2

[2185] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2186] Current content systems for the elderly do not provide content that takes into account the user's emotions, resulting in a poor user experience. Other issues include the difficulty of restoring past photos and videos, and the lack of a virtual reality space that is easy for the elderly to use.

[2187] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2188] In this invention, the server includes an information processing device that retrieves data from a database according to content selected by the user and provides it to the device being used, an emotion analysis device that analyzes the user's voice, facial expression, and movement data to recognize emotions in real time, and a device that recommends optimal content based on the emotion data. This makes it possible to provide optimal content according to the user's emotions, improving the quality of the user experience. It also makes it possible to realize an environment that makes it easy to restore photos and videos and provide virtual reality spaces.

[2189] The term "information processing device means" refers to a device that retrieves data from a database according to content selected by a user and provides the data to the device being used.

[2190] The "device means" refers to a device that allows a user to select content and send a request to an information processing device. It also includes a function to play and display data provided by the information processing device.

[2191] "Emotion analysis means" refers to a device or program that analyzes the user's voice, facial expression, and movement data and recognizes emotions in real time.

[2192] "Means" generally refers to a method, apparatus, or program designed to achieve a specific purpose.

[2193] "Data" refers to information or content acquired and used via information processing device means, and includes various formats such as video, images, audio, and text.

[2194] "Virtual space providing means" refers to a device or program that enables a user to visit past locations in a virtual reality space.

[2195] This invention is a multi-functional content system for use by the elderly of the silver generation, and by combining it with an emotion analysis function, it realizes a system that provides optimal content to users. This system consists of four main components: an information processing device, a user device, a user, and an emotion analysis function.

[2196] Information processing device

[2197] The information processing device provides the following functions:

[2198] 1. Content management: The information processing device stores various content (videos, exercise videos, image restoration, virtual space, will writing tools, and life purpose suggestions) in a database and provides this content in response to user requests. This process uses a database such as MySQL.

[2199] 2. User authentication: When a user logs in through their device, the user is authenticated by checking against an authentication database. Authentication is performed using LDAP or OAuth2.

[2200] 3. Image restoration: The information processing device restores the images and videos uploaded by the user from their device using image processing technology (e.g., OpenCV).

[2201] 4. Virtual space generation: The information processing device generates a virtual reality space, allowing users to virtually visit places from the past. This process uses Unreal Engine and Unity.

[2202] 5. Emotional data management: The information processing device stores the emotional data recognized by the emotion analysis function in a database and provides personalized content.

[2203] Equipment used

[2204] A user device is a device that provides an interface for users to operate the system. It has the following functions:

[2205] 1. Content selection: The user selects the desired content from the menus of video, exercise video, restoration request, virtual space, will writing, and purpose in life suggestions.

[2206] 2. Request transmission: Based on the selected content, an appropriate request is sent to the information processing device. An HTTP request is used.

[2207] 3. Data playback and display: Receives data sent from the information processing device, plays back images and exercise videos, and displays restored images and virtual spaces.

[2208] 4. Interface provision: Provides the user with the interface necessary for the will writing tool and life purpose suggestions, and sends the input information to the information processing device.

[2209] 5. Emotion data transmission: Emotion data recognized by the emotion analysis function is transmitted to the information processing device.

[2210] User

[2211] Users are seniors and their caregivers who:

[2212] 1. Login: Enter your account information into the interface of your device to log in to the system.

[2213] 2. Content usage: Select the desired content from each menu and operate it through the interface of the device you are using.

[2214] 3. Upload: Upload the images or videos you want to restore from your device to the information processing device.

[2215] 4. Experience: Watch the video, try out the exercises, check the restored images, and visit the virtual space. You can also write a will and accept suggestions for a new purpose in life.

[2216] 5. Emotion input: Emotion data is provided in real time through the camera and microphone built into the device being used.

[2217] Sentiment analysis function

[2218] The sentiment analysis feature provides the following features:

[2219] 1. Emotion recognition: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time, using technologies such as OpenFace and DeepFace.

[2220] 2. Emotion data generation: Recognized emotion data is generated and sent to the information processing device via the device being used.

[2221] 3. Content recommendation: Based on the recognized emotions, the information processing device recommends the most suitable content.

[2222] Example scenario

[2223] As a specific example, a scenario in which a user watches a video will be described. First, the user logs in to the device being used and selects the "Watch Video" menu from the home screen. The device being used sends a content request to the information processing device. The information processing device retrieves a video list from a database and sends it to the device being used. The device being used displays the list, and the user selects a video of interest. The emotion analysis function recognizes the user's emotions in real time and sends that data to the information processing device. Based on the recognized emotion data, the information processing device selects a video that best suits the user's mood and sends it to the device being used. The device being used plays the video, and the user watches it.

[2224] Example prompt:

[2225] "Please explain the process of watching videos using a multi-functional content system for seniors in the silver generation. Please explain how a user logs in, selects a video, and how the emotion analysis function works."

[2226] In this way, this system provides an interface that is easy for users to operate, and the emotion analysis function recognizes the user's emotions in real time to provide optimal content, thereby improving the quality of life for the elderly and reducing the burden on caregivers.

[2227] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2228] The program processing flow of this system and specific steps

[2229] Processing Steps

[2230] Step 1: User Login

[2231] Step 2: Menu selection

[2232] Step 3: Submit a content request

[2233] Step 4: Providing and displaying content

[2234] Step 5: Collect and send emotion data

[2235] Step 6: Optimize and reselect content

[2236] Specific explanation of each processing step

[2237] Step 1: User Login

[2238] Input: Account information (ID and password)

[2239] Output: Authentication success or failure response

[2240] The user enters their account information on the login screen of their device and presses the login button. The device sends the entered information to the server as an HTTP POST request. The server verifies the authentication information using an LDAP or MySQL database, and if authentication is successful, returns a successful authentication response to the device. If authentication is unsuccessful, it returns an error message.

[2241] Specific operation: The user enters "ID: user123" and "Password: pass456" and clicks the login button. The device sends this information to the server, which queries the LDAP database for authentication.

[2242] Step 2: Menu selection

[2243] Input: User-selected menu item

[2244] Output: Selected menu information

[2245] The user selects the content they want to use (e.g., watching a video) from the menu displayed on the home screen. The device sends the selected menu information to the server via an HTTP GET request.

[2246] Specific operation: The user selects the "Watch Video" menu from the home screen and clicks the "Watch Video" button on the screen. The device sends a request including the selection information to the server.

[2247] Step 3: Submit a content request

[2248] Input: Selected menu information

[2249] Output: A list of matching content

[2250] The terminal transmits the selected menu information to the server, and the server retrieves the corresponding content list from the database and transmits the list to the terminal.

[2251] Specific operation: The device sends a "video viewing" request to the server. The server retrieves the video list from the MySQL database and sends it to the device in JSON format.

[2252] Step 4: Providing and displaying content

[2253] Input: Content list received from the server

[2254] Output: User selects and retrieves content

[2255] The device displays the content list received from the server to the user. The user selects the content of interest from the list, and the device sends the selection information to the server. The server retrieves the relevant content from the database and sends it to the device. The device plays and displays the content.

[2256] Specific operation: The user selects "Video A" from the displayed list and clicks the play button. The device sends the ID of Video A to the server, and the server retrieves the video data and sends the playback data to the device.

[2257] Step 5: Collect and send emotion data

[2258] Input: User's voice, facial expression, and movement data

[2259] Output: Generated emotion data

[2260] While the device is playing video, the camera and microphone collect the user's voice, facial expressions, and movement data. The emotion analysis function analyzes this data and generates emotion data in real time. The emotion data is then sent to the server via an HTTP POST request.

[2261] Specific operation: The device detects the user's facial expressions and voice, and the emotion analysis function recognizes that the user is "having fun." Emotional data is generated and sent to the server.

[2262] Step 6: Optimize and reselect content

[2263] Input: Received emotion data

[2264] Output: Recommend and serve new content

[2265] The server recommends new content based on the received emotion data. The device displays the recommended content to the user, and the user selects it again. The reselected content is then retrieved from the server and displayed.

[2266] Specific operation: The server analyzes the emotional data of "enjoyed" and recommends the more suitable "Video B." The device presents "Video B" to the user, who selects and plays it.

[2267] This will enable optimal content to be provided according to the user's emotions, improving the quality of life for the elderly.

[2268] (Application example 2)

[2269] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2270] When elderly people use content, it is often difficult to operate, and the content provided is often not suited to their individual emotions or state. This leads to problems that reduce the quality of life of the elderly and increase the burden on caregivers. In addition, there is a lack of appropriate exercise guidance to support psychological and physical health, and functions to recreate past memories in virtual space.

[2271] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes information collection device means for acquiring data from the data collection management device in accordance with information selected by the user and providing the data to the information device, information device means for the user to select information and send a request to the information collection management device, information device means for playing and displaying the data provided from the information collection management device, emotion recognition means for recognizing the user's emotion in real time and sending the data to the information collection management device, and information recommendation means for recommending optimal information based on the recognized emotion data. This provides an interface that is easy for the user to operate, and makes it possible to provide optimal content by having the emotion engine recognize the user's emotion in real time.

[2272] "User" refers to an individual or senior citizen who uses the system.

[2273] An "information accumulation and management device" is a device that stores and manages data and provides it as needed.

[2274] "Information equipment" refers to devices that are operated by users, such as smart glasses and smartphones.

[2275] The "information accumulation device means" is a function or system for acquiring data from the information accumulation management device and providing it to the information device.

[2276] The "information device means" is a function or system that allows a user to select information and send a request to the information accumulation management device.

[2277] The "information device display means" is a function or system for reproducing data provided from the information accumulation management device and displaying it on an information device.

[2278] The "emotion recognition means" is a function or system for recognizing the user's emotions in real time and transmitting the data to the information collection and management device.

[2279] The "information recommendation means" is a function or system for recommending optimal information to a user based on the recognized emotion data.

[2280] "Image restoration means" refers to a function or system for restoring images and videos uploaded by users.

[2281] The "virtual space generation means" is a function or system for generating a virtual space in which the user can visit places from the past.

[2282] This invention is a multi-functional content system for use by elderly people in the silver generation, which provides optimal content according to the user's emotions by combining it with an emotion recognition engine. This system consists of four main components: an information accumulation and management device (server), information equipment (terminal), users, and the emotion recognition engine.

[2283] Information accumulation and management device (server)

[2284] The server provides the following functionality:

[2285] 1. Information accumulation device means: Acquires data from a database according to information selected by the user and provides it to the information device.

[2286] 2. Image restoration tool: It has the function to restore images and videos uploaded by users.

[2287] 3. Virtual space generation means: A past location is generated as a virtual reality space and provided to the user.

[2288] 4. Emotion data management: Emotion data recognized by the emotion recognition engine is stored in a database and personalized content is provided.

[2289] Information equipment (terminals)

[2290] A terminal is a device that provides an interface for users to interact with the system. It has the following functions:

[2291] 1. Information device means: The user selects information and sends a request to the server.

[2292] 2. Information device display means: Receives data provided by the server, plays videos and exercise videos, and displays restored photos and virtual spaces.

[2293] 3. Input interface: Provides users with the interface necessary for the will-writing tool and emotion-based purpose-in-life suggestions, and sends the input content to the server.

[2294] 4. Emotion recognition means: Using the device's built-in camera and microphone, the user's emotional data is acquired in real time and sent to the server.

[2295] User

[2296] Users are seniors and their caregivers who:

[2297] 1. Login: Enter your account information into the terminal interface to log in to the system.

[2298] 2. Content usage: Select the desired information from each menu and operate it through the device interface.

[2299] 3. Upload: Upload the images or videos you want to restore from your device to the server.

[2300] 4. Experience: Watch the video, try out the exercises, check the restored photos, visit the virtual space, and even write a will and accept suggestions for a new purpose in life.

[2301] 5. Emotion input: Provide emotion data in real time via camera or microphone.

[2302] Emotion Recognition Engine

[2303] The emotion recognition engine provides the following features:

[2304] 1. Emotion recognition means: Analyzes the user's voice, facial expressions, and movement data to recognize emotions in real time.

[2305] 2. Emotion data transmission: Generate recognized emotion data and transmit it to the server via the terminal.

[2306] 3. Information recommendation method: Based on the recognized emotions, the server recommends the most appropriate information.

[2307] Specific examples

[2308] For example, suppose a user is wearing smart glasses and sitting in their living room in a relaxed mood. The emotion recognition engine recognizes the user's relaxed state in real time. Based on this, the server will recommend a landscape video that is most suitable for the relaxed mood, and a beautiful natural scene will be displayed on the smart glasses' display. The user can enjoy this visual experience while remaining relaxed.

[2309] Prompt Sentence Examples

[2310] "How can we recommend beautiful landscape videos to elderly people who are in a relaxed state?"

[2311] "Design a system that uses an emotion recognition engine to recognize emotions in real time and display content based on that in smart glasses."

[2312] The above is a specific embodiment for carrying out the present invention.

[2313] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2314] Step 1:

[2315] A user logs in to an information device (e.g., smart glasses). As input, the user enters account information, and the server authenticates the user based on this information. As output, if authentication is successful, the user can access the home screen.

[2316] Step 2:

[2317] The user selects the desired content menu (e.g., watching a video, watching a gymnastics video, visiting a virtual space) from the home screen. The user's selection is received as input, and this data is sent from the device to the server. As output, the server provides the device with a list of the corresponding content.

[2318] Step 3:

[2319] The server uses an emotion recognition engine to obtain the user's emotion data in real time. It receives data from the camera and microphone as input, analyzes the emotion using an emotion recognition model, and generates the recognized emotion data as output, which is then sent to the server.

[2320] Step 4:

[2321] The server recommends optimal content based on the received emotional data. It receives the emotional data and content request as input, selects optimal content from a database based on the received emotional data, and sends the selected content to the device as output.

[2322] Step 5:

[2323] The terminal plays the content sent from the server and displays it to the user. As input, it takes the received content data and converts it into a format for display. As output, the user can visually confirm the played content on the display of the smart glasses.

[2324] Step 6:

[2325] When a user uploads an image or video they want to repair, their device sends these files to the server. As input, it takes the image or video file and uploads it to the server. As output, the server starts the repair process.

[2326] Step 7:

[2327] The server sends the repaired image or video file to the device. It receives the file to be repaired as input, processes it using an image repair algorithm, and provides the repaired file to the device as output.

[2328] Step 8:

[2329] A user sends a request to visit a virtual space from a terminal. The terminal receives the request data for visiting the virtual space as input and sends it to the server. As output, the server generates a virtual space and provides it to the terminal.

[2330] Step 9:

[2331] The terminal displays the virtual space provided by the server and provides the user with an experience. It receives virtual space data as input and converts it into a display format. As an output, the user can enjoy the experience in the virtual space.

[2332] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2333] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2334] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2335] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2336] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2337] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2338] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2339] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2340] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2341] The emotion identification model 59 inputs user input into a pr...

Claims

1. a server means for acquiring data from a database according to content selected by a user and providing the data to the terminal; A terminal means for a user to select content and send a request to a server; a terminal means for reproducing and displaying data provided by the server; A system including:

2. A method for users to upload the photos and video files they want to restore and for the server to process the restoration. A terminal means for acquiring the restored file from the server and displaying it is included. The system of claim 1 .

3. a metaverse means by which users can visit past locations in a virtual reality space; The server generates a metaverse space and provides it to the terminal. The system of claim 1 .

Images