System

Abstract

A system is provided.SOLUTION: A system comprising: an audio input means; a conversion means for converting received audio to text; a storage means for storing the converted text in a database; a AI means for automatically refining and adjusting a family member's schedule; an interface means for enabling a user to arrange a schedule in a fun manner using gamification; and a notifying means for sending notifications regarding the stored schedule.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When raising children, parents and other family members need to manage their own and their children's schedules, but this coordination can be extremely cumbersome, time-consuming, and labor-intensive. There is also a need for a way to not only centrally manage the schedules of all family members, but also to make it enjoyable. Furthermore, there is a lack of a system for storing and managing records of children's growth, along with photos, and for easily accessing them. There is a need for a system that solves these problems and allows for more efficient and enjoyable family schedule management. [Means for solving the problem]

[0005] This invention provides a system that includes a voice input means, a conversion means for converting received voice data into text data, a storage means for saving the converted text data in a database, an AI means for automatically refining and adjusting family members' schedules, an interface means for allowing users to enjoyably organize their schedules using gamification, and a notification means for sending notifications about saved schedules. The system also includes a management means for managing image data by linking it to past family schedules, and an extraction means for automatically extracting schedules from messaging tools such as LINE. This system allows users to easily create schedules using voice input, and the AI ​​automatically adjusts schedules and suggests optimal schedules, streamlining tedious schedule management. Furthermore, gamification makes schedule management fun and simplifies the management of image data linked to past schedules.

[0006] The "voice input means" is a means for receiving voice data uttered by the user.

[0007] The "conversion means" is a means for converting received voice data into text data.

[0008] The "storage means" is a means for storing the converted text data in a database.

[0009] "AI Measures" are measures that include artificial intelligence technology to automatically detail and coordinate schedules for family members.

[0010] "Interface means" refers to screens and operating means that allow users to organize their schedules in a fun way using gamification.

[0011] The "notification means" is a means for sending a notification regarding a saved schedule to the user's terminal.

[0012] The "management means" is a means for managing image data in association with past family schedules.

[0013] The "extraction method" is a method for automatically extracting schedules from messaging tools such as LINE. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] System Overview

[0036] This invention is a system that provides a child-rearing support schedule app, and includes a voice input means, a conversion means, a storage means, an AI means, an interface means, a notification means, a management means, and an extraction means. This system allows users to easily create, manage, and adjust schedules, and also provides a function for managing past records and child growth.

[0037] Explanation of voice input and conversion methods

[0038] When a user uses the voice input means to create a new plan, he or she inputs, for example, "A picnic in the park at 2 PM next Saturday." The voice input means receives this voice data and transmits it to the conversion means. The conversion means converts the voice data into text data, generating the text data "A picnic in the park at 2 PM next Saturday."

[0039] Explanation of storage and AI methods

[0040] The text data generated by the conversion means is stored in a database using a storage means. The stored schedule data is automatically detailed by the AI ​​means. The AI ​​means analyzes overlaps with other schedules and optimal time slots, and proposes optimal schedules to the user.

[0041] Description of interface and notification methods

[0042] The interface means provides gamified screens and operation means to allow users to enjoyably organize their schedules. Users can intuitively operate their schedules by, for example, dragging and dropping schedule icons. Notifications about saved schedules are sent to the user's device using the notification means, allowing users to manage their schedules without missing important schedules.

[0043] Description of management and extraction methods

[0044] The management means provides a function to manage image data by linking it to past family schedules. For example, photos from a family trip can be uploaded and saved linked to the date of the trip. The extraction means also has a function to automatically extract schedules from messaging tools such as LINE and add them to the database. This allows for efficient schedule management by automatically registering schedules based on the contents of messages.

[0045] Specific examples

[0046] Example 1: Creating an appointment by voice input

[0047] The user can say, "Make a doctor's appointment next Friday at 3pm." The device sends this voice data to the server, which converts the speech into text and stores it in a database. AI tools check for conflicts with other appointments, generate an optimal schedule, and even send a notification to the device so the user can confirm their appointment.

[0048] Example 2: Photo management and schedule linking

[0049] A user uploads photos from their child's sports day. The device sends the photos to the server, which stores them in association with the date of the sports day. When the user later references the date of the sports day, related photos are displayed, allowing the user to easily reminisce about their memories.

[0050] Through these means and processes, the present invention allows users to more efficiently manage their family schedules and assist in raising their children in an enjoyable way.

[0051] The processing flow will be explained below.

[0052] Step 1:

[0053] The user starts the application and uses the voice input means to input plans by voice, such as "Picnic in the park at 2:00 p.m. next Saturday."

[0054] Step 2:

[0055] The device receives the audio data and temporarily stores it in local storage.

[0056] Step 3:

[0057] The device calls the speech recognition API and converts the received voice data into text data, generating the text data "Picnic in the park next Saturday at 2 PM."

[0058] Step 4:

[0059] The terminal transmits the generated text data to the server.

[0060] Step 5:

[0061] The server analyzes the received text data and stores it in the database as a new appointment.

[0062] Step 6:

[0063] The AI ​​tool retrieves saved schedule data, automatically checks whether it overlaps with other appointments, analyzes the optimal time slot, and adjusts the schedule accordingly.

[0064] Step 7:

[0065] The server generates optimized schedule data and sends a notification to the terminal.

[0066] Step 8:

[0067] The device receives the notification and displays to the user that a schedule has been registered for "picnic in the park next Saturday at 2:00 PM."

[0068] Step 9:

[0069] Users open the gamification interface on the app and drag and drop event icons to coordinate with other events.

[0070] Step 10:

[0071] The terminal transmits the changed schedule information to the server, and the saved schedule is updated.

[0072] Step 11:

[0073] The user opens the app's "Photo Management" menu to upload photos from a family trip.

[0074] Step 12:

[0075] The device sends the selected photo to the server, which stores the photo and links it to the corresponding event.

[0076] Step 13:

[0077] The saved photos and linked schedules are displayed to the user through the interface means of the terminal, allowing the user to view the photos and easily manage the growth record.

[0078] Step 14:

[0079] The server analyzes message data to automatically extract schedules from messaging tools such as LINE.

[0080] Step 15:

[0081] The server extracts the schedule from the parsed message and stores it as a new schedule in the database.

[0082] Step 16:

[0083] The new event will be saved and coordinated with other events using AI means, and the event will be notified to the device, where the user can see that a new event has been added.

[0084] Example 1

[0085] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0086] Conventional schedule management systems do not fully utilize voice input, automatic data analysis, or an intuitive user interface, requiring users to spend time and effort creating and managing schedules. Furthermore, comprehensive schedule management is difficult because it is not easy to manage image data linked to past family events or to extract schedules from messaging tools.

[0087] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0088] In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, a storage means for saving the converted text data in a data storage, an AI means for checking and optimizing schedule overlaps, an interface means for allowing a user to intuitively organize their schedule, a notification means for sending notifications about the saved schedule to the terminal, a management means for managing image data in association with past family schedules, and an extraction means for automatically extracting schedules from a message tool. This enables users to quickly create schedules through voice input, propose optimized schedules, easily manage image data linked to past events, and automatically extract schedules from a message tool.

[0089] "Voice input means" refers to a device or software that captures voice data spoken by a user.

[0090] The "conversion means" is software or hardware that converts received voice data into text data.

[0091] "Storage means" refers to software or hardware for recording the converted text data in data storage.

[0092] "AI tools" are artificial intelligence algorithms and models that check for overlaps and optimize schedules.

[0093] "Interface means" refers to software that provides a user interface that allows users to intuitively operate and manage their schedules.

[0094] The "notification means" is software or hardware for notifying the user terminal of information related to the saved schedule.

[0095] The "management means" is software or hardware for managing image data in association with the family's past schedules.

[0096] The "extraction means" is software or a function that automatically extracts schedules from message tools.

[0097] This invention is a system for providing a childcare support schedule app, designed to enable users to efficiently create, manage, and adjust schedules. The system includes a voice input unit, a conversion unit, a storage unit, an AI unit, an interface unit, a notification unit, a management unit, and an extraction unit.

[0098] When a user creates a schedule, they use a voice input means. For example, they might say, "Picnic in the park next Saturday at 2 PM." The voice input means captures this voice data and sends it from the device to the server. The server converts the voice data into text data using the Google Cloud Speech-to-Text API as a conversion means. The converted text data is stored in data storage on the server, for example, in MySQL.

[0099] The stored data is analyzed by AI tools, which use generative AI models like TensorFlow and PyTorch to analyze and optimize schedules and identify overlaps with other appointments. An optimized schedule is then generated and sent from the server to the device.

[0100] Users can check and manage their schedules through an interface on their device. The interface uses React and Vue.js to provide an intuitive interface, allowing users to easily adjust their schedules using drag and drop. Notifications based on saved schedules are sent to the user's device in real time using services such as Firebase Cloud Messaging.

[0101] Furthermore, the management tool can be used to manage image data by linking it to past schedules. When a user uploads a photo of their child's sports day, it can be linked to the date of the event and saved for easy reference later. The extraction tool also includes a function to automatically extract schedules from messaging tools such as LINE and add them to the database.

[0102] For example, if a user says, "Make a doctor's appointment next Friday at 3pm," the device will send this voice data to the server, which will convert the voice to text and store it in a database. AI tools will check for conflicts with other appointments, generate an optimal schedule, and send a notification to the device so the user can confirm the appointment.

[0103] Example prompt sentence:

[0104] "Make a doctor's appointment next Friday at 3pm."

[0105] This allows users to quickly create schedules using voice input, propose optimized schedules, manage image data linked to past events, and automatically extract schedules from messaging tools.

[0106] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0107] Step 1:

[0108] A user creates a schedule using voice input. The user launches the app on the device and says, "Picnic in the park next Saturday at 2 PM." The input is voice data, which is captured by the device's microphone. The device then sends this voice data to the server.

[0109] Step 2:

[0110] The server converts the received voice data into text data. The Google Cloud Speech-to-Text API is used as the conversion method to convert the voice data into text data. The input is voice data, and the output is text data such as "Picnic in the park next Saturday at 2 PM." The server generates the converted text data and passes it on to the next process.

[0111] Step 3:

[0112] The server saves the converted text data in data storage. A MySQL database is used as the storage method. The input is text data, which is saved along with the associated user ID and timestamp. The saved text data is "Picnic in the park next Saturday at 2pm" and is saved in the database.

[0113] Step 4:

[0114] The server uses AI tools to analyze the schedule data, check for overlaps with other appointments, and optimize them. TensorFlow or PyTorch is used as the AI ​​tool. The input is stored text data and existing schedule data, and the AI ​​model analyzes the data and generates optimal schedule proposals. The output is the optimized schedule proposals.

[0115] Step 5:

[0116] The server sends the optimized schedule to the terminal. The input is the optimized schedule proposal, and the output is the schedule information to be sent to the user terminal. As a result, the terminal uses the notification means to send a notification to the user saying, "A picnic in the park has been scheduled for next Saturday at 2 PM."

[0117] Step 6:

[0118] The user checks and manipulates the schedule using an interface on the terminal. The user can adjust the schedule by performing operations such as drag and drop on the interface. The input is the schedule information sent from the server, and the output is the schedule content checked or edited by the user. Specific operations include the user dragging icons and changing the time.

[0119] Step 7:

[0120] The user uploads a photo and associates it with a past event. The user selects "Sports Day Photos" from the device interface and uploads the photo. The device sends the photo file to the server, which associates it with the date of the sports day and stores it in a database. The input is the photo data, and the output is the associated date and event information.

[0121] Step 8:

[0122] The server automatically extracts schedules from messaging tools. As a means of extraction, it uses the API of messaging tools such as LINE to analyze schedule information from user messages. The input is message data, and the output is the extracted schedule information. The analyzed data is saved in data storage so that the user can view it later.

[0123] This allows users to quickly create schedules using voice input, efficiently manage images associated with past events, and schedules automatically extracted from message tools.

[0124] (Application example 1)

[0125] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0126] While conventional schedule management systems allow users to easily create schedules using voice input, they lack the ability to display content related to past events or to optimize schedules in real time. Furthermore, few systems support automatic schedule extraction from messaging tools, making efficient schedule management difficult. Furthermore, they lack an interface and gamification elements that allow users to organize their schedules in a fun way, making them difficult to use on a daily basis.

[0127] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0128] In this invention, the server includes a voice input device, a conversion device that converts received voice data into text data, a storage device that stores the converted text data in a database, an AI device that automatically details and coordinates family members' schedules, an interface device that uses gamification to help users organize their schedules in a fun way, a notification device that sends notifications about saved schedules, a content management device that automatically displays content linked to past events, an extraction device that automatically extracts schedules from a messaging tool, and a schedule optimization device that optimizes and notifies users of schedules. This allows users to efficiently manage their schedules and easily check related content and past events. Furthermore, real-time schedule optimization significantly improves everyday usability.

[0129] A "voice input device" is a device that allows a user to input instructions by voice.

[0130] A "conversion device" is a device that has the function of converting voice data into text data.

[0131] A "storage device" is a device for storing converted text data in a database.

[0132] An "AI device" is a device that uses artificial intelligence to automatically detail and optimally coordinate the schedules of family members.

[0133] An "interface device" is a device that uses gamification to provide an operating screen that allows users to organize their schedules in a fun way.

[0134] A "notification device" is a device that sends notifications about saved appointments to a user's terminal.

[0135] A "content management device" is a device that automatically displays and manages content linked to past events.

[0136] An "extraction device" is a device that has the function of automatically extracting schedules from a message tool.

[0137] A "schedule optimization device" is a device that analyzes existing schedules, adjusts them to the optimal form, and notifies the user.

[0138] System Overview

[0139] This invention relates to a child-rearing support system that includes a voice input device, a conversion device, a storage device, an AI device, an interface device, a notification device, a content management device, an extraction device, and a schedule optimization device. This system allows users to easily create schedules through voice input and efficiently manage and adjust them thereafter.

[0140] Hardware and software used

[0141] The server converts data from the voice input device into text data using a speech recognition API (e.g., Google Cloud Speech-to-Text). The storage device uses a database management system (e.g., MySQL). The AI ​​device uses a machine learning model (e.g., TensorFlow) to optimize the schedule. The interface device uses web technology (e.g., React.js) to provide a user interface incorporating gamification. The notification device uses the smartphone's push notification function (e.g., Firebase Cloud Messaging).

[0142] Natural language processing explanation

[0143] 1. The user speaks, "Picnic in the park next Saturday at 2 PM." The server receives this voice data via the voice input device.

[0144] 2. The received voice data is converted into text data using a conversion device.

[0145] 3. The converted text data is stored in a database using a storage device.

[0146] 4. The AI ​​device on the server analyzes the saved schedule data, checks for overlaps with other appointments, and generates the optimal schedule.

[0147] 5. Users can check their schedules through an interface device. The gamified screen allows users to enjoy drag-and-drop operations.

[0148] 6. When an appointment is approaching, users will be notified through the notification device, so they will not miss any important appointments.

[0149] 7. Using content management devices, photos and videos related to past events are automatically displayed. For example, when a user uploads a photo of a sports day, detailed information related to that sports day is displayed.

[0150] 8. The extraction device automatically extracts schedules from messaging tools such as LINE and stores them in a database, eliminating the need to manually enter message content.

[0151] 9. The schedule optimization device performs real-time schedule optimization and notifies the user of the optimal schedule.

[0152] Examples of specific examples and prompts

[0153] Examples:

[0154] The user can say, "Make a doctor's appointment next Friday at 3 p.m." The server converts the speech into text using a speech recognition API and stores it in a database. The AI ​​device then checks for overlapping appointments, generates an optimal schedule, and notifies the user via a smartphone push notification. Additionally, related doctor visit records and photos are displayed via the content management device.

[0155] Example prompt sentence:

[0156] "I'm going to the aquarium with my family on the weekend."

[0157] "Picnic next Friday afternoon"

[0158] "Kids' dance lessons on Saturday mornings at 9am"

[0159] This allows users to easily create and efficiently manage appointments through voice input, optimizing schedules, and automatically displaying records of past events for added convenience.

[0160] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0161] Step 1:

[0162] The user speaks "Picnic in the park next Saturday at 2 PM." Input: User's voice data. Output: Voice data.

[0163] Specific operation: The user speaks their schedule into the smartphone's voice input device. The voice data is received by the device.

[0164] Step 2:

[0165] The device uses a speech recognition API to convert voice data into text data. Input: Voice data. Output: Text data.

[0166] Specific operation: The device's voice recognition API (e.g., Google Cloud Speech-to-Text) analyzes the voice data and converts it into text format.

[0167] Step 3:

[0168] The converted text data is sent to the server and saved to the storage device. Input: Text data. Output: Saved data.

[0169] Specific operation: The terminal sends text data to the server, and the server stores the text data in a database management system (e.g., MySQL).

[0170] Step 4:

[0171] The AI ​​device analyzes the saved schedule data, checks for overlaps with other appointments, and generates an optimal schedule. Input: Saved data. Output: Optimized schedule.

[0172] How it works: The AI ​​device uses a machine learning model (e.g., TensorFlow) to analyze stored schedule data and generate an optimal schedule.

[0173] Step 5:

[0174] The interface device displays a screen that allows the user to intuitively operate the schedule. Input: Optimized schedule. Output: User interface.

[0175] Specific operation: The server uses web technologies (e.g. React.js) to generate an interface that allows users to edit appointments using drag and drop, and displays it on the device.

[0176] Step 6:

[0177] When an appointment is approaching, the notification device sends a notification to the user's device. Input: Optimized schedule. Output: Notification.

[0178] Specific operation: The server uses a push notification service (e.g., Firebase Cloud Messaging) to send a notification of the event to the user's device.

[0179] Step 7:

[0180] When a user uploads photos or videos linked to past events to a content management device, the related content is displayed. Input: Photo or video data. Output: Display of related content.

[0181] Specific operation: The user uploads photos and videos from their device to the server, which then uses a content management device to link them with related past events and store and display them in a database.

[0182] Step 8:

[0183] The extraction device automatically extracts schedule data from messaging tools such as LINE and stores the data in a database. Input: LINE messages. Output: Schedule data.

[0184] Specific operation: The server analyzes the message content using the API of the message tool and saves the extracted schedule data in a database.

[0185] Step 9:

[0186] The schedule optimization device optimizes the schedule in real time and notifies the user of the results. Input: Saved schedule data. Output: Optimized schedule notification.

[0187] Specific operation: The server uses machine learning models to perform real-time analysis, recalculate an optimized schedule, and send notifications to the user device.

[0188] The above are the specific processing steps of the system that realizes the application example.

[0189] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0190] System Overview

[0191] This invention is a system for providing a childcare support schedule app, and includes a voice input unit, a conversion unit, a storage unit, an AI unit, an interface unit, a notification unit, a management unit, an extraction unit, and an emotion engine. This system allows users to easily create, manage, and adjust schedules. It also provides a function for managing past records and child growth, and provides a more personalized experience by recognizing the user's emotions and responding accordingly.

[0192] Explanation of voice input and conversion methods

[0193] When a user uses the voice input means to create a new plan, the user inputs a voice such as "Picnic in the park at 2 PM next Saturday." The voice input means receives this voice data and transmits it to the conversion means. The conversion means converts the voice data into text data, generating the text data "Picnic in the park at 2 PM next Saturday."

[0194] Emotion Engine Explained

[0195] The emotion engine has the ability to analyze the emotions expressed when a user speaks and adjust the system's behavior based on those emotions. For example, if a user speaks of feeling stressed, the emotion engine will recognize this and suggest schedules and notifications that will help them relax. It also accumulates emotion data and suggests optimal schedules based on past emotion patterns.

[0196] Explanation of storage and AI methods

[0197] The text data generated by the conversion means is stored in a database using a storage means. The stored schedule data is automatically detailed by the AI ​​means. The AI ​​means analyzes overlaps with other schedules and optimal time slots to adjust the schedule.

[0198] Description of interface and notification methods

[0199] The interface means provides gamified screens and operation means to allow users to enjoyably organize their schedules. Users can intuitively operate their schedules, for example, by dragging and dropping schedule icons. Notifications about saved schedules are sent to the user's device using the notification means and are automatically adjusted to suit the emotion engine.

[0200] Description of management and extraction methods

[0201] The management means provides a function to manage image data by linking it to past family schedules. For example, photos from a family trip can be uploaded and saved linked to the date of the trip. The extraction means also has a function to automatically extract schedules from messaging tools such as LINE and add them to the database. This allows for efficient schedule management by automatically registering schedules based on the contents of messages.

[0202] Specific examples

[0203] Example 1: Voice-activated scheduling and emotion recognition

[0204] The user voice-inputs, "Make a doctor's appointment next Friday at 3 p.m." The device sends this voice data to the server, which converts the voice into text and stores it in a database. AI tools check for conflicts with other appointments and generate an optimal schedule. The emotion engine analyzes the voice data and, if the user is nervous, suggests relaxing appointments (e.g., an aromatherapy massage the next day).

[0205] Example 2: Photo management and schedule linking

[0206] A user uploads photos from their child's sports day. The device sends the photos to the server, which stores them in association with the date of the sports day. When the user later references the date of the sports day, related photos are displayed, allowing the user to easily reminisce about their memories.

[0207] Example 3: Extracting schedules from messages and reflecting emotional data

[0208] If a LINE message is sent saying, "I want to go see a movie this weekend," the server analyzes the message and automatically adds the event to the database. Furthermore, the emotion engine recognizes the user's emotions from the context of the message and sends encouraging notifications based on those emotions.

[0209] Through these means and processes, this invention allows users to manage their family schedules more efficiently and support child-rearing in a fun way. In addition, the introduction of an emotion engine provides a more personal and emotionally sensitive user experience.

[0210] The processing flow will be explained below.

[0211] Step 1:

[0212] The user starts the application and uses the voice input means to input plans by voice, such as "Picnic in the park at 2:00 p.m. next Saturday."

[0213] Step 2:

[0214] The device receives the audio data and temporarily stores it in local storage.

[0215] Step 3:

[0216] The terminal transmits the voice data to the server, and the voice data is converted into text data by the conversion means, generating text data such as "Picnic in the park at 2:00 PM next Saturday."

[0217] Step 4:

[0218] The terminal transmits the generated text data to the server.

[0219] Step 5:

[0220] The server analyzes the received text data and stores it in the database as a new appointment.

[0221] Step 6:

[0222] AI tools retrieve saved schedule data, automatically check whether it overlaps with other appointments, and adjust the schedule.

[0223] Step 7:

[0224] The emotion engine analyzes the voice input data and recognizes the user's emotions. For example, if it recognizes that the user is feeling stressed, it stores that emotion data.

[0225] Step 8:

[0226] The server generates optimized schedule data and determines appropriate notification content based on feedback from the emotion engine.

[0227] Step 9:

[0228] The server sends a notification to the device, and the user confirms that a plan to "have a picnic in the park next Saturday at 2 p.m." has been registered, and receives a notification message that takes emotion into consideration.

[0229] Step 10:

[0230] Users open the gamification interface on the app and drag and drop event icons to coordinate with other events.

[0231] Step 11:

[0232] The terminal transmits the changed schedule information to the server, and the saved schedule is updated.

[0233] Step 12:

[0234] The user opens the app's "Photo Management" menu to upload photos from a family trip.

[0235] Step 13:

[0236] The terminal transmits the selected photo to the server, which stores the photo and links it to the corresponding appointment.

[0237] Step 14:

[0238] The saved photos and linked schedules are displayed to the user through the interface means of the terminal, allowing the user to view the photos and easily manage the growth record.

[0239] Step 15:

[0240] The server analyzes message data to automatically extract schedules from messaging tools such as LINE.

[0241] Step 16:

[0242] The server extracts the schedule from the parsed message and stores it as a new schedule in the database.

[0243] Step 17:

[0244] The new event will be saved and coordinated with other events using AI means, and the event will be notified to the device, where the user can see that a new event has been added.

[0245] Step 18:

[0246] The emotion engine accumulates the user's emotional data and proposes optimal schedules based on past emotional patterns. During times when the user frequently feels stressed, it suggests relaxing activities.

[0247] Example 2

[0248] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0249] Conventional schedule management systems required users to manually input schedules, which resulted in issues with overlaps and time-consuming adjustments to optimal schedules. Furthermore, they only provided simple schedule management without taking into account the user's emotional state, preventing them from providing a personalized experience. Furthermore, the system did not automatically extract schedules from past schedule data or messaging tools, which often required time and effort from users.

[0250] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, an emotion engine for analyzing the user's emotions, and an AI means for adjusting the system's operation based on the analyzed emotion data. This allows the user to easily input plans by voice and enables optimal schedule management according to the user's emotions. In addition, past schedule data management and automatic extraction functions from message tools are also provided, realizing a more efficient and personalized experience.

[0251] The "voice input means" is a device or function that allows the user to input a schedule by voice.

[0252] The "conversion means" is a device or program for converting received voice data into text data.

[0253] "Storage means" refers to a device or function for storing the converted text data in a database.

[0254] An "emotion engine" is a device or program that analyzes the emotion expressed by the user when inputting voice and generates emotion data.

[0255] "AI means" means a device or program that adjusts the system's behavior and refines and optimizes schedules based on analyzed emotional data.

[0256] An "interface means" is a device or program that provides an operation screen and functions using gamification elements so that the user can organize their schedule in a fun way.

[0257] "Notification means" is a device or function for sending notifications regarding saved schedules to the user's terminal.

[0258] The "management means" is a device or function for managing image data in association with past family schedules.

[0259] The "extraction means" is a device or program for automatically extracting schedules from the message tool and adding them to the database.

[0260] System Overview

[0261] This invention is a schedule management system for supporting child-rearing, which includes a voice input means, a conversion means, a storage means, an emotion engine, an AI means, an interface means, a notification means, a management means, and an extraction means. This system allows users to easily create, manage, and adjust schedules, and also provides personalized suggestions based on emotion recognition.

[0262] Hardware and Software Configuration

[0263] Voice input method: Uses the microphone on a smartphone or tablet device. For software, a voice input API is used.

[0264] Conversion method: Converting voice data into text data using speech recognition software (e.g., Google Speech-to-Text API).

[0265] Storage method: Store text data and emotion data in a database (e.g., MySQL).

[0266] Emotion engine: Analyzes emotions from audio data using an emotion analysis library (e.g., IBM Watson Tone Analyzer).

[0267] AI methods: Use machine learning models (e.g., TensorFlow) to adjust and optimize schedules.

[0268] Interface solutions: Provide a user interface incorporating gamification elements, such as a UI component that allows dragging and dropping appointment icons.

[0269] Notification method: Uses an API with push notification functionality to send notifications to the user's device.

[0270] Management method: A photo storage service for managing image data and linking that data to a database.

[0271] Extraction method: Use the API of a messaging tool (e.g., LINE) to automatically extract schedules from messages.

[0272] System Operation

[0273] First, the user uses the device's voice input means to input their plans by voice. For example, they might say, "Picnic in the park next Saturday at 2 p.m." The device collects this voice data and sends it to a server via the Internet. The server uses voice recognition software to convert the voice data into text data. The converted text data is then stored in a database.

[0274] The server then uses an emotion engine to analyze the emotion of the voice data. The analysis results are stored as emotion data. For example, if the user is speaking with a happy expression, the emotion data is registered as "enjoyment."

[0275] Based on the stored text and emotion data, the server uses AI methods to optimize the schedule, for example by checking whether appointments overlap with other appointments and adjusting them if necessary, and then storing the adjusted schedule back in the database.

[0276] The interface means is designed to allow users to enjoyably organize their schedules, and allows intuitive operation. For example, users can easily change their schedules by dragging and dropping appointment icons.

[0277] Based on the saved schedule data, the notification means generates and sends notifications to the user's device. The notification content takes into account emotional data, providing a more personalized experience.

[0278] Finally, the management and extraction methods can also integrate past schedule data and information from messaging tools. For example, it is possible to extract content such as "I want to go see a movie this weekend" from a LINE message and automatically register it as a schedule.

[0279] Prompt Sentence Examples

[0280] When a user says, "Make an appointment to see the dentist tomorrow at 3 PM," the device sends this voice data to the server. The server generates text data saying, "Make an appointment to see the dentist tomorrow at 3 PM," and stores it in a database along with emotional data analyzed by the emotion engine. The AI ​​means uses this information to coordinate with other appointments and sends a notification to the device.

[0281] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0282] Step 1:

[0283] The user performs voice input. The user speaks to the terminal, "Picnic in the park next Saturday at 2 PM." This voice input is the input data. Voice data is generated as output.

[0284] Step 2:

[0285] The device sends the voice data to the server. The device collects the voice data and sends it to the server via an internet connection. This transfers the voice data to the server. The input is voice data, and the output is the transfer of voice data to the server.

[0286] Step 3:

[0287] The server converts the voice data into text data. The server uses speech recognition software (e.g., Google Speech-to-Text API) to analyze the transmitted voice data and generate text data such as "Picnic in the park next Saturday at 2 p.m." The input is voice data, and the output is text data.

[0288] Step 4:

[0289] The server analyzes emotions using an emotion engine. The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the emotions in the voice and generate emotion data. For example, if someone is talking about enjoying a picnic, the emotion data "enjoyed" is generated. The input is text data, and the output is emotion data.

[0290] Step 5:

[0291] The server stores the text data and emotion data in a database. The server uses a storage means to record the converted text data "Picnic in the park next Saturday at 2 PM" and emotion data "fun" in the database. The input is text data and emotion data, and the output is storage in the database.

[0292] Step 6:

[0293] The server generates a detailed schedule using AI. It uses a machine learning model (e.g., TensorFlow) to analyze whether the schedule overlaps with the user's other plans and proposes and adjusts the optimal schedule. For example, if the schedule overlaps with other plans, it suggests a different time slot. The input is text data and existing schedule data, and the output is the optimized schedule data.

[0294] Step 7:

[0295] The device notifies the user of the results. The device receives the notification data generated by the server and notifies the user through push notifications or interface means. For example, a notification such as "A picnic in the park has been scheduled for next Saturday at 2 PM" is sent. The input is the notification data, and the output is the notification sent to the user.

[0296] (Application example 2)

[0297] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0298] In today's industrial world, there is a demand for reducing the burden on factory workers and for efficient work schedule management. While conventional systems can manage schedules based on voice instructions, they have the problem of not optimizing schedules by taking into account the emotional state of workers. This can result in workers' stress and fatigue being ignored, which can lead to reduced productivity and a worsening work environment.

[0299] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0300] In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, a storage means for saving the converted text data in a database, an emotion engine for analyzing the emotions of workers and adjusting the operation of the system based on those emotions, an AI means for automatically refining and adjusting the schedule of workers, an interface means for intuitively operating the schedule, and a notification means for sending notifications regarding the saved schedule. This enables flexible and efficient schedule management that takes into account the emotional state of workers.

[0301] "Audio input means" refers to devices or techniques for receiving audio data.

[0302] "Conversion means" refers to a device or technology that converts received voice data into text data.

[0303] "Storage means" refers to the device or technology used to store the converted text data in a database.

[0304] An "emotion engine" refers to a device or technology that analyzes emotions from a worker's voice and adjusts the system's operation based on those emotions.

[0305] "AI means" refers to artificial intelligence technology that automatically details and adjusts worker schedules.

[0306] "Interface means" refers to the screens and means that allow users to intuitively operate their schedules.

[0307] "Notification Means" means any device or technology that sends notifications to a user regarding saved appointments.

[0308] "Management means" refers to devices and technologies for managing image data in association with past schedules.

[0309] "Extraction means" refers to a device or technology that automatically extracts schedules from a message tool.

[0310] This invention is a system that efficiently manages the work schedules of factory workers and improves their working environment by taking into account their emotional state. The system consists of the following components:

[0311] 1. Voice input and conversion methods

[0312] A worker inputs a command by voice, such as "Maintenance at 10:00 AM next Monday." This voice data is received through the voice input means and converted into text data using the conversion means. This converted text data is stored in a database by the storage means described below.

[0313] 2. Preservation means

[0314] The converted text data is stored in a database system such as SQLite. The database serves as the basis for schedule management and stores instructions and emotional data for each worker.

[0315] 3. Emotion Engine

[0316] The emotion engine analyzes emotions during voice input and determines states such as stress, fatigue, joy, etc. For example, if it determines that a worker is feeling stressed, the emotion engine will suggest relaxing tasks.

[0317] 4. AI means

[0318] Based on the stored text data, the AI ​​tool refines and optimizes the worker's schedule, taking into account the worker's emotional state and adjusting the optimal time slots and work order.

[0319] 5. Interface Methods

[0320] The system provides an intuitive user interface that allows users to easily change their work schedule with drag and drop, and displays recommended tasks based on their emotional state.

[0321] 6. Means of notification

[0322] It sends real-time notifications about saved schedules to workers via smartphones, tablets, head-mounted displays, etc.

[0323] 7. Control measures

[0324] Image data can be managed by linking it to past work. For example, photos taken during an inspection can be uploaded and saved by linking them to the date and time.

[0325] 8. Extraction means

[0326] It has a function to automatically extract schedules from message tools. For example, it can analyze content such as "I would like to carry out equipment inspection this weekend" in a message tool and automatically reflect it in the schedule.

[0327] Hardware and software examples

[0328] The system uses a smartphone or head-mounted display for voice input, the Google Speech Recognition API for sentiment analysis, SQLite for database management, and simple random selection or specialized libraries for sentiment analysis.

[0329] Examples and prompts

[0330] Example 1:

[0331] When a worker says, "I'd like to do maintenance next Monday at 10 a.m.", the system converts it into text and AI tools analyze their emotions: if they're feeling tired, they're suggested to do other relaxing tasks.

[0332] Example 2:

[0333] When workers upload photos of their inspections to the system, the photos are stored in the database along with the specified date and time, and the photos are displayed when the time is later referenced.

[0334] Example prompt sentence:

[0335] "Equipment maintenance next Saturday at 2pm"

[0336] "I'd like to set up the new device tomorrow at 3pm."

[0337] "Please inspect the warehouse in the morning."

[0338] This makes it possible to manage schedules within the factory efficiently and flexibly.

[0339] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0340] Step 1:

[0341] The user inputs a command by voice, for example, "Maintenance at 10:00 AM next Monday." The terminal receives this voice data and sends it to the next processing step.

[0342] Input: Audio data

[0343] Output: Raw audio data

[0344] Step 2:

[0345] The device sends the received voice data to a conversion means, which uses the Google Speech Recognition API to convert the voice data into text data. The converted text is then sent to the next processing step.

[0346] Input: Audio data

[0347] Output: Text data

[0348] Step 3:

[0349] The server receives the converted text data and stores it in a database using a storage method. This database uses SQLite. The stored data is used in the next processing step.

[0350] Input: Text data

[0351] Output: Text data stored in the database

[0352] Step 4:

[0353] The server passes the stored text data to an emotion engine, which analyzes the worker's emotional state. For example, it can determine stress, fatigue, or happiness from the tone and speed of the voice. The analysis results are used in the next processing step.

[0354] Input: Text data

[0355] Output: Emotion analysis results

[0356] Step 5:

[0357] The server receives the analysis results of the emotion engine and generates an optimal schedule using AI means. Taking into account the emotional state, it adjusts relaxing activities and optimal time slots. The generated schedule is stored in a database and sent to the interface means.

[0358] Input: Sentiment analysis results, existing schedule information in the database

[0359] Output: Optimized schedule

[0360] Step 6:

[0361] The server provides users with an intuitive interface, allowing them to change and check schedules using drag and drop. Changes are reflected in the database in real time.

[0362] Input: Optimized schedule, user input

[0363] Output: Schedule data that the user confirmed or edited

[0364] Step 7:

[0365] The server uses a notification means to send notifications about the saved schedule to the user in real time, and the notifications are displayed on devices such as smartphones and tablets.

[0366] Input: Schedules stored in the database

[0367] Output: Notification to user terminal

[0368] Step 8:

[0369] Specifically, the user uploads the inspection photos they have collected, and the device sends them to the server. The server uses a management method to link the photos with the inspection date and time and saves them in a database. When the user wants to refer to them later, the photos are displayed.

[0370] Input: Uploaded photo data, corresponding date and time

[0371] Output: A database entry with a photo and a date and time stamp.

[0372] Step 9:

[0373] Furthermore, when a user confirms or adds an appointment through a messaging tool such as LINE, the server automatically extracts the appointment from the message using an extraction method. The extraction results are stored in a database and used by the emotion engine and AI methods.

[0374] Input: Message data

[0375] Output: Extracted schedule data

[0376] Through these steps, the system can automatically generate schedules based on workers' voice input and provide an optimal working environment through emotion recognition.

[0377] Example prompt sentence:

[0378] "Equipment maintenance next Saturday at 2pm"

[0379] "I'd like to set up the new device tomorrow at 3pm."

[0380] "Please inspect the warehouse in the morning."

[0381] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0382] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0383] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0384] [Second embodiment]

[0385] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0386] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0387] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0388] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0389] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0390] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0391] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0392] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0393] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0394] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0395] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0396] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0397] System Overview

[0398] This invention is a system that provides a child-rearing support schedule app, and includes a voice input means, a conversion means, a storage means, an AI means, an interface means, a notification means, a management means, and an extraction means. This system allows users to easily create, manage, and adjust schedules, and also provides a function for managing past records and child growth.

[0399] Explanation of voice input and conversion methods

[0400] When a user uses the voice input means to create a new plan, he or she inputs, for example, "A picnic in the park at 2 PM next Saturday." The voice input means receives this voice data and transmits it to the conversion means. The conversion means converts the voice data into text data, generating the text data "A picnic in the park at 2 PM next Saturday."

[0401] Explanation of storage and AI methods

[0402] The text data generated by the conversion means is stored in a database using a storage means. The stored schedule data is automatically detailed by the AI ​​means. The AI ​​means analyzes overlaps with other schedules and optimal time slots, and proposes optimal schedules to the user.

[0403] Description of interface and notification methods

[0404] The interface means provides gamified screens and operation means to allow users to enjoyably organize their schedules. Users can intuitively operate their schedules by, for example, dragging and dropping schedule icons. Notifications about saved schedules are sent to the user's device using the notification means, allowing users to manage their schedules without missing important schedules.

[0405] Description of management and extraction methods

[0406] The management means provides a function to manage image data by linking it to past family schedules. For example, photos from a family trip can be uploaded and saved linked to the date of the trip. The extraction means also has a function to automatically extract schedules from messaging tools such as LINE and add them to the database. This allows for efficient schedule management by automatically registering schedules based on the contents of messages.

[0407] Specific examples

[0408] Example 1: Creating an appointment by voice input

[0409] The user can say, "Make a doctor's appointment next Friday at 3pm." The device sends this voice data to the server, which converts the speech into text and stores it in a database. AI tools check for conflicts with other appointments, generate an optimal schedule, and even send a notification to the device so the user can confirm their appointment.

[0410] Example 2: Photo management and schedule linking

[0411] A user uploads photos from their child's sports day. The device sends the photos to the server, which stores them in association with the date of the sports day. When the user later references the date of the sports day, related photos are displayed, allowing the user to easily reminisce about their memories.

[0412] Through these means and processes, the present invention allows users to more efficiently manage their family schedules and assist in raising their children in an enjoyable way.

[0413] The processing flow will be explained below.

[0414] Step 1:

[0415] The user starts the application and uses the voice input means to input plans by voice, such as "Picnic in the park at 2:00 p.m. next Saturday."

[0416] Step 2:

[0417] The device receives the audio data and temporarily stores it in local storage.

[0418] Step 3:

[0419] The device calls the speech recognition API and converts the received voice data into text data, generating the text data "Picnic in the park next Saturday at 2 PM."

[0420] Step 4:

[0421] The terminal transmits the generated text data to the server.

[0422] Step 5:

[0423] The server analyzes the received text data and stores it in the database as a new appointment.

[0424] Step 6:

[0425] The AI ​​tool retrieves saved schedule data, automatically checks whether it overlaps with other appointments, analyzes the optimal time slot, and adjusts the schedule accordingly.

[0426] Step 7:

[0427] The server generates optimized schedule data and sends a notification to the terminal.

[0428] Step 8:

[0429] The device receives the notification and displays to the user that a schedule has been registered for "picnic in the park next Saturday at 2:00 PM."

[0430] Step 9:

[0431] Users open the gamification interface on the app and drag and drop event icons to coordinate with other events.

[0432] Step 10:

[0433] The terminal transmits the changed schedule information to the server, and the saved schedule is updated.

[0434] Step 11:

[0435] The user opens the app's "Photo Management" menu to upload photos from a family trip.

[0436] Step 12:

[0437] The device sends the selected photo to the server, which stores the photo and links it to the corresponding event.

[0438] Step 13:

[0439] The saved photos and linked schedules are displayed to the user through the interface means of the terminal, allowing the user to view the photos and easily manage the growth record.

[0440] Step 14:

[0441] The server analyzes message data to automatically extract schedules from messaging tools such as LINE.

[0442] Step 15:

[0443] The server extracts the schedule from the parsed message and stores it as a new schedule in the database.

[0444] Step 16:

[0445] The new event will be saved and coordinated with other events using AI means, and the event will be notified to the device, where the user can see that a new event has been added.

[0446] Example 1

[0447] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0448] Conventional schedule management systems do not fully utilize voice input, automatic data analysis, or an intuitive user interface, requiring users to spend time and effort creating and managing schedules. Furthermore, comprehensive schedule management is difficult because it is not easy to manage image data linked to past family events or to extract schedules from messaging tools.

[0449] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0450] In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, a storage means for saving the converted text data in a data storage, an AI means for checking and optimizing schedule overlaps, an interface means for allowing a user to intuitively organize their schedule, a notification means for sending notifications about the saved schedule to the terminal, a management means for managing image data in association with past family schedules, and an extraction means for automatically extracting schedules from a message tool. This enables users to quickly create schedules through voice input, propose optimized schedules, easily manage image data linked to past events, and automatically extract schedules from a message tool.

[0451] "Voice input means" refers to a device or software that captures voice data spoken by a user.

[0452] The "conversion means" is software or hardware that converts received voice data into text data.

[0453] "Storage means" refers to software or hardware for recording the converted text data in data storage.

[0454] "AI tools" are artificial intelligence algorithms and models that check for overlaps and optimize schedules.

[0455] "Interface means" refers to software that provides a user interface that allows users to intuitively operate and manage their schedules.

[0456] The "notification means" is software or hardware for notifying the user terminal of information related to the saved schedule.

[0457] The "management means" is software or hardware for managing image data in association with the family's past schedules.

[0458] The "extraction means" is software or a function that automatically extracts schedules from message tools.

[0459] This invention is a system for providing a childcare support schedule app, designed to enable users to efficiently create, manage, and adjust schedules. The system includes a voice input unit, a conversion unit, a storage unit, an AI unit, an interface unit, a notification unit, a management unit, and an extraction unit.

[0460] When a user creates a schedule, they use a voice input means. For example, they might say, "Picnic in the park next Saturday at 2 PM." The voice input means captures this voice data and sends it from the device to the server. The server converts the voice data into text data using the Google Cloud Speech-to-Text API as a conversion means. The converted text data is stored in data storage on the server, for example, in MySQL.

[0461] The stored data is analyzed by AI tools, which use generative AI models like TensorFlow and PyTorch to analyze and optimize schedules and identify overlaps with other appointments. An optimized schedule is then generated and sent from the server to the device.

[0462] Users can check and manage their schedules through an interface on their device. The interface uses React and Vue.js to provide an intuitive interface, allowing users to easily adjust their schedules using drag and drop. Notifications based on saved schedules are sent to the user's device in real time using services such as Firebase Cloud Messaging.

[0463] Furthermore, the management tool can be used to manage image data by linking it to past schedules. When a user uploads a photo of their child's sports day, it can be linked to the date of the event and saved for easy reference later. The extraction tool also includes a function to automatically extract schedules from messaging tools such as LINE and add them to the database.

[0464] For example, if a user says, "Make a doctor's appointment next Friday at 3pm," the device will send this voice data to the server, which will convert the voice to text and store it in a database. AI tools will check for conflicts with other appointments, generate an optimal schedule, and send a notification to the device so the user can confirm the appointment.

[0465] Example prompt sentence:

[0466] "Make a doctor's appointment next Friday at 3pm."

[0467] This allows users to quickly create schedules using voice input, propose optimized schedules, manage image data linked to past events, and automatically extract schedules from messaging tools.

[0468] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0469] Step 1:

[0470] A user creates a schedule using voice input. The user launches the app on the device and says, "Picnic in the park next Saturday at 2 PM." The input is voice data, which is captured by the device's microphone. The device then sends this voice data to the server.

[0471] Step 2:

[0472] The server converts the received voice data into text data. The Google Cloud Speech-to-Text API is used as the conversion method to convert the voice data into text data. The input is voice data, and the output is text data such as "Picnic in the park next Saturday at 2 PM." The server generates the converted text data and passes it on to the next process.

[0473] Step 3:

[0474] The server saves the converted text data in data storage. A MySQL database is used as the storage method. The input is text data, which is saved along with the associated user ID and timestamp. The saved text data is "Picnic in the park next Saturday at 2pm" and is saved in the database.

[0475] Step 4:

[0476] The server uses AI tools to analyze the schedule data, check for overlaps with other appointments, and optimize them. TensorFlow or PyTorch is used as the AI ​​tool. The input is stored text data and existing schedule data, and the AI ​​model analyzes the data and generates optimal schedule proposals. The output is the optimized schedule proposals.

[0477] Step 5:

[0478] The server sends the optimized schedule to the terminal. The input is the optimized schedule proposal, and the output is the schedule information to be sent to the user terminal. As a result, the terminal uses the notification means to send a notification to the user saying, "A picnic in the park has been scheduled for next Saturday at 2 PM."

[0479] Step 6:

[0480] The user checks and manipulates the schedule using an interface on the terminal. The user can adjust the schedule by performing operations such as drag and drop on the interface. The input is the schedule information sent from the server, and the output is the schedule content checked or edited by the user. Specific operations include the user dragging icons and changing the time.

[0481] Step 7:

[0482] The user uploads a photo and associates it with a past event. The user selects "Sports Day Photos" from the device interface and uploads the photo. The device sends the photo file to the server, which associates it with the date of the sports day and stores it in a database. The input is the photo data, and the output is the associated date and event information.

[0483] Step 8:

[0484] The server automatically extracts schedules from messaging tools. As a means of extraction, it uses the API of messaging tools such as LINE to analyze schedule information from user messages. The input is message data, and the output is the extracted schedule information. The analyzed data is saved in data storage so that the user can view it later.

[0485] This allows users to quickly create schedules using voice input, efficiently manage images associated with past events, and schedules automatically extracted from message tools.

[0486] (Application example 1)

[0487] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0488] While conventional schedule management systems allow users to easily create schedules using voice input, they lack the ability to display content related to past events or to optimize schedules in real time. Furthermore, few systems support automatic schedule extraction from messaging tools, making efficient schedule management difficult. Furthermore, they lack an interface and gamification elements that allow users to organize their schedules in a fun way, making them difficult to use on a daily basis.

[0489] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0490] In this invention, the server includes a voice input device, a conversion device that converts received voice data into text data, a storage device that stores the converted text data in a database, an AI device that automatically details and coordinates family members' schedules, an interface device that uses gamification to help users organize their schedules in a fun way, a notification device that sends notifications about saved schedules, a content management device that automatically displays content linked to past events, an extraction device that automatically extracts schedules from a messaging tool, and a schedule optimization device that optimizes and notifies users of schedules. This allows users to efficiently manage their schedules and easily check related content and past events. Furthermore, real-time schedule optimization significantly improves everyday usability.

[0491] A "voice input device" is a device that allows a user to input instructions by voice.

[0492] A "conversion device" is a device that has the function of converting voice data into text data.

[0493] A "storage device" is a device for storing converted text data in a database.

[0494] An "AI device" is a device that uses artificial intelligence to automatically detail and optimally coordinate the schedules of family members.

[0495] An "interface device" is a device that uses gamification to provide an operating screen that allows users to organize their schedules in a fun way.

[0496] A "notification device" is a device that sends notifications about saved appointments to a user's terminal.

[0497] A "content management device" is a device that automatically displays and manages content linked to past events.

[0498] An "extraction device" is a device that has the function of automatically extracting schedules from a message tool.

[0499] A "schedule optimization device" is a device that analyzes existing schedules, adjusts them to the optimal form, and notifies the user.

[0500] System Overview

[0501] This invention relates to a child-rearing support system that includes a voice input device, a conversion device, a storage device, an AI device, an interface device, a notification device, a content management device, an extraction device, and a schedule optimization device. This system allows users to easily create schedules through voice input and efficiently manage and adjust them thereafter.

[0502] Hardware and software used

[0503] The server converts data from the voice input device into text data using a speech recognition API (e.g., Google Cloud Speech-to-Text). The storage device uses a database management system (e.g., MySQL). The AI ​​device uses a machine learning model (e.g., TensorFlow) to optimize the schedule. The interface device uses web technology (e.g., React.js) to provide a user interface incorporating gamification. The notification device uses the smartphone's push notification function (e.g., Firebase Cloud Messaging).

[0504] Natural language processing explanation

[0505] 1. The user speaks, "Picnic in the park next Saturday at 2 PM." The server receives this voice data via the voice input device.

[0506] 2. The received voice data is converted into text data using a conversion device.

[0507] 3. The converted text data is stored in a database using a storage device.

[0508] 4. The AI ​​device on the server analyzes the saved schedule data, checks for overlaps with other appointments, and generates the optimal schedule.

[0509] 5. Users can check their schedules through an interface device. The gamified screen allows users to enjoy drag-and-drop operations.

[0510] 6. When an appointment is approaching, users will be notified through the notification device, so they will not miss any important appointments.

[0511] 7. Using content management devices, photos and videos related to past events are automatically displayed. For example, when a user uploads a photo of a sports day, detailed information related to that sports day is displayed.

[0512] 8. The extraction device automatically extracts schedules from messaging tools such as LINE and stores them in a database, eliminating the need to manually enter message content.

[0513] 9. The schedule optimization device performs real-time schedule optimization and notifies the user of the optimal schedule.

[0514] Examples of specific examples and prompts

[0515] Examples:

[0516] The user can say, "Make a doctor's appointment next Friday at 3 p.m." The server converts the speech into text using a speech recognition API and stores it in a database. The AI ​​device then checks for overlapping appointments, generates an optimal schedule, and notifies the user via a smartphone push notification. Additionally, related doctor visit records and photos are displayed via the content management device.

[0517] Example prompt sentence:

[0518] "I'm going to the aquarium with my family on the weekend."

[0519] "Picnic next Friday afternoon"

[0520] "Kids' dance lessons on Saturday mornings at 9am"

[0521] This allows users to easily create and efficiently manage appointments through voice input, optimizing schedules, and automatically displaying records of past events for added convenience.

[0522] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0523] Step 1:

[0524] The user speaks "Picnic in the park next Saturday at 2 PM." Input: User's voice data. Output: Voice data.

[0525] Specific operation: The user speaks their schedule into the smartphone's voice input device. The voice data is received by the device.

[0526] Step 2:

[0527] The device uses a speech recognition API to convert voice data into text data. Input: Voice data. Output: Text data.

[0528] Specific operation: The device's voice recognition API (e.g., Google Cloud Speech-to-Text) analyzes the voice data and converts it into text format.

[0529] Step 3:

[0530] The converted text data is sent to the server and saved to the storage device. Input: Text data. Output: Saved data.

[0531] Specific operation: The terminal sends text data to the server, and the server stores the text data in a database management system (e.g., MySQL).

[0532] Step 4:

[0533] The AI ​​device analyzes the saved schedule data, checks for overlaps with other appointments, and generates an optimal schedule. Input: Saved data. Output: Optimized schedule.

[0534] How it works: The AI ​​device uses a machine learning model (e.g., TensorFlow) to analyze stored schedule data and generate an optimal schedule.

[0535] Step 5:

[0536] The interface device displays a screen that allows the user to intuitively operate the schedule. Input: Optimized schedule. Output: User interface.

[0537] Specific operation: The server uses web technologies (e.g. React.js) to generate an interface that allows users to edit appointments using drag and drop, and displays it on the device.

[0538] Step 6:

[0539] When an appointment is approaching, the notification device sends a notification to the user's device. Input: Optimized schedule. Output: Notification.

[0540] Specific operation: The server uses a push notification service (e.g., Firebase Cloud Messaging) to send a notification of the event to the user's device.

[0541] Step 7:

[0542] When a user uploads photos or videos linked to past events to a content management device, the related content is displayed. Input: Photo or video data. Output: Display of related content.

[0543] Specific operation: The user uploads photos and videos from their device to the server, which then uses a content management device to link them with related past events and store and display them in a database.

[0544] Step 8:

[0545] The extraction device automatically extracts schedule data from messaging tools such as LINE and stores the data in a database. Input: LINE messages. Output: Schedule data.

[0546] Specific operation: The server analyzes the message content using the API of the message tool and saves the extracted schedule data in a database.

[0547] Step 9:

[0548] The schedule optimization device optimizes the schedule in real time and notifies the user of the results. Input: Saved schedule data. Output: Optimized schedule notification.

[0549] Specific operation: The server uses machine learning models to perform real-time analysis, recalculate an optimized schedule, and send notifications to the user device.

[0550] The above are the specific processing steps of the system that realizes the application example.

[0551] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0552] System Overview

[0553] This invention is a system for providing a childcare support schedule app, and includes a voice input unit, a conversion unit, a storage unit, an AI unit, an interface unit, a notification unit, a management unit, an extraction unit, and an emotion engine. This system allows users to easily create, manage, and adjust schedules. It also provides a function for managing past records and child growth, and provides a more personalized experience by recognizing the user's emotions and responding accordingly.

[0554] Explanation of voice input and conversion methods

[0555] When a user uses the voice input means to create a new plan, the user inputs a voice such as "Picnic in the park at 2 PM next Saturday." The voice input means receives this voice data and transmits it to the conversion means. The conversion means converts the voice data into text data, generating the text data "Picnic in the park at 2 PM next Saturday."

[0556] Emotion Engine Explained

[0557] The emotion engine has the ability to analyze the emotions expressed when a user speaks and adjust the system's behavior based on those emotions. For example, if a user speaks of feeling stressed, the emotion engine will recognize this and suggest schedules and notifications that will help them relax. It also accumulates emotion data and suggests optimal schedules based on past emotion patterns.

[0558] Explanation of storage and AI methods

[0559] The text data generated by the conversion means is stored in a database using a storage means. The stored schedule data is automatically detailed by the AI ​​means. The AI ​​means analyzes overlaps with other schedules and optimal time slots to adjust the schedule.

[0560] Description of interface and notification methods

[0561] The interface means provides gamified screens and operation means to allow users to enjoyably organize their schedules. Users can intuitively operate their schedules, for example, by dragging and dropping schedule icons. Notifications about saved schedules are sent to the user's device using the notification means and are automatically adjusted to suit the emotion engine.

[0562] Description of management and extraction methods

[0563] The management means provides a function to manage image data by linking it to past family schedules. For example, photos from a family trip can be uploaded and saved linked to the date of the trip. The extraction means also has a function to automatically extract schedules from messaging tools such as LINE and add them to the database. This allows for efficient schedule management by automatically registering schedules based on the contents of messages.

[0564] Specific examples

[0565] Example 1: Voice-activated scheduling and emotion recognition

[0566] The user voice-inputs, "Make a doctor's appointment next Friday at 3 p.m." The device sends this voice data to the server, which converts the voice into text and stores it in a database. AI tools check for conflicts with other appointments and generate an optimal schedule. The emotion engine analyzes the voice data and, if the user is nervous, suggests relaxing appointments (e.g., an aromatherapy massage the next day).

[0567] Example 2: Photo management and schedule linking

[0568] A user uploads photos from their child's sports day. The device sends the photos to the server, which stores them in association with the date of the sports day. When the user later references the date of the sports day, related photos are displayed, allowing the user to easily reminisce about their memories.

[0569] Example 3: Extracting schedules from messages and reflecting emotional data

[0570] If a LINE message is sent saying, "I want to go see a movie this weekend," the server analyzes the message and automatically adds the event to the database. Furthermore, the emotion engine recognizes the user's emotions from the context of the message and sends encouraging notifications based on those emotions.

[0571] Through these means and processes, this invention allows users to manage their family schedules more efficiently and support child-rearing in a fun way. In addition, the introduction of an emotion engine provides a more personal and emotionally sensitive user experience.

[0572] The processing flow will be explained below.

[0573] Step 1:

[0574] The user starts the application and uses the voice input means to input plans by voice, such as "Picnic in the park at 2:00 p.m. next Saturday."

[0575] Step 2:

[0576] The device receives the audio data and temporarily stores it in local storage.

[0577] Step 3:

[0578] The terminal transmits the voice data to the server, and the voice data is converted into text data by the conversion means, generating text data such as "Picnic in the park at 2:00 PM next Saturday."

[0579] Step 4:

[0580] The terminal transmits the generated text data to the server.

[0581] Step 5:

[0582] The server analyzes the received text data and stores it in the database as a new appointment.

[0583] Step 6:

[0584] AI tools retrieve saved schedule data, automatically check whether it overlaps with other appointments, and adjust the schedule.

[0585] Step 7:

[0586] The emotion engine analyzes the voice input data and recognizes the user's emotions. For example, if it recognizes that the user is feeling stressed, it stores that emotion data.

[0587] Step 8:

[0588] The server generates optimized schedule data and determines appropriate notification content based on feedback from the emotion engine.

[0589] Step 9:

[0590] The server sends a notification to the device, and the user confirms that a plan to "have a picnic in the park next Saturday at 2 p.m." has been registered, and receives a notification message that takes emotion into consideration.

[0591] Step 10:

[0592] Users open the gamification interface on the app and drag and drop event icons to coordinate with other events.

[0593] Step 11:

[0594] The terminal transmits the changed schedule information to the server, and the saved schedule is updated.

[0595] Step 12:

[0596] The user opens the app's "Photo Management" menu to upload photos from a family trip.

[0597] Step 13:

[0598] The terminal transmits the selected photo to the server, which stores the photo and links it to the corresponding appointment.

[0599] Step 14:

[0600] The saved photos and linked schedules are displayed to the user through the interface means of the terminal, allowing the user to view the photos and easily manage the growth record.

[0601] Step 15:

[0602] The server analyzes message data to automatically extract schedules from messaging tools such as LINE.

[0603] Step 16:

[0604] The server extracts the schedule from the parsed message and stores it as a new schedule in the database.

[0605] Step 17:

[0606] The new event will be saved and coordinated with other events using AI means, and the event will be notified to the device, where the user can see that a new event has been added.

[0607] Step 18:

[0608] The emotion engine accumulates the user's emotional data and proposes optimal schedules based on past emotional patterns. During times when the user frequently feels stressed, it suggests relaxing activities.

[0609] Example 2

[0610] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0611] Conventional schedule management systems required users to manually input schedules, which resulted in issues with overlaps and time-consuming adjustments to optimal schedules. Furthermore, they only provided simple schedule management without taking into account the user's emotional state, preventing them from providing a personalized experience. Furthermore, the system did not automatically extract schedules from past schedule data or messaging tools, which often required time and effort from users.

[0612] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, an emotion engine for analyzing the user's emotions, and an AI means for adjusting the system's operation based on the analyzed emotion data. This allows the user to easily input plans by voice and enables optimal schedule management according to the user's emotions. In addition, past schedule data management and automatic extraction functions from message tools are also provided, realizing a more efficient and personalized experience.

[0613] The "voice input means" is a device or function that allows the user to input a schedule by voice.

[0614] The "conversion means" is a device or program for converting received voice data into text data.

[0615] "Storage means" refers to a device or function for storing the converted text data in a database.

[0616] An "emotion engine" is a device or program that analyzes the emotion expressed by the user when inputting voice and generates emotion data.

[0617] "AI means" means a device or program that adjusts the system's behavior and refines and optimizes schedules based on analyzed emotional data.

[0618] An "interface means" is a device or program that provides an operation screen and functions using gamification elements so that the user can organize their schedule in a fun way.

[0619] "Notification means" is a device or function for sending notifications regarding saved schedules to the user's terminal.

[0620] The "management means" is a device or function for managing image data in association with past family schedules.

[0621] The "extraction means" is a device or program for automatically extracting schedules from the message tool and adding them to the database.

[0622] System Overview

[0623] This invention is a schedule management system for supporting child-rearing, which includes a voice input means, a conversion means, a storage means, an emotion engine, an AI means, an interface means, a notification means, a management means, and an extraction means. This system allows users to easily create, manage, and adjust schedules, and also provides personalized suggestions based on emotion recognition.

[0624] Hardware and Software Configuration

[0625] Voice input method: Uses the microphone on a smartphone or tablet device. For software, a voice input API is used.

[0626] Conversion method: Converting voice data into text data using speech recognition software (e.g., Google Speech-to-Text API).

[0627] Storage method: Store text data and emotion data in a database (e.g., MySQL).

[0628] Emotion engine: Analyzes emotions from audio data using an emotion analysis library (e.g., IBM Watson Tone Analyzer).

[0629] AI methods: Use machine learning models (e.g., TensorFlow) to adjust and optimize schedules.

[0630] Interface solutions: Provide a user interface incorporating gamification elements, such as a UI component that allows dragging and dropping appointment icons.

[0631] Notification method: Uses an API with push notification functionality to send notifications to the user's device.

[0632] Management method: A photo storage service for managing image data and linking that data to a database.

[0633] Extraction method: Use the API of a messaging tool (e.g., LINE) to automatically extract schedules from messages.

[0634] System Operation

[0635] First, the user uses the device's voice input means to input their plans by voice. For example, they might say, "Picnic in the park next Saturday at 2 p.m." The device collects this voice data and sends it to a server via the Internet. The server uses voice recognition software to convert the voice data into text data. The converted text data is then stored in a database.

[0636] The server then uses an emotion engine to analyze the emotion of the voice data. The analysis results are stored as emotion data. For example, if the user is speaking with a happy expression, the emotion data is registered as "enjoyment."

[0637] Based on the stored text and emotion data, the server uses AI methods to optimize the schedule, for example by checking whether appointments overlap with other appointments and adjusting them if necessary, and then storing the adjusted schedule back in the database.

[0638] The interface means is designed to allow users to enjoyably organize their schedules, and allows intuitive operation. For example, users can easily change their schedules by dragging and dropping appointment icons.

[0639] Based on the saved schedule data, the notification means generates and sends notifications to the user's device. The notification content takes into account emotional data, providing a more personalized experience.

[0640] Finally, the management and extraction methods can also integrate past schedule data and information from messaging tools. For example, it is possible to extract content such as "I want to go see a movie this weekend" from a LINE message and automatically register it as a schedule.

[0641] Prompt Sentence Examples

[0642] When a user says, "Make an appointment to see the dentist tomorrow at 3 PM," the device sends this voice data to the server. The server generates text data saying, "Make an appointment to see the dentist tomorrow at 3 PM," and stores it in a database along with emotional data analyzed by the emotion engine. The AI ​​means uses this information to coordinate with other appointments and sends a notification to the device.

[0643] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0644] Step 1:

[0645] The user performs voice input. The user speaks to the terminal, "Picnic in the park next Saturday at 2 PM." This voice input is the input data. Voice data is generated as output.

[0646] Step 2:

[0647] The device sends the voice data to the server. The device collects the voice data and sends it to the server via an internet connection. This transfers the voice data to the server. The input is voice data, and the output is the transfer of voice data to the server.

[0648] Step 3:

[0649] The server converts the voice data into text data. The server uses speech recognition software (e.g., Google Speech-to-Text API) to analyze the transmitted voice data and generate text data such as "Picnic in the park next Saturday at 2 p.m." The input is voice data, and the output is text data.

[0650] Step 4:

[0651] The server analyzes emotions using an emotion engine. The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the emotions in the voice and generate emotion data. For example, if someone is talking about enjoying a picnic, the emotion data "enjoyed" is generated. The input is text data, and the output is emotion data.

[0652] Step 5:

[0653] The server stores the text data and emotion data in a database. The server uses a storage means to record the converted text data "Picnic in the park next Saturday at 2 PM" and emotion data "fun" in the database. The input is text data and emotion data, and the output is storage in the database.

[0654] Step 6:

[0655] The server generates a detailed schedule using AI. It uses a machine learning model (e.g., TensorFlow) to analyze whether the schedule overlaps with the user's other plans and proposes and adjusts the optimal schedule. For example, if the schedule overlaps with other plans, it suggests a different time slot. The input is text data and existing schedule data, and the output is the optimized schedule data.

[0656] Step 7:

[0657] The device notifies the user of the results. The device receives the notification data generated by the server and notifies the user through push notifications or interface means. For example, a notification such as "A picnic in the park has been scheduled for next Saturday at 2 PM" is sent. The input is the notification data, and the output is the notification sent to the user.

[0658] (Application example 2)

[0659] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0660] In today's industrial world, there is a demand for reducing the burden on factory workers and for efficient work schedule management. While conventional systems can manage schedules based on voice instructions, they have the problem of not optimizing schedules by taking into account the emotional state of workers. This can result in workers' stress and fatigue being ignored, which can lead to reduced productivity and a worsening work environment.

[0661] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0662] In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, a storage means for saving the converted text data in a database, an emotion engine for analyzing the emotions of workers and adjusting the operation of the system based on those emotions, an AI means for automatically refining and adjusting the schedule of workers, an interface means for intuitively operating the schedule, and a notification means for sending notifications regarding the saved schedule. This enables flexible and efficient schedule management that takes into account the emotional state of workers.

[0663] "Audio input means" refers to devices or techniques for receiving audio data.

[0664] "Conversion means" refers to a device or technology that converts received voice data into text data.

[0665] "Storage means" refers to the device or technology used to store the converted text data in a database.

[0666] An "emotion engine" refers to a device or technology that analyzes emotions from a worker's voice and adjusts the system's operation based on those emotions.

[0667] "AI means" refers to artificial intelligence technology that automatically details and adjusts worker schedules.

[0668] "Interface means" refers to the screens and means that allow users to intuitively operate their schedules.

[0669] "Notification Means" means any device or technology that sends notifications to a user regarding saved appointments.

[0670] "Management means" refers to devices and technologies for managing image data in association with past schedules.

[0671] "Extraction means" refers to a device or technology that automatically extracts schedules from a message tool.

[0672] This invention is a system that efficiently manages the work schedules of factory workers and improves their working environment by taking into account their emotional state. The system consists of the following components:

[0673] 1. Voice input and conversion methods

[0674] A worker inputs a command by voice, such as "Maintenance at 10:00 AM next Monday." This voice data is received through the voice input means and converted into text data using the conversion means. This converted text data is stored in a database by the storage means described below.

[0675] 2. Preservation means

[0676] The converted text data is stored in a database system such as SQLite. The database serves as the basis for schedule management and stores instructions and emotional data for each worker.

[0677] 3. Emotion Engine

[0678] The emotion engine analyzes emotions during voice input and determines states such as stress, fatigue, joy, etc. For example, if it determines that a worker is feeling stressed, the emotion engine will suggest relaxing tasks.

[0679] 4. AI means

[0680] Based on the stored text data, the AI ​​tool refines and optimizes the worker's schedule, taking into account the worker's emotional state and adjusting the optimal time slots and work order.

[0681] 5. Interface Methods

[0682] The system provides an intuitive user interface that allows users to easily change their work schedule with drag and drop, and displays recommended tasks based on their emotional state.

[0683] 6. Means of notification

[0684] It sends real-time notifications about saved schedules to workers via smartphones, tablets, head-mounted displays, etc.

[0685] 7. Control measures

[0686] Image data can be managed by linking it to past work. For example, photos taken during an inspection can be uploaded and saved by linking them to the date and time.

[0687] 8. Extraction means

[0688] It has a function to automatically extract schedules from message tools. For example, it can analyze content such as "I would like to carry out equipment inspection this weekend" in a message tool and automatically reflect it in the schedule.

[0689] Hardware and software examples

[0690] The system uses a smartphone or head-mounted display for voice input, the Google Speech Recognition API for sentiment analysis, SQLite for database management, and simple random selection or specialized libraries for sentiment analysis.

[0691] Examples and prompts

[0692] Example 1:

[0693] When a worker says, "I'd like to do maintenance next Monday at 10 a.m.", the system converts it into text and AI tools analyze their emotions: if they're feeling tired, they're suggested to do other relaxing tasks.

[0694] Example 2:

[0695] When workers upload photos of their inspections to the system, the photos are stored in the database along with the specified date and time, and the photos are displayed when the time is later referenced.

[0696] Example prompt sentence:

[0697] "Equipment maintenance next Saturday at 2pm"

[0698] "I'd like to set up the new device tomorrow at 3pm."

[0699] "Please inspect the warehouse in the morning."

[0700] This makes it possible to manage schedules within the factory efficiently and flexibly.

[0701] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0702] Step 1:

[0703] The user inputs a command by voice, for example, "Maintenance at 10:00 AM next Monday." The terminal receives this voice data and sends it to the next processing step.

[0704] Input: Audio data

[0705] Output: Raw audio data

[0706] Step 2:

[0707] The device sends the received voice data to a conversion means, which uses the Google Speech Recognition API to convert the voice data into text data. The converted text is then sent to the next processing step.

[0708] Input: Audio data

[0709] Output: Text data

[0710] Step 3:

[0711] The server receives the converted text data and stores it in a database using a storage method. This database uses SQLite. The stored data is used in the next processing step.

[0712] Input: Text data

[0713] Output: Text data stored in the database

[0714] Step 4:

[0715] The server passes the stored text data to an emotion engine, which analyzes the worker's emotional state. For example, it can determine stress, fatigue, or happiness from the tone and speed of the voice. The analysis results are used in the next processing step.

[0716] Input: Text data

[0717] Output: Emotion analysis results

[0718] Step 5:

[0719] The server receives the analysis results of the emotion engine and generates an optimal schedule using AI means. Taking into account the emotional state, it adjusts relaxing activities and optimal time slots. The generated schedule is stored in a database and sent to the interface means.

[0720] Input: Sentiment analysis results, existing schedule information in the database

[0721] Output: Optimized schedule

[0722] Step 6:

[0723] The server provides users with an intuitive interface, allowing them to change and check schedules using drag and drop. Changes are reflected in the database in real time.

[0724] Input: Optimized schedule, user input

[0725] Output: Schedule data that the user confirmed or edited

[0726] Step 7:

[0727] The server uses a notification means to send notifications about the saved schedule to the user in real time, and the notifications are displayed on devices such as smartphones and tablets.

[0728] Input: Schedules stored in the database

[0729] Output: Notification to user terminal

[0730] Step 8:

[0731] Specifically, the user uploads the inspection photos they have collected, and the device sends them to the server. The server uses a management method to link the photos with the inspection date and time and saves them in a database. When the user wants to refer to them later, the photos are displayed.

[0732] Input: Uploaded photo data, corresponding date and time

[0733] Output: A database entry with a photo and a date and time stamp.

[0734] Step 9:

[0735] Furthermore, when a user confirms or adds an appointment through a messaging tool such as LINE, the server automatically extracts the appointment from the message using an extraction method. The extraction results are stored in a database and used by the emotion engine and AI methods.

[0736] Input: Message data

[0737] Output: Extracted schedule data

[0738] Through these steps, the system can automatically generate schedules based on workers' voice input and provide an optimal working environment through emotion recognition.

[0739] Example prompt sentence:

[0740] "Equipment maintenance next Saturday at 2pm"

[0741] "I'd like to set up the new device tomorrow at 3pm."

[0742] "Please inspect the warehouse in the morning."

[0743] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0744] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0745] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0746] [Third embodiment]

[0747] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0748] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0749] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0750] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0751] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0752] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0753] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0754] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0755] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0756] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0757] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0758] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0759] System Overview

[0760] This invention is a system that provides a child-rearing support schedule app, and includes a voice input means, a conversion means, a storage means, an AI means, an interface means, a notification means, a management means, and an extraction means. This system allows users to easily create, manage, and adjust schedules, and also provides a function for managing past records and child growth.

[0761] Explanation of voice input and conversion methods

[0762] When a user uses the voice input means to create a new plan, he or she inputs, for example, "A picnic in the park at 2 PM next Saturday." The voice input means receives this voice data and transmits it to the conversion means. The conversion means converts the voice data into text data, generating the text data "A picnic in the park at 2 PM next Saturday."

[0763] Explanation of storage and AI methods

[0764] The text data generated by the conversion means is stored in a database using a storage means. The stored schedule data is automatically detailed by the AI ​​means. The AI ​​means analyzes overlaps with other schedules and optimal time slots, and proposes optimal schedules to the user.

[0765] Description of interface and notification methods

[0766] The interface means provides gamified screens and operation means to allow users to enjoyably organize their schedules. Users can intuitively operate their schedules by, for example, dragging and dropping schedule icons. Notifications about saved schedules are sent to the user's device using the notification means, allowing users to manage their schedules without missing important schedules.

[0767] Description of management and extraction methods

[0768] The management means provides a function to manage image data by linking it to past family schedules. For example, photos from a family trip can be uploaded and saved linked to the date of the trip. The extraction means also has a function to automatically extract schedules from messaging tools such as LINE and add them to the database. This allows for efficient schedule management by automatically registering schedules based on the contents of messages.

[0769] Specific examples

[0770] Example 1: Creating an appointment by voice input

[0771] The user can say, "Make a doctor's appointment next Friday at 3pm." The device sends this voice data to the server, which converts the speech into text and stores it in a database. AI tools check for conflicts with other appointments, generate an optimal schedule, and even send a notification to the device so the user can confirm their appointment.

[0772] Example 2: Photo management and schedule linking

[0773] A user uploads photos from their child's sports day. The device sends the photos to the server, which stores them in association with the date of the sports day. When the user later references the date of the sports day, related photos are displayed, allowing the user to easily reminisce about their memories.

[0774] Through these means and processes, the present invention allows users to more efficiently manage their family schedules and assist in raising their children in an enjoyable way.

[0775] The processing flow will be explained below.

[0776] Step 1:

[0777] The user starts the application and uses the voice input means to input plans by voice, such as "Picnic in the park at 2:00 p.m. next Saturday."

[0778] Step 2:

[0779] The device receives the audio data and temporarily stores it in local storage.

[0780] Step 3:

[0781] The device calls the speech recognition API and converts the received voice data into text data, generating the text data "Picnic in the park next Saturday at 2 PM."

[0782] Step 4:

[0783] The terminal transmits the generated text data to the server.

[0784] Step 5:

[0785] The server analyzes the received text data and stores it in the database as a new appointment.

[0786] Step 6:

[0787] The AI ​​tool retrieves saved schedule data, automatically checks whether it overlaps with other appointments, analyzes the optimal time slot, and adjusts the schedule accordingly.

[0788] Step 7:

[0789] The server generates optimized schedule data and sends a notification to the terminal.

[0790] Step 8:

[0791] The device receives the notification and displays to the user that a schedule has been registered for "picnic in the park next Saturday at 2:00 PM."

[0792] Step 9:

[0793] Users open the gamification interface on the app and drag and drop event icons to coordinate with other events.

[0794] Step 10:

[0795] The terminal transmits the changed schedule information to the server, and the saved schedule is updated.

[0796] Step 11:

[0797] The user opens the app's "Photo Management" menu to upload photos from a family trip.

[0798] Step 12:

[0799] The device sends the selected photo to the server, which stores the photo and links it to the corresponding event.

[0800] Step 13:

[0801] The saved photos and linked schedules are displayed to the user through the interface means of the terminal, allowing the user to view the photos and easily manage the growth record.

[0802] Step 14:

[0803] The server analyzes message data to automatically extract schedules from messaging tools such as LINE.

[0804] Step 15:

[0805] The server extracts the schedule from the parsed message and stores it as a new schedule in the database.

[0806] Step 16:

[0807] The new event will be saved and coordinated with other events using AI means, and the event will be notified to the device, where the user can see that a new event has been added.

[0808] Example 1

[0809] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0810] Conventional schedule management systems do not fully utilize voice input, automatic data analysis, or an intuitive user interface, requiring users to spend time and effort creating and managing schedules. Furthermore, comprehensive schedule management is difficult because it is not easy to manage image data linked to past family events or to extract schedules from messaging tools.

[0811] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0812] In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, a storage means for saving the converted text data in a data storage, an AI means for checking and optimizing schedule overlaps, an interface means for allowing a user to intuitively organize their schedule, a notification means for sending notifications about the saved schedule to the terminal, a management means for managing image data in association with past family schedules, and an extraction means for automatically extracting schedules from a message tool. This enables users to quickly create schedules through voice input, propose optimized schedules, easily manage image data linked to past events, and automatically extract schedules from a message tool.

[0813] "Voice input means" refers to a device or software that captures voice data spoken by a user.

[0814] The "conversion means" is software or hardware that converts received voice data into text data.

[0815] "Storage means" refers to software or hardware for recording the converted text data in data storage.

[0816] "AI tools" are artificial intelligence algorithms and models that check for overlaps and optimize schedules.

[0817] "Interface means" refers to software that provides a user interface that allows users to intuitively operate and manage their schedules.

[0818] The "notification means" is software or hardware for notifying the user terminal of information related to the saved schedule.

[0819] The "management means" is software or hardware for managing image data in association with the family's past schedules.

[0820] The "extraction means" is software or a function that automatically extracts schedules from message tools.

[0821] This invention is a system for providing a childcare support schedule app, designed to enable users to efficiently create, manage, and adjust schedules. The system includes a voice input unit, a conversion unit, a storage unit, an AI unit, an interface unit, a notification unit, a management unit, and an extraction unit.

[0822] When a user creates a schedule, they use a voice input means. For example, they might say, "Picnic in the park next Saturday at 2 PM." The voice input means captures this voice data and sends it from the device to the server. The server converts the voice data into text data using the Google Cloud Speech-to-Text API as a conversion means. The converted text data is stored in data storage on the server, for example, in MySQL.

[0823] The stored data is analyzed by AI tools, which use generative AI models like TensorFlow and PyTorch to analyze and optimize schedules and identify overlaps with other appointments. An optimized schedule is then generated and sent from the server to the device.

[0824] Users can check and manage their schedules through an interface on their device. The interface uses React and Vue.js to provide an intuitive interface, allowing users to easily adjust their schedules using drag and drop. Notifications based on saved schedules are sent to the user's device in real time using services such as Firebase Cloud Messaging.

[0825] Furthermore, the management tool can be used to manage image data by linking it to past schedules. When a user uploads a photo of their child's sports day, it can be linked to the date of the event and saved for easy reference later. The extraction tool also includes a function to automatically extract schedules from messaging tools such as LINE and add them to the database.

[0826] For example, if a user says, "Make a doctor's appointment next Friday at 3pm," the device will send this voice data to the server, which will convert the voice to text and store it in a database. AI tools will check for conflicts with other appointments, generate an optimal schedule, and send a notification to the device so the user can confirm the appointment.

[0827] Example prompt sentence:

[0828] "Make a doctor's appointment next Friday at 3pm."

[0829] This allows users to quickly create schedules using voice input, propose optimized schedules, manage image data linked to past events, and automatically extract schedules from messaging tools.

[0830] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0831] Step 1:

[0832] A user creates a schedule using voice input. The user launches the app on the device and says, "Picnic in the park next Saturday at 2 PM." The input is voice data, which is captured by the device's microphone. The device then sends this voice data to the server.

[0833] Step 2:

[0834] The server converts the received voice data into text data. The Google Cloud Speech-to-Text API is used as the conversion method to convert the voice data into text data. The input is voice data, and the output is text data such as "Picnic in the park next Saturday at 2 PM." The server generates the converted text data and passes it on to the next process.

[0835] Step 3:

[0836] The server saves the converted text data in data storage. A MySQL database is used as the storage method. The input is text data, which is saved along with the associated user ID and timestamp. The saved text data is "Picnic in the park next Saturday at 2pm" and is saved in the database.

[0837] Step 4:

[0838] The server uses AI tools to analyze the schedule data, check for overlaps with other appointments, and optimize them. TensorFlow or PyTorch is used as the AI ​​tool. The input is stored text data and existing schedule data, and the AI ​​model analyzes the data and generates optimal schedule proposals. The output is the optimized schedule proposals.

[0839] Step 5:

[0840] The server sends the optimized schedule to the terminal. The input is the optimized schedule proposal, and the output is the schedule information to be sent to the user terminal. As a result, the terminal uses the notification means to send a notification to the user saying, "A picnic in the park has been scheduled for next Saturday at 2 PM."

[0841] Step 6:

[0842] The user checks and manipulates the schedule using an interface on the terminal. The user can adjust the schedule by performing operations such as drag and drop on the interface. The input is the schedule information sent from the server, and the output is the schedule content checked or edited by the user. Specific operations include the user dragging icons and changing the time.

[0843] Step 7:

[0844] The user uploads a photo and associates it with a past event. The user selects "Sports Day Photos" from the device interface and uploads the photo. The device sends the photo file to the server, which associates it with the date of the sports day and stores it in a database. The input is the photo data, and the output is the associated date and event information.

[0845] Step 8:

[0846] The server automatically extracts schedules from messaging tools. As a means of extraction, it uses the API of messaging tools such as LINE to analyze schedule information from user messages. The input is message data, and the output is the extracted schedule information. The analyzed data is saved in data storage so that the user can view it later.

[0847] This allows users to quickly create schedules using voice input, efficiently manage images associated with past events, and schedules automatically extracted from message tools.

[0848] (Application example 1)

[0849] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0850] While conventional schedule management systems allow users to easily create schedules using voice input, they lack the ability to display content related to past events or to optimize schedules in real time. Furthermore, few systems support automatic schedule extraction from messaging tools, making efficient schedule management difficult. Furthermore, they lack an interface and gamification elements that allow users to organize their schedules in a fun way, making them difficult to use on a daily basis.

[0851] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0852] In this invention, the server includes a voice input device, a conversion device that converts received voice data into text data, a storage device that stores the converted text data in a database, an AI device that automatically details and coordinates family members' schedules, an interface device that uses gamification to help users organize their schedules in a fun way, a notification device that sends notifications about saved schedules, a content management device that automatically displays content linked to past events, an extraction device that automatically extracts schedules from a messaging tool, and a schedule optimization device that optimizes and notifies users of schedules. This allows users to efficiently manage their schedules and easily check related content and past events. Furthermore, real-time schedule optimization significantly improves everyday usability.

[0853] A "voice input device" is a device that allows a user to input instructions by voice.

[0854] A "conversion device" is a device that has the function of converting voice data into text data.

[0855] A "storage device" is a device for storing converted text data in a database.

[0856] An "AI device" is a device that uses artificial intelligence to automatically detail and optimally coordinate the schedules of family members.

[0857] An "interface device" is a device that uses gamification to provide an operating screen that allows users to organize their schedules in a fun way.

[0858] A "notification device" is a device that sends notifications about saved appointments to a user's terminal.

[0859] A "content management device" is a device that automatically displays and manages content linked to past events.

[0860] An "extraction device" is a device that has the function of automatically extracting schedules from a message tool.

[0861] A "schedule optimization device" is a device that analyzes existing schedules, adjusts them to the optimal form, and notifies the user.

[0862] System Overview

[0863] This invention relates to a child-rearing support system that includes a voice input device, a conversion device, a storage device, an AI device, an interface device, a notification device, a content management device, an extraction device, and a schedule optimization device. This system allows users to easily create schedules through voice input and efficiently manage and adjust them thereafter.

[0864] Hardware and software used

[0865] The server converts data from the voice input device into text data using a speech recognition API (e.g., Google Cloud Speech-to-Text). The storage device uses a database management system (e.g., MySQL). The AI ​​device uses a machine learning model (e.g., TensorFlow) to optimize the schedule. The interface device uses web technology (e.g., React.js) to provide a user interface incorporating gamification. The notification device uses the smartphone's push notification function (e.g., Firebase Cloud Messaging).

[0866] Natural language processing explanation

[0867] 1. The user speaks, "Picnic in the park next Saturday at 2 PM." The server receives this voice data via the voice input device.

[0868] 2. The received voice data is converted into text data using a conversion device.

[0869] 3. The converted text data is stored in a database using a storage device.

[0870] 4. The AI ​​device on the server analyzes the saved schedule data, checks for overlaps with other appointments, and generates the optimal schedule.

[0871] 5. Users can check their schedules through an interface device. The gamified screen allows users to enjoy drag-and-drop operations.

[0872] 6. When an appointment is approaching, users will be notified through the notification device, so they will not miss any important appointments.

[0873] 7. Using content management devices, photos and videos related to past events are automatically displayed. For example, when a user uploads a photo of a sports day, detailed information related to that sports day is displayed.

[0874] 8. The extraction device automatically extracts schedules from messaging tools such as LINE and stores them in a database, eliminating the need to manually enter message content.

[0875] 9. The schedule optimization device performs real-time schedule optimization and notifies the user of the optimal schedule.

[0876] Examples of specific examples and prompts

[0877] Examples:

[0878] The user can say, "Make a doctor's appointment next Friday at 3 p.m." The server converts the speech into text using a speech recognition API and stores it in a database. The AI ​​device then checks for overlapping appointments, generates an optimal schedule, and notifies the user via a smartphone push notification. Additionally, related doctor visit records and photos are displayed via the content management device.

[0879] Example prompt sentence:

[0880] "I'm going to the aquarium with my family on the weekend."

[0881] "Picnic next Friday afternoon"

[0882] "Kids' dance lessons on Saturday mornings at 9am"

[0883] This allows users to easily create and efficiently manage appointments through voice input, optimizing schedules, and automatically displaying records of past events for added convenience.

[0884] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0885] Step 1:

[0886] The user speaks "Picnic in the park next Saturday at 2 PM." Input: User's voice data. Output: Voice data.

[0887] Specific operation: The user speaks their schedule into the smartphone's voice input device. The voice data is received by the device.

[0888] Step 2:

[0889] The device uses a speech recognition API to convert voice data into text data. Input: Voice data. Output: Text data.

[0890] Specific operation: The device's voice recognition API (e.g., Google Cloud Speech-to-Text) analyzes the voice data and converts it into text format.

[0891] Step 3:

[0892] The converted text data is sent to the server and saved to the storage device. Input: Text data. Output: Saved data.

[0893] Specific operation: The terminal sends text data to the server, and the server stores the text data in a database management system (e.g., MySQL).

[0894] Step 4:

[0895] The AI ​​device analyzes the saved schedule data, checks for overlaps with other appointments, and generates an optimal schedule. Input: Saved data. Output: Optimized schedule.

[0896] How it works: The AI ​​device uses a machine learning model (e.g., TensorFlow) to analyze stored schedule data and generate an optimal schedule.

[0897] Step 5:

[0898] The interface device displays a screen that allows the user to intuitively operate the schedule. Input: Optimized schedule. Output: User interface.

[0899] Specific operation: The server uses web technologies (e.g. React.js) to generate an interface that allows users to edit appointments using drag and drop, and displays it on the device.

[0900] Step 6:

[0901] When an appointment is approaching, the notification device sends a notification to the user's device. Input: Optimized schedule. Output: Notification.

[0902] Specific operation: The server uses a push notification service (e.g., Firebase Cloud Messaging) to send a notification of the event to the user's device.

[0903] Step 7:

[0904] When a user uploads photos or videos linked to past events to a content management device, the related content is displayed. Input: Photo or video data. Output: Display of related content.

[0905] Specific operation: The user uploads photos and videos from their device to the server, which then uses a content management device to link them with related past events and store and display them in a database.

[0906] Step 8:

[0907] The extraction device automatically extracts schedule data from messaging tools such as LINE and stores the data in a database. Input: LINE messages. Output: Schedule data.

[0908] Specific operation: The server analyzes the message content using the API of the message tool and saves the extracted schedule data in a database.

[0909] Step 9:

[0910] The schedule optimization device optimizes the schedule in real time and notifies the user of the results. Input: Saved schedule data. Output: Optimized schedule notification.

[0911] Specific operation: The server uses machine learning models to perform real-time analysis, recalculate an optimized schedule, and send notifications to the user device.

[0912] The above are the specific processing steps of the system that realizes the application example.

[0913] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0914] System Overview

[0915] This invention is a system for providing a childcare support schedule app, and includes a voice input unit, a conversion unit, a storage unit, an AI unit, an interface unit, a notification unit, a management unit, an extraction unit, and an emotion engine. This system allows users to easily create, manage, and adjust schedules. It also provides a function for managing past records and child growth, and provides a more personalized experience by recognizing the user's emotions and responding accordingly.

[0916] Explanation of voice input and conversion methods

[0917] When a user uses the voice input means to create a new plan, the user inputs a voice such as "Picnic in the park at 2 PM next Saturday." The voice input means receives this voice data and transmits it to the conversion means. The conversion means converts the voice data into text data, generating the text data "Picnic in the park at 2 PM next Saturday."

[0918] Emotion Engine Explained

[0919] The emotion engine has the ability to analyze the emotions expressed when a user speaks and adjust the system's behavior based on those emotions. For example, if a user speaks of feeling stressed, the emotion engine will recognize this and suggest schedules and notifications that will help them relax. It also accumulates emotion data and suggests optimal schedules based on past emotion patterns.

[0920] Explanation of storage and AI methods

[0921] The text data generated by the conversion means is stored in a database using a storage means. The stored schedule data is automatically detailed by the AI ​​means. The AI ​​means analyzes overlaps with other schedules and optimal time slots to adjust the schedule.

[0922] Description of interface and notification methods

[0923] The interface means provides gamified screens and operation means to allow users to enjoyably organize their schedules. Users can intuitively operate their schedules, for example, by dragging and dropping schedule icons. Notifications about saved schedules are sent to the user's device using the notification means and are automatically adjusted to suit the emotion engine.

[0924] Description of management and extraction methods

[0925] The management means provides a function to manage image data by linking it to past family schedules. For example, photos from a family trip can be uploaded and saved linked to the date of the trip. The extraction means also has a function to automatically extract schedules from messaging tools such as LINE and add them to the database. This allows for efficient schedule management by automatically registering schedules based on the contents of messages.

[0926] Specific examples

[0927] Example 1: Voice-activated scheduling and emotion recognition

[0928] The user voice-inputs, "Make a doctor's appointment next Friday at 3 p.m." The device sends this voice data to the server, which converts the voice into text and stores it in a database. AI tools check for conflicts with other appointments and generate an optimal schedule. The emotion engine analyzes the voice data and, if the user is nervous, suggests relaxing appointments (e.g., an aromatherapy massage the next day).

[0929] Example 2: Photo management and schedule linking

[0930] A user uploads photos from their child's sports day. The device sends the photos to the server, which stores them in association with the date of the sports day. When the user later references the date of the sports day, related photos are displayed, allowing the user to easily reminisce about their memories.

[0931] Example 3: Extracting schedules from messages and reflecting emotional data

[0932] If a LINE message is sent saying, "I want to go see a movie this weekend," the server analyzes the message and automatically adds the event to the database. Furthermore, the emotion engine recognizes the user's emotions from the context of the message and sends encouraging notifications based on those emotions.

[0933] Through these means and processes, this invention allows users to manage their family schedules more efficiently and support child-rearing in a fun way. In addition, the introduction of an emotion engine provides a more personal and emotionally sensitive user experience.

[0934] The processing flow will be explained below.

[0935] Step 1:

[0936] The user starts the application and uses the voice input means to input plans by voice, such as "Picnic in the park at 2:00 p.m. next Saturday."

[0937] Step 2:

[0938] The device receives the audio data and temporarily stores it in local storage.

[0939] Step 3:

[0940] The terminal transmits the voice data to the server, and the voice data is converted into text data by the conversion means, generating text data such as "Picnic in the park at 2:00 PM next Saturday."

[0941] Step 4:

[0942] The terminal transmits the generated text data to the server.

[0943] Step 5:

[0944] The server analyzes the received text data and stores it in the database as a new appointment.

[0945] Step 6:

[0946] AI tools retrieve saved schedule data, automatically check whether it overlaps with other appointments, and adjust the schedule.

[0947] Step 7:

[0948] The emotion engine analyzes the voice input data and recognizes the user's emotions. For example, if it recognizes that the user is feeling stressed, it stores that emotion data.

[0949] Step 8:

[0950] The server generates optimized schedule data and determines appropriate notification content based on feedback from the emotion engine.

[0951] Step 9:

[0952] The server sends a notification to the device, and the user confirms that a plan to "have a picnic in the park next Saturday at 2 p.m." has been registered, and receives a notification message that takes emotion into consideration.

[0953] Step 10:

[0954] Users open the gamification interface on the app and drag and drop event icons to coordinate with other events.

[0955] Step 11:

[0956] The terminal transmits the changed schedule information to the server, and the saved schedule is updated.

[0957] Step 12:

[0958] The user opens the app's "Photo Management" menu to upload photos from a family trip.

[0959] Step 13:

[0960] The terminal transmits the selected photo to the server, which stores the photo and links it to the corresponding appointment.

[0961] Step 14:

[0962] The saved photos and linked schedules are displayed to the user through the interface means of the terminal, allowing the user to view the photos and easily manage the growth record.

[0963] Step 15:

[0964] The server analyzes message data to automatically extract schedules from messaging tools such as LINE.

[0965] Step 16:

[0966] The server extracts the schedule from the parsed message and stores it as a new schedule in the database.

[0967] Step 17:

[0968] The new event will be saved and coordinated with other events using AI means, and the event will be notified to the device, where the user can see that a new event has been added.

[0969] Step 18:

[0970] The emotion engine accumulates the user's emotional data and proposes optimal schedules based on past emotional patterns. During times when the user frequently feels stressed, it suggests relaxing activities.

[0971] Example 2

[0972] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0973] Conventional schedule management systems required users to manually input schedules, which resulted in issues with overlaps and time-consuming adjustments to optimal schedules. Furthermore, they only provided simple schedule management without taking into account the user's emotional state, preventing them from providing a personalized experience. Furthermore, the system did not automatically extract schedules from past schedule data or messaging tools, which often required time and effort from users.

[0974] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, an emotion engine for analyzing the user's emotions, and an AI means for adjusting the system's operation based on the analyzed emotion data. This allows the user to easily input plans by voice and enables optimal schedule management according to the user's emotions. In addition, past schedule data management and automatic extraction functions from message tools are also provided, realizing a more efficient and personalized experience.

[0975] The "voice input means" is a device or function that allows the user to input a schedule by voice.

[0976] The "conversion means" is a device or program for converting received voice data into text data.

[0977] "Storage means" refers to a device or function for storing the converted text data in a database.

[0978] An "emotion engine" is a device or program that analyzes the emotion expressed by the user when inputting voice and generates emotion data.

[0979] "AI means" means a device or program that adjusts the system's behavior and refines and optimizes schedules based on analyzed emotional data.

[0980] An "interface means" is a device or program that provides an operation screen and functions using gamification elements so that the user can organize their schedule in a fun way.

[0981] "Notification means" is a device or function for sending notifications regarding saved schedules to the user's terminal.

[0982] The "management means" is a device or function for managing image data in association with past family schedules.

[0983] The "extraction means" is a device or program for automatically extracting schedules from the message tool and adding them to the database.

[0984] System Overview

[0985] This invention is a schedule management system for supporting child-rearing, which includes a voice input means, a conversion means, a storage means, an emotion engine, an AI means, an interface means, a notification means, a management means, and an extraction means. This system allows users to easily create, manage, and adjust schedules, and also provides personalized suggestions based on emotion recognition.

[0986] Hardware and Software Configuration

[0987] Voice input method: Uses the microphone on a smartphone or tablet device. For software, a voice input API is used.

[0988] Conversion method: Converting voice data into text data using speech recognition software (e.g., Google Speech-to-Text API).

[0989] Storage method: Store text data and emotion data in a database (e.g., MySQL).

[0990] Emotion engine: Analyzes emotions from audio data using an emotion analysis library (e.g., IBM Watson Tone Analyzer).

[0991] AI methods: Use machine learning models (e.g., TensorFlow) to adjust and optimize schedules.

[0992] Interface solutions: Provide a user interface incorporating gamification elements, such as a UI component that allows dragging and dropping appointment icons.

[0993] Notification method: Uses an API with push notification functionality to send notifications to the user's device.

[0994] Management method: A photo storage service for managing image data and linking that data to a database.

[0995] Extraction method: Use the API of a messaging tool (e.g., LINE) to automatically extract schedules from messages.

[0996] System Operation

[0997] First, the user uses the device's voice input means to input their plans by voice. For example, they might say, "Picnic in the park next Saturday at 2 p.m." The device collects this voice data and sends it to a server via the Internet. The server uses voice recognition software to convert the voice data into text data. The converted text data is then stored in a database.

[0998] The server then uses an emotion engine to analyze the emotion of the voice data. The analysis results are stored as emotion data. For example, if the user is speaking with a happy expression, the emotion data is registered as "enjoyment."

[0999] Based on the stored text and emotion data, the server uses AI methods to optimize the schedule, for example by checking whether appointments overlap with other appointments and adjusting them if necessary, and then storing the adjusted schedule back in the database.

[1000] The interface means is designed to allow users to enjoyably organize their schedules, and allows intuitive operation. For example, users can easily change their schedules by dragging and dropping appointment icons.

[1001] Based on the saved schedule data, the notification means generates and sends notifications to the user's device. The notification content takes into account emotional data, providing a more personalized experience.

[1002] Finally, the management and extraction methods can also integrate past schedule data and information from messaging tools. For example, it is possible to extract content such as "I want to go see a movie this weekend" from a LINE message and automatically register it as a schedule.

[1003] Prompt Sentence Examples

[1004] When a user says, "Make an appointment to see the dentist tomorrow at 3 PM," the device sends this voice data to the server. The server generates text data saying, "Make an appointment to see the dentist tomorrow at 3 PM," and stores it in a database along with emotional data analyzed by the emotion engine. The AI ​​means uses this information to coordinate with other appointments and sends a notification to the device.

[1005] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1006] Step 1:

[1007] The user performs voice input. The user speaks to the terminal, "Picnic in the park next Saturday at 2 PM." This voice input is the input data. Voice data is generated as output.

[1008] Step 2:

[1009] The device sends the voice data to the server. The device collects the voice data and sends it to the server via an internet connection. This transfers the voice data to the server. The input is voice data, and the output is the transfer of voice data to the server.

[1010] Step 3:

[1011] The server converts the voice data into text data. The server uses speech recognition software (e.g., Google Speech-to-Text API) to analyze the transmitted voice data and generate text data such as "Picnic in the park next Saturday at 2 p.m." The input is voice data, and the output is text data.

[1012] Step 4:

[1013] The server analyzes emotions using an emotion engine. The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the emotions in the voice and generate emotion data. For example, if someone is talking about enjoying a picnic, the emotion data "enjoyed" is generated. The input is text data, and the output is emotion data.

[1014] Step 5:

[1015] The server stores the text data and emotion data in a database. The server uses a storage means to record the converted text data "Picnic in the park next Saturday at 2 PM" and emotion data "fun" in the database. The input is text data and emotion data, and the output is storage in the database.

[1016] Step 6:

[1017] The server generates a detailed schedule using AI. It uses a machine learning model (e.g., TensorFlow) to analyze whether the schedule overlaps with the user's other plans and proposes and adjusts the optimal schedule. For example, if the schedule overlaps with other plans, it suggests a different time slot. The input is text data and existing schedule data, and the output is the optimized schedule data.

[1018] Step 7:

[1019] The device notifies the user of the results. The device receives the notification data generated by the server and notifies the user through push notifications or interface means. For example, a notification such as "A picnic in the park has been scheduled for next Saturday at 2 PM" is sent. The input is the notification data, and the output is the notification sent to the user.

[1020] (Application example 2)

[1021] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1022] In today's industrial world, there is a demand for reducing the burden on factory workers and for efficient work schedule management. While conventional systems can manage schedules based on voice instructions, they have the problem of not optimizing schedules by taking into account the emotional state of workers. This can result in workers' stress and fatigue being ignored, which can lead to reduced productivity and a worsening work environment.

[1023] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1024] In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, a storage means for saving the converted text data in a database, an emotion engine for analyzing the emotions of workers and adjusting the operation of the system based on those emotions, an AI means for automatically refining and adjusting the schedule of workers, an interface means for intuitively operating the schedule, and a notification means for sending notifications regarding the saved schedule. This enables flexible and efficient schedule management that takes into account the emotional state of workers.

[1025] "Audio input means" refers to devices or techniques for receiving audio data.

[1026] "Conversion means" refers to a device or technology that converts received voice data into text data.

[1027] "Storage means" refers to the device or technology used to store the converted text data in a database.

[1028] An "emotion engine" refers to a device or technology that analyzes emotions from a worker's voice and adjusts the system's operation based on those emotions.

[1029] "AI means" refers to artificial intelligence technology that automatically details and adjusts worker schedules.

[1030] "Interface means" refers to the screens and means that allow users to intuitively operate their schedules.

[1031] "Notification Means" means any device or technology that sends notifications to a user regarding saved appointments.

[1032] "Management means" refers to devices and technologies for managing image data in association with past schedules.

[1033] "Extraction means" refers to a device or technology that automatically extracts schedules from a message tool.

[1034] This invention is a system that efficiently manages the work schedules of factory workers and improves their working environment by taking into account their emotional state. The system consists of the following components:

[1035] 1. Voice input and conversion methods

[1036] A worker inputs a command by voice, such as "Maintenance at 10:00 AM next Monday." This voice data is received through the voice input means and converted into text data using the conversion means. This converted text data is stored in a database by the storage means described below.

[1037] 2. Preservation means

[1038] The converted text data is stored in a database system such as SQLite. The database serves as the basis for schedule management and stores instructions and emotional data for each worker.

[1039] 3. Emotion Engine

[1040] The emotion engine analyzes emotions during voice input and determines states such as stress, fatigue, joy, etc. For example, if it determines that a worker is feeling stressed, the emotion engine will suggest relaxing tasks.

[1041] 4. AI means

[1042] Based on the stored text data, the AI ​​tool refines and optimizes the worker's schedule, taking into account the worker's emotional state and adjusting the optimal time slots and work order.

[1043] 5. Interface Methods

[1044] The system provides an intuitive user interface that allows users to easily change their work schedule with drag and drop, and displays recommended tasks based on their emotional state.

[1045] 6. Means of notification

[1046] It sends real-time notifications about saved schedules to workers via smartphones, tablets, head-mounted displays, etc.

[1047] 7. Control measures

[1048] Image data can be managed by linking it to past work. For example, photos taken during an inspection can be uploaded and saved by linking them to the date and time.

[1049] 8. Extraction means

[1050] It has a function to automatically extract schedules from message tools. For example, it can analyze content such as "I would like to carry out equipment inspection this weekend" in a message tool and automatically reflect it in the schedule.

[1051] Hardware and software examples

[1052] The system uses a smartphone or head-mounted display for voice input, the Google Speech Recognition API for sentiment analysis, SQLite for database management, and simple random selection or specialized libraries for sentiment analysis.

[1053] Examples and prompts

[1054] Example 1:

[1055] When a worker says, "I'd like to do maintenance next Monday at 10 a.m.", the system converts it into text and AI tools analyze their emotions: if they're feeling tired, they're suggested to do other relaxing tasks.

[1056] Example 2:

[1057] When workers upload photos of their inspections to the system, the photos are stored in the database along with the specified date and time, and the photos are displayed when the time is later referenced.

[1058] Example prompt sentence:

[1059] "Equipment maintenance next Saturday at 2pm"

[1060] "I'd like to set up the new device tomorrow at 3pm."

[1061] "Please inspect the warehouse in the morning."

[1062] This makes it possible to manage schedules within the factory efficiently and flexibly.

[1063] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1064] Step 1:

[1065] The user inputs a command by voice, for example, "Maintenance at 10:00 AM next Monday." The terminal receives this voice data and sends it to the next processing step.

[1066] Input: Audio data

[1067] Output: Raw audio data

[1068] Step 2:

[1069] The device sends the received voice data to a conversion means, which uses the Google Speech Recognition API to convert the voice data into text data. The converted text is then sent to the next processing step.

[1070] Input: Audio data

[1071] Output: Text data

[1072] Step 3:

[1073] The server receives the converted text data and stores it in a database using a storage method. This database uses SQLite. The stored data is used in the next processing step.

[1074] Input: Text data

[1075] Output: Text data stored in the database

[1076] Step 4:

[1077] The server passes the stored text data to an emotion engine, which analyzes the worker's emotional state. For example, it can determine stress, fatigue, or happiness from the tone and speed of the voice. The analysis results are used in the next processing step.

[1078] Input: Text data

[1079] Output: Emotion analysis results

[1080] Step 5:

[1081] The server receives the analysis results of the emotion engine and generates an optimal schedule using AI means. Taking into account the emotional state, it adjusts relaxing activities and optimal time slots. The generated schedule is stored in a database and sent to the interface means.

[1082] Input: Sentiment analysis results, existing schedule information in the database

[1083] Output: Optimized schedule

[1084] Step 6:

[1085] The server provides users with an intuitive interface, allowing them to change and check schedules using drag and drop. Changes are reflected in the database in real time.

[1086] Input: Optimized schedule, user input

[1087] Output: Schedule data that the user confirmed or edited

[1088] Step 7:

[1089] The server uses a notification means to send notifications about the saved schedule to the user in real time, and the notifications are displayed on devices such as smartphones and tablets.

[1090] Input: Schedules stored in the database

[1091] Output: Notification to user terminal

[1092] Step 8:

[1093] Specifically, the user uploads the inspection photos they have collected, and the device sends them to the server. The server uses a management method to link the photos with the inspection date and time and saves them in a database. When the user wants to refer to them later, the photos are displayed.

[1094] Input: Uploaded photo data, corresponding date and time

[1095] Output: A database entry with a photo and a date and time stamp.

[1096] Step 9:

[1097] Furthermore, when a user confirms or adds an appointment through a messaging tool such as LINE, the server automatically extracts the appointment from the message using an extraction method. The extraction results are stored in a database and used by the emotion engine and AI methods.

[1098] Input: Message data

[1099] Output: Extracted schedule data

[1100] Through these steps, the system can automatically generate schedules based on workers' voice input and provide an optimal working environment through emotion recognition.

[1101] Example prompt sentence:

[1102] "Equipment maintenance next Saturday at 2pm"

[1103] "I'd like to set up the new device tomorrow at 3pm."

[1104] "Please inspect the warehouse in the morning."

[1105] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1106] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1107] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1108] [Fourth embodiment]

[1109] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1110] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1111] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1112] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1113] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1114] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1115] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1116] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1117] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1118] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1119] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1120] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1121] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1122] System Overview

[1123] This invention is a system that provides a child-rearing support schedule app, and includes a voice input means, a conversion means, a storage means, an AI means, an interface means, a notification means, a management means, and an extraction means. This system allows users to easily create, manage, and adjust schedules, and also provides a function for managing past records and child growth.

[1124] Explanation of voice input and conversion methods

[1125] When a user uses the voice input means to create a new plan, he or she inputs, for example, "A picnic in the park at 2 PM next Saturday." The voice input means receives this voice data and transmits it to the conversion means. The conversion means converts the voice data into text data, generating the text data "A picnic in the park at 2 PM next Saturday."

[1126] Explanation of storage and AI methods

[1127] The text data generated by the conversion means is stored in a database using a storage means. The stored schedule data is automatically detailed by the AI ​​means. The AI ​​means analyzes overlaps with other schedules and optimal time slots, and proposes optimal schedules to the user.

[1128] Description of interface and notification methods

[1129] The interface means provides gamified screens and operation means to allow users to enjoyably organize their schedules. Users can intuitively operate their schedules by, for example, dragging and dropping schedule icons. Notifications about saved schedules are sent to the user's device using the notification means, allowing users to manage their schedules without missing important schedules.

[1130] Description of management and extraction methods

[1131] The management means provides a function to manage image data by linking it to past family schedules. For example, photos from a family trip can be uploaded and saved linked to the date of the trip. The extraction means also has a function to automatically extract schedules from messaging tools such as LINE and add them to the database. This allows for efficient schedule management by automatically registering schedules based on the contents of messages.

[1132] Specific examples

[1133] Example 1: Creating an appointment by voice input

[1134] The user can say, "Make a doctor's appointment next Friday at 3pm." The device sends this voice data to the server, which converts the speech into text and stores it in a database. AI tools check for conflicts with other appointments, generate an optimal schedule, and even send a notification to the device so the user can confirm their appointment.

[1135] Example 2: Photo management and schedule linking

[1136] A user uploads photos from their child's sports day. The device sends the photos to the server, which stores them in association with the date of the sports day. When the user later references the date of the sports day, related photos are displayed, allowing the user to easily reminisce about their memories.

[1137] Through these means and processes, the present invention allows users to more efficiently manage their family schedules and assist in raising their children in an enjoyable way.

[1138] The processing flow will be explained below.

[1139] Step 1:

[1140] The user starts the application and uses the voice input means to input plans by voice, such as "Picnic in the park at 2:00 p.m. next Saturday."

[1141] Step 2:

[1142] The device receives the audio data and temporarily stores it in local storage.

[1143] Step 3:

[1144] The device calls the speech recognition API and converts the received voice data into text data, generating the text data "Picnic in the park next Saturday at 2 PM."

[1145] Step 4:

[1146] The terminal transmits the generated text data to the server.

[1147] Step 5:

[1148] The server analyzes the received text data and stores it in the database as a new appointment.

[1149] Step 6:

[1150] The AI ​​tool retrieves saved schedule data, automatically checks whether it overlaps with other appointments, analyzes the optimal time slot, and adjusts the schedule accordingly.

[1151] Step 7:

[1152] The server generates optimized schedule data and sends a notification to the terminal.

[1153] Step 8:

[1154] The device receives the notification and displays to the user that a schedule has been registered for "picnic in the park next Saturday at 2:00 PM."

[1155] Step 9:

[1156] Users open the gamification interface on the app and drag and drop event icons to coordinate with other events.

[1157] Step 10:

[1158] The terminal transmits the changed schedule information to the server, and the saved schedule is updated.

[1159] Step 11:

[1160] The user opens the app's "Photo Management" menu to upload photos from a family trip.

[1161] Step 12:

[1162] The device sends the selected photo to the server, which stores the photo and links it to the corresponding event.

[1163] Step 13:

[1164] The saved photos and linked schedules are displayed to the user through the interface means of the terminal, allowing the user to view the photos and easily manage the growth record.

[1165] Step 14:

[1166] The server analyzes message data to automatically extract schedules from messaging tools such as LINE.

[1167] Step 15:

[1168] The server extracts the schedule from the parsed message and stores it as a new schedule in the database.

[1169] Step 16:

[1170] The new event will be saved and coordinated with other events using AI means, and the event will be notified to the device, where the user can see that a new event has been added.

[1171] Example 1

[1172] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1173] Conventional schedule management systems do not fully utilize voice input, automatic data analysis, or an intuitive user interface, requiring users to spend time and effort creating and managing schedules. Furthermore, comprehensive schedule management is difficult because it is not easy to manage image data linked to past family events or to extract schedules from messaging tools.

[1174] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1175] In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, a storage means for saving the converted text data in a data storage, an AI means for checking and optimizing schedule overlaps, an interface means for allowing a user to intuitively organize their schedule, a notification means for sending notifications about the saved schedule to the terminal, a management means for managing image data in association with past family schedules, and an extraction means for automatically extracting schedules from a message tool. This enables users to quickly create schedules through voice input, propose optimized schedules, easily manage image data linked to past events, and automatically extract schedules from a message tool.

[1176] "Voice input means" refers to a device or software that captures voice data spoken by a user.

[1177] The "conversion means" is software or hardware that converts received voice data into text data.

[1178] "Storage means" refers to software or hardware for recording the converted text data in data storage.

[1179] "AI tools" are artificial intelligence algorithms and models that check for overlaps and optimize schedules.

[1180] "Interface means" refers to software that provides a user interface that allows users to intuitively operate and manage their schedules.

[1181] The "notification means" is software or hardware for notifying the user terminal of information related to the saved schedule.

[1182] The "management means" is software or hardware for managing image data in association with the family's past schedules.

[1183] The "extraction means" is software or a function that automatically extracts schedules from message tools.

[1184] This invention is a system for providing a childcare support schedule app, designed to enable users to efficiently create, manage, and adjust schedules. The system includes a voice input unit, a conversion unit, a storage unit, an AI unit, an interface unit, a notification unit, a management unit, and an extraction unit.

[1185] When a user creates a schedule, they use a voice input means. For example, they might say, "Picnic in the park next Saturday at 2 PM." The voice input means captures this voice data and sends it from the device to the server. The server converts the voice data into text data using the Google Cloud Speech-to-Text API as a conversion means. The converted text data is stored in data storage on the server, for example, in MySQL.

[1186] The stored data is analyzed by AI tools, which use generative AI models like TensorFlow and PyTorch to analyze and optimize schedules and identify overlaps with other appointments. An optimized schedule is then generated and sent from the server to the device.

[1187] Users can check and manage their schedules through an interface on their device. The interface uses React and Vue.js to provide an intuitive interface, allowing users to easily adjust their schedules using drag and drop. Notifications based on saved schedules are sent to the user's device in real time using services such as Firebase Cloud Messaging.

[1188] Furthermore, the management tool can be used to manage image data by linking it to past schedules. When a user uploads a photo of their child's sports day, it can be linked to the date of the event and saved for easy reference later. The extraction tool also includes a function to automatically extract schedules from messaging tools such as LINE and add them to the database.

[1189] For example, if a user says, "Make a doctor's appointment next Friday at 3pm," the device will send this voice data to the server, which will convert the voice to text and store it in a database. AI tools will check for conflicts with other appointments, generate an optimal schedule, and send a notification to the device so the user can confirm the appointment.

[1190] Example prompt sentence:

[1191] "Make a doctor's appointment next Friday at 3pm."

[1192] This allows users to quickly create schedules using voice input, propose optimized schedules, manage image data linked to past events, and automatically extract schedules from messaging tools.

[1193] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1194] Step 1:

[1195] A user creates a schedule using voice input. The user launches the app on the device and says, "Picnic in the park next Saturday at 2 PM." The input is voice data, which is captured by the device's microphone. The device then sends this voice data to the server.

[1196] Step 2:

[1197] The server converts the received voice data into text data. The Google Cloud Speech-to-Text API is used as the conversion method to convert the voice data into text data. The input is voice data, and the output is text data such as "Picnic in the park next Saturday at 2 PM." The server generates the converted text data and passes it on to the next process.

[1198] Step 3:

[1199] The server saves the converted text data in data storage. A MySQL database is used as the storage method. The input is text data, which is saved along with the associated user ID and timestamp. The saved text data is "Picnic in the park next Saturday at 2pm" and is saved in the database.

[1200] Step 4:

[1201] The server uses AI tools to analyze the schedule data, check for overlaps with other appointments, and optimize them. TensorFlow or PyTorch is used as the AI ​​tool. The input is stored text data and existing schedule data, and the AI ​​model analyzes the data and generates optimal schedule proposals. The output is the optimized schedule proposals.

[1202] Step 5:

[1203] The server sends the optimized schedule to the terminal. The input is the optimized schedule proposal, and the output is the schedule information to be sent to the user terminal. As a result, the terminal uses the notification means to send a notification to the user saying, "A picnic in the park has been scheduled for next Saturday at 2 PM."

[1204] Step 6:

[1205] The user checks and manipulates the schedule using an interface on the terminal. The user can adjust the schedule by performing operations such as drag and drop on the interface. The input is the schedule information sent from the server, and the output is the schedule content checked or edited by the user. Specific operations include the user dragging icons and changing the time.

[1206] Step 7:

[1207] The user uploads a photo and associates it with a past event. The user selects "Sports Day Photos" from the device interface and uploads the photo. The device sends the photo file to the server, which associates it with the date of the sports day and stores it in a database. The input is the photo data, and the output is the associated date and event information.

[1208] Step 8:

[1209] The server automatically extracts schedules from messaging tools. As a means of extraction, it uses the API of messaging tools such as LINE to analyze schedule information from user messages. The input is message data, and the output is the extracted schedule information. The analyzed data is saved in data storage so that the user can view it later.

[1210] This allows users to quickly create schedules using voice input, efficiently manage images associated with past events, and schedules automatically extracted from message tools.

[1211] (Application example 1)

[1212] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1213] While conventional schedule management systems allow users to easily create schedules using voice input, they lack the ability to display content related to past events or to optimize schedules in real time. Furthermore, few systems support automatic schedule extraction from messaging tools, making efficient schedule management difficult. Furthermore, they lack an interface and gamification elements that allow users to organize their schedules in a fun way, making them difficult to use on a daily basis.

[1214] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1215] In this invention, the server includes a voice input device, a conversion device that converts received voice data into text data, a storage device that stores the converted text data in a database, an AI device that automatically details and coordinates family members' schedules, an interface device that uses gamification to help users organize their schedules in a fun way, a notification device that sends notifications about saved schedules, a content management device that automatically displays content linked to past events, an extraction device that automatically extracts schedules from a messaging tool, and a schedule optimization device that optimizes and notifies users of schedules. This allows users to efficiently manage their schedules and easily check related content and past events. Furthermore, real-time schedule optimization significantly improves everyday usability.

[1216] A "voice input device" is a device that allows a user to input instructions by voice.

[1217] A "conversion device" is a device that has the function of converting voice data into text data.

[1218] A "storage device" is a device for storing converted text data in a database.

[1219] An "AI device" is a device that uses artificial intelligence to automatically detail and optimally coordinate the schedules of family members.

[1220] An "interface device" is a device that uses gamification to provide an operating screen that allows users to organize their schedules in a fun way.

[1221] A "notification device" is a device that sends notifications about saved appointments to a user's terminal.

[1222] A "content management device" is a device that automatically displays and manages content linked to past events.

[1223] An "extraction device" is a device that has the function of automatically extracting schedules from a message tool.

[1224] A "schedule optimization device" is a device that analyzes existing schedules, adjusts them to the optimal form, and notifies the user.

[1225] System Overview

[1226] This invention relates to a child-rearing support system that includes a voice input device, a conversion device, a storage device, an AI device, an interface device, a notification device, a content management device, an extraction device, and a schedule optimization device. This system allows users to easily create schedules through voice input and efficiently manage and adjust them thereafter.

[1227] Hardware and software used

[1228] The server converts data from the voice input device into text data using a speech recognition API (e.g., Google Cloud Speech-to-Text). The storage device uses a database management system (e.g., MySQL). The AI ​​device uses a machine learning model (e.g., TensorFlow) to optimize the schedule. The interface device uses web technology (e.g., React.js) to provide a user interface incorporating gamification. The notification device uses the smartphone's push notification function (e.g., Firebase Cloud Messaging).

[1229] Natural language processing explanation

[1230] 1. The user speaks, "Picnic in the park next Saturday at 2 PM." The server receives this voice data via the voice input device.

[1231] 2. The received voice data is converted into text data using a conversion device.

[1232] 3. The converted text data is stored in a database using a storage device.

[1233] 4. The AI ​​device on the server analyzes the saved schedule data, checks for overlaps with other appointments, and generates the optimal schedule.

[1234] 5. Users can check their schedules through an interface device. The gamified screen allows users to enjoy drag-and-drop operations.

[1235] 6. When an appointment is approaching, users will be notified through the notification device, so they will not miss any important appointments.

[1236] 7. Using content management devices, photos and videos related to past events are automatically displayed. For example, when a user uploads a photo of a sports day, detailed information related to that sports day is displayed.

[1237] 8. The extraction device automatically extracts schedules from messaging tools such as LINE and stores them in a database, eliminating the need to manually enter message content.

[1238] 9. The schedule optimization device performs real-time schedule optimization and notifies the user of the optimal schedule.

[1239] Examples of specific examples and prompts

[1240] Examples:

[1241] The user can say, "Make a doctor's appointment next Friday at 3 p.m." The server converts the speech into text using a speech recognition API and stores it in a database. The AI ​​device then checks for overlapping appointments, generates an optimal schedule, and notifies the user via a smartphone push notification. Additionally, related doctor visit records and photos are displayed via the content management device.

[1242] Example prompt sentence:

[1243] "I'm going to the aquarium with my family on the weekend."

[1244] "Picnic next Friday afternoon"

[1245] "Kids' dance lessons on Saturday mornings at 9am"

[1246] This allows users to easily create and efficiently manage appointments through voice input, optimizing schedules, and automatically displaying records of past events for added convenience.

[1247] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1248] Step 1:

[1249] The user speaks "Picnic in the park next Saturday at 2 PM." Input: User's voice data. Output: Voice data.

[1250] Specific operation: The user speaks their schedule into the smartphone's voice input device. The voice data is received by the device.

[1251] Step 2:

[1252] The device uses a speech recognition API to convert voice data into text data. Input: Voice data. Output: Text data.

[1253] Specific operation: The device's voice recognition API (e.g., Google Cloud Speech-to-Text) analyzes the voice data and converts it into text format.

[1254] Step 3:

[1255] The converted text data is sent to the server and saved to the storage device. Input: Text data. Output: Saved data.

[1256] Specific operation: The terminal sends text data to the server, and the server stores the text data in a database management system (e.g., MySQL).

[1257] Step 4:

[1258] The AI ​​device analyzes the saved schedule data, checks for overlaps with other appointments, and generates an optimal schedule. Input: Saved data. Output: Optimized schedule.

[1259] How it works: The AI ​​device uses a machine learning model (e.g., TensorFlow) to analyze stored schedule data and generate an optimal schedule.

[1260] Step 5:

[1261] The interface device displays a screen that allows the user to intuitively operate the schedule. Input: Optimized schedule. Output: User interface.

[1262] Specific operation: The server uses web technologies (e.g. React.js) to generate an interface that allows users to edit appointments using drag and drop, and displays it on the device.

[1263] Step 6:

[1264] When an appointment is approaching, the notification device sends a notification to the user's device. Input: Optimized schedule. Output: Notification.

[1265] Specific operation: The server uses a push notification service (e.g., Firebase Cloud Messaging) to send a notification of the event to the user's device.

[1266] Step 7:

[1267] When a user uploads photos or videos linked to past events to a content management device, the related content is displayed. Input: Photo or video data. Output: Display of related content.

[1268] Specific operation: The user uploads photos and videos from their device to the server, which then uses a content management device to link them with related past events and store and display them in a database.

[1269] Step 8:

[1270] The extraction device automatically extracts schedule data from messaging tools such as LINE and stores the data in a database. Input: LINE messages. Output: Schedule data.

[1271] Specific operation: The server analyzes the message content using the API of the message tool and saves the extracted schedule data in a database.

[1272] Step 9:

[1273] The schedule optimization device optimizes the schedule in real time and notifies the user of the results. Input: Saved schedule data. Output: Optimized schedule notification.

[1274] Specific operation: The server uses machine learning models to perform real-time analysis, recalculate an optimized schedule, and send notifications to the user device.

[1275] The above are the specific processing steps of the system that realizes the application example.

[1276] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1277] System Overview

[1278] This invention is a system for providing a childcare support schedule app, and includes a voice input unit, a conversion unit, a storage unit, an AI unit, an interface unit, a notification unit, a management unit, an extraction unit, and an emotion engine. This system allows users to easily create, manage, and adjust schedules. It also provides a function for managing past records and child growth, and provides a more personalized experience by recognizing the user's emotions and responding accordingly.

[1279] Explanation of voice input and conversion methods

[1280] When a user uses the voice input means to create a new plan, the user inputs a voice such as "Picnic in the park at 2 PM next Saturday." The voice input means receives this voice data and transmits it to the conversion means. The conversion means converts the voice data into text data, generating the text data "Picnic in the park at 2 PM next Saturday."

[1281] Emotion Engine Explained

[1282] The emotion engine has the ability to analyze the emotions expressed when a user speaks and adjust the system's behavior based on those emotions. For example, if a user speaks of feeling stressed, the emotion engine will recognize this and suggest schedules and notifications that will help them relax. It also accumulates emotion data and suggests optimal schedules based on past emotion patterns.

[1283] Explanation of storage and AI methods

[1284] The text data generated by the conversion means is stored in a database using a storage means. The stored schedule data is automatically detailed by the AI ​​means. The AI ​​means analyzes overlaps with other schedules and optimal time slots to adjust the schedule.

[1285] Description of interface and notification methods

[1286] The interface means provides gamified screens and operation means to allow users to enjoyably organize their schedules. Users can intuitively operate their schedules, for example, by dragging and dropping schedule icons. Notifications about saved schedules are sent to the user's device using the notification means and are automatically adjusted to suit the emotion engine.

[1287] Description of management and extraction methods

[1288] The management means provides a function to manage image data by linking it to past family schedules. For example, photos from a family trip can be uploaded and saved linked to the date of the trip. The extraction means also has a function to automatically extract schedules from messaging tools such as LINE and add them to the database. This allows for efficient schedule management by automatically registering schedules based on the contents of messages.

[1289] Specific examples

[1290] Example 1: Voice-activated scheduling and emotion recognition

[1291] The user voice-inputs, "Make a doctor's appointment next Friday at 3 p.m." The device sends this voice data to the server, which converts the voice into text and stores it in a database. AI tools check for conflicts with other appointments and generate an optimal schedule. The emotion engine analyzes the voice data and, if the user is nervous, suggests relaxing appointments (e.g., an aromatherapy massage the next day).

[1292] Example 2: Photo management and schedule linking

[1293] A user uploads photos from their child's sports day. The device sends the photos to the server, which stores them in association with the date of the sports day. When the user later references the date of the sports day, related photos are displayed, allowing the user to easily reminisce about their memories.

[1294] Example 3: Extracting schedules from messages and reflecting emotional data

[1295] If a LINE message is sent saying, "I want to go see a movie this weekend," the server analyzes the message and automatically adds the event to the database. Furthermore, the emotion engine recognizes the user's emotions from the context of the message and sends encouraging notifications based on those emotions.

[1296] Through these means and processes, this invention allows users to manage their family schedules more efficiently and support child-rearing in a fun way. In addition, the introduction of an emotion engine provides a more personal and emotionally sensitive user experience.

[1297] The processing flow will be explained below.

[1298] Step 1:

[1299] The user starts the application and uses the voice input means to input plans by voice, such as "Picnic in the park at 2:00 p.m. next Saturday."

[1300] Step 2:

[1301] The device receives the audio data and temporarily stores it in local storage.

[1302] Step 3:

[1303] The terminal transmits the voice data to the server, and the voice data is converted into text data by the conversion means, generating text data such as "Picnic in the park at 2:00 PM next Saturday."

[1304] Step 4:

[1305] The terminal transmits the generated text data to the server.

[1306] Step 5:

[1307] The server analyzes the received text data and stores it in the database as a new appointment.

[1308] Step 6:

[1309] AI tools retrieve saved schedule data, automatically check whether it overlaps with other appointments, and adjust the schedule.

[1310] Step 7:

[1311] The emotion engine analyzes the voice input data and recognizes the user's emotions. For example, if it recognizes that the user is feeling stressed, it stores that emotion data.

[1312] Step 8:

[1313] The server generates optimized schedule data and determines appropriate notification content based on feedback from the emotion engine.

[1314] Step 9:

[1315] The server sends a notification to the device, and the user confirms that a plan to "have a picnic in the park next Saturday at 2 p.m." has been registered, and receives a notification message that takes emotion into consideration.

[1316] Step 10:

[1317] Users open the gamification interface on the app and drag and drop event icons to coordinate with other events.

[1318] Step 11:

[1319] The terminal transmits the changed schedule information to the server, and the saved schedule is updated.

[1320] Step 12:

[1321] The user opens the app's "Photo Management" menu to upload photos from a family trip.

[1322] Step 13:

[1323] The terminal transmits the selected photo to the server, which stores the photo and links it to the corresponding appointment.

[1324] Step 14:

[1325] The saved photos and linked schedules are displayed to the user through the interface means of the terminal, allowing the user to view the photos and easily manage the growth record.

[1326] Step 15:

[1327] The server analyzes message data to automatically extract schedules from messaging tools such as LINE.

[1328] Step 16:

[1329] The server extracts the schedule from the parsed message and stores it as a new schedule in the database.

[1330] Step 17:

[1331] The new event will be saved and coordinated with other events using AI means, and the event will be notified to the device, where the user can see that a new event has been added.

[1332] Step 18:

[1333] The emotion engine accumulates the user's emotional data and proposes optimal schedules based on past emotional patterns. During times when the user frequently feels stressed, it suggests relaxing activities.

[1334] Example 2

[1335] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1336] Conventional schedule management systems required users to manually input schedules, which resulted in issues with overlaps and time-consuming adjustments to optimal schedules. Furthermore, they only provided simple schedule management without taking into account the user's emotional state, preventing them from providing a personalized experience. Furthermore, the system did not automatically extract schedules from past schedule data or messaging tools, which often required time and effort from users.

[1337] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, an emotion engine for analyzing the user's emotions, and an AI means for adjusting the system's operation based on the analyzed emotion data. This allows the user to easily input plans by voice and enables optimal schedule management according to the user's emotions. In addition, past schedule data management and automatic extraction functions from message tools are also provided, realizing a more efficient and personalized experience.

[1338] The "voice input means" is a device or function that allows the user to input a schedule by voice.

[1339] The "conversion means" is a device or program for converting received voice data into text data.

[1340] "Storage means" refers to a device or function for storing the converted text data in a database.

[1341] An "emotion engine" is a device or program that analyzes the emotion expressed by the user when inputting voice and generates emotion data.

[1342] "AI means" means a device or program that adjusts the system's behavior and refines and optimizes schedules based on analyzed emotional data.

[1343] An "interface means" is a device or program that provides an operation screen and functions using gamification elements so that the user can organize their schedule in a fun way.

[1344] "Notification means" is a device or function for sending notifications regarding saved schedules to the user's terminal.

[1345] The "management means" is a device or function for managing image data in association with past family schedules.

[1346] The "extraction means" is a device or program for automatically extracting schedules from the message tool and adding them to the database.

[1347] System Overview

[1348] This invention is a schedule management system for supporting child-rearing, which includes a voice input means, a conversion means, a storage means, an emotion engine, an AI means, an interface means, a notification means, a management means, and an extraction means. This system allows users to easily create, manage, and adjust schedules, and also provides personalized suggestions based on emotion recognition.

[1349] Hardware and Software Configuration

[1350] Voice input method: Uses the microphone on a smartphone or tablet device. For software, a voice input API is used.

[1351] Conversion method: Converting voice data into text data using speech recognition software (e.g., Google Speech-to-Text API).

[1352] Storage method: Store text data and emotion data in a database (e.g., MySQL).

[1353] Emotion engine: Analyzes emotions from audio data using an emotion analysis library (e.g., IBM Watson Tone Analyzer).

[1354] AI methods: Use machine learning models (e.g., TensorFlow) to adjust and optimize schedules.

[1355] Interface solutions: Provide a user interface incorporating gamification elements, such as a UI component that allows dragging and dropping appointment icons.

[1356] Notification method: Uses an API with push notification functionality to send notifications to the user's device.

[1357] Management method: A photo storage service for managing image data and linking that data to a database.

[1358] Extraction method: Use the API of a messaging tool (e.g., LINE) to automatically extract schedules from messages.

[1359] System Operation

[1360] First, the user uses the device's voice input means to input their plans by voice. For example, they might say, "Picnic in the park next Saturday at 2 p.m." The device collects this voice data and sends it to a server via the Internet. The server uses voice recognition software to convert the voice data into text data. The converted text data is then stored in a database.

[1361] The server then uses an emotion engine to analyze the emotion of the voice data. The analysis results are stored as emotion data. For example, if the user is speaking with a happy expression, the emotion data is registered as "enjoyment."

[1362] Based on the stored text and emotion data, the server uses AI methods to optimize the schedule, for example by checking whether appointments overlap with other appointments and adjusting them if necessary, and then storing the adjusted schedule back in the database.

[1363] The interface means is designed to allow users to enjoyably organize their schedules, and allows intuitive operation. For example, users can easily change their schedules by dragging and dropping appointment icons.

[1364] Based on the saved schedule data, the notification means generates and sends notifications to the user's device. The notification content takes into account emotional data, providing a more personalized experience.

[1365] Finally, the management and extraction methods can also integrate past schedule data and information from messaging tools. For example, it is possible to extract content such as "I want to go see a movie this weekend" from a LINE message and automatically register it as a schedule.

[1366] Prompt Sentence Examples

[1367] When a user says, "Make an appointment to see the dentist tomorrow at 3 PM," the device sends this voice data to the server. The server generates text data saying, "Make an appointment to see the dentist tomorrow at 3 PM," and stores it in a database along with emotional data analyzed by the emotion engine. The AI ​​means uses this information to coordinate with other appointments and sends a notification to the device.

[1368] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1369] Step 1:

[1370] The user performs voice input. The user speaks to the terminal, "Picnic in the park next Saturday at 2 PM." This voice input is the input data. Voice data is generated as output.

[1371] Step 2:

[1372] The device sends the voice data to the server. The device collects the voice data and sends it to the server via an internet connection. This transfers the voice data to the server. The input is voice data, and the output is the transfer of voice data to the server.

[1373] Step 3:

[1374] The server converts the voice data into text data. The server uses speech recognition software (e.g., Google Speech-to-Text API) to analyze the transmitted voice data and generate text data such as "Picnic in the park next Saturday at 2 p.m." The input is voice data, and the output is text data.

[1375] Step 4:

[1376] The server analyzes emotions using an emotion engine. The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the emotions in the voice and generate emotion data. For example, if someone is talking about enjoying a picnic, the emotion data "enjoyed" is generated. The input is text data, and the output is emotion data.

[1377] Step 5:

[1378] The server stores the text data and emotion data in a database. The server uses a storage means to record the converted text data "Picnic in the park next Saturday at 2 PM" and emotion data "fun" in the database. The input is text data and emotion data, and the output is storage in the database.

[1379] Step 6:

[1380] The server generates a detailed schedule using AI. It uses a machine learning model (e.g., TensorFlow) to analyze whether the schedule overlaps with the user's other plans and proposes and adjusts the optimal schedule. For example, if the schedule overlaps with other plans, it suggests a different time slot. The input is text data and existing schedule data, and the output is the optimized schedule data.

[1381] Step 7:

[1382] The device notifies the user of the results. The device receives the notification data generated by the server and notifies the user through push notifications or interface means. For example, a notification such as "A picnic in the park has been scheduled for next Saturday at 2 PM" is sent. The input is the notification data, and the output is the notification sent to the user.

[1383] (Application example 2)

[1384] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1385] In today's industrial world, there is a demand for reducing the burden on factory workers and for efficient work schedule management. While conventional systems can manage schedules based on voice instructions, they have the problem of not optimizing schedules by taking into account the emotional state of workers. This can result in workers' stress and fatigue being ignored, which can lead to reduced productivity and a worsening work environment.

[1386] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1387] In this invention, the server includes a voice input means, a conversion means for converting received voice data into text data, a storage means for saving the converted text data in a database, an emotion engine for analyzing the emotions of workers and adjusting the operation of the system based on those emotions, an AI means for automatically refining and adjusting the schedule of workers, an interface means for intuitively operating the schedule, and a notification means for sending notifications regarding the saved schedule. This enables flexible and efficient schedule management that takes into account the emotional state of workers.

[1388] "Audio input means" refers to devices or techniques for receiving audio data.

[1389] "Conversion means" refers to a device or technology that converts received voice data into text data.

[1390] "Storage means" refers to the device or technology used to store the converted text data in a database.

[1391] An "emotion engine" refers to a device or technology that analyzes emotions from a worker's voice and adjusts the system's operation based on those emotions.

[1392] "AI means" refers to artificial intelligence technology that automatically details and adjusts worker schedules.

[1393] "Interface means" refers to the screens and means that allow users to intuitively operate their schedules.

[1394] "Notification Means" means any device or technology that sends notifications to a user regarding saved appointments.

[1395] "Management means" refers to devices and technologies for managing image data in association with past schedules.

[1396] "Extraction means" refers to a device or technology that automatically extracts schedules from a message tool.

[1397] This invention is a system that efficiently manages the work schedules of factory workers and improves their working environment by taking into account their emotional state. The system consists of the following components:

[1398] 1. Voice input and conversion methods

[1399] A worker inputs a command by voice, such as "Maintenance at 10:00 AM next Monday." This voice data is received through the voice input means and converted into text data using the conversion means. This converted text data is stored in a database by the storage means described below.

[1400] 2. Preservation means

[1401] The converted text data is stored in a database system such as SQLite. The database serves as the basis for schedule management and stores instructions and emotional data for each worker.

[1402] 3. Emotion Engine

[1403] The emotion engine analyzes emotions during voice input and determines states such as stress, fatigue, joy, etc. For example, if it determines that a worker is feeling stressed, the emotion engine will suggest relaxing tasks.

[1404] 4. AI means

[1405] Based on the stored text data, the AI ​​tool refines and optimizes the worker's schedule, taking into account the worker's emotional state and adjusting the optimal time slots and work order.

[1406] 5. Interface Methods

[1407] The system provides an intuitive user interface that allows users to easily change their work schedule with drag and drop, and displays recommended tasks based on their emotional state.

[1408] 6. Means of notification

[1409] It sends real-time notifications about saved schedules to workers via smartphones, tablets, head-mounted displays, etc.

[1410] 7. Control measures

[1411] Image data can be managed by linking it to past work. For example, photos taken during an inspection can be uploaded and saved by linking them to the date and time.

[1412] 8. Extraction means

[1413] It has a function to automatically extract schedules from message tools. For example, it can analyze content such as "I would like to carry out equipment inspection this weekend" in a message tool and automatically reflect it in the schedule.

[1414] Hardware and software examples

[1415] The system uses a smartphone or head-mounted display for voice input, the Google Speech Recognition API for sentiment analysis, SQLite for database management, and simple random selection or specialized libraries for sentiment analysis.

[1416] Examples and prompts

[1417] Example 1:

[1418] When a worker says, "I'd like to do maintenance next Monday at 10 a.m.", the system converts it into text and AI tools analyze their emotions: if they're feeling tired, they're suggested to do other relaxing tasks.

[1419] Example 2:

[1420] When workers upload photos of their inspections to the system, the photos are stored in the database along with the specified date and time, and the photos are displayed when the time is later referenced.

[1421] Example prompt sentence:

[1422] "Equipment maintenance next Saturday at 2pm"

[1423] "I'd like to set up the new device tomorrow at 3pm."

[1424] "Please inspect the warehouse in the morning."

[1425] This makes it possible to manage schedules within the factory efficiently and flexibly.

[1426] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1427] Step 1:

[1428] The user inputs a command by voice, for example, "Maintenance at 10:00 AM next Monday." The terminal receives this voice data and sends it to the next processing step.

[1429] Input: Audio data

[1430] Output: Raw audio data

[1431] Step 2:

[1432] The device sends the received voice data to a conversion means, which uses the Google Speech Recognition API to convert the voice data into text data. The converted text is then sent to the next processing step.

[1433] Input: Audio data

[1434] Output: Text data

[1435] Step 3:

[1436] The server receives the converted text data and stores it in a database using a storage method. This database uses SQLite. The stored data is used in the next processing step.

[1437] Input: Text data

[1438] Output: Text data stored in the database

[1439] Step 4:

[1440] The server passes the stored text data to an emotion engine, which analyzes the worker's emotional state. For example, it can determine stress, fatigue, or happiness from the tone and speed of the voice. The analysis results are used in the next processing step.

[1441] Input: Text data

[1442] Output: Emotion analysis results

[1443] Step 5:

[1444] The server receives the analysis results of the emotion engine and generates an optimal schedule using AI means. Taking into account the emotional state, it adjusts relaxing activities and optimal time slots. The generated schedule is stored in a database and sent to the interface means.

[1445] Input: Sentiment analysis results, existing schedule information in the database

[1446] Output: Optimized schedule

[1447] Step 6:

[1448] The server provides users with an intuitive interface, allowing them to change and check schedules using drag and drop. Changes are reflected in the database in real time.

[1449] Input: Optimized schedule, user input

[1450] Output: Schedule data that the user confirmed or edited

[1451] Step 7:

[1452] The server uses a notification means to send notifications about the saved schedule to the user in real time, and the notifications are displayed on devices such as smartphones and tablets.

[1453] Input: Schedules stored in the database

[1454] Output: Notification to user terminal

[1455] Step 8:

[1456] Specifically, the user uploads the inspection photos they have collected, and the device sends them to the server. The server uses a management method to link the photos with the inspection date and time and saves them in a database. When the user wants to refer to them later, the photos are displayed.

[1457] Input: Uploaded photo data, corresponding date and time

[1458] Output: A database entry with a photo and a date and time stamp.

[1459] Step 9:

[1460] Furthermore, when a user confirms or adds an appointment through a messaging tool such as LINE, the server automatically extracts the appointment from the message using an extraction method. The extraction results are stored in a database and used by the emotion engine and AI methods.

[1461] Input: Message data

[1462] Output: Extracted schedule data

[1463] Through these steps, the system can automatically generate schedules based on workers' voice input and provide an optimal working environment through emotion recognition.

[1464] Example prompt sentence:

[1465] "Equipment maintenance next Saturday at 2pm"

[1466] "I'd like to set up the new device tomorrow at 3pm."

[1467] "Please inspect the warehouse in the morning."

[1468] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1469] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1470] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1471] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1472] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1473] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1474] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1475] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1476] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1477] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1478] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1479] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1480] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1481] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1482] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1483] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1484] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1485] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1486] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1487] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1488] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1489] The following is further disclosed regarding the above embodiment.

[1490] (Claim 1)

[1491] A voice input means;

[1492] A conversion means for converting received voice data into text data;

[1493] a storage means for storing the converted text data in a database;

[1494] AI means to automatically detail and coordinate family member schedules;

[1495] A gamified interface that allows users to organize their schedules in a fun way.

[1496] Notification methods to send notifications about saved events

[1497] A system including:

[1498] (Claim 2)

[1499] 2. The system according to claim 1, further comprising a management unit for managing image data in association with past schedules of family members.

[1500] (Claim 3)

[1501] The system according to claim 1, further comprising an extraction means for automatically extracting schedules from a messaging tool such as LINE.

[1502] "Example 1"

[1503] (Claim 1)

[1504] A voice input means;

[1505] A conversion means for converting received voice data into text data;

[1506] a storage means for storing the converted text data in a data storage;

[1507] AI methods to check and optimize schedule overlaps;

[1508] An interface means that allows users to intuitively organize their schedules;

[1509] a notification means for sending a notification regarding the saved schedule to the terminal;

[1510] A system including:

[1511] (Claim 2)

[1512] 2. The system according to claim 1, further comprising a management unit for managing image data in association with past schedules of family members.

[1513] (Claim 3)

[1514] 10. The system of claim 1, further comprising an extracting means for automatically extracting a schedule from the message tool.

[1515] "Application Example 1"

[1516] (Claim 1)

[1517] A voice input device;

[1518] a conversion device that converts the received voice data into text data;

[1519] a storage device for storing the converted text data in a database;

[1520] An AI device that automatically details and coordinates family members' schedules,

[1521] An interface device that uses gamification to help users organize their schedules in a fun way,

[1522] a notification device for sending notifications about saved appointments;

[1523] A content management device that automatically displays content linked to past events;

[1524] An extraction device that automatically extracts schedules from a message tool;

[1525] Schedule optimization device that optimizes and notifies schedules

[1526] A system including:

[1527] (Claim 2)

[1528] 10. The system of claim 1, further comprising a function for managing image data and related content linked to past events.

[1529] (Claim 3)

[1530] 10. The system of claim 1, further comprising the functionality of converting voice input from a messaging tool into text and automatically extracting schedules.

[1531] "Example 2: Combining Emotion Engines"

[1532] (Claim 1)

[1533] A voice input means;

[1534] A conversion means for converting received voice data into text data;

[1535] a storage means for storing the converted text data in a database;

[1536] an emotion engine that analyzes the user's emotions;

[1537] AI means for adjusting the system's behavior based on the analyzed emotion data; and

[1538] A gamified interface that allows users to organize their schedules in a fun way.

[1539] Notification methods to send notifications about saved events

[1540] A system including:

[1541] (Claim 2)

[1542] 2. The system according to claim 1, further comprising a management unit for managing image data in association with past schedules of family members.

[1543] (Claim 3)

[1544] 10. The system of claim 1, further comprising an extracting means for automatically extracting a schedule from the message tool.

[1545] "Application example 2 when combining emotion engines"

[1546] (Claim 1)

[1547] A voice input means;

[1548] A conversion means for converting received voice data into text data;

[1549] a storage means for storing the converted text data in a database;

[1550] An emotion engine that analyzes the emotions of workers and adjusts the system's behavior based on those emotions;

[1551] AI means to automatically detail and adjust worker schedules,

[1552] An interface that allows you to intuitively operate your schedule,

[1553] Notification methods to send notifications about saved events

[1554] A system including:

[1555] (Claim 2)

[1556] 2. The system according to claim 1, further comprising a management unit for managing image data in association with past schedules.

[1557] (Claim 3)

[1558] 10. The system of claim 1, further comprising an extracting means for automatically extracting a schedule from the message tool. [Explanation of symbols]

[1559] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A voice input means; A conversion means for converting received voice data into text data; a storage means for storing the converted text data in a database; AI means to automatically detail and coordinate family member schedules; A gamified interface that allows users to organize their schedules in a fun way. Notification methods to send notifications about saved events A system including:

2. The system according to claim 1, further comprising a management unit for managing image data in association with past schedules of family members.

3. The system according to claim 1, further comprising an extraction unit that automatically extracts a schedule from a messaging tool such as LINE.

Images