System

Abstract

To provide a system.SOLUTION: A system is provided, comprising means for acquiring calendar information of a user and family information including family plans, means for collecting event information on events related to preselected genre and scheduled to be held on free days of the user according to the calendar information, means for filtering events scheduled to be held during hours not conflicting with the family plans from the collected event information, and means for notifying the user of the filtered event information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With the current system, it is difficult to efficiently collect event information and adjust schedules to fit the schedule and preferences of families. Additionally, there are problems with missing events and lack of opportunities to experience them. [Means for solving the problem]

[0005] This invention proposes a system that allows users to register school and work schedules on a calendar and input basic family information, and then provides event notifications that combine family information and favorite genres with free events on the calendar. This is expected to prevent events from being missed and increase opportunities to experience them. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In the system of the present invention, users can register school and work schedules in a calendar. This registration can be done by taking a photo of a school newsletter or shift schedule. For example, if a user takes a photo of a school report card with a smartphone camera, the system extracts text from the image and registers the schedule in the calendar based on that information.

[0029] "Example 2"

[0030] Users also enter basic family information into the system, including family members' addresses, ages, grades, and heights. Height, in particular, can be measured automatically using a smartphone. For example, the smartphone's camera and augmented reality (AR) technology can be used to measure the heights of family members and enter that information into the system.

[0031] "Example 3"

[0032] Furthermore, the system will provide event notifications based on free calendar appointments, combining family information and the user's favorite genres. For example, if a user selects "art" as their favorite genre, the system will collect information about art-related events and notify the user if the event is held at a time that does not overlap with family schedules. This prevents users from missing out on events and increases opportunities for family experiences.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: The user takes a photo of their school or work schedule with their smartphone camera, such as a school report card or a work shift schedule.

[0036] Step 2: The system extracts text from the captured image using OCR (Optical Character Recognition) technology.

[0037] Step 3: The system uses the extracted text information to add an event to the user's calendar.

[0038] "Example 2"

[0039] Step 1: The user enters basic family information into the system, including information such as the family member's address, age, grade, and height.

[0040] Step 2: The user uses their smartphone camera and AR (augmented reality) technology to measure the heights of their family members.

[0041] Step 3: The system updates basic information about the family based on the height information measured.

[0042] "Example 3"

[0043] Step 1: The user selects their favorite genre for the system. For example, they select "art."

[0044] Step 2: The system collects information about events related to the selected genre.

[0045] Step 3: The system compares the collected event information with the user's calendar, and if the event is held at a time that does not conflict with family schedules, the system notifies the user.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0048] Traditional calendar registration systems require users to manually enter appointments, which is time-consuming and prone to input errors. Furthermore, there is no system that automatically generates event notifications based on basic family information or preferred genres, so users may miss many events. Furthermore, there is a lack of efficient ways to manage basic family information.

[0049] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0050] In this invention, the server includes a means for sending the captured image to the server, a means for the server to extract text from the image, and a means for the server to analyze the extracted text. This allows users to automatically register events on their calendar simply by taking a photo of a school newsletter or shift schedule. Furthermore, by entering basic family information, event notifications are automatically generated based on available calendar schedules, allowing users to make reservations, preventing users from missing out on events and increasing opportunities to experience them.

[0051] "Means for registering school and work schedules on a calendar" refers to a function that allows users to enter and manage school and work schedules on an electronic calendar.

[0052] "Means that allow for the capture of school newsletters and shift schedules by photographing them" refers to a function that allows users to use a camera on a smartphone or tablet to take a photo of a school newsletter or shift schedule and import that image into the system.

[0053] The "means for transmitting captured images to a server" is a function that allows a user to upload image data captured by the user to a server via the Internet.

[0054] The "means for the server to extract text from images" refers to a function for the server to extract text information from image data received using optical character recognition (OCR) technology.

[0055] "Means for analyzing text extracted by the server" refers to a function in which the server analyzes the extracted text data using natural language processing (NLP) technology and converts it into a format for registration on the calendar.

[0056] The "means for the server to register the analysis results in the calendar" is a function that enables the server to register the analyzed text data in a calendar application and add an event to the user's calendar.

[0057] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[0058] "Means for automatically measuring height using a mobile information terminal" refers to a function that allows a user to use a mobile information terminal to automatically measure the height of family members and input that data into the system.

[0059] "A means for receiving event notifications that combine family information and favorite genres with available time slots on the calendar, allowing users to make reservations" is a function in which the system generates event notifications based on basic family information and favorite genres, based on available time slots on the calendar, and allows users to reserve those events.

[0060] This invention is a system that allows users to register school and work schedules on a calendar. Users can easily register schedules by taking photos of school newsletters or shift schedules. This system is implemented using a mobile information terminal such as a smartphone or tablet and a server.

[0061] A user uses the smartphone camera to take a picture of a school newsletter or shift schedule. The image is sent to a server via a smartphone application. The server extracts text from the image using image processing software (e.g., a common cloud-based image recognition API). The extracted text is analyzed using natural language processing (NLP) software (e.g., a common NLP library) and converted into a format suitable for a calendar event. The server then registers the analyzed data with a calendar application (e.g., a common calendar API).

[0062] As a concrete example, consider the case where a user takes a photo of their school report card. The user takes a photo of the report card with their smartphone camera and sends the image to a server through an application. The server uses a cloud-based image recognition API to extract text from the image and analyzes the text using an NLP library. The analyzed information is then registered in the user's calendar using a calendar API. For example, if the report card contains information such as "PTA meeting on October 15th at 2:00 PM," this is registered as "PTA meeting" on the calendar.

[0063] Example prompt sentence:

[0064] "Take a photo of your school report card and put the event on your calendar."

[0065] This system allows users to efficiently manage their schedules by eliminating the need to manually enter schedules. By entering basic family information, the system automatically generates event notifications based on available calendars and allows users to make reservations, eliminating the need to miss out on events and increasing opportunities to experience them.

[0066] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0067] Step 1:

[0068] The user takes a photo of a school newsletter or shift schedule.

[0069] Specifically, the user launches the camera app on their smartphone, frames the school newsletter or shift schedule, and presses the shutter button. The input is the physical paper copy of the school newsletter or shift schedule, and the output is the image data stored on the smartphone.

[0070] Step 2:

[0071] The device sends the captured image to the server.

[0072] Specifically, the smartphone application uploads image data to the server as an HTTP request. The input is the image data stored on the smartphone, and the output is the image data sent to the server.

[0073] Step 3:

[0074] The server extracts the text from the image.

[0075] Specifically, the server uses a cloud-based image recognition API to extract text information from the received image data. The input is the image data sent to the server, and the output is the extracted text data.

[0076] Step 4:

[0077] The server parses the extracted text.

[0078] Specifically, the server uses a natural language processing (NLP) library to parse the extracted text data and convert it into a format suitable for calendar entry. The input is the extracted text data, and the output is the parsed text data.

[0079] Step 5:

[0080] The server registers the analysis results in the calendar.

[0081] Specifically, the server uses a calendar API to register the parsed text data in the user's calendar. The input is the parsed text data, and the output is the schedule registered in the calendar.

[0082] Step 6:

[0083] The user enters basic information about their family.

[0084] Specifically, the user enters basic information about the family members, such as their address, age, grade, height, etc. The input is the basic information entered by the user, and the output is the basic information stored on the server.

[0085] Step 7:

[0086] The server generates an event notification based on the available schedules in the calendar.

[0087] Specifically, the server detects free time on the calendar based on the basic information and favorite genres of the family members, and generates appropriate event notifications. The inputs are free time on the calendar, basic information of the family members, and favorite genres, and the output is the generated event notification.

[0088] Step 8:

[0089] A user books an event.

[0090] Specifically, the user checks the event notification sent from the server through the application, selects the desired event, and reserves it. The input is the event notification sent from the server, and the output is the reserved event information.

[0091] (Application example 1)

[0092] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] Conventional calendar registration systems required users to manually enter school and work schedules, which was time-consuming and labor-intensive. Also, manually registering shift schedule information into the calendar could result in input errors or missing information. Furthermore, managing basic family information and event notifications was cumbersome, making efficient schedule management difficult.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0095] In this invention, the server includes means for registering school and work schedules in a calendar, means for enabling the registration by capturing school newsletters and shift schedules by photographing them, means for inputting basic family information, the basic family information being address, age, grade, and height, with height being automatically measured using a smartphone, means for receiving event notifications that combine family information and favorite genres in available calendar schedules and making reservations, and means for extracting text from images of shift schedules, analyzing shift information, and automatically registering it in the calendar. This allows users to easily register school and work schedules in the calendar, prevents input errors and omissions in shift information, and enables efficient schedule management.

[0096] "Means for registering school and work schedules on a calendar" refers to a function that allows users to enter and manage school and work schedules on an electronic calendar.

[0097] "The means by which the registration can be made by taking a photograph of the school newsletter or shift schedule" refers to a function that allows a user to take a photograph of the school newsletter or shift schedule with a smartphone camera, extract text from the image, and automatically register the schedule in the calendar.

[0098] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[0099] "The basic information of the family members is their address, age, grade, and height, and height can be automatically measured using a smartphone" means that basic information of the family members includes their address, age, grade, and height, and height in particular can be automatically measured using a smartphone function.

[0100] "A means to receive event notifications that combine family information and favorite genres with available time slots on the calendar, allowing reservations" is a function that allows the system to send event notifications that take into account basic family information and favorite genres based on available time slots on the calendar, and allows reservations for those events.

[0101] "Means for extracting text from an image of a shift schedule, analyzing the shift information, and automatically registering it in a calendar" is a function that analyzes an image of a shift schedule, extracts text information, and automatically registers that shift information in a calendar.

[0102] The system for implementing this invention allows users to efficiently register school and work schedules on a calendar and prevents input errors and omissions of shift information. A specific embodiment of this system will be described below.

[0103] First, the user takes a photo of the school newsletter or shift schedule using their smartphone. The smartphone has OpenCV installed as image processing software, which reads the captured image. Next, text is extracted from the image using Tesseract OCR. Through this text extraction process, the information in the shift schedule or school newsletter is obtained as digital data.

[0104] The acquired text data is sent to a server and analyzed. The analysis process identifies lines containing shift information and school schedules and extracts information such as dates, start times, and end times. Natural language processing technology is used for this analysis process.

[0105] The analyzed information is automatically registered in the user's calendar using a calendar API (e.g., Google (registered trademark) Calendar API), which saves the user the trouble of manually entering events.

[0106] The system also has the ability to manage basic family information. Users can input information such as address, age, grade, and height, and height can be automatically measured using a smartphone sensor. This information is stored on the server, and event notifications tailored to the family's preferences are sent based on available calendar availability.

[0107] As a concrete example, consider a scenario in which a store clerk takes a photo of a shift schedule and the shift information is automatically registered in a calendar. In this case, the store clerk takes a photo of the shift schedule with their smartphone, and the system extracts text from the image, analyzes the shift information, and registers it in the calendar. This prevents input errors and omissions of shift information.

[0108] Example prompt sentence:

[0109] Please create an app that takes a photo of a shift schedule and automatically registers it in a calendar. Include a function to extract text from the shift schedule image, parse the shift date, start time, and end time, and register it in the calendar.

[0110] In this way, users can easily register school and work schedules on their calendar, preventing input errors and omissions in shift information and enabling efficient schedule management.

[0111] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0112] Step 1:

[0113] A user takes a photo of a school newsletter or shift schedule using a smartphone. The input is the captured image, and the output is an image file. Specifically, the user launches the smartphone's camera app and takes a photo of the school newsletter or shift schedule.

[0114] Step 2:

[0115] The device uses OpenCV to read the captured image. The input is an image file and the output is image data. Specifically, the device uses the OpenCV library to read the image file into memory.

[0116] Step 3:

[0117] The device uses Tesseract OCR to extract text from an image. The input is image data, and the output is extracted text data. Specifically, the device invokes the Tesseract OCR engine to parse and extract text from the image data.

[0118] Step 4:

[0119] The terminal transmits the extracted text data to the server. The input is the text data, and the output is the text data transmitted to the server. In specific operations, the terminal transmits the text data to the server via network communication.

[0120] Step 5:

[0121] The server analyzes the received text data and identifies shift information and school schedules. The input is the received text data, and the output is the analyzed shift information and schedule information. Specifically, the server analyzes the text data using natural language processing technology and extracts information such as date, start time, and end time.

[0122] Step 6:

[0123] The server uses a calendar API to register the analyzed information in the calendar. The input is the analyzed shift information and schedule information, and the output is the schedule registered in the calendar. Specifically, the server calls a calendar API such as the Google Calendar API and registers the analyzed information in the calendar.

[0124] Step 7:

[0125] The server manages basic information about family members and generates event notifications based on the free time on the calendar. The input is the basic information about family members and the free time on the calendar, and the output is the generated event notification. Specifically, the server takes into account information such as the address, age, grade, and height of family members and generates event notifications based on the free time on the calendar.

[0126] Step 8:

[0127] The server sends the generated event notification to the user. The input is the generated event notification, and the output is the event notification sent to the user. In concrete terms, the server sends the event notification to the user's terminal via network communication.

[0128] In this way, users can easily register school and work schedules on their calendar, preventing input errors and omissions in shift information and enabling efficient schedule management.

[0129] Example 2

[0130] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0131] Conventional calendar systems require users to manually enter school and work schedules, which is time-consuming and prone to input errors. Similar problems arise when manually entering basic family information. Furthermore, event notifications are not based on individual users' interests or family information, meaning users may miss events that are beneficial to them. There is a need to solve these problems and improve user convenience.

[0132] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0133] In this invention, the server includes a means for registering school and work schedules in a calendar, a means for capturing documents and tables by taking pictures, a means for inputting basic family information, a means for measuring heights using a mobile device's camera and augmented reality technology and inputting the data into the system, a means for generating event notifications by inputting prompt sentences into a generative AI model, and a means for receiving event notifications that combine family information and favorite genres with available calendar entries and allowing reservations. This allows users to easily and accurately register schedules and easily input basic family information. Furthermore, event notifications based on the user's interests and family information are provided, allowing users to participate in useful events without missing them.

[0134] The "means for registering school and work schedules in a calendar" is a function that allows a user to input and manage school and work schedules in an electronic calendar.

[0135] "Means that allow documents and tables to be captured by image capture" refers to a function that allows a user to use the camera on a smartphone or tablet to capture a document or table on paper and import the contents into the system as digital data.

[0136] The "means for inputting basic family information" is a function that allows the user to input basic information such as the address, age, grade, and height of family members into the system.

[0137] "Means for measuring height using a mobile device's camera and augmented reality technology and inputting that data into the system" refers to a function that allows users to measure the height of their family members using a smartphone or tablet's camera and AR technology, and automatically input the measurement results into the system.

[0138] "Means for generating event notifications by inputting prompt sentences into a generative AI model" refers to a function that allows a user to input specific prompt sentences into a generative AI model, which in turn generates event notifications based on the user's interests and family information.

[0139] "A means for receiving event notifications that combine family information and favorite genres in available calendar slots, allowing reservations" is a function that enables the system to detect available times on the calendar, send event notifications to the user that combine basic family information and the user's interests, and allow the event to be reserved.

[0140] The present invention is a system that allows a user to input basic family information, register school and work schedules in a calendar, and generate event notifications. A specific embodiment of this system will be described below.

[0141] Hardware and software used

[0142] Hardware

[0143] Mobile device: Smartphone or tablet (with camera)

[0144] software

[0145] Applications that use AR technology: For example, ARKit for iOS, ARCore for ANDROID (registered trademark)

[0146] Generative AI model: An AI model that generates event notifications based on user prompts.

[0147] System program processing

[0148] Enter basic family information

[0149] Users launch a dedicated smartphone app and enter basic information about their family members, including their address, age, grade, height, etc. Height, in particular, is automatically measured using the smartphone's camera and AR technology.

[0150] Specific examples

[0151] Users launch the app and tap the "Enter Family Information" button on the home screen. They then enter their family member's address, age, and grade in the form that appears. To measure their height, they point their smartphone camera at a family member and adjust the camera according to the on-screen guide. AR technology automatically measures their height and enters that data into the system.

[0152] Registering school and work schedules

[0153] Users can register school and work schedules in the calendar, and can take pictures of documents and tables, and import the contents into the system as digital data.

[0154] Specific examples

[0155] Users use their smartphone camera to take a photo of a school schedule or work shift schedule, and the system uses image recognition technology to extract appointments from the image and automatically register them in the calendar.

[0156] Generate event notifications

[0157] By inputting prompts into the generative AI model, event notifications are generated based on the user's interests and family information. Event notifications that combine family information and favorite genres are sent to the user's calendar, allowing the user to make reservations for the event.

[0158] Specific examples

[0159] Enter the following prompt into the generative AI model:

[0160] "Please enter basic information about your family members, including their address, age, grade, and height. Height can be measured using your smartphone camera and AR technology."

[0161] Based on this prompt, the generative AI model generates and provides event notifications to the user based on their interests and family information.

[0162] In this way, users can easily and accurately register their schedules and input basic family information. In addition, event notifications based on the user's interests and family information are provided, so users can participate in useful events without missing them.

[0163] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0164] Step 1:

[0165] The user enters basic information about their family.

[0166] Specifically, the user launches the dedicated smartphone app and taps the "Enter Family Information" button on the home screen. Next, they enter the address, age, and grade of their family members in the form that appears. The entered data is saved on the device.

[0167] Input: Address, age, grade

[0168] Output: Basic information of family members stored on the device

[0169] Step 2:

[0170] The user measures their height using the mobile device's camera and augmented reality technology.

[0171] Specifically, users go to the "Height Measurement" screen and point their smartphone camera at a family member. They adjust the camera according to the on-screen guide, and the AR technology automatically measures their height. The measurement results are saved on the device.

[0172] Input: Camera image

[0173] Output: Height data saved on the device

[0174] Step 3:

[0175] The device sends basic information and height data of family members to the server.

[0176] Specifically, when the user taps the "Send Data" button, the device sends the saved basic information and height data of the family to the server, which then stores the received data in a database.

[0177] Input: Basic information of family, height data

[0178] Output: Data stored on the server

[0179] Step 4:

[0180] Users add school and work events to their calendars.

[0181] Specifically, users use their smartphone camera to take a photo of a school schedule or work shift schedule, and the system uses image recognition technology to extract appointments from the image and automatically register them in the calendar.

[0182] Input: Schedule or shift schedule image

[0183] Output: Events registered in the calendar

[0184] Step 5:

[0185] Input a prompt sentence into the generative AI model to generate an event notification.

[0186] Specifically, the user inputs the following prompt into the generative AI model: "Please enter basic information about your family, including address, age, grade, and height. Height can be measured using your smartphone's camera and AR technology." Based on this prompt, the generative AI model generates event notifications based on the user's interests and family information.

[0187] Input: prompt statement

[0188] Output: Generated event notifications

[0189] Step 6:

[0190] An event notification that combines family information and favorite genres will be sent to the user's calendar, and the user can make a reservation for the event.

[0191] Specifically, the server detects free time on the calendar and sends an event notification to the user that combines basic information about the family and the user's interests. The user receives the notification and reserves the event.

[0192] Input: Calendar availability, family information, user interests

[0193] Output: Event notifications sent to users, subscribed events

[0194] (Application example 2)

[0195] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0196] In modern society, it is important to efficiently manage family schedules and provide opportunities to participate in events. However, conventional systems require the manual input of basic family information, and measuring height is particularly cumbersome. Additionally, when trying on clothes in physical stores, finding the right size can be time-consuming and time-consuming. To solve these issues, a more efficient and automated system is needed.

[0197] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for registering school and work schedules in a calendar, means for enabling the registration by capturing school newsletters and shift schedules by photographing them, means for inputting basic family information, the basic family information being address, age, grade, and height, means for automatically measuring height using a smartphone, means for receiving event notifications that combine family information and favorite genres in available calendar appointments and allowing reservations, means for automatically measuring family heights using a smartphone camera and augmented reality technology and suggesting optimal clothing sizes, and means for performing try-on simulations using augmented reality technology. This makes it possible to efficiently manage family schedules, increase opportunities to participate in events, and simplify trying on clothing in physical stores.

[0198] "Means for registering school and work schedules on a calendar" is a function that allows users to enter and manage their school or work schedules on a digital calendar.

[0199] "The means by which the registration can be made by taking a photograph of a school newsletter or shift schedule" refers to a function that allows a user to take a photograph of a newsletter distributed by school or a workplace shift schedule with a smartphone camera and automatically register that information in the calendar.

[0200] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[0201] "The basic information of the family members is their address, age, grade, and height, and height can be automatically measured using a smartphone" refers to a function that includes basic information of the family members, such as address, age, grade, and height, and in particular, automatically measures height using a smartphone's camera and augmented reality technology.

[0202] "A means to receive event notifications that combine family information and favorite genres based on available times on the calendar, allowing you to make reservations" is a function that allows you to receive notifications about events that combine basic family information and favorite genres based on available times on the calendar, and to make reservations for those events.

[0203] "Means to automatically measure the heights of family members using a smartphone camera and augmented reality technology and suggest the most suitable clothing sizes" is a function that uses a smartphone camera and augmented reality technology to automatically measure the heights of family members and suggests the most suitable clothing sizes based on that data.

[0204] "Means for performing fitting simulation using augmented reality technology" refers to a function that uses augmented reality technology to virtually simulate the state of trying on clothing before the user actually tries it on.

[0205] A system for carrying out the present invention has the following configuration: A server includes means for registering school and work schedules on a calendar, means for enabling the registration by capturing school newsletters and shift schedules by photographing them, means for inputting basic information about family members, the basic information about the family members being address, age, grade, and height, which can be automatically measured using a smartphone, means for receiving event notifications that combine family information and favorite genres with available calendar appointments and enabling reservations, means for automatically measuring the heights of family members using a smartphone camera and augmented reality technology and suggesting optimal clothing sizes, and means for performing a try-on simulation using augmented reality technology.

[0206] Hardware and software used

[0207] Hardware: Smartphone (with camera)

[0208] Software: OpenCV (image processing library), MediaPipe (pose estimation library)

[0209] Data processing and calculation

[0210] 1. Camera input: Capture video in real time using the user's smartphone camera.

[0211] 2. Pose estimation: Use MediaPipe to detect the position of the user's shoulders and ankles.

[0212] 3. Height calculation: Calculate the user's height from the position of the shoulders and ankles.

[0213] 4. Size suggestion: Based on the calculated height, we suggest the best clothing size for you.

[0214] 5. Try-on simulation: Using augmented reality technology, users can virtually try on clothes before actually trying them on.

[0215] Specific examples

[0216] For example, consider a scenario where a user measures their height using a smartphone and an app suggests a "medium" size shirt. The user takes a full-body photo using the smartphone camera and uses MediaPipe to detect the position of their shoulders and ankles. OpenCV is used to calculate their height from these positions, and the app then suggests the optimal clothing size based on that data. Furthermore, augmented reality technology can be used to simulate the user virtually trying on the shirt.

[0217] Prompt Sentence Examples

[0218] "Develop an application that uses a smartphone camera to measure height and suggest optimal clothing sizes. Use MediaPipe for pose estimation and OpenCV for image processing."

[0219] In this way, it becomes possible to efficiently manage family schedules, increase opportunities to participate in events, and simplify trying on clothes in physical stores.

[0220] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0221] Step 1:

[0222] The user activates the smartphone camera and takes a full-body image. The input is the smartphone camera image, and the output is real-time video data. Based on this video data, the system adjusts the user's entire body to fit within the frame.

[0223] Step 2:

[0224] The device uses MediaPipe to estimate the user's posture from the camera image. The input is the video data obtained in step 1, and the output is the position information of the shoulders and ankles. Specifically, the device uses MediaPipe's posture estimation model to detect the user's shoulder and ankle landmarks.

[0225] Step 3:

[0226] The device uses OpenCV to calculate the user's height from the shoulder and ankle positions. The input is the shoulder and ankle position information obtained in step 2, and the output is the user's height data. Specifically, the distance between the shoulder and ankle landmarks is calculated, and the actual height is calculated using a known scale factor.

[0227] Step 4:

[0228] The device then suggests the optimal clothing size based on the calculated height data. The input is the height data obtained in step 3, and the output is the suggested clothing size information. Specifically, the device compares the height data with an existing size chart to determine the optimal size.

[0229] Step 5:

[0230] The device uses augmented reality technology to provide the user with a virtual try-on simulation. The input is the clothing size information obtained in step 4, and the output is a video of the user virtually trying on the clothing. Specifically, a 3D model of the selected clothing is overlaid on the user's video, and a try-on simulation is performed in real time.

[0231] Step 6:

[0232] The server generates an event notification that matches the available calendar time slots based on the user's basic information and preferred genres, and sends it to the device. The input is the user's basic information and available calendar time slots, and the output is an event notification. Specifically, the server retrieves the user's basic information and preferred genres from a database, searches for events that match the available time slots, and generates a notification.

[0233] Step 7:

[0234] The terminal displays the event notification received to the user and prompts them to make a reservation. The input is the event notification obtained in step 6, and the output is the user's reservation action. Specifically, the notification is displayed in a pop-up format, and the user completes the event reservation by pressing the reservation button.

[0235] The above steps will make it possible to manage family schedules more efficiently, increase opportunities to participate in events, and simplify trying on clothes in physical stores.

[0236] Example 3

[0237] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0238] In conventional calendar systems, users have to collect event information individually and check it against their family schedules, which can lead to missed events and reduced opportunities for family experiences. In addition, there is a lack of a way to efficiently collect event information related to the genre selected by the user and notify them at the appropriate time.

[0239] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0240] In this invention, the server includes means for registering school and work schedules in a calendar, means for enabling the registration by capturing images of school newsletters and shift schedules, means for inputting basic family information, the basic family information being address, age, grade, and height, with height being automatically measured using a mobile device, means for receiving event notifications that combine family information and a favorite genre in available calendar schedules and allowing reservations, means for collecting event information related to a genre selected by the user from an external information source, means for comparing the collected event information with calendar information and identifying events to be held in times that do not overlap with family schedules, and means for notifying the user of the identified event information, thereby enabling the user to efficiently collect event information and not miss events to be held in times that do not overlap with family schedules.

[0241] The "means for registering school or work schedules in a calendar" refers to a method by which a user can input school or work schedules into an electronic calendar.

[0242] "Methods that can be achieved by capturing images" refers to a method of taking a photo of paper documents such as school newsletters or shift schedules with a camera, analyzing the image, and automatically registering the schedule in the calendar.

[0243] The "means for inputting basic family information" is a method by which a user inputs basic information such as the address, age, grade, and height of a family member into the system.

[0244] "Means that can be measured automatically using a mobile device" refers to a method of automatically measuring the height of family members using a mobile device such as a smartphone or tablet.

[0245] "A means for notifying users of events that combine family information and their favorite genres with available time on their calendar, allowing them to make reservations" is a method for notifying users of events related to family information and the genre selected by the user based on available time on the user's calendar, allowing them to make reservations.

[0246] "Means for collecting event information related to the genre selected by the user from external information sources" refers to a method for obtaining event information related to the genre of interest selected by the user from an external database or API.

[0247] The "means for comparing the collected event information with calendar information and identifying events that will be held during times that do not overlap with family schedules" is a method for comparing the collected event information with the user's calendar information and finding events that will be held during times that do not overlap with family schedules.

[0248] The "means for notifying the user of the specified event information" refers to a means for notifying the user of the specified event information by email, push notification, or the like.

[0249] The present invention is a system for notifying a user of events related to a genre selected by the user based on the user's calendar information and family information. A specific embodiment of this system will be described below.

[0250] Hardware and software used

[0251] This system is implemented using a server, terminals (smartphones and PCs), and various APIs. Specifically, the following hardware and software are used:

[0252] Server: Processes and manages data. Implemented in programming languages ​​such as Python or Java (registered trademark).

[0253] Terminal: A device that a user uses to access the system and input information. This includes smartphones and PCs.

[0254] Google Calendar API: Used to retrieve user calendar information.

[0255] MySQL (registered trademark): A database for managing family information.

[0256] Eventbrite API: Used to collect event information.

[0257] SendGrid API: Used to notify users of event information via email.

[0258] Firebase Cloud Messaging: Used to push event information to users.

[0259] The Python pandas library: Used to filter and match data.

[0260] Specific operation of the system

[0261] 1. The user selects their favorite genre.

[0262] The user accesses the system interface using a terminal and selects a genre (e.g., "art") from a provided genre list. This selection information is sent to the server.

[0263] 2. The server retrieves the user's calendar information.

[0264] The server uses the Google Calendar API to retrieve the user's calendar information, specifically, to read the calendar events using an authentication token to access the user's Google account.

[0265] 3. The server retrieves the family information from the database.

[0266] The server connects to a MySQL database and retrieves the user's family information, which includes the schedules and contacts of all family members.

[0267] 4. The server collects event information related to the selected genre.

[0268] The server uses the Eventbrite API to collect event information related to the selected genre (e.g., "art"), sends API requests, and parses the returned event data.

[0269] 5. The server matches the event information with the calendar information.

[0270] The server uses the Python pandas library to match the collected event information with the user's calendar information, specifically filtering out events that take place at times that don't conflict with family schedules.

[0271] 6. The server notifies the user of the appropriate event.

[0272] The server uses the SendGrid API to notify users of the filtered event information via email, and can also send push notifications using Firebase Cloud Messaging.

[0273] Examples of specific examples and prompts

[0274] For example, if the user selects "art" as a favorite genre and the family has a free Saturday afternoon, the server operates as follows.

[0275] 1. The user selects "Art" on the device.

[0276] 2. The server uses the Google Calendar API to check the user's calendar to see if Saturday afternoon is free.

[0277] 3. The server retrieves the family's schedule from the MySQL database and finds that Saturday afternoon is free.

[0278] 4. The server uses the Eventbrite API to collect art-related events taking place on Saturday afternoons.

[0279] 5. Filter the event information collected by the server using the pandas library to identify events that do not overlap with family schedules.

[0280] 6. The server uses the SendGrid API to notify the user of the identified event information via email and also sends a push notification using Firebase Cloud Messaging.

[0281] Specific prompt examples:

[0282] Get the user's calendar information and notify them of art-related events that are scheduled at times that don't conflict with family schedules.

[0283] In this way, the user can receive information about art-related events that are held at times that do not overlap with family schedules. The flow of the specification process in the third embodiment will be described with reference to FIG.

[0284] Step 1:

[0285] The user selects the genre they like.

[0286] The user accesses the system interface using a terminal and selects a genre (e.g., "art") from a provided genre list. This selection information is sent to the server.

[0287] Input: User genre selection

[0288] Output: Genre information sent to the server

[0289] Step 2:

[0290] The server retrieves the user's calendar information.

[0291] The server uses the Google Calendar API to retrieve the user's calendar information, specifically, to read the calendar events using an authentication token to access the user's Google account.

[0292] Input: User's authentication token

[0293] Output: User's calendar information

[0294] Step 3:

[0295] The server retrieves the family information from the database.

[0296] The server connects to a MySQL database and retrieves the user's family information, which includes the schedules and contacts of all family members.

[0297] Input: Database connection information

[0298] Output: Family information

[0299] Step 4:

[0300] The server collects event information related to the selected genre.

[0301] The server uses the Eventbrite API to collect event information related to the selected genre (e.g., "art"), sends API requests, and parses the returned event data.

[0302] Input: Genre information

[0303] Output: Event information

[0304] Step 5:

[0305] The server matches the event information with the calendar information.

[0306] The server uses the Python pandas library to match the collected event information with the user's calendar information, specifically filtering out events that take place at times that don't conflict with family schedules.

[0307] Input: Event information, calendar information

[0308] Output: Filtered event information

[0309] Step 6:

[0310] The server notifies the user of appropriate events.

[0311] The server uses the SendGrid API to notify users of the filtered event information via email, and can also send push notifications using Firebase Cloud Messaging.

[0312] Input: Filtered event information

[0313] Output: Notification to the user (email or push notification)

[0314] In this way, the user can receive information about art-related events that are held at times that do not overlap with family schedules.

[0315] (Application example 3)

[0316] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0317] In modern households, it is difficult to find events that the whole family can enjoy, and it is also difficult to plan events at times when family schedules do not overlap. Furthermore, there is a lack of ways to efficiently collect event information and notify people at the right time, which makes it easy for events to be missed. There is a need to solve these problems and increase opportunities for family experiences.

[0318] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0319] In this invention, the server includes: means for registering school and work schedules in a calendar; means for enabling the registration by capturing school newsletters and shift schedules by photographing them; means for inputting basic family information; means for the basic family information being address, age, grade, and height, the height of which can be automatically measured using a smartphone; means for receiving an event notification that combines family information and favorite genres in an available schedule on the calendar and allowing reservations; means for suggesting events that the whole family can enjoy in the event notification; means for notifying the user if the event is held at a time that does not overlap with family schedules; means for collecting the event information from an external database; means for notifying the user of the event information; and means for notifying the user by push notification or email notification. This makes it possible to efficiently find events that the whole family can enjoy and notify the user at an appropriate time.

[0320] "Means for registering school and work schedules in a calendar" is a function that allows users to enter school and work schedules into an electronic calendar.

[0321] "Means that allow the registration by taking a photograph of school newsletters or shift schedules" refers to a function that allows you to take a photograph of a notice distributed by school or a workplace shift schedule with your smartphone camera and automatically update the information on the calendar.

[0322] "Means for entering basic family information" is a function for entering basic information such as family member's address, age, grade, height, etc. into the system.

[0323] "The basic information of the family members is their address, age, grade, and height, and height can be automatically measured using a smartphone" refers to a function that inputs the address, age, grade, and height of the family members, and automatically measures height in particular using the smartphone's sensor.

[0324] "A means of receiving event notifications that combine family information and favorite genres in the available time slots on the calendar, allowing reservations to be made" is a function that notifies users of events related to basic family information and genres selected by the user based on available time slots on the calendar, and allows them to make reservations for those events.

[0325] "Means for the event notification to suggest events that the whole family can enjoy" is a function in which the system automatically selects events that the whole family can be interested in and enjoy, and suggests them to the user.

[0326] The "means for notifying if the event is held at a time that does not overlap with family schedules" is a function that selects an event that is held at a time that does not overlap with family schedules and notifies the user of that information.

[0327] The "means for collecting the event information from an external database" is a function for acquiring event information from an external database or API.

[0328] The "means for notifying the user of the event information" is a function for notifying the user of the collected event information.

[0329] "Means for the notification to be made by push notification or email notification" refers to a function that sends event information to users via push notification or email.

[0330] The system for implementing this invention proposes events that the whole family can enjoy based on the user's calendar information and basic family information, and notifies them at the appropriate time. A specific embodiment of this system will be described below.

[0331] System Configuration

[0332] 1. Hardware Configuration

[0333] Smartphone: A device that allows users to input calendar information and receive event notifications. Smartphones are equipped with cameras and sensors, allowing them to take photos of school newsletters and shift schedules, and automatically measure height.

[0334] Server: A central processing unit that collects event information and suggests events based on the user's calendar information and basic family information.

[0335] 2. Software Configuration

[0336] Google Calendar API: An API for obtaining a user's calendar information.

[0337] External Database API: An API for collecting event information.

[0338] Notification system: A system for sending push notifications and email notifications.

[0339] Data processing and calculation

[0340] 1. Get calendar information

[0341] The server uses the Google Calendar API to retrieve the user's calendar information, which allows it to determine the user's free time.

[0342] 2. Enter basic family information

[0343] Users use their smartphones to enter basic information about their family members, such as their address, age, grade, and height, which is then automatically measured using the smartphone's sensors.

[0344] 3. Collecting event information

[0345] The server uses an external database API to collect event information related to the genre selected by the user.

[0346] 4. Event Proposal

[0347] The server selects events that the whole family can enjoy based on the collected event information, the user's calendar information, and basic family information. The selected events are held during the user's free time.

[0348] 5. Event Notifications

[0349] The server notifies the user of the selected event information via push notification or email notification.

[0350] Specific examples

[0351] For example, if a user selects "art" as their favorite genre and has free time on their family calendar, the system will notify them of events such as:

[0352] Event name: "Monet Exhibition"

[0353] Date and time: October 15, 2023, 14:00 - 16:00

[0354] Prompt Sentence Examples

[0355] An example of a prompt sentence to input to the generative AI model is as follows:

[0356] I'm developing an application that suggests events that the whole family can enjoy based on the user's free time on their calendar. I'd like to implement a function that collects event information related to the user's favorite genre (e.g., art, music, sports, etc.) and notifies them of events that are held at times that don't conflict with other family schedules. Could you please give me a specific example of a program and the process?

[0357] In this way, the system can efficiently find events that the whole family can enjoy based on the user's calendar information and basic family information, and notify them at the appropriate time.

[0358] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0359] Step 1:

[0360] A user uses a smartphone to add school or work appointments to their calendar.

[0361] Input: School newsletters, shift schedule photos, manually entered schedule information

[0362] Data processing: Images taken with a smartphone camera are converted into text data using OCR (optical character recognition) technology and registered in the calendar.

[0363] Output: Schedule information registered in the calendar

[0364] Step 2:

[0365] A user uses a smartphone to enter basic information about their family.

[0366] Input: Address, age, grade, height (automatically measured using smartphone sensor)

[0367] Data processing: The entered information is saved in a database, and height is automatically measured using the smartphone sensor if necessary.

[0368] Output: Basic information of family members stored in the database

[0369] Step 3:

[0370] The server retrieves the user's calendar information using the Google Calendar API.

[0371] Input: User's Google Calendar account information

[0372] Data processing: Using the Google Calendar API, schedule information is obtained from the user's calendar and free time is identified.

[0373] Output: User's calendar free / busy information

[0374] Step 4:

[0375] The server uses an external database API to collect event information related to the genre selected by the user.

[0376] Input: User-selected genre (e.g., art, music, sports)

[0377] Data processing: Using an external database API, event information related to the specified genre is obtained.

[0378] Output: Collected event information

[0379] Step 5:

[0380] Based on the event information collected by the server, the user's calendar information, and basic family information, the system selects events that the whole family can enjoy.

[0381] Input: Collected event information, user calendar availability, basic family information

[0382] Data processing: Event information is compared with available time on the calendar and basic family information to select events that the whole family can enjoy.

[0383] Output: Selected event information

[0384] Step 6:

[0385] The server notifies the user of the selected event information via push notification or email notification.

[0386] Input: Selected event information, user contact information

[0387] Data processing: The selected event information is converted into a push notification or email notification format and sent to the user.

[0388] Output: Event notification sent to the user

[0389] In this way, the system can efficiently find events that the whole family can enjoy based on the user's calendar information and basic family information, and notify them at the appropriate time.

[0390] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0391] "Example 1"

[0392] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system has the ability to recognize emotions from a user's tone of voice, facial expressions, behavioral patterns, etc. Specifically, the system analyzes the tone of voice as the user speaks through the smartphone's microphone and recognizes emotions such as joy, anger, sadness, and surprise. It also captures the user's facial expressions through the smartphone's camera and recognizes emotions from smiles, frowns, surprise, etc. Furthermore, it uses the smartphone's sensors to analyze the user's behavioral patterns and infers emotions from activity levels, movement patterns, usage patterns, etc.

[0393] "Example 2"

[0394] A function is also provided to adjust the priority of event notifications based on the emotions recognized by the emotion engine. For example, if the user is expressing joy, event notifications in the user's preferred genre are displayed preferentially. Conversely, if the user is expressing anger or sadness, event notifications in the user's preferred genre are displayed less. This makes it possible to provide optimal event notifications according to the user's emotions. "Example 3"

[0395] The emotion engine also has the ability to learn changes in a user's emotions and reflect the results in adjusting the priority of event notifications. For example, if a user often expresses joy after seeing an event notification for a specific genre, the engine will learn to prioritize displaying event notifications for that genre. This allows the system to more accurately understand the relationship between a user's emotions and event notifications, making it possible to provide the most appropriate event notifications for the user.

[0396] The processing flow of each embodiment will be described below.

[0397] "Example 1"

[0398] Step 1: The user speaks through the smartphone microphone.

[0399] Step 2: The system analyzes the tone of the voice and recognizes emotions such as joy, anger, sadness, and surprise.

[0400] Step 3: Capture the user's facial expression using the smartphone camera.

[0401] Step 4: The system recognizes emotions such as smiles, frowns, and surprised expressions.

[0402] Step 5: Use smartphone sensors to analyze user behavior patterns and infer emotions based on activity levels, movement patterns, usage patterns, etc.

[0403] "Example 2"

[0404] Step 1: Adjust the priority of event notifications based on the emotions recognized by the emotion engine.

[0405] Step 2: If the user is expressing joy, notifications of upcoming events in the user's preferred genre are displayed preferentially.

[0406] Step 3: If the user expresses anger or sadness, subtly display event notifications in the user's preferred genre.

[0407] "Example 3"

[0408] Step 1: The emotion engine learns the user's emotional changes.

[0409] Step 2: If a user often expresses joy after seeing event notifications of a specific genre, the system learns to prioritize displaying event notifications of that genre. Step 3: The system more accurately understands the relationship between user emotions and event notifications, and provides the most appropriate event notifications for the user.

[0410] Example 1

[0411] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0412] Conventional calendar registration systems require users to manually input schedules, which is time-consuming and prone to input errors. It is also difficult to provide services that take users' feelings into consideration, making it difficult to improve user satisfaction. Furthermore, the lack of event notification and reservation functions that utilize basic family information makes managing the schedules of the entire family cumbersome.

[0413] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0414] In this invention, the server includes means for registering school and work schedules in a calendar, means for enabling the registration by capturing school newsletters and shift schedules by photographing them, means for inputting basic information about family members, the basic information about the family members being address, age, grade, and height, the height of which can be automatically measured using a mobile information terminal, means for receiving event notifications that combine family information and favorite genres in the available schedules on the calendar and allowing reservations to be made, means for extracting text from images, means for recognizing emotions from voice, means for recognizing emotions from facial expressions, and means for inferring emotions from behavioral patterns.This allows users to easily and accurately register schedules in the calendar, enables services that take emotions into consideration, and makes it easier to manage the schedules of the entire family.

[0415] "Means for registering school and work schedules in a calendar" is a function that allows users to automatically register school and work schedules in an electronic calendar.

[0416] "The means by which the registration can be made by taking a photograph of the school newsletter or shift schedule" is a function that allows the user to take a photograph of the school newsletter or shift schedule and automatically import that information into the electronic calendar.

[0417] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[0418] "The basic information of the family members is address, age, grade, and height, and height can be automatically measured using a mobile information terminal" refers to a function that includes address, age, grade, and height as basic information of the family members, and in particular automatically measures height using a mobile information terminal.

[0419] "A means to receive event notifications that combine family information and favorite genres based on available times on the calendar, and make reservations" is a function that allows you to receive event notifications that combine basic family information and favorite genres based on available times on the calendar, and to make reservations for those events.

[0420] The "means for extracting text from an image" is a function for automatically extracting text information from a captured image.

[0421] "Means for recognizing emotions from voice" is a function for analyzing the user's voice and recognizing emotions from that voice.

[0422] "Means for recognizing emotions from facial expressions" is a function that analyzes the user's facial expressions and recognizes emotions from those expressions.

[0423] "Means for inferring emotions from behavioral patterns" is a function that analyzes user behavioral data and infers emotions from those behavioral patterns.

[0424] This invention is a system that allows users to easily register school and work schedules on a calendar and provides services that recognize the user's emotions. This system is realized by combining a mobile information terminal such as a smartphone with a server.

[0425] 1. Extract text from images and add it to your calendar

[0426] Hardware and software used

[0427] Hardware: Smartphone camera

[0428] Software: OCR software (e.g., Google Cloud Vision API), calendar app (e.g., Google Calendar)

[0429] Processing flow

[0430] A user uses their smartphone camera to take a photo of a school newsletter or shift schedule. The device sends the image to a server, which uses OCR software to extract text from the image. The server then analyzes the extracted text and registers the schedule information in a calendar app.

[0431] Specific examples

[0432] Users take a photo of their school report card and the system registers the information in their calendar.

[0433] Example prompt: "Please take a photo of your school report card and add it to your calendar."

[0434] 2. Emotion Recognition by Emotion Engine

[0435] Hardware and software used

[0436] Hardware: Smartphone microphone, camera, sensors (e.g. accelerometer, GPS)

[0437] Software: Voice analysis software (e.g., IBM Watson (registered trademark) Tone Analyzer), facial expression analysis software (e.g., Microsoft (registered trademark) Azure (registered trademark) Face API), behavior analysis software

[0438] Processing flow

[0439] When a user speaks through the smartphone's microphone, the device sends the recorded voice data to a server, which uses voice analysis software to recognize emotions from the voice. When the user takes a photo of their facial expression using the smartphone's camera, the device sends the facial image to the server, which then uses facial expression analysis software to recognize emotions from the facial expression. In addition, the device uses the smartphone's sensors to collect user behavior data and sends it to the server. The server uses behavior analysis software to infer emotions from behavioral patterns.

[0440] Specific examples

[0441] The user speaks into the smartphone and the system recognizes the emotion of joy from the tone of voice.

[0442] Example prompt: "Speak into your smartphone and identify your emotions."

[0443] 3. Family Information and Event Notifications

[0444] Hardware and software used

[0445] Hardware: Smartphone

[0446] Software: Calendar app, event notification system

[0447] Processing flow

[0448] Users enter basic family information (address, age, grade, height) into the system. Height can be automatically measured using a smartphone. Based on available calendar availability, users will receive notifications of upcoming events that combine their family information with their favorite genres, and can make reservations for those events.

[0449] Specific examples

[0450] Users enter basic family information, and the system sends event notifications, such as art and programming, based on calendar availability.

[0451] Example prompt: "Enter basic family information to receive event notifications."

[0452] In this way, the system of the present invention allows users to accurately register events on a calendar without much effort, makes it possible to provide services that take emotions into consideration, and makes it easier to manage the schedules of all family members.

[0453] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0454] Extract text from images and add it to your calendar

[0455] Processing Steps

[0456] Step 1:

[0457] Users use their smartphone camera to take photos of school newsletters and shift schedules.

[0458] Specific operation: The user launches the camera app, points the camera at an object, and presses the shutter button.

[0459] Input: School newsletter or shift schedule image

[0460] Output: Captured image data

[0461] Step 2:

[0462] The device sends the captured image to the server.

[0463] Specific operation: The device uploads image data to the server using an HTTP POST request.

[0464] Input: Photographed image data

[0465] Output: Image data sent to the server

[0466] Step 3:

[0467] The server extracts the text from the image.

[0468] Specific operation: The server sends image data to OCR software (e.g., Google Cloud Vision API) and receives the returned text data.

[0469] Input: Image data sent to the server

[0470] Output: Extracted text data

[0471] Step 4:

[0472] The server parses the text and generates the schedule.

[0473] What it does: The server uses natural language processing (NLP) algorithms to extract information from the text, such as dates, times, and event names.

[0474] Input: Extracted text data

[0475] Output: Generated schedule information

[0476] Step 5:

[0477] The server registers the event in the calendar.

[0478] Specific operation: The server uses a calendar API (e.g., Google Calendar API) to add the event information to the calendar.

[0479] Input: Generated schedule information

[0480] Output: Events registered in the calendar

[0481] Emotion recognition by emotion engine

[0482] Processing Steps

[0483] Step 1:

[0484] The user inputs voice.

[0485] Specific behavior: The user launches a voice input app and speaks into the microphone.

[0486] Input: User's voice

[0487] Output: Recorded audio data

[0488] Step 2:

[0489] The terminal transmits the voice data to the server.

[0490] Specific operation: The device uploads audio data to the server using an HTTP POST request.

[0491] Input: Recorded audio data

[0492] Output: Audio data sent to the server

[0493] Step 3:

[0494] The server recognizes emotions from the voice.

[0495] Specific operation: The server sends voice data to voice analysis software (e.g., IBM Watson Tone Analyzer) and receives the returned emotion data.

[0496] Input: Audio data sent to the server

[0497] Output: Recognized emotion data

[0498] Step 4:

[0499] The user captures their facial expression.

[0500] Specific action: The user launches the camera app and takes a picture of their face.

[0501] Input: User's facial expression image

[0502] Output: Photographed facial expression image data

[0503] Step 5:

[0504] The terminal transmits the facial expression data to the server.

[0505] Specific operation: The device uploads image data to the server using an HTTP POST request.

[0506] Input: Photographed facial expression image data

[0507] Output: Facial expression image data sent to the server

[0508] Step 6:

[0509] The server recognizes emotions from facial expressions.

[0510] Specific operation: The server sends facial expression images to facial expression analysis software (e.g., Microsoft Azure Face API) and receives the returned emotional data.

[0511] Input: Facial expression image data sent to the server

[0512] Output: Recognized emotion data

[0513] Step 7:

[0514] The device collects behavioral data and sends it to a server.

[0515] Specific operation: The device periodically collects sensor data and uploads it to the server via an HTTP POST request.

[0516] Input: User behavior data (e.g., accelerometer, GPS data)

[0517] Output: Behavioral data sent to the server

[0518] Step 8:

[0519] The server infers emotions from behavioral patterns.

[0520] How it works: The server analyzes the behavioral data and applies an algorithm to infer emotions.

[0521] Input: Behavioral data sent to the server

[0522] Output: Inferred emotion data

[0523] Family Information and Event Notifications

[0524] Processing Steps

[0525] Step 1:

[0526] The user enters basic information about their family.

[0527] Specific behavior: The user launches the app and enters information such as address, age, grade, and height.

[0528] Input: Basic information of family members (address, age, grade, height)

[0529] Output: Basic information data of the entered family

[0530] Step 2:

[0531] The device will automatically measure your height.

[0532] Specific operation: The user measures their height using a smartphone.

[0533] Input: User's height measurement data

[0534] Output: Measured height data

[0535] Step 3:

[0536] The server checks the availability of the calendar.

[0537] Specific operation: The server uses the calendar API to obtain free time.

[0538] Input: Calendar free / busy data

[0539] Output: Retrieved free / busy data

[0540] Step 4:

[0541] The server generates an event notification that combines family information and favorite genres.

[0542] Specific operation: The server generates an event notification based on family information and the user's preferred genres.

[0543] Input: Family information data, user's favorite genre data

[0544] Output: Generated event notification data

[0545] Step 5:

[0546] The server sends an event notification to the user.

[0547] Specific operation: The server sends event notification data to the user's terminal.

[0548] Input: Generated event notification data

[0549] Output: Event notification sent to the user

[0550] Step 6:

[0551] A user books an event.

[0552] What happens: The user receives a notification and takes action to book the event.

[0553] Input: Event notification

[0554] Output: Scheduled event data

[0555] (Application example 1)

[0556] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0557] Conventional calendar registration systems required users to manually enter schedules, which was time-consuming and prone to input errors. Furthermore, the service provided did not take user feelings into consideration, and efforts were not made to improve the user experience. Furthermore, it was difficult to centrally manage the schedules of all family members, resulting in missed events and missed opportunities.

[0558] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes: means for registering school and work schedules in a calendar; means for enabling the registration by capturing school newsletters and shift schedules by photographing them; means for inputting basic family information; means for the basic family information being address, age, grade, and height, with height being automatically measured using a smartphone; means for receiving event notifications that combine family information and favorite genres in the free schedules on the calendar and allowing reservations; means for recognizing the user's emotions; means for recognizing the emotions using the smartphone's camera, microphone, and sensors; and means for suggesting appropriate products and services based on the recognized emotions. This allows the user to easily and accurately register schedules, enables the provision of services based on emotions, centrally manages the schedules of all family members, and prevents missing events and missing out on experience opportunities.

[0559] "Means for registering school and work schedules in a calendar" is a function that allows users to enter school and work schedules into an electronic calendar.

[0560] "The means by which the registration can be made by taking a photograph of the school newsletter or shift schedule" refers to a function that allows a user to take a photograph of the school newsletter or shift schedule with a smartphone camera, extract text from the image, and automatically register the schedule in the calendar.

[0561] "Means for entering basic family information" refers to a function for entering basic information such as family member's address, age, grade, height, etc. into the system.

[0562] "The basic information of the family member is the address, age, grade, and height, and the height can be automatically measured using a smartphone" refers to a function that includes the address, age, grade, and height as basic information of the family member, and in particular automatically measures height using a smartphone sensor.

[0563] "A means to receive event notifications that combine family information and favorite genres with available time on the calendar, allowing reservations" is a function that sends event notifications that take into account basic family information and favorite genres based on available time on the calendar, and allows reservations for those events.

[0564] "Means for recognizing user emotions" refers to a function for identifying emotions from the user's tone of voice, facial expressions, behavioral patterns, etc.

[0565] "Means for performing emotion recognition using the smartphone's camera, microphone, and sensors" refers to a function for recognizing a user's emotions using the smartphone's camera, microphone, and sensors.

[0566] "Means for suggesting appropriate products and services based on recognized emotions" is a function that analyzes the user's emotions and suggests appropriate products and services based on the results.

[0567] The system for implementing this invention allows a user to register school or work schedules in a calendar, and suggests appropriate products and services through emotion recognition. Specific embodiments of the system will be described below.

[0568] System configuration

[0569] The system consists of a smartphone, a server, and related software. The smartphone is equipped with a camera, microphone, and sensors, and these hardware components are used to recognize the user's emotions. The server processes and stores the data, and provides appropriate notifications and suggestions to the user.

[0570] Program processing

[0571] 1. Register an appointment:

[0572] When a user takes a picture of a school newsletter or shift schedule with their smartphone camera, the text is extracted from the image. The extracted text is sent to the server and automatically registered in the calendar. This process uses OCR (Optical Character Recognition) technology.

[0573] 2. Enter family information:

[0574] Users enter basic information such as their family members' addresses, ages, grades, and heights via their smartphones. Heights are automatically measured using the smartphone's sensors. This information is then stored on a server.

[0575] 3. Emotion recognition:

[0576] The smartphone's camera, microphone, and sensors are used to collect the user's facial expressions, tone of voice, and behavioral patterns. This data is input into an emotion recognition model to identify the user's emotions. Emotion recognition uses software such as OpenCV (camera image processing), Keras (emotion recognition model), and SpeechRecognition (voice recognition).

[0577] 4. Product or service suggestions:

[0578] Based on the recognized emotion, the server will suggest appropriate products and services to the user. For example, if the user is feeling stressed, it will suggest relaxation-related products and services. This suggestion is based on the user's available time on the calendar.

[0579] Specific examples

[0580] For example, if a user takes a photo of their school report card with their smartphone, the system will extract the text from the image and add it to their calendar. Or, if a user says, "I want to know more about this product," the smartphone will recognize their voice and display relevant information.

[0581] Prompt Sentence Examples

[0582] Example prompts to input to the generative AI model:

[0583] "We want to develop an application that recognizes a customer's facial expressions and voice while they are looking at products, and suggests appropriate products and services. Please generate specific suggestions based on the following information."

[0584] In this way, a system is realized that allows users to register schedules accurately and hassle-free, enables services to be provided based on emotions, centrally manages the schedules of all family members, and prevents events from being missed or experiences from being missed.

[0585] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0586] Step 1:

[0587] A user takes a picture of a school newsletter or shift schedule with their smartphone camera. The input is the captured image. The smartphone uses OCR (Optical Character Recognition) technology to extract text from the image. The output is the extracted text data.

[0588] Step 2:

[0589] The smartphone sends the extracted text data to the server. The input is the text data. The server analyzes the received text data and registers the event in the calendar. The output is updated calendar data.

[0590] Step 3:

[0591] The user inputs basic information about their family members (address, age, grade, height) via their smartphone. The input is the basic information entered by the user. The smartphone automatically measures the height using a sensor. The output is the measured height data and other basic information.

[0592] Step 4:

[0593] The smartphone sends basic family information to the server. The input is the basic family information. The server stores the received information in a database. The output is the stored family information data.

[0594] Step 5:

[0595] While a user is using a smartphone, the smartphone's camera, microphone, and sensors collect the user's facial expressions, tone of voice, and behavioral patterns. The input is the collected facial expression data, voice data, and behavioral data. The smartphone inputs this data into an emotion recognition model to identify the user's emotions. The output is the identified emotion data.

[0596] Step 6:

[0597] The smartphone sends the identified emotion data to the server. The input is the emotion data. The server analyzes the received emotion data and suggests appropriate products or services to the user. The output is information about the suggested products or services.

[0598] Step 7:

[0599] The server checks the free time on the user's calendar and generates an event notification by combining family information and favorite genres. The input is calendar data, family information, and favorite genres. The server sends the generated event notification to the user. The output is an event notification.

[0600] Step 8:

[0601] The user checks the received event notification and reserves the event that interests them. The input is the event notification. The smartphone sends the user's reservation information to the server. The output is the reserved event information.

[0602] In this way, a system is realized that allows users to accurately register schedules without hassle, enables services to be provided based on emotions, manages the schedules of all family members in a unified manner, and prevents events from being missed or opportunities to experience things being lost.

[0603] Example 2

[0604] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0605] Conventional calendar systems require users to manually enter school and work schedules, which is time-consuming, and they also have the problem of not being able to adjust the priority of event notifications based on basic family information or emotions. Furthermore, they are unable to provide optimal event notifications based on the user's emotions, making it difficult to maximize the user's experience.

[0606] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0607] In this invention, the server includes means for registering school and work schedules in a calendar, means for enabling the registration by capturing school newsletters and shift schedules by photographing them, means for inputting basic information about family members, the basic information about the family members being address, age, grade, and height, the height being automatically measurable using a mobile information terminal, means for recognizing the user's emotions using an emotion engine, means for adjusting the priority of event notifications based on the recognized emotions, and means for receiving event notifications that combine family information and favorite genres into available calendar schedules and enabling reservations to be made. This reduces the user's effort and makes it possible to provide optimal event notifications according to their emotions.

[0608] "Means for registering school and work schedules in a calendar" is a function that allows a user to input and manage school and work schedules in a digital calendar.

[0609] "Means for enabling the registration by photographing the school newsletter or shift schedule" refers to a function that allows the user to photograph the school newsletter or shift schedule, analyze the image data, and automatically register the schedule information on the calendar.

[0610] The "means for inputting basic family information" is a function that allows the user to input basic information such as the address, age, grade, and height of family members into the system.

[0611] "The basic information of the family members is address, age, grade, and height, and height can be automatically measured using a mobile information terminal" means a function that includes address, age, grade, and height as basic information of the family members, and in particular automatically measures height using a mobile information terminal.

[0612] The "means for recognizing user emotions using an emotion engine" is a function for analyzing user behavior data and input data to identify the user's emotions.

[0613] The "means for adjusting the priority of event notifications based on the recognized emotions" is a function for adjusting the display order and importance of event notifications based on the user's emotions recognized by the emotion engine.

[0614] "A means for receiving event notifications that combine family information and favorite genres with available time slots on a calendar, allowing reservations" is a function that sends event notifications that take into account basic family information and the user's favorite genres based on available time slots on a calendar, allowing the user to reserve the event.

[0615] MODE FOR CARRYING OUT THE INVENTION

[0616] This invention is a system that allows a user to input basic information about their family, recognizes the user's emotions using an emotion engine, and adjusts the priority of event notifications based on the emotions. Specific embodiments of this system are described below.

[0617] Enter basic family information

[0618] The user logs into the system and moves to a screen where they can enter basic family information. Here, the user enters information such as address, age, grade, and height. Height, in particular, can be measured automatically using a mobile information terminal (smartphone). The user launches the smartphone's camera app and uses AR technology to overlay a tape measure on the camera image. This function can be realized using, for example, Apple's ARKit or Google's ARCore.

[0619] Emotion recognition

[0620] The server collects user behavioral data and input data, including social media posts and activity history within the system. The server uses an emotion engine to analyze this data and identify the user's emotions. For example, it determines whether the user is feeling "happy" or "sad."

[0621] Adjusting the priority of event notifications

[0622] The server receives the results of the emotion engine and prioritizes event notifications based on the user's emotions. For example, if the user is feeling happy, event notifications such as music concerts and movie premieres will be displayed first. Conversely, if the user is feeling sad, notifications for relaxing yoga classes and quiet reading groups will be displayed less. The server then sends event notifications to the user based on this priority.

[0623] Specific examples

[0624] For example, when a user enters basic information about their family members, they can use their smartphone camera to measure their heights. The user launches the camera app and uses AR technology to overlay a tape measure onto the camera image. This automatically measures the heights of their family members and enters that data into the system.

[0625] For example, if a user posts on social media, "I'm so happy today!", the server analyzes the content of the post and determines that the user is feeling happy. Based on this, the server prioritizes displaying event notifications such as music concerts and movie premieres.

[0626] Example prompts for generative AI models

[0627] Below are some example prompts to input to a generative AI model:

[0628] Describe the system's steps, where users enter basic family information. Specifically, explain how the smartphone camera and AR technology are used to measure height. Also, explain how the emotion engine prioritizes event notifications based on the emotions it recognizes.

[0629] Using this prompt, the generative AI model can generate a detailed description of the system.

[0630] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0631] Step 1:

[0632] A user logs in to the system. The user enters their user ID and password on the login screen, and if authentication is successful, they are taken to the main screen. The input is the user ID and password, and the output is the authentication result. If authentication is successful, the user can proceed to the next step.

[0633] Step 2:

[0634] The user is taken to a screen where they can enter basic information about their family. Here, they enter information such as address, age, grade, height, etc. The input is basic information such as address, age, grade, height, etc., and the output is that this information is saved in a database. Once the user has completed the input, they can proceed to the next step.

[0635] Step 3:

[0636] A user uses the smartphone camera to measure the height of a family member. The user launches the camera app and uses AR technology to overlay a tape measure on the camera image. The input is the camera image, and the output is the measured height data. The device sends this data to the system and stores it in a database.

[0637] Step 4:

[0638] The server collects user behavioral data and input data. This includes social media posts and activity history within the system. The input is user behavioral data and input data, and the output is that this data is stored in a database. The server prepares this data for analysis.

[0639] Step 5:

[0640] The server uses an emotion engine to analyze the collected data and identify the user's emotion. The input is the collected behavioral and input data, and the output is the user's emotional state. For example, it determines whether the user is feeling "happy" or "sad."

[0641] Step 6:

[0642] The server receives the results of the emotion engine and prioritizes event notifications based on the user's emotions. The input is the user's emotional state, and the output is prioritized event notifications. For example, if the user is feeling "happy," event notifications such as music concerts and movie premieres will be displayed with priority.

[0643] Step 7:

[0644] The server sends event notifications to users according to their priorities. The input is a prioritized event notification, and the output is a notification sent to the user's device. The user can receive this notification, check the details of the event, and make a reservation.

[0645] (Application example 2)

[0646] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0647] In conventional calendar systems, family schedules and event notifications were managed separately, making it impossible to provide optimal event notifications based on the user's emotions. It was also time-consuming to manually enter basic family information, and measuring height was particularly tedious. Furthermore, the priority of event notifications was not adjusted according to the user's emotions, which meant that event notifications were not tailored to the user's interests.

[0648] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0649] In this invention, the server includes: means for registering school and work schedules in a calendar; means for enabling the registration by capturing school newsletters and shift schedules by photographing them; means for inputting basic family information, the basic family information being address, age, grade, and height, with height being automatically measured using a smartphone; means for receiving event notifications that combine family information and favorite genres in the free schedules on the calendar and allowing reservations; means for recognizing the user's emotions using the smartphone's camera and microphone; and means for adjusting the priority of event notifications based on the recognized emotions. This makes it possible to provide optimal event notifications according to the user's emotions, simplify the input of basic family information, and provide event notifications that match the user's interests and concerns.

[0650] "Means for registering school and work schedules on a calendar" refers to a function that allows users to enter and manage school and work schedules on an electronic calendar.

[0651] "The means by which the registration can be made by taking a photograph of the school newsletter or shift schedule" refers to a function that allows a user to take a photograph of the school newsletter or shift schedule with a smartphone camera and automatically register that information in the calendar.

[0652] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[0653] "The basic information of the family members is their address, age, grade, and height, and height can be automatically measured using a smartphone" refers to a function that includes basic information of the family members, such as address, age, grade, and height, and in particular, automatically measures height using a smartphone's camera and AR technology.

[0654] "A means to receive event notifications that combine family information and favorite genres with available time on the calendar, allowing reservations" is a function that allows users to receive and reserve event notifications that take into account basic family information and the user's favorite genres based on available time on the calendar.

[0655] "Means for recognizing a user's emotions using a smartphone's camera or microphone" refers to a function that uses a smartphone's camera or microphone to analyze a user's facial expressions and voice and recognize their emotions.

[0656] The "means for adjusting the priority of event notifications based on the recognized emotions" is a function that recognizes the user's emotions and adjusts the display order and importance of event notifications according to those emotions.

[0657] The system for implementing this invention has the following configuration. First, the user uses a smartphone to enter basic family information. This basic information includes address, age, grade, and height, and height in particular is automatically measured using the smartphone's camera and AR technology. This allows the user to enter accurate information without any effort.

[0658] Next, the smartphone's camera and microphone are used to recognize the user's emotions. An emotion recognition engine (e.g., Microsoft Azure Cognitive Services) analyzes the user's facial expressions and voice to recognize emotions such as joy, anger, and sadness in real time. This emotion data is stored in a cloud database (e.g., Firebase Realtime Database).

[0659] In addition, event notifications are sent based on the user's free time on the calendar, combining family information and the user's favorite genres. The event notification system (e.g., Firebase Cloud Messaging) adjusts the priority of notifications according to the user's emotions. For example, if the user is expressing joy, event notifications in the user's favorite genres are displayed preferentially. Conversely, if the user is expressing anger or sadness, event notifications in the user's favorite genres are displayed less.

[0660] For example, when a user is at a shopping mall with their family, they can use their smartphone camera to measure their family members' heights and input the data into the system. If the user shows signs of joy while shopping, the system will prioritize notifications for their favorite fashion events.

[0661] An example of a prompt sentence is as follows:

[0662] Users are asked to enter basic information about their family members. The smartphone camera is used to measure their heights and enter this information into the system. The smartphone's camera and microphone are then used to recognize the user's emotions, and event notification priorities are adjusted based on the emotions. For example, if a user expresses joy, event notifications in the user's preferred genres will be displayed first.

[0663] In this way, it is possible to provide optimal event notifications according to the user's emotions, simplify the input of basic family information, and provide event notifications that match the user's interests and concerns.

[0664] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0665] Step 1:

[0666] The user uses a smartphone to enter basic information about their family members. The information entered includes address, age, grade, and height. In particular, height is measured automatically using the smartphone's camera and AR technology. A full-body photo of the family member is taken with the smartphone camera, and their height is measured using AR technology, and the data is sent to a cloud database. Input: Basic information about the family members (address, age, grade, height). Output: Basic information about the family members saved in the cloud database.

[0667] Step 2:

[0668] The device uses the smartphone's camera and microphone to recognize the user's emotions. Using an emotion recognition engine (e.g., Microsoft Azure Cognitive Services), it analyzes the user's facial expressions and voice in real time to recognize emotions such as joy, anger, and sadness. The recognized emotion data is stored in a cloud database. Input: Data on the user's facial expressions and voice. Output: Emotion data stored in the cloud database.

[0669] Step 3:

[0670] The server retrieves available calendar appointments and generates an event notification that combines family information and the user's favorite genres. An event notification system (e.g., Firebase Cloud Messaging) is used to adjust the priority of notifications according to the user's emotions. For example, if a user expresses joy, event notifications in the user's favorite genres are displayed preferentially. Input: Available calendar appointments, family information, user's favorite genres, emotion data. Output: Event notification with adjusted priority.

[0671] Step 4:

[0672] The device displays the event notification with the adjusted priority to the user. The user can check the notification, select the event of interest and make a reservation. The reservation information is stored in the cloud database and reflected in the calendar. Input: Event notification with adjusted priority. Output: Reservation information stored in the cloud database, event reflected in the calendar.

[0673] Step 5:

[0674] The server periodically analyzes the user's emotional data and event participation history, and generates new event notifications based on the user's interests. A generative AI model is used to predict the user's preferences and suggest the most suitable events. Input: Emotional data, event participation history. Output: New event notifications.

[0675] In this way, it is possible to provide optimal event notifications according to the user's emotions, simplify the input of basic family information, and provide event notifications that match the user's interests and concerns.

[0676] Example 3

[0677] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0678] With conventional calendar systems, users had to individually collect event information and manually register it in their calendars, which led to missed or duplicated events. Furthermore, event notifications were not based on the user's emotions or preferences, leading to low user satisfaction. Furthermore, it was difficult to provide event notifications that took into account the schedules of all family members, often limiting opportunities for family experiences.

[0679] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0680] In this invention, the server includes: a means for registering school and work schedules in a calendar; a means for enabling the registration by capturing images of documents and tables; a means for inputting basic family information; the basic family information including address, age, grade, and height, with height automatically measured using a mobile device; a means for receiving event notifications that combine family information and favorite genres in free calendar appointments and allowing reservations; a means for analyzing user emotion data and adjusting the priority of event notifications; and a means for the emotion data to include feedback and emotion data from past notifications. This prevents users from missing events and allows them to receive optimal event notifications that take into account the schedules of all family members. Furthermore, since event notifications are provided based on the user's emotions and preferences, user satisfaction is improved.

[0681] "Means for registering school or work schedules in a calendar" is a function that allows users to enter school or work schedules into an electronic calendar.

[0682] "Means possible by capturing images of documents and tables" refers to a function that captures images of paper documents and tables with a camera, analyzes the images, and captures them as digital data.

[0683] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[0684] "Means that can be measured automatically using a mobile device" refers to a function that automatically measures physical data such as height using a mobile device such as a smartphone or tablet.

[0685] "A means of receiving event notifications that combine family information and favorite genres with available time on the calendar, allowing reservations" is a function that notifies users of event information based on available time on the user's calendar, taking into account family information and the user's favorite genres, and allows them to reserve the event.

[0686] "Means for analyzing user emotional data and adjusting the priority of event notifications" refers to a function that analyzes a user's past emotional data and adjusts the display order and importance of event notifications based on the results.

[0687] "Feedback and emotion data on past notifications" refers to the emotion and evaluation data that a user has shown on notifications they have received in the past.

[0688] This invention is a system that provides event notifications that combine family information and favorite genres selected by the user based on the free time in the user's calendar. It also has a function that uses an emotion engine to learn changes in the user's emotions and reflects the results in adjusting the priority of event notifications.

[0689] Hardware and software used

[0690] Server: Cloud computing service (e.g., Amazon Web Services)

[0691] Database: Relational database service (e.g. Amazon RDS)

[0692] Emotion engine: Natural language processing API (e.g., Google Cloud Natural Language API)

[0693] Event information collection: Web scraping tools (e.g., Beautiful Soup, Scrapy)

[0694] Notification system: Cloud messaging service (e.g., Firebase Cloud Messaging)

[0695] Program processing

[0696] Retrieving User Information

[0697] The server retrieves the user's calendar information and family information from the database. Specifically, it calls the user's Google Calendar API to retrieve event information. It also retrieves family information by executing a query from the database.

[0698] Collecting event information

[0699] The server uses a web scraping tool to collect related event information based on the user's favorite genre. For example, if the user selects "art," the server collects art-related event information from the official museum website and event information sites.

[0700] Event filtering

[0701] The server compares the collected event information with the available times on the user's calendar and filters out events that are held at times that do not overlap with family schedules. For example, it selects art exhibitions that are held at times when the user's calendar is available.

[0702] Priority adjustment by emotion engine

[0703] The server uses a natural language processing API to analyze users' past emotional data and learn how their emotions change in response to events of a specific genre. For example, if a user previously expressed joy when receiving notifications about an art exhibition, the server will prioritize notifications for events of that genre.

[0704] Sending notifications

[0705] The server then uses a cloud messaging service to notify users of the filtered event information, for example by sending a notification to their smartphone saying, "There's an art exhibition this weekend."

[0706] Specific examples

[0707] User A has selected "art" as his favorite genre.

[0708] The server calls User A's Google Calendar API and confirms that he is free on Saturday afternoon this weekend.

[0709] The server uses Beautiful Soup to collect information about art exhibitions being held this weekend from the museum's official website.

[0710] The server filters out art exhibitions that are taking place at times that don't conflict with family schedules.

[0711] Using a natural language processing API, the server learns that User A expressed joy when receiving notifications about art exhibitions in the past.

[0712] The server uses a cloud messaging service to send a notification to User A saying, "There's an art exhibition this weekend."

[0713] Prompt Sentence Examples

[0714] Obtain the user's calendar information and family information, and collect relevant event information based on the user's favorite genres. Compare the collected event information with the user's available calendar schedules and filter out events that take place at times that do not overlap with family schedules. Furthermore, use an emotion engine to learn changes in the user's emotions and reflect the results in adjusting the priority of event notifications. Finally, notify the user of the filtered event information.

[0715] The flow of the identification process in the third embodiment will be described with reference to FIG.

[0716] Step 1:

[0717] Retrieving User Information

[0718] The server retrieves the user's calendar information and family information from the database. Specifically, it calls the user's Google Calendar API to retrieve event information. It also retrieves family information by executing a query from the database.

[0719] Input: User ID, Google Calendar API token

[0720] Data processing: Use the Google Calendar API to obtain schedule information and query family information from the database.

[0721] Output: User's calendar information, family information

[0722] Step 2:

[0723] Collecting event information

[0724] The server uses a web scraping tool to collect related event information based on the user's favorite genre. For example, if the user selects "art," the server collects art-related event information from the official museum website and event information sites.

[0725] Input: User's favorite genre

[0726] Data processing: Use web scraping tools (Beautiful Soup, Scrapy) to collect relevant event information.

[0727] Output: Collected event information

[0728] Step 3:

[0729] Event filtering

[0730] The server compares the collected event information with the available times on the user's calendar and filters out events that are held at times that do not overlap with family schedules. For example, it selects art exhibitions that are held at times when the user's calendar is available.

[0731] Input: User's calendar information, family information, collected event information

[0732] Data processing: Compare the user's calendar information with family schedule information and list events that take place at different times.

[0733] Output: Filtered event information

[0734] Step 4:

[0735] Priority adjustment by emotion engine

[0736] The server uses a natural language processing API to analyze users' past emotional data and learn how their emotions change in response to events of a specific genre. For example, if a user previously expressed joy when receiving notifications about an art exhibition, the server will prioritize notifications for events of that genre.

[0737] Input: User's past emotional data, filtered event information

[0738] Data processing: Analyze sentiment data using natural language processing APIs and adjust the priority of event notifications.

[0739] Output: Priority adjusted event information

[0740] Step 5:

[0741] Sending notifications

[0742] The server then uses a cloud messaging service to notify users of the filtered event information, for example by sending a notification to their smartphone saying, "There's an art exhibition this weekend."

[0743] Input: Priority adjusted event information

[0744] Data processing: Send notifications using a cloud messaging service (Firebase Cloud Messaging).

[0745] Output: Notification sent to the user's phone

[0746] (Application example 3)

[0747] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0748] With conventional calendar systems, users had to search for event information themselves and adjust it based on their family schedules, which was time-consuming and made it easy for events to be missed.In addition, the priority of event notifications was not adjusted based on the user's emotions or interests, making it difficult to provide the most appropriate event information for the user.

[0749] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: means for registering school and work schedules in a calendar; means for enabling the registration by capturing school newsletters and shift schedules by photographing them; means for inputting basic family information; means for the basic family information being address, age, grade, and height, with height being automatically measured using a smartphone; means for receiving event notifications that combine family information and favorite genres in available calendar appointments and allowing reservations; means for using an emotion engine to learn changes in the user's emotions and adjust the priority of event notifications; and means for the emotion engine to adjust the priority of event notifications based on the user's past reactions. This allows users to receive optimal event information without hassle, prevents missing out on events, and increases opportunities for family experiences.

[0750] "Means for registering school and work schedules in a calendar" is a function that allows users to enter school and work schedules into a digital calendar.

[0751] "Means that allow the registration by taking a photograph of a school newsletter or shift schedule" refers to a function that takes a photograph of a newsletter distributed by a school or a workplace shift schedule with a smartphone camera, automatically extracts a schedule from the image, and registers it on a calendar.

[0752] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[0753] "The basic information of the family member is the address, age, grade, and height, and the height can be automatically measured using a smartphone" refers to a function for inputting the address, age, grade, and height of the family member, and in particular, a function for automatically measuring height using a smartphone sensor.

[0754] "A means for notifying users of upcoming events that combine family information and their favorite genres with available time on the calendar, allowing them to make reservations" is a function that detects available time on the calendar, notifies users of event information based on basic family information and the genre selected by the user, and allows them to reserve the event.

[0755] "Means of using an emotion engine to learn changes in users' emotions and adjust the priority of event notifications" is a function that analyzes users' emotional data, learns what types of events users respond positively to, and adjusts the priority of event notifications based on the results.

[0756] "Means for the emotion engine to adjust the priority of event notifications based on the user's past reactions" refers to a function that records the emotional reactions shown by the user to event notifications received in the past and adjusts the priority of future event notifications based on that data.

[0757] The system for implementing this invention is configured using a user's smartphone, a server, and an external API. Specific embodiments are described below.

[0758] 1. System Configuration

[0759] The system consists of the following main components:

[0760] Smartphone: A device that allows users to register school and work schedules on a calendar and enter basic family information. It also has the ability to take photos of school newsletters and shift schedules using the smartphone's camera and extract schedules from those images.

[0761] Server: Collects event information, analyzes user emotion data using an emotion engine, and adjusts the priority of event notifications.

[0762] External API: Used to collect event information. For example, using the API of an event information provider.

[0763] 2. Program Processing

[0764] The server performs the following process.

[0765] Get free events from the calendar: Uses the Google Calendar API to get free events from the user's calendar.

[0766] Event information collection: Use external APIs to collect event information related to the genre selected by the user.

[0767] Prioritization using an emotion engine: Use an emotion engine (e.g., Microsoft Azure Emotion API) to analyze user emotion data and adjust the priority of event notifications.

[0768] 3. Specific Examples

[0769] For example, if a user selects "art" as their favorite genre and wants to be notified of art exhibitions that are being held at times that don't conflict with family schedules, the emotion engine will learn from the user's past reactions and prioritize displaying notifications about art exhibitions.

[0770] 4. Examples of prompts

[0771] Below are some example prompts to input to the generative AI model:

[0772] Create a Python program that retrieves free events from the user's calendar, collects event information based on family information and the user's favorite genre (e.g., art), adjusts the priority using an emotion engine, and notifies the user.

[0773] In this way, users can easily receive the most appropriate event information, prevent missing out on events, and increase opportunities for family experiences.

[0774] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0775] Step 1:

[0776] Users use their smartphones to register school and work schedules in a calendar. They take pictures of school newsletters and shift schedules with their smartphone cameras and upload the images to the system. The system uses image recognition technology to extract schedule information from the captured image and automatically register it in the calendar. The input is the captured image, and the output is the schedule information registered in the calendar.

[0777] Step 2:

[0778] The user inputs basic family information using a smartphone. The user inputs information such as address, age, grade, and height, and height is automatically measured using the smartphone's sensor. The input is basic family information, and the output is basic family information stored in the system.

[0779] Step 3:

[0780] The server uses the Google Calendar API to retrieve free events from the user's calendar. The server finds free time within a specified period and retrieves that information. The input is the user's calendar information, and the output is a list of free events.

[0781] Step 4:

[0782] The server uses an external API to collect event information related to the user's favorite genre. The server then calls the API of an event information service based on the user's interests to obtain related event information. The input is the user's favorite genre information, and the output is the collected event information.

[0783] Step 5:

[0784] The server uses an emotion engine to analyze the user's emotional data. The server records the user's emotional responses to past event notifications and learns the user's emotional changes based on that data. The input is the user's past emotional data, and the output is the learned emotion model.

[0785] Step 6:

[0786] The server uses the emotion engine to adjust the priority of event notifications. Based on the learned emotion model, the server adjusts the priority of notifications to events to which the user has a positive reaction. The input is the learned emotion model and collected event information, and the output is a list of event notifications with adjusted priorities.

[0787] Step 7:

[0788] The server sends the priority-adjusted event notification list to the user's smartphone. The user receives the notifications on their smartphone, checks the events of interest, and can make reservations. The input is the priority-adjusted event notification list, and the output is the event notification displayed on the user's smartphone.

[0789] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0790] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0791] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[0792] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0793] [Second embodiment]

[0794] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0795] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0796] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0797] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0798] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0799] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0800] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0801] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0802] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0803] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0804] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0805] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0806] "Example 1"

[0807] In the system of the present invention, users can register school and work schedules in a calendar. This registration can be done by taking a photo of a school newsletter or shift schedule. For example, if a user takes a photo of a school report card with a smartphone camera, the system extracts text from the image and registers the schedule in the calendar based on that information.

[0808] "Example 2"

[0809] Users also enter basic family information into the system, including family members' addresses, ages, grades, and heights. Height, in particular, can be measured automatically using a smartphone. For example, the smartphone's camera and augmented reality (AR) technology can be used to measure the heights of family members and enter that information into the system.

[0810] "Example 3"

[0811] Furthermore, the system will provide event notifications based on free calendar appointments, combining family information and the user's favorite genres. For example, if a user selects "art" as their favorite genre, the system will collect information about art-related events and notify the user if the event is held at a time that does not overlap with family schedules. This prevents users from missing out on events and increases opportunities for family experiences.

[0812] The processing flow of each embodiment will be described below.

[0813] "Example 1"

[0814] Step 1: The user takes a photo of their school or work schedule with their smartphone camera, such as a school report card or a work shift schedule.

[0815] Step 2: The system extracts text from the captured image using OCR (Optical Character Recognition) technology.

[0816] Step 3: The system uses the extracted text information to add an event to the user's calendar.

[0817] "Example 2"

[0818] Step 1: The user enters basic family information into the system, including information such as the family member's address, age, grade, and height.

[0819] Step 2: The user uses their smartphone camera and AR (augmented reality) technology to measure the heights of their family members.

[0820] Step 3: The system updates basic information about the family based on the height information measured.

[0821] "Example 3"

[0822] Step 1: The user selects their favorite genre for the system. For example, they select "art."

[0823] Step 2: The system collects information about events related to the selected genre.

[0824] Step 3: The system compares the collected event information with the user's calendar, and if the event is held at a time that does not conflict with family schedules, the system notifies the user.

[0825] Example 1

[0826] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0827] Traditional calendar registration systems require users to manually enter appointments, which is time-consuming and prone to input errors. Furthermore, there is no system that automatically generates event notifications based on basic family information or preferred genres, so users may miss many events. Furthermore, there is a lack of efficient ways to manage basic family information.

[0828] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0829] In this invention, the server includes a means for sending the captured image to the server, a means for the server to extract text from the image, and a means for the server to analyze the extracted text. This allows users to automatically register events on their calendar simply by taking a photo of a school newsletter or shift schedule. Furthermore, by entering basic family information, event notifications are automatically generated based on available calendar schedules, allowing users to make reservations, preventing users from missing out on events and increasing opportunities to experience them.

[0830] "Means for registering school and work schedules on a calendar" refers to a function that allows users to enter and manage school and work schedules on an electronic calendar.

[0831] "Means that allow for the capture of school newsletters and shift schedules by photographing them" refers to a function that allows users to use a camera on a smartphone or tablet to take a photo of a school newsletter or shift schedule and import that image into the system.

[0832] The "means for transmitting captured images to a server" is a function that allows a user to upload image data captured by the user to a server via the Internet.

[0833] The "means for the server to extract text from images" refers to a function for the server to extract text information from image data received using optical character recognition (OCR) technology.

[0834] "Means for analyzing text extracted by the server" refers to a function in which the server analyzes the extracted text data using natural language processing (NLP) technology and converts it into a format for registration on the calendar.

[0835] The "means for the server to register the analysis results in the calendar" is a function that enables the server to register the analyzed text data in a calendar application and add an event to the user's calendar.

[0836] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[0837] "Means for automatically measuring height using a mobile information terminal" refers to a function that allows a user to use a mobile information terminal to automatically measure the height of family members and input that data into the system.

[0838] "A means for receiving event notifications that combine family information and favorite genres with available time slots on the calendar, allowing users to make reservations" is a function in which the system generates event notifications based on basic family information and favorite genres, based on available time slots on the calendar, and allows users to reserve those events.

[0839] This invention is a system that allows users to register school and work schedules on a calendar. Users can easily register schedules by taking photos of school newsletters or shift schedules. This system is implemented using a mobile information terminal such as a smartphone or tablet and a server.

[0840] A user uses the smartphone camera to take a picture of a school newsletter or shift schedule. The image is sent to a server via a smartphone application. The server extracts text from the image using image processing software (e.g., a common cloud-based image recognition API). The extracted text is analyzed using natural language processing (NLP) software (e.g., a common NLP library) and converted into a format suitable for a calendar event. The server then registers the analyzed data with a calendar application (e.g., a common calendar API).

[0841] As a concrete example, consider the case where a user takes a photo of their school report card. The user takes a photo of the report card with their smartphone camera and sends the image to a server through an application. The server uses a cloud-based image recognition API to extract text from the image and analyzes the text using an NLP library. The analyzed information is then registered in the user's calendar using a calendar API. For example, if the report card contains information such as "PTA meeting on October 15th at 2:00 PM," this is registered as "PTA meeting" on the calendar.

[0842] Example prompt sentence:

[0843] "Take a photo of your school report card and put the event on your calendar."

[0844] This system allows users to efficiently manage their schedules by eliminating the need to manually enter schedules. By entering basic family information, the system automatically generates event notifications based on available calendars and allows users to make reservations, eliminating the need to miss out on events and increasing opportunities to experience them.

[0845] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0846] Step 1:

[0847] The user takes a photo of a school newsletter or shift schedule.

[0848] Specifically, the user launches the camera app on their smartphone, frames the school newsletter or shift schedule, and presses the shutter button. The input is the physical paper copy of the school newsletter or shift schedule, and the output is the image data stored on the smartphone.

[0849] Step 2:

[0850] The device sends the captured image to the server.

[0851] Specifically, the smartphone application uploads image data to the server as an HTTP request. The input is the image data stored on the smartphone, and the output is the image data sent to the server.

[0852] Step 3:

[0853] The server extracts the text from the image.

[0854] Specifically, the server uses a cloud-based image recognition API to extract text information from the received image data. The input is the image data sent to the server, and the output is the extracted text data.

[0855] Step 4:

[0856] The server parses the extracted text.

[0857] Specifically, the server uses a natural language processing (NLP) library to parse the extracted text data and convert it into a format suitable for calendar entry. The input is the extracted text data, and the output is the parsed text data.

[0858] Step 5:

[0859] The server registers the analysis results in the calendar.

[0860] Specifically, the server uses a calendar API to register the parsed text data in the user's calendar. The input is the parsed text data, and the output is the schedule registered in the calendar.

[0861] Step 6:

[0862] The user enters basic information about their family.

[0863] Specifically, the user enters basic information about the family members, such as their address, age, grade, height, etc. The input is the basic information entered by the user, and the output is the basic information stored on the server.

[0864] Step 7:

[0865] The server generates an event notification based on the available schedules in the calendar.

[0866] Specifically, the server detects free time on the calendar based on the basic information and favorite genres of the family members, and generates appropriate event notifications. The inputs are free time on the calendar, basic information of the family members, and favorite genres, and the output is the generated event notification.

[0867] Step 8:

[0868] A user books an event.

[0869] Specifically, the user checks the event notification sent from the server through the application, selects the desired event, and reserves it. The input is the event notification sent from the server, and the output is the reserved event information.

[0870] (Application example 1)

[0871] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0872] Conventional calendar registration systems required users to manually enter school and work schedules, which was time-consuming and labor-intensive. Also, manually registering shift schedule information into the calendar could result in input errors or missing information. Furthermore, managing basic family information and event notifications was cumbersome, making efficient schedule management difficult.

[0873] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0874] In this invention, the server includes means for registering school and work schedules in a calendar, means for enabling the registration by capturing school newsletters and shift schedules by photographing them, means for inputting basic family information, the basic family information being address, age, grade, and height, with height being automatically measured using a smartphone, means for receiving event notifications that combine family information and favorite genres in available calendar schedules and making reservations, and means for extracting text from images of shift schedules, analyzing shift information, and automatically registering it in the calendar. This allows users to easily register school and work schedules in the calendar, prevents input errors and omissions in shift information, and enables efficient schedule management.

[0875] "Means for registering school and work schedules on a calendar" refers to a function that allows users to enter and manage school and work schedules on an electronic calendar.

[0876] "The means by which the registration can be made by taking a photograph of the school newsletter or shift schedule" refers to a function that allows a user to take a photograph of the school newsletter or shift schedule with a smartphone camera, extract text from the image, and automatically register the schedule in the calendar.

[0877] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[0878] "The basic information of the family members is their address, age, grade, and height, and height can be automatically measured using a smartphone" means that basic information of the family members includes their address, age, grade, and height, and height in particular can be automatically measured using a smartphone function.

[0879] "A means to receive event notifications that combine family information and favorite genres with available time slots on the calendar, allowing reservations" is a function that allows the system to send event notifications that take into account basic family information and favorite genres based on available time slots on the calendar, and allows reservations for those events.

[0880] "Means for extracting text from an image of a shift schedule, analyzing the shift information, and automatically registering it in a calendar" is a function that analyzes an image of a shift schedule, extracts text information, and automatically registers that shift information in a calendar.

[0881] The system for implementing this invention allows users to efficiently register school and work schedules on a calendar and prevents input errors and omissions of shift information. A specific embodiment of this system will be described below.

[0882] First, the user takes a photo of the school newsletter or shift schedule using their smartphone. The smartphone has OpenCV installed as image processing software, which reads the captured image. Next, text is extracted from the image using Tesseract OCR. Through this text extraction process, the information in the shift schedule or school newsletter is obtained as digital data.

[0883] The acquired text data is sent to a server and analyzed. The analysis process identifies lines containing shift information and school schedules and extracts information such as dates, start times, and end times. Natural language processing technology is used for this analysis process.

[0884] The analyzed information is automatically registered in the user's calendar using a calendar API (e.g., Google Calendar API), which saves the user the trouble of manually entering events.

[0885] The system also has the ability to manage basic family information. Users can input information such as address, age, grade, and height, and height can be automatically measured using a smartphone sensor. This information is stored on the server, and event notifications tailored to the family's preferences are sent based on available calendar availability.

[0886] As a concrete example, consider a scenario in which a store clerk takes a photo of a shift schedule and the shift information is automatically registered in a calendar. In this case, the store clerk takes a photo of the shift schedule with their smartphone, and the system extracts text from the image, analyzes the shift information, and registers it in the calendar. This prevents input errors and omissions of shift information.

[0887] Example prompt sentence:

[0888] Please create an app that takes a photo of a shift schedule and automatically registers it in a calendar. Include a function to extract text from the shift schedule image, parse the shift date, start time, and end time, and register it in the calendar.

[0889] In this way, users can easily register school and work schedules on their calendar, preventing input errors and omissions in shift information and enabling efficient schedule management.

[0890] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0891] Step 1:

[0892] A user takes a photo of a school newsletter or shift schedule using a smartphone. The input is the captured image, and the output is an image file. Specifically, the user launches the smartphone's camera app and takes a photo of the school newsletter or shift schedule.

[0893] Step 2:

[0894] The device uses OpenCV to read the captured image. The input is an image file and the output is image data. Specifically, the device uses the OpenCV library to read the image file into memory.

[0895] Step 3:

[0896] The device uses Tesseract OCR to extract text from an image. The input is image data, and the output is extracted text data. Specifically, the device invokes the Tesseract OCR engine to parse and extract text from the image data.

[0897] Step 4:

[0898] The terminal transmits the extracted text data to the server. The input is the text data, and the output is the text data transmitted to the server. In specific operations, the terminal transmits the text data to the server via network communication.

[0899] Step 5:

[0900] The server analyzes the received text data and identifies shift information and school schedules. The input is the received text data, and the output is the analyzed shift information and schedule information. Specifically, the server analyzes the text data using natural language processing technology and extracts information such as date, start time, and end time.

[0901] Step 6:

[0902] The server uses a calendar API to register the analyzed information in the calendar. The input is the analyzed shift information and schedule information, and the output is the schedule registered in the calendar. Specifically, the server calls a calendar API such as the Google Calendar API and registers the analyzed information in the calendar.

[0903] Step 7:

[0904] The server manages basic information about family members and generates event notifications based on the free time on the calendar. The input is the basic information about family members and the free time on the calendar, and the output is the generated event notification. Specifically, the server takes into account information such as the address, age, grade, and height of family members and generates event notifications based on the free time on the calendar.

[0905] Step 8:

[0906] The server sends the generated event notification to the user. The input is the generated event notification, and the output is the event notification sent to the user. In concrete terms, the server sends the event notification to the user's terminal via network communication.

[0907] In this way, users can easily register school and work schedules on their calendar, preventing input errors and omissions in shift information and enabling efficient schedule management.

[0908] Example 2

[0909] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0910] Conventional calendar systems require users to manually enter school and work schedules, which is time-consuming and prone to input errors. Similar problems arise when manually entering basic family information. Furthermore, event notifications are not based on individual users' interests or family information, meaning users may miss events that are beneficial to them. There is a need to solve these problems and improve user convenience.

[0911] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0912] In this invention, the server includes a means for registering school and work schedules in a calendar, a means for capturing documents and tables by taking pictures, a means for inputting basic family information, a means for measuring heights using a mobile device's camera and augmented reality technology and inputting the data into the system, a means for generating event notifications by inputting prompt sentences into a generative AI model, and a means for receiving event notifications that combine family information and favorite genres with available calendar entries and allowing reservations. This allows users to easily and accurately register schedules and easily input basic family information. Furthermore, event notifications based on the user's interests and family information are provided, allowing users to participate in useful events without missing them.

[0913] The "means for registering school and work schedules in a calendar" is a function that allows a user to input and manage school and work schedules in an electronic calendar.

[0914] "Means that allow documents and tables to be captured by image capture" refers to a function that allows a user to use the camera on a smartphone or tablet to capture a document or table on paper and import the contents into the system as digital data.

[0915] The "means for inputting basic family information" is a function that allows the user to input basic information such as the address, age, grade, and height of family members into the system.

[0916] "Means for measuring height using a mobile device's camera and augmented reality technology and inputting that data into the system" refers to a function that allows users to measure the height of their family members using a smartphone or tablet's camera and AR technology, and automatically input the measurement results into the system.

[0917] "Means for generating event notifications by inputting prompt sentences into a generative AI model" refers to a function that allows a user to input specific prompt sentences into a generative AI model, which in turn generates event notifications based on the user's interests and family information.

[0918] "A means for receiving event notifications that combine family information and favorite genres in available calendar slots, allowing reservations" is a function that enables the system to detect available times on the calendar, send event notifications to the user that combine basic family information and the user's interests, and allow the event to be reserved.

[0919] The present invention is a system that allows a user to input basic family information, register school and work schedules in a calendar, and generate event notifications. A specific embodiment of this system will be described below.

[0920] Hardware and software used

[0921] Hardware

[0922] Mobile device: Smartphone or tablet (with camera)

[0923] software

[0924] Applications that use AR technology: For example, ARKit for iOS, ARCore for Android

[0925] Generative AI model: An AI model that generates event notifications based on user prompts.

[0926] System program processing

[0927] Enter basic family information

[0928] Users launch a dedicated smartphone app and enter basic information about their family members, including their address, age, grade, height, etc. Height, in particular, is automatically measured using the smartphone's camera and AR technology.

[0929] Specific examples

[0930] Users launch the app and tap the "Enter Family Information" button on the home screen. They then enter their family member's address, age, and grade in the form that appears. To measure their height, they point their smartphone camera at a family member and adjust the camera according to the on-screen guide. AR technology automatically measures their height and enters that data into the system.

[0931] Registering school and work schedules

[0932] Users can register school and work schedules in the calendar, and can take pictures of documents and tables, and import the contents into the system as digital data.

[0933] Specific examples

[0934] Users use their smartphone camera to take a photo of a school schedule or work shift schedule, and the system uses image recognition technology to extract appointments from the image and automatically register them in the calendar.

[0935] Generate event notifications

[0936] By inputting prompts into the generative AI model, event notifications are generated based on the user's interests and family information. Event notifications that combine family information and favorite genres are sent to the user's calendar, allowing the user to make reservations for the event.

[0937] Specific examples

[0938] Enter the following prompt into the generative AI model:

[0939] "Please enter basic information about your family members, including their address, age, grade, and height. Height can be measured using your smartphone camera and AR technology."

[0940] Based on this prompt, the generative AI model generates and provides event notifications to the user based on their interests and family information.

[0941] In this way, users can easily and accurately register their schedules and input basic family information. In addition, event notifications based on the user's interests and family information are provided, so users can participate in useful events without missing them.

[0942] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0943] Step 1:

[0944] The user enters basic information about their family.

[0945] Specifically, the user launches the dedicated smartphone app and taps the "Enter Family Information" button on the home screen. Next, they enter the address, age, and grade of their family members in the form that appears. The entered data is saved on the device.

[0946] Input: Address, age, grade

[0947] Output: Basic information of family members stored on the device

[0948] Step 2:

[0949] The user measures their height using the mobile device's camera and augmented reality technology.

[0950] Specifically, users go to the "Height Measurement" screen and point their smartphone camera at a family member. They adjust the camera according to the on-screen guide, and the AR technology automatically measures their height. The measurement results are saved on the device.

[0951] Input: Camera image

[0952] Output: Height data saved on the device

[0953] Step 3:

[0954] The device sends basic information and height data of family members to the server.

[0955] Specifically, when the user taps the "Send Data" button, the device sends the saved basic information and height data of the family to the server, which then stores the received data in a database.

[0956] Input: Basic information of family, height data

[0957] Output: Data stored on the server

[0958] Step 4:

[0959] Users add school and work events to their calendars.

[0960] Specifically, users use their smartphone camera to take a photo of a school schedule or work shift schedule, and the system uses image recognition technology to extract appointments from the image and automatically register them in the calendar.

[0961] Input: Schedule or shift schedule image

[0962] Output: Events registered in the calendar

[0963] Step 5:

[0964] Input a prompt sentence into the generative AI model to generate an event notification.

[0965] Specifically, the user inputs the following prompt into the generative AI model: "Please enter basic information about your family, including address, age, grade, and height. Height can be measured using your smartphone's camera and AR technology." Based on this prompt, the generative AI model generates event notifications based on the user's interests and family information.

[0966] Input: prompt statement

[0967] Output: Generated event notifications

[0968] Step 6:

[0969] An event notification that combines family information and favorite genres will be sent to the user's calendar, and the user can make a reservation for the event.

[0970] Specifically, the server detects free time on the calendar and sends an event notification to the user that combines basic information about the family and the user's interests. The user receives the notification and reserves the event.

[0971] Input: Calendar availability, family information, user interests

[0972] Output: Event notifications sent to users, subscribed events

[0973] (Application example 2)

[0974] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0975] In modern society, it is important to efficiently manage family schedules and provide opportunities to participate in events. However, conventional systems require the manual input of basic family information, and measuring height is particularly cumbersome. Additionally, when trying on clothes in physical stores, finding the right size can be time-consuming and time-consuming. To solve these issues, a more efficient and automated system is needed.

[0976] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for registering school and work schedules in a calendar, means for enabling the registration by capturing school newsletters and shift schedules by photographing them, means for inputting basic family information, the basic family information being address, age, grade, and height, means for automatically measuring height using a smartphone, means for receiving event notifications that combine family information and favorite genres in available calendar appointments and allowing reservations, means for automatically measuring family heights using a smartphone camera and augmented reality technology and suggesting optimal clothing sizes, and means for performing try-on simulations using augmented reality technology. This makes it possible to efficiently manage family schedules, increase opportunities to participate in events, and simplify trying on clothing in physical stores.

[0977] "Means for registering school and work schedules on a calendar" is a function that allows users to enter and manage their school or work schedules on a digital calendar.

[0978] "The means by which the registration can be made by taking a photograph of a school newsletter or shift schedule" refers to a function that allows a user to take a photograph of a newsletter distributed by school or a workplace shift schedule with a smartphone camera and automatically register that information in the calendar.

[0979] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[0980] "The basic information of the family members is their address, age, grade, and height, and height can be automatically measured using a smartphone" refers to a function that includes basic information of the family members, such as address, age, grade, and height, and in particular, automatically measures height using a smartphone's camera and augmented reality technology.

[0981] "A means to receive event notifications that combine family information and favorite genres based on available times on the calendar, allowing you to make reservations" is a function that allows you to receive notifications about events that combine basic family information and favorite genres based on available times on the calendar, and to make reservations for those events.

[0982] "Means to automatically measure the heights of family members using a smartphone camera and augmented reality technology and suggest the most suitable clothing sizes" is a function that uses a smartphone camera and augmented reality technology to automatically measure the heights of family members and suggests the most suitable clothing sizes based on that data.

[0983] "Means for performing fitting simulation using augmented reality technology" refers to a function that uses augmented reality technology to virtually simulate the state of trying on clothing before the user actually tries it on.

[0984] A system for carrying out the present invention has the following configuration: A server includes means for registering school and work schedules on a calendar, means for enabling the registration by capturing school newsletters and shift schedules by photographing them, means for inputting basic information about family members, the basic information about the family members being address, age, grade, and height, which can be automatically measured using a smartphone, means for receiving event notifications that combine family information and favorite genres with available calendar appointments and enabling reservations, means for automatically measuring the heights of family members using a smartphone camera and augmented reality technology and suggesting optimal clothing sizes, and means for performing a try-on simulation using augmented reality technology.

[0985] Hardware and software used

[0986] Hardware: Smartphone (with camera)

[0987] Software: OpenCV (image processing library), MediaPipe (pose estimation library)

[0988] Data processing and calculation

[0989] 1. Camera input: Capture video in real time using the user's smartphone camera.

[0990] 2. Pose estimation: Use MediaPipe to detect the position of the user's shoulders and ankles.

[0991] 3. Height calculation: Calculate the user's height from the position of the shoulders and ankles.

[0992] 4. Size suggestion: Based on the calculated height, we suggest the best clothing size for you.

[0993] 5. Try-on simulation: Using augmented reality technology, users can virtually try on clothes before actually trying them on.

[0994] Specific examples

[0995] For example, consider a scenario where a user measures their height using a smartphone and an app suggests a "medium" size shirt. The user takes a full-body photo using the smartphone camera and uses MediaPipe to detect the position of their shoulders and ankles. OpenCV is used to calculate their height from these positions, and the app then suggests the optimal clothing size based on that data. Furthermore, augmented reality technology can be used to simulate the user virtually trying on the shirt.

[0996] Prompt Sentence Examples

[0997] "Develop an application that uses a smartphone camera to measure height and suggest optimal clothing sizes. Use MediaPipe for pose estimation and OpenCV for image processing."

[0998] In this way, it becomes possible to efficiently manage family schedules, increase opportunities to participate in events, and simplify trying on clothes in physical stores.

[0999] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1000] Step 1:

[1001] The user activates the smartphone camera and takes a full-body image. The input is the smartphone camera image, and the output is real-time video data. Based on this video data, the system adjusts the user's entire body to fit within the frame.

[1002] Step 2:

[1003] The device uses MediaPipe to estimate the user's posture from the camera image. The input is the video data obtained in step 1, and the output is the position information of the shoulders and ankles. Specifically, the device uses MediaPipe's posture estimation model to detect the user's shoulder and ankle landmarks.

[1004] Step 3:

[1005] The device uses OpenCV to calculate the user's height from the shoulder and ankle positions. The input is the shoulder and ankle position information obtained in step 2, and the output is the user's height data. Specifically, the distance between the shoulder and ankle landmarks is calculated, and the actual height is calculated using a known scale factor.

[1006] Step 4:

[1007] The device then suggests the optimal clothing size based on the calculated height data. The input is the height data obtained in step 3, and the output is the suggested clothing size information. Specifically, the device compares the height data with an existing size chart to determine the optimal size.

[1008] Step 5:

[1009] The device uses augmented reality technology to provide the user with a virtual try-on simulation. The input is the clothing size information obtained in step 4, and the output is a video of the user virtually trying on the clothing. Specifically, a 3D model of the selected clothing is overlaid on the user's video, and a try-on simulation is performed in real time.

[1010] Step 6:

[1011] The server generates an event notification that matches the available calendar time slots based on the user's basic information and preferred genres, and sends it to the device. The input is the user's basic information and available calendar time slots, and the output is an event notification. Specifically, the server retrieves the user's basic information and preferred genres from a database, searches for events that match the available time slots, and generates a notification.

[1012] Step 7:

[1013] The terminal displays the event notification received to the user and prompts them to make a reservation. The input is the event notification obtained in step 6, and the output is the user's reservation action. Specifically, the notification is displayed in a pop-up format, and the user completes the event reservation by pressing the reservation button.

[1014] The above steps will make it possible to manage family schedules more efficiently, increase opportunities to participate in events, and simplify trying on clothes in physical stores.

[1015] Example 3

[1016] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1017] In conventional calendar systems, users have to collect event information individually and check it against their family schedules, which can lead to missed events and reduced opportunities for family experiences. In addition, there is a lack of a way to efficiently collect event information related to the genre selected by the user and notify them at the appropriate time.

[1018] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1019] In this invention, the server includes means for registering school and work schedules in a calendar, means for enabling the registration by capturing images of school newsletters and shift schedules, means for inputting basic family information, the basic family information being address, age, grade, and height, with height being automatically measured using a mobile device, means for receiving event notifications that combine family information and a favorite genre in available calendar schedules and allowing reservations, means for collecting event information related to a genre selected by the user from an external information source, means for comparing the collected event information with calendar information and identifying events to be held in times that do not overlap with family schedules, and means for notifying the user of the identified event information, thereby enabling the user to efficiently collect event information and not miss events to be held in times that do not overlap with family schedules.

[1020] The "means for registering school or work schedules in a calendar" refers to a method by which a user can input school or work schedules into an electronic calendar.

[1021] "Methods that can be achieved by capturing images" refers to a method of taking a photo of paper documents such as school newsletters or shift schedules with a camera, analyzing the image, and automatically registering the schedule in the calendar.

[1022] The "means for inputting basic family information" is a method by which a user inputs basic information such as the address, age, grade, and height of a family member into the system.

[1023] "Means that can be measured automatically using a mobile device" refers to a method of automatically measuring the height of family members using a mobile device such as a smartphone or tablet.

[1024] "A means for notifying users of events that combine family information and their favorite genres with available time on their calendar, allowing them to make reservations" is a method for notifying users of events related to family information and the genre selected by the user based on available time on the user's calendar, allowing them to make reservations.

[1025] "Means for collecting event information related to the genre selected by the user from external information sources" refers to a method for obtaining event information related to the genre of interest selected by the user from an external database or API.

[1026] The "means for comparing the collected event information with calendar information and identifying events that will be held during times that do not overlap with family schedules" is a method for comparing the collected event information with the user's calendar information and finding events that will be held during times that do not overlap with family schedules.

[1027] The "means for notifying the user of the specified event information" refers to a means for notifying the user of the specified event information by email, push notification, or the like.

[1028] The present invention is a system for notifying a user of events related to a genre selected by the user based on the user's calendar information and family information. A specific embodiment of this system will be described below.

[1029] Hardware and software used

[1030] This system is implemented using a server, terminals (smartphones and PCs), and various APIs. Specifically, the following hardware and software are used:

[1031] Server: Processes and manages data. Implemented in a programming language such as Python or Java.

[1032] Terminal: A device that a user uses to access the system and input information. This includes smartphones and PCs.

[1033] Google Calendar API: Used to retrieve user calendar information.

[1034] MySQL: A database for managing family information.

[1035] Eventbrite API: Used to collect event information.

[1036] SendGrid API: Used to notify users of event information via email.

[1037] Firebase Cloud Messaging: Used to push event information to users.

[1038] The Python pandas library: Used to filter and match data.

[1039] Specific operation of the system

[1040] 1. The user selects their favorite genre

[1041] The user accesses the system interface using a terminal and selects a genre (e.g., "art") from a provided genre list. This selection information is sent to the server.

[1042] 2. The server retrieves the user's calendar information.

[1043] The server uses the Google Calendar API to retrieve the user's calendar information, specifically, to read the calendar events using an authentication token to access the user's Google account.

[1044] 3. The server retrieves the family information from the database

[1045] The server connects to a MySQL database and retrieves the user's family information, which includes the schedules and contacts of all family members.

[1046] 4. The server collects event information related to the selected genre.

[1047] The server uses the Eventbrite API to collect event information related to the selected genre (e.g., "art"), sends API requests, and parses the returned event data.

[1048] 5. The server matches the event information with the calendar information

[1049] The server uses the Python pandas library to match the collected event information with the user's calendar information, specifically filtering out events that take place at times that don't conflict with family schedules.

[1050] 6. The server notifies the user of appropriate events

[1051] The server uses the SendGrid API to notify users of the filtered event information via email, and can also send push notifications using Firebase Cloud Messaging.

[1052] Examples of specific examples and prompts

[1053] For example, if the user selects "art" as a favorite genre and the family has a free Saturday afternoon, the server operates as follows.

[1054] 1. The user selects "Art" on the device.

[1055] 2. The server uses the Google Calendar API to check the user's calendar to see if Saturday afternoon is free.

[1056] 3. The server retrieves the family's schedule from the MySQL database and finds that Saturday afternoon is free.

[1057] 4. The server uses the Eventbrite API to collect art-related events taking place on Saturday afternoons.

[1058] 5. Filter the event information collected by the server using the pandas library to identify events that do not overlap with family schedules.

[1059] 6. The server uses the SendGrid API to notify the user of the identified event information via email and also sends a push notification using Firebase Cloud Messaging.

[1060] Specific prompt examples:

[1061] Get the user's calendar information and notify them of art-related events that are scheduled at times that don't conflict with family schedules.

[1062] In this way, the user can receive information about art-related events that are held at times that do not overlap with family schedules. The flow of the specification process in the third embodiment will be described with reference to FIG.

[1063] Step 1:

[1064] The user selects the genre they like.

[1065] The user accesses the system interface using a terminal and selects a genre (e.g., "art") from a provided genre list. This selection information is sent to the server.

[1066] Input: User genre selection

[1067] Output: Genre information sent to the server

[1068] Step 2:

[1069] The server retrieves the user's calendar information.

[1070] The server uses the Google Calendar API to retrieve the user's calendar information, specifically, to read the calendar events using an authentication token to access the user's Google account.

[1071] Input: User's authentication token

[1072] Output: User's calendar information

[1073] Step 3:

[1074] The server retrieves the family information from the database.

[1075] The server connects to a MySQL database and retrieves the user's family information, which includes the schedules and contacts of all family members.

[1076] Input: Database connection information

[1077] Output: Family information

[1078] Step 4:

[1079] The server collects event information related to the selected genre.

[1080] The server uses the Eventbrite API to collect event information related to the selected genre (e.g., "art"), sends API requests, and parses the returned event data.

[1081] Input: Genre information

[1082] Output: Event information

[1083] Step 5:

[1084] The server matches the event information with the calendar information.

[1085] The server uses the Python pandas library to match the collected event information with the user's calendar information, specifically filtering out events that take place at times that don't conflict with family schedules.

[1086] Input: Event information, calendar information

[1087] Output: Filtered event information

[1088] Step 6:

[1089] The server notifies the user of appropriate events.

[1090] The server uses the SendGrid API to notify users of the filtered event information via email, and can also send push notifications using Firebase Cloud Messaging.

[1091] Input: Filtered event information

[1092] Output: Notification to the user (email or push notification)

[1093] In this way, the user can receive information about art-related events that are held at times that do not overlap with family schedules.

[1094] (Application example 3)

[1095] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1096] In modern households, it is difficult to find events that the whole family can enjoy, and it is also difficult to plan events at times when family schedules do not overlap. Furthermore, there is a lack of ways to efficiently collect event information and notify people at the right time, which makes it easy for events to be missed. There is a need to solve these problems and increase opportunities for family experiences.

[1097] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1098] In this invention, the server includes: means for registering school and work schedules in a calendar; means for enabling the registration by capturing school newsletters and shift schedules by photographing them; means for inputting basic family information; means for the basic family information being address, age, grade, and height, the height of which can be automatically measured using a smartphone; means for receiving an event notification that combines family information and favorite genres in an available schedule on the calendar and allowing reservations; means for suggesting events that the whole family can enjoy in the event notification; means for notifying the user if the event is held at a time that does not overlap with family schedules; means for collecting the event information from an external database; means for notifying the user of the event information; and means for notifying the user by push notification or email notification. This makes it possible to efficiently find events that the whole family can enjoy and notify the user at an appropriate time.

[1099] "Means for registering school and work schedules in a calendar" is a function that allows users to enter school and work schedules into an electronic calendar.

[1100] "Means that allow the registration by taking a photograph of school newsletters or shift schedules" refers to a function that allows you to take a photograph of a notice distributed by school or a workplace shift schedule with your smartphone camera and automatically update the information on the calendar.

[1101] "Means for entering basic family information" is a function for entering basic information such as family member's address, age, grade, height, etc. into the system.

[1102] "The basic information of the family members is their address, age, grade, and height, and height can be automatically measured using a smartphone" refers to a function that inputs the address, age, grade, and height of the family members, and automatically measures height in particular using the smartphone's sensor.

[1103] "A means of receiving event notifications that combine family information and favorite genres in the available time slots on the calendar, allowing reservations to be made" is a function that notifies users of events related to basic family information and genres selected by the user based on available time slots on the calendar, and allows them to make reservations for those events.

[1104] "Means for the event notification to suggest events that the whole family can enjoy" is a function in which the system automatically selects events that the whole family can be interested in and enjoy, and suggests them to the user.

[1105] The "means for notifying if the event is held at a time that does not overlap with family schedules" is a function that selects an event that is held at a time that does not overlap with family schedules and notifies the user of that information.

[1106] The "means for collecting the event information from an external database" is a function for acquiring event information from an external database or API.

[1107] The "means for notifying the user of the event information" is a function for notifying the user of the collected event information.

[1108] "Means for the notification to be made by push notification or email notification" refers to a function that sends event information to users via push notification or email.

[1109] The system for implementing this invention proposes events that the whole family can enjoy based on the user's calendar information and basic family information, and notifies them at the appropriate time. A specific embodiment of this system will be described below.

[1110] System Configuration

[1111] 1. Hardware Configuration

[1112] Smartphone: A device that allows users to input calendar information and receive event notifications. Smartphones are equipped with cameras and sensors, allowing them to take photos of school newsletters and shift schedules, and automatically measure height.

[1113] Server: A central processing unit that collects event information and suggests events based on the user's calendar information and basic family information.

[1114] 2. Software Configuration

[1115] Google Calendar API: An API for obtaining a user's calendar information.

[1116] External Database API: An API for collecting event information.

[1117] Notification system: A system for sending push notifications and email notifications.

[1118] Data processing and calculation

[1119] 1. Get calendar information

[1120] The server uses the Google Calendar API to retrieve the user's calendar information, which allows it to determine the user's free time.

[1121] 2. Enter basic family information

[1122] Users use their smartphones to enter basic information about their family members, such as their address, age, grade, and height, which is then automatically measured using the smartphone's sensors.

[1123] 3. Collecting event information

[1124] The server uses an external database API to collect event information related to the genre selected by the user.

[1125] 4. Event Proposal

[1126] The server selects events that the whole family can enjoy based on the collected event information, the user's calendar information, and basic family information. The selected events are held during the user's free time.

[1127] 5. Event Notifications

[1128] The server notifies the user of the selected event information via push notification or email notification.

[1129] Specific examples

[1130] For example, if a user selects "art" as their favorite genre and has free time on their family calendar, the system will notify them of events such as:

[1131] Event name: "Monet Exhibition"

[1132] Date and time: October 15, 2023, 14:00 - 16:00

[1133] Prompt Sentence Examples

[1134] An example of a prompt sentence to input to the generative AI model is as follows:

[1135] I'm developing an application that suggests events that the whole family can enjoy based on the user's free time on their calendar. I'd like to implement a function that collects event information related to the user's favorite genre (e.g., art, music, sports, etc.) and notifies them of events that are held at times that don't conflict with other family schedules. Could you please give me a specific example of a program and the process?

[1136] In this way, the system can efficiently find events that the whole family can enjoy based on the user's calendar information and basic family information, and notify them at the appropriate time.

[1137] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1138] Step 1:

[1139] A user uses a smartphone to add school or work appointments to their calendar.

[1140] Input: School newsletters, shift schedule photos, manually entered schedule information

[1141] Data processing: Images taken with a smartphone camera are converted into text data using OCR (optical character recognition) technology and registered in the calendar.

[1142] Output: Schedule information registered in the calendar

[1143] Step 2:

[1144] A user uses a smartphone to enter basic information about their family.

[1145] Input: Address, age, grade, height (automatically measured using smartphone sensor)

[1146] Data processing: The entered information is saved in a database, and height is automatically measured using the smartphone sensor if necessary.

[1147] Output: Basic information of family members stored in the database

[1148] Step 3:

[1149] The server retrieves the user's calendar information using the Google Calendar API.

[1150] Input: User's Google Calendar account information

[1151] Data processing: Using the Google Calendar API, schedule information is obtained from the user's calendar and free time is identified.

[1152] Output: User's calendar free / busy information

[1153] Step 4:

[1154] The server uses an external database API to collect event information related to the genre selected by the user.

[1155] Input: User-selected genre (e.g., art, music, sports)

[1156] Data processing: Using an external database API, event information related to the specified genre is obtained.

[1157] Output: Collected event information

[1158] Step 5:

[1159] Based on the event information collected by the server, the user's calendar information, and basic family information, the system selects events that the whole family can enjoy.

[1160] Input: Collected event information, user calendar availability, basic family information

[1161] Data processing: Event information is compared with available time on the calendar and basic family information to select events that the whole family can enjoy.

[1162] Output: Selected event information

[1163] Step 6:

[1164] The server notifies the user of the selected event information via push notification or email notification.

[1165] Input: Selected event information, user contact information

[1166] Data processing: The selected event information is converted into a push notification or email notification format and sent to the user.

[1167] Output: Event notification sent to the user

[1168] In this way, the system can efficiently find events that the whole family can enjoy based on the user's calendar information and basic family information, and notify them at the appropriate time.

[1169] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1170] "Example 1"

[1171] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system has the ability to recognize emotions from a user's tone of voice, facial expressions, behavioral patterns, etc. Specifically, the system analyzes the tone of voice as the user speaks through the smartphone's microphone and recognizes emotions such as joy, anger, sadness, and surprise. It also captures the user's facial expressions through the smartphone's camera and recognizes emotions from smiles, frowns, surprise, etc. Furthermore, it uses the smartphone's sensors to analyze the user's behavioral patterns and infers emotions from activity levels, movement patterns, usage patterns, etc.

[1172] "Example 2"

[1173] A function is also provided to adjust the priority of event notifications based on the emotions recognized by the emotion engine. For example, if the user is expressing joy, event notifications in the user's preferred genre are displayed preferentially. Conversely, if the user is expressing anger or sadness, event notifications in the user's preferred genre are displayed less. This makes it possible to provide optimal event notifications according to the user's emotions. "Example 3"

[1174] The emotion engine also has the ability to learn changes in a user's emotions and reflect the results in adjusting the priority of event notifications. For example, if a user often expresses joy after seeing an event notification for a specific genre, the engine will learn to prioritize displaying event notifications for that genre. This allows the system to more accurately understand the relationship between a user's emotions and event notifications, making it possible to provide the most appropriate event notifications for the user.

[1175] The processing flow of each embodiment will be described below.

[1176] "Example 1"

[1177] Step 1: The user speaks through the smartphone microphone.

[1178] Step 2: The system analyzes the tone of the voice and recognizes emotions such as joy, anger, sadness, and surprise.

[1179] Step 3: Capture the user's facial expression using the smartphone camera.

[1180] Step 4: The system recognizes emotions such as smiles, frowns, and surprised expressions.

[1181] Step 5: Use smartphone sensors to analyze user behavior patterns and infer emotions based on activity levels, movement patterns, usage patterns, etc.

[1182] "Example 2"

[1183] Step 1: Adjust the priority of event notifications based on the emotions recognized by the emotion engine.

[1184] Step 2: If the user is expressing joy, notifications of upcoming events in the user's preferred genre are displayed preferentially.

[1185] Step 3: If the user expresses anger or sadness, subtly display event notifications in the user's preferred genre.

[1186] "Example 3"

[1187] Step 1: The emotion engine learns the user's emotional changes.

[1188] Step 2: If a user often expresses joy after seeing event notifications of a specific genre, the system learns to prioritize displaying event notifications of that genre. Step 3: The system more accurately understands the relationship between user emotions and event notifications, and provides the most appropriate event notifications for the user.

[1189] Example 1

[1190] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1191] Conventional calendar registration systems require users to manually input schedules, which is time-consuming and prone to input errors. It is also difficult to provide services that take users' feelings into consideration, making it difficult to improve user satisfaction. Furthermore, the lack of event notification and reservation functions that utilize basic family information makes managing the schedules of the entire family cumbersome.

[1192] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1193] In this invention, the server includes means for registering school and work schedules in a calendar, means for enabling the registration by capturing school newsletters and shift schedules by photographing them, means for inputting basic information about family members, the basic information about the family members being address, age, grade, and height, the height of which can be automatically measured using a mobile information terminal, means for receiving event notifications that combine family information and favorite genres in the available schedules on the calendar and allowing reservations to be made, means for extracting text from images, means for recognizing emotions from voice, means for recognizing emotions from facial expressions, and means for inferring emotions from behavioral patterns.This allows users to easily and accurately register schedules in the calendar, enables services that take emotions into consideration, and makes it easier to manage the schedules of the entire family.

[1194] "Means for registering school and work schedules in a calendar" is a function that allows users to automatically register school and work schedules in an electronic calendar.

[1195] "The means by which the registration can be made by taking a photograph of the school newsletter or shift schedule" is a function that allows the user to take a photograph of the school newsletter or shift schedule and automatically import that information into the electronic calendar.

[1196] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[1197] "The basic information of the family members is address, age, grade, and height, and height can be automatically measured using a mobile information terminal" refers to a function that includes address, age, grade, and height as basic information of the family members, and in particular automatically measures height using a mobile information terminal.

[1198] "A means to receive event notifications that combine family information and favorite genres based on available times on the calendar, and make reservations" is a function that allows you to receive event notifications that combine basic family information and favorite genres based on available times on the calendar, and to make reservations for those events.

[1199] The "means for extracting text from an image" is a function for automatically extracting text information from a captured image.

[1200] "Means for recognizing emotions from voice" is a function for analyzing the user's voice and recognizing emotions from that voice.

[1201] "Means for recognizing emotions from facial expressions" is a function that analyzes the user's facial expressions and recognizes emotions from those expressions.

[1202] "Means for inferring emotions from behavioral patterns" is a function that analyzes user behavioral data and infers emotions from those behavioral patterns.

[1203] This invention is a system that allows users to easily register school and work schedules on a calendar and provides services that recognize the user's emotions. This system is realized by combining a mobile information terminal such as a smartphone with a server.

[1204] 1. Extract text from images and add it to your calendar

[1205] Hardware and software used

[1206] Hardware: Smartphone camera

[1207] Software: OCR software (e.g., Google Cloud Vision API), calendar app (e.g., Google Calendar)

[1208] Processing flow

[1209] A user uses their smartphone camera to take a photo of a school newsletter or shift schedule. The device sends the image to a server, which uses OCR software to extract text from the image. The server then analyzes the extracted text and registers the schedule information in a calendar app.

[1210] Specific examples

[1211] Users take a photo of their school report card and the system registers the information in their calendar.

[1212] Example prompt: "Please take a photo of your school report card and add it to your calendar."

[1213] 2. Emotion Recognition by Emotion Engine

[1214] Hardware and software used

[1215] Hardware: Smartphone microphone, camera, sensors (e.g. accelerometer, GPS)

[1216] Software: Voice analysis software (e.g., IBM Watson Tone Analyzer), facial expression analysis software (e.g., Microsoft Azure Face API), behavior analysis software

[1217] Processing flow

[1218] When a user speaks through the smartphone's microphone, the device sends the recorded voice data to a server, which uses voice analysis software to recognize emotions from the voice. When the user takes a photo of their facial expression using the smartphone's camera, the device sends the facial image to the server, which then uses facial expression analysis software to recognize emotions from the facial expression. In addition, the device uses the smartphone's sensors to collect user behavior data and sends it to the server. The server uses behavior analysis software to infer emotions from behavioral patterns.

[1219] Specific examples

[1220] The user speaks into the smartphone and the system recognizes the emotion of joy from the tone of voice.

[1221] Example prompt: "Speak into your smartphone and identify your emotions."

[1222] 3. Family Information and Event Notifications

[1223] Hardware and software used

[1224] Hardware: Smartphone

[1225] Software: Calendar app, event notification system

[1226] Processing flow

[1227] Users enter basic family information (address, age, grade, height) into the system. Height can be automatically measured using a smartphone. Based on available calendar availability, users will receive notifications of upcoming events that combine their family information with their favorite genres, and can make reservations for those events.

[1228] Specific examples

[1229] Users enter basic family information, and the system sends event notifications, such as art and programming, based on calendar availability.

[1230] Example prompt: "Enter basic family information to receive event notifications."

[1231] In this way, the system of the present invention allows users to accurately register events on a calendar without much effort, makes it possible to provide services that take emotions into consideration, and makes it easier to manage the schedules of all family members.

[1232] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1233] Extract text from images and add it to your calendar

[1234] Processing Steps

[1235] Step 1:

[1236] Users use their smartphone camera to take photos of school newsletters and shift schedules.

[1237] Specific operation: The user launches the camera app, points the camera at an object, and presses the shutter button.

[1238] Input: School newsletter or shift schedule image

[1239] Output: Captured image data

[1240] Step 2:

[1241] The device sends the captured image to the server.

[1242] Specific operation: The device uploads image data to the server using an HTTP POST request.

[1243] Input: Photographed image data

[1244] Output: Image data sent to the server

[1245] Step 3:

[1246] The server extracts the text from the image.

[1247] Specific operation: The server sends image data to OCR software (e.g., Google Cloud Vision API) and receives the returned text data.

[1248] Input: Image data sent to the server

[1249] Output: Extracted text data

[1250] Step 4:

[1251] The server parses the text and generates the schedule.

[1252] What it does: The server uses natural language processing (NLP) algorithms to extract information from the text, such as dates, times, and event names.

[1253] Input: Extracted text data

[1254] Output: Generated schedule information

[1255] Step 5:

[1256] The server registers the event in the calendar.

[1257] Specific operation: The server uses a calendar API (e.g., Google Calendar API) to add the event information to the calendar.

[1258] Input: Generated schedule information

[1259] Output: Events registered in the calendar

[1260] Emotion recognition by emotion engine

[1261] Processing Steps

[1262] Step 1:

[1263] The user inputs voice.

[1264] Specific behavior: The user launches a voice input app and speaks into the microphone.

[1265] Input: User's voice

[1266] Output: Recorded audio data

[1267] Step 2:

[1268] The terminal transmits the voice data to the server.

[1269] Specific operation: The device uploads audio data to the server using an HTTP POST request.

[1270] Input: Recorded audio data

[1271] Output: Audio data sent to the server

[1272] Step 3:

[1273] The server recognizes emotions from the voice.

[1274] Specific operation: The server sends voice data to voice analysis software (e.g., IBM Watson Tone Analyzer) and receives the returned emotion data.

[1275] Input: Audio data sent to the server

[1276] Output: Recognized emotion data

[1277] Step 4:

[1278] The user captures their facial expression.

[1279] Specific action: The user launches the camera app and takes a picture of their face.

[1280] Input: User's facial expression image

[1281] Output: Photographed facial expression image data

[1282] Step 5:

[1283] The terminal transmits the facial expression data to the server.

[1284] Specific operation: The device uploads image data to the server using an HTTP POST request.

[1285] Input: Photographed facial expression image data

[1286] Output: Facial expression image data sent to the server

[1287] Step 6:

[1288] The server recognizes emotions from facial expressions.

[1289] Specific operation: The server sends facial expression images to facial expression analysis software (e.g., Microsoft Azure Face API) and receives the returned emotional data.

[1290] Input: Facial expression image data sent to the server

[1291] Output: Recognized emotion data

[1292] Step 7:

[1293] The device collects behavioral data and sends it to a server.

[1294] Specific operation: The device periodically collects sensor data and uploads it to the server via an HTTP POST request.

[1295] Input: User behavior data (e.g., accelerometer, GPS data)

[1296] Output: Behavioral data sent to the server

[1297] Step 8:

[1298] The server infers emotions from behavioral patterns.

[1299] How it works: The server analyzes the behavioral data and applies an algorithm to infer emotions.

[1300] Input: Behavioral data sent to the server

[1301] Output: Inferred emotion data

[1302] Family Information and Event Notifications

[1303] Processing Steps

[1304] Step 1:

[1305] The user enters basic information about their family.

[1306] Specific behavior: The user launches the app and enters information such as address, age, grade, and height.

[1307] Input: Basic information of family members (address, age, grade, height)

[1308] Output: Basic information data of the entered family

[1309] Step 2:

[1310] The device will automatically measure your height.

[1311] Specific operation: The user measures their height using a smartphone.

[1312] Input: User's height measurement data

[1313] Output: Measured height data

[1314] Step 3:

[1315] The server checks the availability of the calendar.

[1316] Specific operation: The server uses the calendar API to obtain free time.

[1317] Input: Calendar free / busy data

[1318] Output: Retrieved free / busy data

[1319] Step 4:

[1320] The server generates an event notification that combines family information and favorite genres.

[1321] Specific operation: The server generates an event notification based on family information and the user's preferred genres.

[1322] Input: Family information data, user's favorite genre data

[1323] Output: Generated event notification data

[1324] Step 5:

[1325] The server sends an event notification to the user.

[1326] Specific operation: The server sends event notification data to the user's terminal.

[1327] Input: Generated event notification data

[1328] Output: Event notification sent to the user

[1329] Step 6:

[1330] A user books an event.

[1331] What happens: The user receives a notification and takes action to book the event.

[1332] Input: Event notification

[1333] Output: Scheduled event data

[1334] (Application example 1)

[1335] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1336] Conventional calendar registration systems required users to manually enter schedules, which was time-consuming and prone to input errors. Furthermore, the service provided did not take user feelings into consideration, and efforts were not made to improve the user experience. Furthermore, it was difficult to centrally manage the schedules of all family members, resulting in missed events and missed opportunities.

[1337] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes: means for registering school and work schedules in a calendar; means for enabling the registration by capturing school newsletters and shift schedules by photographing them; means for inputting basic family information; means for the basic family information being address, age, grade, and height, with height being automatically measured using a smartphone; means for receiving event notifications that combine family information and favorite genres in the free schedules on the calendar and allowing reservations; means for recognizing the user's emotions; means for recognizing the emotions using the smartphone's camera, microphone, and sensors; and means for suggesting appropriate products and services based on the recognized emotions. This allows the user to easily and accurately register schedules, enables the provision of services based on emotions, centrally manages the schedules of all family members, and prevents missing events and missing out on experience opportunities.

[1338] "Means for registering school and work schedules in a calendar" is a function that allows users to enter school and work schedules into an electronic calendar.

[1339] "The means by which the registration can be made by taking a photograph of the school newsletter or shift schedule" refers to a function that allows a user to take a photograph of the school newsletter or shift schedule with a smartphone camera, extract text from the image, and automatically register the schedule in the calendar.

[1340] "Means for entering basic family information" refers to a function for entering basic information such as family member's address, age, grade, height, etc. into the system.

[1341] "The basic information of the family member is the address, age, grade, and height, and the height can be automatically measured using a smartphone" refers to a function that includes the address, age, grade, and height as basic information of the family member, and in particular automatically measures height using a smartphone sensor.

[1342] "A means to receive event notifications that combine family information and favorite genres with available time on the calendar, allowing reservations" is a function that sends event notifications that take into account basic family information and favorite genres based on available time on the calendar, and allows reservations for those events.

[1343] "Means for recognizing user emotions" refers to a function for identifying emotions from the user's tone of voice, facial expressions, behavioral patterns, etc.

[1344] "Means for performing emotion recognition using the smartphone's camera, microphone, and sensors" refers to a function for recognizing a user's emotions using the smartphone's camera, microphone, and sensors.

[1345] "Means for suggesting appropriate products and services based on recognized emotions" is a function that analyzes the user's emotions and suggests appropriate products and services based on the results.

[1346] The system for implementing this invention allows a user to register school or work schedules in a calendar, and suggests appropriate products and services through emotion recognition. Specific embodiments of the system will be described below.

[1347] System configuration

[1348] The system consists of a smartphone, a server, and related software. The smartphone is equipped with a camera, microphone, and sensors, and these hardware components are used to recognize the user's emotions. The server processes and stores the data, and provides appropriate notifications and suggestions to the user.

[1349] Program processing

[1350] 1. Register an appointment:

[1351] When a user takes a picture of a school newsletter or shift schedule with their smartphone camera, the text is extracted from the image. The extracted text is sent to the server and automatically registered in the calendar. This process uses OCR (Optical Character Recognition) technology.

[1352] 2. Enter family information:

[1353] Users enter basic information such as their family members' addresses, ages, grades, and heights via their smartphones. Heights are automatically measured using the smartphone's sensors. This information is then stored on a server.

[1354] 3. Emotion recognition:

[1355] The smartphone's camera, microphone, and sensors are used to collect the user's facial expressions, tone of voice, and behavioral patterns. This data is input into an emotion recognition model to identify the user's emotions. Emotion recognition uses software such as OpenCV (camera image processing), Keras (emotion recognition model), and SpeechRecognition (voice recognition).

[1356] 4. Product or service suggestions:

[1357] Based on the recognized emotion, the server will suggest appropriate products and services to the user. For example, if the user is feeling stressed, it will suggest relaxation-related products and services. This suggestion is based on the user's available time on the calendar.

[1358] Specific examples

[1359] For example, if a user takes a photo of their school report card with their smartphone, the system will extract the text from the image and add it to their calendar. Or, if a user says, "I want to know more about this product," the smartphone will recognize their voice and display relevant information.

[1360] Prompt Sentence Examples

[1361] Example prompts to input to the generative AI model:

[1362] "We want to develop an application that recognizes a customer's facial expressions and voice while they are looking at products, and suggests appropriate products and services. Please generate specific suggestions based on the following information."

[1363] In this way, a system is realized that allows users to register schedules accurately and hassle-free, enables services to be provided based on emotions, centrally manages the schedules of all family members, and prevents events from being missed or experiences from being missed.

[1364] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1365] Step 1:

[1366] A user takes a picture of a school newsletter or shift schedule with their smartphone camera. The input is the captured image. The smartphone uses OCR (Optical Character Recognition) technology to extract text from the image. The output is the extracted text data.

[1367] Step 2:

[1368] The smartphone sends the extracted text data to the server. The input is the text data. The server analyzes the received text data and registers the event in the calendar. The output is updated calendar data.

[1369] Step 3:

[1370] The user inputs basic information about their family members (address, age, grade, height) via their smartphone. The input is the basic information entered by the user. The smartphone automatically measures the height using a sensor. The output is the measured height data and other basic information.

[1371] Step 4:

[1372] The smartphone sends basic family information to the server. The input is the basic family information. The server stores the received information in a database. The output is the stored family information data.

[1373] Step 5:

[1374] While a user is using a smartphone, the smartphone's camera, microphone, and sensors collect the user's facial expressions, tone of voice, and behavioral patterns. The input is the collected facial expression data, voice data, and behavioral data. The smartphone inputs this data into an emotion recognition model to identify the user's emotions. The output is the identified emotion data.

[1375] Step 6:

[1376] The smartphone sends the identified emotion data to the server. The input is the emotion data. The server analyzes the received emotion data and suggests appropriate products or services to the user. The output is information about the suggested products or services.

[1377] Step 7:

[1378] The server checks the free time on the user's calendar and generates an event notification by combining family information and favorite genres. The input is calendar data, family information, and favorite genres. The server sends the generated event notification to the user. The output is an event notification.

[1379] Step 8:

[1380] The user checks the received event notification and reserves the event that interests them. The input is the event notification. The smartphone sends the user's reservation information to the server. The output is the reserved event information.

[1381] In this way, a system is realized that allows users to accurately register schedules without hassle, enables services to be provided based on emotions, manages the schedules of all family members in a unified manner, and prevents events from being missed or opportunities to experience things being lost.

[1382] Example 2

[1383] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1384] Conventional calendar systems require users to manually enter school and work schedules, which is time-consuming, and they also have the problem of not being able to adjust the priority of event notifications based on basic family information or emotions. Furthermore, they are unable to provide optimal event notifications based on the user's emotions, making it difficult to maximize the user's experience.

[1385] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1386] In this invention, the server includes means for registering school and work schedules in a calendar, means for enabling the registration by capturing school newsletters and shift schedules by photographing them, means for inputting basic information about family members, the basic information about the family members being address, age, grade, and height, the height being automatically measurable using a mobile information terminal, means for recognizing the user's emotions using an emotion engine, means for adjusting the priority of event notifications based on the recognized emotions, and means for receiving event notifications that combine family information and favorite genres into available calendar schedules and enabling reservations to be made. This reduces the user's effort and makes it possible to provide optimal event notifications according to their emotions.

[1387] "Means for registering school and work schedules in a calendar" is a function that allows a user to input and manage school and work schedules in a digital calendar.

[1388] "Means for enabling the registration by photographing the school newsletter or shift schedule" refers to a function that allows the user to photograph the school newsletter or shift schedule, analyze the image data, and automatically register the schedule information on the calendar.

[1389] The "means for inputting basic family information" is a function that allows the user to input basic information such as the address, age, grade, and height of family members into the system.

[1390] "The basic information of the family members is address, age, grade, and height, and height can be automatically measured using a mobile information terminal" means a function that includes address, age, grade, and height as basic information of the family members, and in particular automatically measures height using a mobile information terminal.

[1391] The "means for recognizing user emotions using an emotion engine" is a function for analyzing user behavior data and input data to identify the user's emotions.

[1392] The "means for adjusting the priority of event notifications based on the recognized emotions" is a function for adjusting the display order and importance of event notifications based on the user's emotions recognized by the emotion engine.

[1393] "A means for receiving event notifications that combine family information and favorite genres with available time slots on a calendar, allowing reservations" is a function that sends event notifications that take into account basic family information and the user's favorite genres based on available time slots on a calendar, allowing the user to reserve the event.

[1394] MODE FOR CARRYING OUT THE INVENTION

[1395] This invention is a system that allows a user to input basic information about their family, recognizes the user's emotions using an emotion engine, and adjusts the priority of event notifications based on the emotions. Specific embodiments of this system are described below.

[1396] Enter basic family information

[1397] The user logs into the system and moves to a screen where they can enter basic family information. Here, the user enters information such as address, age, grade, and height. Height, in particular, can be measured automatically using a mobile information terminal (smartphone). The user launches the smartphone's camera app and uses AR technology to overlay a tape measure on the camera image. This function can be realized using, for example, Apple's ARKit or Google's ARCore.

[1398] Emotion recognition

[1399] The server collects user behavioral data and input data, including social media posts and activity history within the system. The server uses an emotion engine to analyze this data and identify the user's emotions. For example, it determines whether the user is feeling "happy" or "sad."

[1400] Adjusting the priority of event notifications

[1401] The server receives the results of the emotion engine and prioritizes event notifications based on the user's emotions. For example, if the user is feeling happy, event notifications such as music concerts and movie premieres will be displayed first. Conversely, if the user is feeling sad, notifications for relaxing yoga classes and quiet reading groups will be displayed less. The server then sends event notifications to the user based on this priority.

[1402] Specific examples

[1403] For example, when a user enters basic information about their family members, they can use their smartphone camera to measure their heights. The user launches the camera app and uses AR technology to overlay a tape measure onto the camera image. This automatically measures the heights of their family members and enters that data into the system.

[1404] For example, if a user posts on social media, "I'm so happy today!", the server analyzes the content of the post and determines that the user is feeling happy. Based on this, the server prioritizes displaying event notifications such as music concerts and movie premieres.

[1405] Example prompts for generative AI models

[1406] Below are some example prompts to input to a generative AI model:

[1407] Describe the system's steps, where users enter basic family information. Specifically, explain how the smartphone camera and AR technology are used to measure height. Also, explain how the emotion engine prioritizes event notifications based on the emotions it recognizes.

[1408] Using this prompt, the generative AI model can generate a detailed description of the system.

[1409] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1410] Step 1:

[1411] A user logs in to the system. The user enters their user ID and password on the login screen, and if authentication is successful, they are taken to the main screen. The input is the user ID and password, and the output is the authentication result. If authentication is successful, the user can proceed to the next step.

[1412] Step 2:

[1413] The user is taken to a screen where they can enter basic information about their family. Here, they enter information such as address, age, grade, height, etc. The input is basic information such as address, age, grade, height, etc., and the output is that this information is saved in a database. Once the user has completed the input, they can proceed to the next step.

[1414] Step 3:

[1415] A user uses the smartphone camera to measure the height of a family member. The user launches the camera app and uses AR technology to overlay a tape measure on the camera image. The input is the camera image, and the output is the measured height data. The device sends this data to the system and stores it in a database.

[1416] Step 4:

[1417] The server collects user behavioral data and input data. This includes social media posts and activity history within the system. The input is user behavioral data and input data, and the output is that this data is stored in a database. The server prepares this data for analysis.

[1418] Step 5:

[1419] The server uses an emotion engine to analyze the collected data and identify the user's emotion. The input is the collected behavioral and input data, and the output is the user's emotional state. For example, it determines whether the user is feeling "happy" or "sad."

[1420] Step 6:

[1421] The server receives the results of the emotion engine and prioritizes event notifications based on the user's emotions. The input is the user's emotional state, and the output is prioritized event notifications. For example, if the user is feeling "happy," event notifications such as music concerts and movie premieres will be displayed with priority.

[1422] Step 7:

[1423] The server sends event notifications to users according to their priorities. The input is a prioritized event notification, and the output is a notification sent to the user's device. The user can receive this notification, check the details of the event, and make a reservation.

[1424] (Application example 2)

[1425] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1426] In conventional calendar systems, family schedules and event notifications were managed separately, making it impossible to provide optimal event notifications based on the user's emotions. It was also time-consuming to manually enter basic family information, and measuring height was particularly tedious. Furthermore, the priority of event notifications was not adjusted according to the user's emotions, which meant that event notifications were not tailored to the user's interests.

[1427] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1428] In this invention, the server includes: means for registering school and work schedules in a calendar; means for enabling the registration by capturing school newsletters and shift schedules by photographing them; means for inputting basic family information, the basic family information being address, age, grade, and height, with height being automatically measured using a smartphone; means for receiving event notifications that combine family information and favorite genres in the free schedules on the calendar and allowing reservations; means for recognizing the user's emotions using the smartphone's camera and microphone; and means for adjusting the priority of event notifications based on the recognized emotions. This makes it possible to provide optimal event notifications according to the user's emotions, simplify the input of basic family information, and provide event notifications that match the user's interests and concerns.

[1429] "Means for registering school and work schedules on a calendar" refers to a function that allows users to enter and manage school and work schedules on an electronic calendar.

[1430] "The means by which the registration can be made by taking a photograph of the school newsletter or shift schedule" refers to a function that allows a user to take a photograph of the school newsletter or shift schedule with a smartphone camera and automatically register that information in the calendar.

[1431] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[1432] "The basic information of the family members is their address, age, grade, and height, and height can be automatically measured using a smartphone" refers to a function that includes basic information of the family members, such as address, age, grade, and height, and in particular, automatically measures height using a smartphone's camera and AR technology.

[1433] "A means to receive event notifications that combine family information and favorite genres with available time on the calendar, allowing reservations" is a function that allows users to receive and reserve event notifications that take into account basic family information and the user's favorite genres based on available time on the calendar.

[1434] "Means for recognizing a user's emotions using a smartphone's camera or microphone" refers to a function that uses a smartphone's camera or microphone to analyze a user's facial expressions and voice and recognize their emotions.

[1435] The "means for adjusting the priority of event notifications based on the recognized emotions" is a function that recognizes the user's emotions and adjusts the display order and importance of event notifications according to those emotions.

[1436] The system for implementing this invention has the following configuration. First, the user uses a smartphone to enter basic family information. This basic information includes address, age, grade, and height, and height in particular is automatically measured using the smartphone's camera and AR technology. This allows the user to enter accurate information without any effort.

[1437] Next, the smartphone's camera and microphone are used to recognize the user's emotions. An emotion recognition engine (e.g., Microsoft Azure Cognitive Services) analyzes the user's facial expressions and voice to recognize emotions such as joy, anger, and sadness in real time. This emotion data is stored in a cloud database (e.g., Firebase Realtime Database).

[1438] In addition, event notifications are sent based on the user's free time on the calendar, combining family information and the user's favorite genres. The event notification system (e.g., Firebase Cloud Messaging) adjusts the priority of notifications according to the user's emotions. For example, if the user is expressing joy, event notifications in the user's favorite genres are displayed preferentially. Conversely, if the user is expressing anger or sadness, event notifications in the user's favorite genres are displayed less.

[1439] For example, when a user is at a shopping mall with their family, they can use their smartphone camera to measure their family members' heights and input the data into the system. If the user shows signs of joy while shopping, the system will prioritize notifications for their favorite fashion events.

[1440] An example of a prompt sentence is as follows:

[1441] Users are asked to enter basic information about their family members. The smartphone camera is used to measure their heights and enter this information into the system. The smartphone's camera and microphone are then used to recognize the user's emotions, and event notification priorities are adjusted based on the emotions. For example, if a user expresses joy, event notifications in the user's preferred genres will be displayed first.

[1442] In this way, it is possible to provide optimal event notifications according to the user's emotions, simplify the input of basic family information, and provide event notifications that match the user's interests and concerns.

[1443] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1444] Step 1:

[1445] The user uses a smartphone to enter basic information about their family members. The information entered includes address, age, grade, and height. In particular, height is measured automatically using the smartphone's camera and AR technology. A full-body photo of the family member is taken with the smartphone camera, and their height is measured using AR technology, and the data is sent to a cloud database. Input: Basic information about the family members (address, age, grade, height). Output: Basic information about the family members saved in the cloud database.

[1446] Step 2:

[1447] The device uses the smartphone's camera and microphone to recognize the user's emotions. Using an emotion recognition engine (e.g., Microsoft Azure Cognitive Services), it analyzes the user's facial expressions and voice in real time to recognize emotions such as joy, anger, and sadness. The recognized emotion data is stored in a cloud database. Input: Data on the user's facial expressions and voice. Output: Emotion data stored in the cloud database.

[1448] Step 3:

[1449] The server retrieves available calendar appointments and generates an event notification that combines family information and the user's favorite genres. An event notification system (e.g., Firebase Cloud Messaging) is used to adjust the priority of notifications according to the user's emotions. For example, if a user expresses joy, event notifications in the user's favorite genres are displayed preferentially. Input: Available calendar appointments, family information, user's favorite genres, emotion data. Output: Event notification with adjusted priority.

[1450] Step 4:

[1451] The device displays the event notification with the adjusted priority to the user. The user can check the notification, select the event of interest and make a reservation. The reservation information is stored in the cloud database and reflected in the calendar. Input: Event notification with adjusted priority. Output: Reservation information stored in the cloud database, event reflected in the calendar.

[1452] Step 5:

[1453] The server periodically analyzes the user's emotional data and event participation history, and generates new event notifications based on the user's interests. A generative AI model is used to predict the user's preferences and suggest the most suitable events. Input: Emotional data, event participation history. Output: New event notifications.

[1454] In this way, it is possible to provide optimal event notifications according to the user's emotions, simplify the input of basic family information, and provide event notifications that match the user's interests and concerns.

[1455] Example 3

[1456] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1457] With conventional calendar systems, users had to individually collect event information and manually register it in their calendars, which led to missed or duplicated events. Furthermore, event notifications were not based on the user's emotions or preferences, leading to low user satisfaction. Furthermore, it was difficult to provide event notifications that took into account the schedules of all family members, often limiting opportunities for family experiences.

[1458] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1459] In this invention, the server includes: a means for registering school and work schedules in a calendar; a means for enabling the registration by capturing images of documents and tables; a means for inputting basic family information; the basic family information including address, age, grade, and height, with height automatically measured using a mobile device; a means for receiving event notifications that combine family information and favorite genres in free calendar appointments and allowing reservations; a means for analyzing user emotion data and adjusting the priority of event notifications; and a means for the emotion data to include feedback and emotion data from past notifications. This prevents users from missing events and allows them to receive optimal event notifications that take into account the schedules of all family members. Furthermore, since event notifications are provided based on the user's emotions and preferences, user satisfaction is improved.

[1460] "Means for registering school or work schedules in a calendar" is a function that allows users to enter school or work schedules into an electronic calendar.

[1461] "Means possible by capturing images of documents and tables" refers to a function that captures images of paper documents and tables with a camera, analyzes the images, and captures them as digital data.

[1462] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[1463] "Means that can be measured automatically using a mobile device" refers to a function that automatically measures physical data such as height using a mobile device such as a smartphone or tablet.

[1464] "A means of receiving event notifications that combine family information and favorite genres with available time on the calendar, allowing reservations" is a function that notifies users of event information based on available time on the user's calendar, taking into account family information and the user's favorite genres, and allows them to reserve the event.

[1465] "Means for analyzing user emotional data and adjusting the priority of event notifications" refers to a function that analyzes a user's past emotional data and adjusts the display order and importance of event notifications based on the results.

[1466] "Feedback and emotion data on past notifications" refers to the emotion and evaluation data that a user has shown on notifications they have received in the past.

[1467] This invention is a system that provides event notifications that combine family information and favorite genres selected by the user based on the free time in the user's calendar. It also has a function that uses an emotion engine to learn changes in the user's emotions and reflects the results in adjusting the priority of event notifications.

[1468] Hardware and software used

[1469] Server: Cloud computing service (e.g., Amazon Web Services)

[1470] Database: Relational database service (e.g. Amazon RDS)

[1471] Emotion engine: Natural language processing API (e.g., Google Cloud Natural Language API)

[1472] Event information collection: Web scraping tools (e.g., Beautiful Soup, Scrapy)

[1473] Notification system: Cloud messaging service (e.g., Firebase Cloud Messaging)

[1474] Program processing

[1475] Retrieving User Information

[1476] The server retrieves the user's calendar information and family information from the database. Specifically, it calls the user's Google Calendar API to retrieve event information. It also retrieves family information by executing a query from the database.

[1477] Collecting event information

[1478] The server uses a web scraping tool to collect related event information based on the user's favorite genre. For example, if the user selects "art," the server collects art-related event information from the official museum website and event information sites.

[1479] Event filtering

[1480] The server compares the collected event information with the available times on the user's calendar and filters out events that are held at times that do not overlap with family schedules. For example, it selects art exhibitions that are held at times when the user's calendar is available.

[1481] Priority adjustment by emotion engine

[1482] The server uses a natural language processing API to analyze users' past emotional data and learn how their emotions change in response to events of a specific genre. For example, if a user previously expressed joy when receiving notifications about an art exhibition, the server will prioritize notifications for events of that genre.

[1483] Sending notifications

[1484] The server then uses a cloud messaging service to notify users of the filtered event information, for example by sending a notification to their smartphone saying, "There's an art exhibition this weekend."

[1485] Specific examples

[1486] User A has selected "art" as his favorite genre.

[1487] The server calls User A's Google Calendar API and confirms that he is free on Saturday afternoon this weekend.

[1488] The server uses Beautiful Soup to collect information about art exhibitions being held this weekend from the museum's official website.

[1489] The server filters out art exhibitions that are taking place at times that don't conflict with family schedules.

[1490] Using a natural language processing API, the server learns that User A expressed joy when receiving notifications about art exhibitions in the past.

[1491] The server uses a cloud messaging service to send a notification to User A saying, "There's an art exhibition this weekend."

[1492] Prompt Sentence Examples

[1493] Obtain the user's calendar information and family information, and collect relevant event information based on the user's favorite genres. Compare the collected event information with the user's available calendar schedules and filter out events that take place at times that do not overlap with family schedules. Furthermore, use an emotion engine to learn changes in the user's emotions and reflect the results in adjusting the priority of event notifications. Finally, notify the user of the filtered event information.

[1494] The flow of the identification process in the third embodiment will be described with reference to FIG.

[1495] Step 1:

[1496] Retrieving User Information

[1497] The server retrieves the user's calendar information and family information from the database. Specifically, it calls the user's Google Calendar API to retrieve event information. It also retrieves family information by executing a query from the database.

[1498] Input: User ID, Google Calendar API token

[1499] Data processing: Use the Google Calendar API to obtain schedule information and query family information from the database.

[1500] Output: User's calendar information, family information

[1501] Step 2:

[1502] Collecting event information

[1503] The server uses a web scraping tool to collect related event information based on the user's favorite genre. For example, if the user selects "art," the server collects art-related event information from the official museum website and event information sites.

[1504] Input: User's favorite genre

[1505] Data processing: Use web scraping tools (Beautiful Soup, Scrapy) to collect relevant event information.

[1506] Output: Collected event information

[1507] Step 3:

[1508] Event filtering

[1509] The server compares the collected event information with the available times on the user's calendar and filters out events that are held at times that do not overlap with family schedules. For example, it selects art exhibitions that are held at times when the user's calendar is available.

[1510] Input: User's calendar information, family information, collected event information

[1511] Data processing: Compare the user's calendar information with family schedule information and list events that take place at different times.

[1512] Output: Filtered event information

[1513] Step 4:

[1514] Priority adjustment by emotion engine

[1515] The server uses a natural language processing API to analyze users' past emotional data and learn how their emotions change in response to events of a specific genre. For example, if a user previously expressed joy when receiving notifications about an art exhibition, the server will prioritize notifications for events of that genre.

[1516] Input: User's past emotional data, filtered event information

[1517] Data processing: Analyze sentiment data using natural language processing APIs and adjust the priority of event notifications.

[1518] Output: Priority adjusted event information

[1519] Step 5:

[1520] Sending notifications

[1521] The server then uses a cloud messaging service to notify users of the filtered event information, for example by sending a notification to their smartphone saying, "There's an art exhibition this weekend."

[1522] Input: Priority adjusted event information

[1523] Data processing: Send notifications using a cloud messaging service (Firebase Cloud Messaging).

[1524] Output: Notification sent to the user's phone

[1525] (Application example 3)

[1526] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1527] With conventional calendar systems, users had to search for event information themselves and adjust it based on their family schedules, which was time-consuming and made it easy for events to be missed.In addition, the priority of event notifications was not adjusted based on the user's emotions or interests, making it difficult to provide the most appropriate event information for the user.

[1528] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: means for registering school and work schedules in a calendar; means for enabling the registration by capturing school newsletters and shift schedules by photographing them; means for inputting basic family information; means for the basic family information being address, age, grade, and height, with height being automatically measured using a smartphone; means for receiving event notifications that combine family information and favorite genres in available calendar appointments and allowing reservations; means for using an emotion engine to learn changes in the user's emotions and adjust the priority of event notifications; and means for the emotion engine to adjust the priority of event notifications based on the user's past reactions. This allows users to receive optimal event information without hassle, prevents missing out on events, and increases opportunities for family experiences.

[1529] "Means for registering school and work schedules in a calendar" is a function that allows users to enter school and work schedules into a digital calendar.

[1530] "Means that allow the registration by taking a photograph of a school newsletter or shift schedule" refers to a function that takes a photograph of a newsletter distributed by a school or a workplace shift schedule with a smartphone camera, automatically extracts a schedule from the image, and registers it on a calendar.

[1531] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[1532] "The basic information of the family member is the address, age, grade, and height, and the height can be automatically measured using a smartphone" refers to a function for inputting the address, age, grade, and height of the family member, and in particular, a function for automatically measuring height using a smartphone sensor.

[1533] "A means for notifying users of upcoming events that combine family information and their favorite genres with available time on the calendar, allowing them to make reservations" is a function that detects available time on the calendar, notifies users of event information based on basic family information and the genre selected by the user, and allows them to reserve the event.

[1534] "Means of using an emotion engine to learn changes in users' emotions and adjust the priority of event notifications" is a function that analyzes users' emotional data, learns what types of events users respond positively to, and adjusts the priority of event notifications based on the results.

[1535] "Means for the emotion engine to adjust the priority of event notifications based on the user's past reactions" refers to a function that records the emotional reactions shown by the user to event notifications received in the past and adjusts the priority of future event notifications based on that data.

[1536] The system for implementing this invention is configured using a user's smartphone, a server, and an external API. Specific embodiments are described below.

[1537] 1. System Configuration

[1538] The system consists of the following main components:

[1539] Smartphone: A device that allows users to register school and work schedules on a calendar and enter basic family information. It also has the ability to take photos of school newsletters and shift schedules using the smartphone's camera and extract schedules from those images.

[1540] Server: Collects event information, analyzes user emotion data using an emotion engine, and adjusts the priority of event notifications.

[1541] External API: Used to collect event information. For example, using the API of an event information provider.

[1542] 2. Program Processing

[1543] The server performs the following process.

[1544] Get free events from the calendar: Uses the Google Calendar API to get free events from the user's calendar.

[1545] Event information collection: Use external APIs to collect event information related to the genre selected by the user.

[1546] Prioritization using an emotion engine: Use an emotion engine (e.g., Microsoft Azure Emotion API) to analyze user emotion data and adjust the priority of event notifications.

[1547] 3. Specific Examples

[1548] For example, if a user selects "art" as their favorite genre and wants to be notified of art exhibitions that are being held at times that don't conflict with family schedules, the emotion engine will learn from the user's past reactions and prioritize displaying notifications about art exhibitions.

[1549] 4. Examples of prompts

[1550] Below are some example prompts to input to the generative AI model:

[1551] Create a Python program that retrieves free events from the user's calendar, collects event information based on family information and the user's favorite genre (e.g., art), adjusts the priority using an emotion engine, and notifies the user.

[1552] In this way, users can easily receive the most appropriate event information, prevent missing out on events, and increase opportunities for family experiences.

[1553] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1554] Step 1:

[1555] Users use their smartphones to register school and work schedules in a calendar. They take pictures of school newsletters and shift schedules with their smartphone cameras and upload the images to the system. The system uses image recognition technology to extract schedule information from the captured image and automatically register it in the calendar. The input is the captured image, and the output is the schedule information registered in the calendar.

[1556] Step 2:

[1557] The user inputs basic family information using a smartphone. The user inputs information such as address, age, grade, and height, and height is automatically measured using the smartphone's sensor. The input is basic family information, and the output is basic family information stored in the system.

[1558] Step 3:

[1559] The server uses the Google Calendar API to retrieve free events from the user's calendar. The server finds free time within a specified period and retrieves that information. The input is the user's calendar information, and the output is a list of free events.

[1560] Step 4:

[1561] The server uses an external API to collect event information related to the user's favorite genre. The server then calls the API of an event information service based on the user's interests to obtain related event information. The input is the user's favorite genre information, and the output is the collected event information.

[1562] Step 5:

[1563] The server uses an emotion engine to analyze the user's emotional data. The server records the user's emotional responses to past event notifications and learns the user's emotional changes based on that data. The input is the user's past emotional data, and the output is the learned emotion model.

[1564] Step 6:

[1565] The server uses the emotion engine to adjust the priority of event notifications. Based on the learned emotion model, the server adjusts the priority of notifications to events to which the user has a positive reaction. The input is the learned emotion model and collected event information, and the output is a list of event notifications with adjusted priorities.

[1566] Step 7:

[1567] The server sends the priority-adjusted event notification list to the user's smartphone. The user receives the notifications on their smartphone, checks the events of interest, and can make reservations. The input is the priority-adjusted event notification list, and the output is the event notification displayed on the user's smartphone.

[1568] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1569] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1570] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1571] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1572] [Third embodiment]

[1573] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1574] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1575] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1576] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1577] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1578] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1579] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1580] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1581] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1582] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1583] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1584] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1585] "Example 1"

[1586] In the system of the present invention, users can register school and work schedules in a calendar. This registration can be done by taking a photo of a school newsletter or shift schedule. For example, if a user takes a photo of a school report card with a smartphone camera, the system extracts text from the image and registers the schedule in the calendar based on that information.

[1587] "Example 2"

[1588] Users also enter basic family information into the system, including family members' addresses, ages, grades, and heights. Height, in particular, can be measured automatically using a smartphone. For example, the smartphone's camera and augmented reality (AR) technology can be used to measure the heights of family members and enter that information into the system.

[1589] "Example 3"

[1590] Furthermore, the system will provide event notifications based on free calendar appointments, combining family information and the user's favorite genres. For example, if a user selects "art" as their favorite genre, the system will collect information about art-related events and notify the user if the event is held at a time that does not overlap with family schedules. This prevents users from missing out on events and increases opportunities for family experiences.

[1591] The processing flow of each embodiment will be described below.

[1592] "Example 1"

[1593] Step 1: The user takes a photo of their school or work schedule with their smartphone camera, such as a school report card or a work shift schedule.

[1594] Step 2: The system extracts text from the captured image using OCR (Optical Character Recognition) technology.

[1595] Step 3: The system uses the extracted text information to add an event to the user's calendar.

[1596] "Example 2"

[1597] Step 1: The user enters basic family information into the system, including information such as the family member's address, age, grade, and height.

[1598] Step 2: The user uses their smartphone camera and AR (augmented reality) technology to measure the heights of their family members.

[1599] Step 3: The system updates basic information about the family based on the height information measured.

[1600] "Example 3"

[1601] Step 1: The user selects their favorite genre for the system. For example, they select "art."

[1602] Step 2: The system collects information about events related to the selected genre.

[1603] Step 3: The system compares the collected event information with the user's calendar, and if the event is held at a time that does not conflict with family schedules, the system notifies the user.

[1604] Example 1

[1605] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1606] Traditional calendar registration systems require users to manually enter appointments, which is time-consuming and prone to input errors. Furthermore, there is no system that automatically generates event notifications based on basic family information or preferred genres, so users may miss many events. Furthermore, there is a lack of efficient ways to manage basic family information.

[1607] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1608] In this invention, the server includes a means for sending the captured image to the server, a means for the server to extract text from the image, and a means for the server to analyze the extracted text. This allows users to automatically register events on their calendar simply by taking a photo of a school newsletter or shift schedule. Furthermore, by entering basic family information, event notifications are automatically generated based on available calendar schedules, allowing users to make reservations, preventing users from missing out on events and increasing opportunities to experience them.

[1609] "Means for registering school and work schedules on a calendar" refers to a function that allows users to enter and manage school and work schedules on an electronic calendar.

[1610] "Means that allow for the capture of school newsletters and shift schedules by photographing them" refers to a function that allows users to use a camera on a smartphone or tablet to take a photo of a school newsletter or shift schedule and import that image into the system.

[1611] The "means for transmitting captured images to a server" is a function that allows a user to upload image data captured by the user to a server via the Internet.

[1612] The "means for the server to extract text from images" refers to a function for the server to extract text information from image data received using optical character recognition (OCR) technology.

[1613] "Means for analyzing text extracted by the server" refers to a function in which the server analyzes the extracted text data using natural language processing (NLP) technology and converts it into a format for registration on the calendar.

[1614] The "means for the server to register the analysis results in the calendar" is a function that enables the server to register the analyzed text data in a calendar application and add an event to the user's calendar.

[1615] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[1616] "Means for automatically measuring height using a mobile information terminal" refers to a function that allows a user to use a mobile information terminal to automatically measure the height of family members and input that data into the system.

[1617] "A means for receiving event notifications that combine family information and favorite genres with available time slots on the calendar, allowing users to make reservations" is a function in which the system generates event notifications based on basic family information and favorite genres, based on available time slots on the calendar, and allows users to reserve those events.

[1618] This invention is a system that allows users to register school and work schedules on a calendar. Users can easily register schedules by taking photos of school newsletters or shift schedules. This system is implemented using a mobile information terminal such as a smartphone or tablet and a server.

[1619] A user uses the smartphone camera to take a picture of a school newsletter or shift schedule. The image is sent to a server via a smartphone application. The server extracts text from the image using image processing software (e.g., a common cloud-based image recognition API). The extracted text is analyzed using natural language processing (NLP) software (e.g., a common NLP library) and converted into a format suitable for a calendar event. The server then registers the analyzed data with a calendar application (e.g., a common calendar API).

[1620] As a concrete example, consider the case where a user takes a photo of their school report card. The user takes a photo of the report card with their smartphone camera and sends the image to a server through an application. The server uses a cloud-based image recognition API to extract text from the image and analyzes the text using an NLP library. The analyzed information is then registered in the user's calendar using a calendar API. For example, if the report card contains information such as "PTA meeting on October 15th at 2:00 PM," this is registered as "PTA meeting" on the calendar.

[1621] Example prompt sentence:

[1622] "Take a photo of your school report card and put the event on your calendar."

[1623] This system allows users to efficiently manage their schedules by eliminating the need to manually enter schedules. By entering basic family information, the system automatically generates event notifications based on available calendars and allows users to make reservations, eliminating the need to miss out on events and increasing opportunities to experience them.

[1624] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1625] Step 1:

[1626] The user takes a photo of a school newsletter or shift schedule.

[1627] Specifically, the user launches the camera app on their smartphone, frames the school newsletter or shift schedule, and presses the shutter button. The input is the physical paper copy of the school newsletter or shift schedule, and the output is the image data stored on the smartphone.

[1628] Step 2:

[1629] The device sends the captured image to the server.

[1630] Specifically, the smartphone application uploads image data to the server as an HTTP request. The input is the image data stored on the smartphone, and the output is the image data sent to the server.

[1631] Step 3:

[1632] The server extracts the text from the image.

[1633] Specifically, the server uses a cloud-based image recognition API to extract text information from the received image data. The input is the image data sent to the server, and the output is the extracted text data.

[1634] Step 4:

[1635] The server parses the extracted text.

[1636] Specifically, the server uses a natural language processing (NLP) library to parse the extracted text data and convert it into a format suitable for calendar entry. The input is the extracted text data, and the output is the parsed text data.

[1637] Step 5:

[1638] The server registers the analysis results in the calendar.

[1639] Specifically, the server uses a calendar API to register the parsed text data in the user's calendar. The input is the parsed text data, and the output is the schedule registered in the calendar.

[1640] Step 6:

[1641] The user enters basic information about their family.

[1642] Specifically, the user enters basic information about the family members, such as their address, age, grade, height, etc. The input is the basic information entered by the user, and the output is the basic information stored on the server.

[1643] Step 7:

[1644] The server generates an event notification based on the available schedules in the calendar.

[1645] Specifically, the server detects free time on the calendar based on the basic information and favorite genres of the family members, and generates appropriate event notifications. The inputs are free time on the calendar, basic information of the family members, and favorite genres, and the output is the generated event notification.

[1646] Step 8:

[1647] A user books an event.

[1648] Specifically, the user checks the event notification sent from the server through the application, selects the desired event, and reserves it. The input is the event notification sent from the server, and the output is the reserved event information.

[1649] (Application example 1)

[1650] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1651] Conventional calendar registration systems required users to manually enter school and work schedules, which was time-consuming and labor-intensive. Also, manually registering shift schedule information into the calendar could result in input errors or missing information. Furthermore, managing basic family information and event notifications was cumbersome, making efficient schedule management difficult.

[1652] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1653] In this invention, the server includes means for registering school and work schedules in a calendar, means for enabling the registration by capturing school newsletters and shift schedules by photographing them, means for inputting basic family information, the basic family information being address, age, grade, and height, with height being automatically measured using a smartphone, means for receiving event notifications that combine family information and favorite genres in available calendar schedules and making reservations, and means for extracting text from images of shift schedules, analyzing shift information, and automatically registering it in the calendar. This allows users to easily register school and work schedules in the calendar, prevents input errors and omissions in shift information, and enables efficient schedule management.

[1654] "Means for registering school and work schedules on a calendar" refers to a function that allows users to enter and manage school and work schedules on an electronic calendar.

[1655] "The means by which the registration can be made by taking a photograph of the school newsletter or shift schedule" refers to a function that allows a user to take a photograph of the school newsletter or shift schedule with a smartphone camera, extract text from the image, and automatically register the schedule in the calendar.

[1656] "Means for entering basic family information" is a function that allows users to enter basic information such as family member's address, age, grade, height, etc. into the system.

[1657] "The basic information of the family members is their address, age, grade, and height, and height can be automatically measured using a smartphone" means that basic information of the family members includes their address, age, grade, and height, and height in particular can be automatically measured using a smartphone function.

[1658] "A means to receive event notifications that combine family information and favorite genres with available time slots on the calendar, allowing reservations" is a function that allows the system to send event notifications that take into account basic family information and favorite genres based on available time slots on the calendar, and allows reservations for those events.

[1659] "Means for extracting text from an image of a shift schedule, analyzing the shift information, and automatically registering it in a calendar" is a function that analyzes an image of a shift schedule, extracts text information, and automatically registers that shift information in a calendar.

[1660] The system for implementing this invention allows users to efficiently register school and work schedules on a calendar and prevents input errors and omissions of shift information. A specific embodiment of this system will be described below.

[1661] First, the user takes a photo of the school newsletter or shift schedule using their smartphone. The smartphone has OpenCV installed as image processing software, which reads the captured image. Next, text is extracted from the image using Tesseract OCR. Through this text extraction process, the information in the shift schedule or school newsletter is obtained as digital data.

[1662] The acquired text data is sent to a server and analyzed. The analysis process identifies lines containing shift information and school schedules and extracts information such as dates, start times, and end times. Natural language processing technology is used for this analysis process.

[1663] The analyzed information is automatically registered in the user's calendar using a calendar API (e.g., Google Calendar API), which saves the user the trouble of manually entering events.

[1664] The system also has the ability to manage basic family information. Users can input information such as address, age, grade, and height, and height can be automatically measured using a smartphone sensor. This information is stored on the server, and event notifications tailored to the family's preferences are sent based on available calendar availability.

[1665] As a concrete example, consider a scenario in which a store clerk takes a photo of a shift schedule and the shift information is automatically registered in a calendar. In this case, the store clerk takes a photo of the shift schedule with their smartphone, and the system extracts text from the image, analyzes the shift information, and registers it in the calendar. This prevents input errors and omissions of shift information.

[1666] Example prompt sentence:

[1667] Please create an app that takes a photo of a shift schedule and automatically registers it in a calendar. Include a function to extract text from the shift schedule image, parse the shift date, start time, and end time, and register it in the calendar.

[1668] In this way, users can easily register school and work schedules on their calendar, preventing input errors and omissions in shift information and enabling efficient schedule management.

[1669] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1670] Step 1:

[1671] A user takes a photo of a school newsletter or shift schedule using a smartphone. The input is the captured image, and the output is an image file. Specifically, the user launches the smartphone's camera app and takes a photo of the school newsletter or shift schedule.

[1672] Step 2:

[1673] The device uses OpenCV to read the captured image. The input is an image file and the output is image data. Specifically, the device uses the OpenCV library to read the image file into memory.

[1674] Step 3:

[1675] The device uses Tesseract OCR to extract text from an image. The input is image data, and the output is extracted text data. Specifically, the device invokes the Tesseract OCR engine to parse and extract text from the image data.

[1676] Step 4:

[1677] The terminal transmits the extracted text data to the server. The input is the text data, and the output is the text data transmitted to the server. In specific operations, the terminal transmits the text data to the server via network communication.

[1678] Step 5:

[1679] The server analyzes the received text data and identifies shift information and school schedules. The input is the received text data, and the output is the analyzed shift information and schedule information. Specifically, the server analyzes the text data using natural language processing technology and extracts information such as date, start time, and end time.

[1680] Step 6:

[1681] The server uses a calendar API to register the analyzed information in the calendar. The input is the analyzed shift information and schedule information, and the output is the schedule registered in the calendar. Specifically, the server calls a calendar API such as the Google Calendar API and registers the analyzed information in the calendar.

[1682] Step 7:

[1683] The server manages basic information about family members and generates event notifications based on the free time on the calendar. The input is the basic information about family members and the free time on the calendar, and the output is the generated event notification. Specifically, the server takes into account information such as the address, age, grade, and height of family members and generates event notifications based on the free time on the calendar.

[1684] Step 8:

[1685] The server sends the generated event notification to the user. The input is the generated event notification, and the output is the event notification sent to the user. In concrete terms, the server sends the event notification to the user's terminal via network communication.

[1686] In this way, users can easily register school and work schedules on their calendar, preventing input errors and omissions in shift information and enabling efficient schedule management.

[1687] Example 2

[1688] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1689] Conventional calendar systems require users to manually enter school and work schedules, which is time-consuming and prone to input errors. Similar problems arise when manually entering basic family information. Furthermore, event notifications are not based on individual users' interests or family information, meaning users may miss events that are beneficial to them. There is a need to solve these problems and improve user convenience.

[1690] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1691] In this invention, the server includes a means for registering school and work schedules in a calendar, a means for capturing documents and tables by taking pictures, a means for inputting basic family information, a means for measuring heights using a mobile device's camera and augmented reality technology and inputting the data into the system, a means for generating event notifications by inputting prompt sentences into a generative AI model, and a means for receiving event notifications that combine family information and favorite genres with available calendar entries and allowing reservations. This allows users to easily and accurately register schedules and easily input basic family information. Furthermore, event notifications based on the user's interests and family information are provided, allowing users to participate in useful events without missing them.

[1692] The "means for registering school and work schedules in a calendar" is a function that allows a user to input and manage school and work schedules in an electronic calendar.

[1693] "Means that allow documents and tables to be captured by image capture" refers to a function that allows a user to use the camera on a smartphone or tablet to capture a document or table on paper and import the contents into the system as digital data.

[1694] The "means for inputting basic family information" is a function that allows the user to input basic information such as the address, age, grade, and height of family members into the system.

[1695] "Means for measuring height using a mobile device's camera and augmented reality technology and inputting that data into the system" refers to a function that allows users to measure the height of their family members using a smartphone or tablet's camera and AR technology, and automatically input the measurement results into the system.

[1696] "Means for generating event notifications by inputting prompt sentences into a generative AI model" refers to a function that allows a user to input specific prompt sentences into a generative AI model, which in turn generates event notifications based on the user's interests and family information.

[1697] "A means for receiving event notifications that combine family information and favorite genres in available calendar slots, allowing reservations" is a function that enables the system to detect available times on the calendar, send event notifications to the user that combine basic family information and the user's interests, and allow the event to be reserved.

[1698] The present invention is a system that allows a user to input basic family information, register school and work schedules in a calendar, and generate event notifications. A specific embodiment of this system will be described below.

[1699] Hardware and software used

[1700] Hardware

[1701] Mobile device: Smartphone or tablet (with camera)

[1702] software

[1703] Applications that use AR technology: For example, ARKit for iOS, ARCore for Android

[1704] Generative AI model: An AI model that ge...

Claims

1. means for obtaining a user's calendar information and family information including family schedules; means for collecting event information related to a preselected genre that is to be held on the user's free days based on the calendar information; A means for filtering out, from the collected event information, event information to be held during a time period that does not overlap with family schedules; means for notifying the user of the filtered event information; A system including:

2. 2. The system according to claim 1, wherein the means for notifying the user notifies the user of the filtered event information by email.

3. The system according to claim 1 , further comprising: means for recognizing an emotion of the user and setting a priority of the event information to be notified based on the recognized emotion.

Images