system

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting a mountain climbing plan from any of URL paper images of other company applications provided by a mountain climber; means for using a generative data model that analyzes the collected mountain climbing plan and converts the mountain climbing plan into a digital format; means for centrally managing the analyzed mountain climbing plan AI; means for periodically acquiring position information of the mountain climber from a mobile terminal and collating the position information with the collected mountain climbing plan; and means for notifying the mountain climber when the mountain climbing plan does not match the current position information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, mountain accidents have been increasing, creating a need for systems that enable police and local governments to quickly identify lost climbers. However, climbers' climbing plans are currently managed across multiple applications and paper media, making it difficult to centrally understand them, resulting in scattered information. Furthermore, there are issues with inconsistencies between plans and actual behavior, or with difficulty in identifying stalled climbers early on, reducing the efficiency of search and rescue efforts. This invention aims to solve these issues by centrally managing climbing plans and climbers' location information and monitoring them in real time. [Means for solving the problem]

[0005] The present invention solves these problems with a system that includes: means for collecting mountain climbing plans from either voice data provided by climbers, URLs of other companies' apps, or paper images; means for using a generative AI model to analyze the collected mountain climbing plans and convert them into a digital format; means for centrally managing the analyzed mountain climbing plan data; means for periodically obtaining the climber's location information from the mobile device and comparing it with the collected mountain climbing plans; means for generating an alert and notifying the climber if there is a discrepancy between the mountain climbing plan and the current location information; and means for notifying mountain huts and other climbers if the climber is delayed or does not descend the mountain as planned.

[0006] "Climber" refers to an individual whose purpose is to climb a mountain.

[0007] "Mountain climbing plan" refers to information including plans and action plans for mountain climbing that a climber has made in advance.

[0008] "Audio data" refers to data in which sound is recorded in digital format.

[0009] "Third-party app URL" refers to a web link for an application provided by another company or organization.

[0010] "Paper image" refers to an image file that stores information printed on paper in digital format.

[0011] A "generative AI model" refers to an algorithm or system that uses artificial intelligence to analyze and convert data such as audio and images.

[0012] "Digital format" refers to data that is represented in a form that can be processed by a computer.

[0013] "Centralized management" refers to the centralized management of multiple different pieces of information in one place.

[0014] "Mobile device" refers to a portable electronic device such as a mobile phone or smartphone.

[0015] "Location information" refers to information that indicates a physical location or coordinates identified using technology such as GPS.

[0016] "Mismatch" refers to a difference between planned or scheduled behavior and actual behavior.

[0017] An "alert" is a notification or warning that notifies you when an abnormality or emergency occurs.

[0018] "Stagnation" refers to a state in which a climber remains in the same place for a period of time without carrying out any planned actions.

[0019] A "mountain hut" refers to a building set up in the mountains for the purpose of rest and shelter for climbers. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] This invention is a system that centrally manages and monitors climbers' location information and climbing plans in real time. This system collects climbing plan data provided by climbers in various formats (voice, URLs of other companies' apps, paper images), analyzes it using a generative AI model, converts it into a digital format, and centrally manages it. As a result, it ensures the safety of climbers and enables the prevention and rapid response of accidents.

[0042] Server program processing

[0043] Data collection and analysis

[0044] The server receives the voice data provided by the climber, the URL of the third-party app, and the paper image. This data is analyzed using the generative AI model and converted into a standard digital format. For example, if a climber says, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[0045] Centralized data management

[0046] The server stores the analyzed climbing plans in a digital format in a database and manages them centrally, allowing each climber's plan and progress to be visualized in one place and checked in real time.

[0047] Location Tracking

[0048] The server receives location information periodically sent from the climber's mobile device. This location information is checked against the climbing plan to see if it matches the plan. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[0049] Conflict notification and stagnation management

[0050] If there is a discrepancy between the location information and the climbing plan, the server immediately generates an alert and sends a notification to the climber's mobile device. If the climber is delayed or does not descend as planned, notifications are sent to the mountain hut and other climbers so that a prompt response can be taken.

[0051] Terminal program processing

[0052] User authentication and mountain climbing plan input

[0053] The device requires users to create an account and log in when they first start the device. After that, users can input their hiking plans using voice commands, URLs from other companies' apps, or images of paper. This data is sent to a server for analysis.

[0054] Sending location information

[0055] The device activates its GPS function when the climb begins, acquires location information at regular intervals, and sends it to the server. This location information is then checked against the climbing plan to confirm its consistency.

[0056] Receive notifications

[0057] If the device detects a discrepancy between the plan and location information, or if the climber is delayed, it will immediately receive an alert notification and display it to the user in a pop-up.

[0058] User Examples

[0059] Initial setup and climbing plan input

[0060] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by verbally inputting, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the climbing plan is sent to the server and analyzed.

[0061] Start climbing and share your location

[0062] When a user arrives at the trailhead, they press the "Start Climbing" button on the app. The device activates its GPS function and sends its location information to the server every 15 minutes. The location information is compared with the climbing plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[0063] Specific examples

[0064] A user vocally inputs, "Tomorrow I'm going to climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and stores this information. If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending it to the user's device. The user receives the notification and takes appropriate measures to ensure their safety. In this way, the system can prevent accidents and quickly identify stranded individuals.

[0065] The processing flow will be explained below.

[0066] User creates an account

[0067] Step 1:

[0068] A user downloads the mobile app and launches it for the first time.

[0069] Step 2:

[0070] The user enters basic information such as name, email address, and password, and clicks the registration button.

[0071] Step 3:

[0072] The terminal transmits the user's registration information to the server.

[0073] Step 4:

[0074] The server stores the received user information in a database.

[0075] Step 5:

[0076] The server sends a confirmation message to the terminal to confirm the completion of registration, and the terminal displays the confirmation message.

[0077] The user inputs a mountain climbing plan

[0078] Step 1:

[0079] The user logs in and selects "Enter Mountain Climbing Plan" from the main menu.

[0080] Step 2:

[0081] The user selects one of the following methods: "Voice input," "URL of a third-party app," or "Image upload."

[0082] Step 3:

[0083] For example, a user might say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 p.m."

[0084] Step 4:

[0085] The device records the audio data and sends it to the server.

[0086] Step 5:

[0087] The server analyzes the voice data using a generative AI model and converts it into text data.

[0088] Step 6:

[0089] The server stores the parsed text data in a database and associates it with the user's account.

[0090] User starts climbing

[0091] Step 1:

[0092] The user arrives at the trailhead and launches the app.

[0093] Step 2:

[0094] The user presses the "Start climbing" button.

[0095] Step 3:

[0096] The device will activate the GPS function and begin acquiring location information.

[0097] Step 4:

[0098] The device periodically (e.g., every 15 minutes) sends its location information to the server.

[0099] Location monitoring and discrepancy notification during mountain climbing

[0100] Step 1:

[0101] The server compares the received location information with the mountain climbing plan.

[0102] Step 2:

[0103] The server compares the climbing plan (e.g., scheduled arrival at the mountain hut at 13:00) with the current location information.

[0104] Step 3:

[0105] If the server has not reached the hut by the scheduled time (13:00), it detects a discrepancy.

[0106] Step 4:

[0107] If the server detects a mismatch, it generates an alert message and sends it to the terminal.

[0108] Step 5:

[0109] The device displays a warning to the user saying, "You have not arrived at the mountain hut by the scheduled time. Please check that your surroundings are safe."

[0110] Notification of stagnation and those who have not yet descended

[0111] Step 1:

[0112] The server periodically compares the location information of all users with their mountain climbing plans.

[0113] Step 2:

[0114] If the server detects stagnation or failure to descend, it generates an alert list.

[0115] Step 3:

[0116] The server sends alerts of stagnation or failure to descend the mountain to the relevant user's device.

[0117] Step 4:

[0118] The device displays a warning to the user saying, "You have been stuck at your current location for an extended period of time. Please make sure you are safe."

[0119] Step 5:

[0120] The server also sends relevant notifications to mountain huts and other climbers, encouraging them to take prompt action.

[0121] The server centrally manages data

[0122] Step 1:

[0123] The server stores all climbers' climbing plans, location information, and conflict data in a database.

[0124] Step 2:

[0125] The server updates and displays the location information and mountain climbing plan received in real time.

[0126] Step 3:

[0127] If the server needs search and rescue, it will provide relevant information to the police and local authorities.

[0128] Specific examples

[0129] Initial setup and climbing plan input

[0130] Step 1:

[0131] A user launches the app for the first time and creates an account by entering their name, email address, password, etc.

[0132] Step 2:

[0133] The user says in a voice message, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 p.m."

[0134] Step 3:

[0135] The server analyzes the audio and converts it into text data.

[0136] Step 4:

[0137] The server stores the parsed data in a database and associates it with the user's account.

[0138] Start climbing and share your location

[0139] Step 1:

[0140] The user arrives at the trailhead and presses the "Start Climbing" button on the app.

[0141] Step 2:

[0142] The device will enable the GPS function and send location information to the server every 15 minutes.

[0143] Step 3:

[0144] The server compares the location information with the climbing plan to check for any discrepancies.

[0145] Step 4:

[0146] If things aren't going according to plan, the user is immediately alerted.

[0147] Inconsistency warnings and actions

[0148] Step 1:

[0149] The server detects that the user has not yet arrived at the mountain hut even though it is now 13:00.

[0150] Step 2:

[0151] The server detects the discrepancy and generates an alert that is sent to the user terminal.

[0152] Step 3:

[0153] The terminal displays a warning to the user that "You have not arrived at the mountain hut even though the scheduled time has passed."

[0154] Step 4:

[0155] The user checks the notification to check safety and understand the situation.

[0156] Example 1

[0157] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0158] When climbing mountains, ensuring the safety of climbers and responding quickly are essential, but conventional methods have made managing climbing plans and current locations cumbersome. Furthermore, there are insufficient means for responding quickly when climbers are not following plan or when there is a possibility of getting lost. Therefore, a system is needed that efficiently and centrally manages climbers' location information and climbing plans, and monitors them in real time.

[0159] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0160] In this invention, the server includes a means for collecting mountain climbing plans from either voice data provided by climbers, links to other companies' applications, or paper images, a means for using a generative artificial intelligence model to analyze the collected mountain climbing plans and convert them into a digital format, and a means for storing the analyzed mountain climbing plan data in a database for centralized management. This allows for efficient management of climbers' plans and current location information, and makes it possible to quickly detect and respond to any abnormalities.

[0161] A "mountaineer" is a person who goes mountain climbing and refers to a user of this system.

[0162] "Audio data" refers to information input by voice by a climber that has been recorded as digital data.

[0163] "Links to third-party applications" refers to URLs that connect to pages or information within applications provided by other companies.

[0164] "Paper images" refer to image data that has been digitized from paper media containing mountain climbing plans.

[0165] A "mountain climbing plan" is a detailed description of information such as the climber's planned time to reach the summit, route, and destination.

[0166] A "generative artificial intelligence model" refers to a machine learning model that analyzes natural language and converts it into digital data.

[0167] A "digital format" is a unified format for expressing mountain climbing plans as digital data.

[0168] A "database" refers to a system or device for centrally managing and storing analyzed mountain climbing plan data and location information.

[0169] "Mobile devices" refer to electronic devices that climbers can carry with them, such as smartphones or devices with GPS functionality.

[0170] "Location information" refers to data indicating a climber's current location, typically obtained by GPS or other positioning systems.

[0171] "Detecting anomalies" refers to checking for discrepancies when they arise between the mountain climbing plan and the current location information.

[0172] A "notification" is a message sent to the climber or a designated third party regarding detected abnormalities or important information.

[0173] "Designated locations" refers to locations or facilities that have been registered in advance as part of mountain climbing plans or as emergency contact points.

[0174] "Other climbers" refers to other climbers using this system, who may be subject to notification in the event of an emergency.

[0175] This invention is a management system for mountain climbing plans and location information, with the aim of ensuring the safety of climbers and providing prompt response. This system is composed of a server, terminals, and climbers.

[0176] The server has a means for receiving voice data provided by the climber, links to third-party applications, and paper image data. Specifically, it uses a generative AI model (e.g., GPT-4) to analyze this data and convert it into a standard digital format. If the climber verbally inputs, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the server uses the generative AI model to convert the voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut." This analysis uses the prompt, "Please analyze this climber's voice data and convert his / her climbing plan into a digital format."

[0177] The server is equipped with a system for storing analyzed digital climbing plans in a database and managing them centrally. This database aggregates each climber's plans and progress, allowing them to be viewed in real time. The server also has the function of periodically receiving location information from the climber's mobile device. This location information is obtained using a GPS module.

[0178] The server compares the received location information with the climbing plan data to see if it matches the plan. If a discrepancy is detected, the server detects the anomaly and generates an alert to notify the climber's mobile device. If a climber is delayed or does not descend as planned, notifications are sent to the mountain hut and other climbers, allowing for a prompt response.

[0179] The device, on the other hand, has functions for user authentication and input of mountain climbing plans. When the user starts the device for the first time, they create an account by entering information such as their name, email address, and password, and then log in. After that, the user can enter their mountain climbing plan using voice, a link to a third-party application, or an image of paper, and send it to the server.

[0180] The device also activates its GPS function when the climb begins and acquires and transmits location information to the server at regular intervals (for example, every 15 minutes). This location information is compared with the climbing plan to confirm consistency. Furthermore, the device has the ability to instantly receive an alert notification and display it to the user in a pop-up if a discrepancy between the plan and location information is detected, or if the climber is stalled.

[0181] In this way, the present invention efficiently manages climbers' plans and current locations, and quickly detects and responds to abnormalities, ensuring the safety of climbers. It also enables prompt action through a notification function to mountain huts and other climbers.

[0182] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0183] Server program processing

[0184] Step 1: Data collection

[0185] The server receives audio data provided by the climber, links to other companies' applications, and paper image data. This input data is sent to the server through various data collection methods. For example, if a climber records audio data, the recording is uploaded to the server. Specifically, the server receives an HTTP request and saves the data in a local directory.

[0186] Step 2: Data analysis

[0187] The server inputs the received data into a generative AI model (e.g., GPT-4) and parses it into a digital format. The input data is an audio file, and the output is analyzed text data. The prompt "Please analyze this climber's audio data and convert the climbing plan into a digital format" is used and input into the generative AI model. Specifically, the server sends the audio data to a text conversion API and receives the conversion result.

[0188] Step 3: Store in the database

[0189] The server stores the analyzed digitally formatted mountain climbing plan in a database. The input data is the text data of the analysis results, and the output data is registered in the database. Specifically, the server issues an SQL query and performs an insert process into the database.

[0190] Step 4: Receiving and analyzing location information

[0191] The server receives location information periodically sent from the climber's mobile device. The input data is GPS information, and the output is the current location, which is registered in the database. Specifically, the server receives an HTTP request and saves the location information in the database.

[0192] Step 5: Detect and notify discrepancies

[0193] The server compares the received location information with the climbing plan data and detects any discrepancies. The input data is the current location and climbing plan data, and the output generates an alert notification if an abnormality is detected. Specifically, the server uses an SQL query to retrieve the necessary data from the database, compares it with the plan, and calls a notification API if an abnormality is found.

[0194] Step 6: Alert response actions

[0195] The server notifies the relevant parties of the generated alert. The input data is the alert information, and the output is a notification. Specifically, the server notifies the mountain hut and other climbers using the notification API.

[0196] Terminal program processing

[0197] Step 1: Initial setup and user authentication

[0198] When a user starts up a device for the first time, they create an account by entering information such as their name, email address, and password. The input data is user information, and the output is an account registered on the server. Specifically, the device sends the entered data to the server as an HTTP POST request.

[0199] Step 2: Enter and submit your climbing plan

[0200] Users input their mountain climbing plans using voice, links to third-party applications, and paper images. The input data includes voice data and image data, which are sent to the server as output. Specifically, after collecting the data, the device sends it to the server as an HTTP POST request.

[0201] Step 3: Collecting and sending location information

[0202] When the user presses the "Start Climbing" button on the app to begin climbing, the device activates its GPS function. GPS information is the input data, and location information is sent to the server every 15 minutes as output. Specifically, the device starts a timer, periodically acquires GPS data, and sends it to the server as an HTTP POST request.

[0203] Step 4: Receive and view alert notifications

[0204] If a mismatch is detected between the user's location information and the mountain climbing plan, the device immediately receives an alert notification. The input data is the alert information, and the output is a pop-up notification that is displayed to the user. Specifically, the device receives a push notification from the server and displays the notification content on the screen.

[0205] (Application example 1)

[0206] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0207] In modern society, serious risks and problems often arise when plans and actual actions do not match in situations such as mountain climbing and food delivery. For climbers, the risk of getting lost increases if they do not proceed along the mountain path as planned, requiring a quick response. Meanwhile, in food delivery, if delivery employees are unable to deliver as scheduled, customer satisfaction decreases and work efficiency deteriorates. To prevent such situations from occurring, it is necessary to confirm the match between plans and location information in real time. However, current technology does not provide sufficient means to do this efficiently.

[0208] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0209] In this invention, the server includes: means for collecting mountain climbing plans from either voice data, URLs of other companies' apps, or paper images provided by climbers; means for using a generative AI model to analyze the collected mountain climbing plans and convert them into a digital format; means for centrally managing the analyzed mountain climbing plan data; means for periodically obtaining climbers' location information from a mobile device and comparing it with the collected mountain climbing plans; means for generating an alert and notifying the climber if the mountain climbing plan and current location information do not match; means for notifying mountain huts and other climbers if the climber is delayed or does not descend the mountain as planned; means for collecting delivery plans from either voice memos, URLs of other companies' apps, or images of handwritten notes provided by delivery employees; means for using a generative AI model to analyze the collected delivery plans and convert them into a digital format; means for centrally managing the analyzed delivery plan data; means for periodically obtaining delivery employee location information from a mobile device and comparing it with the collected delivery plans; means for generating an alert and notifying the delivery employee if the delivery plan and current location information do not match; and means for notifying other relevant parties if the delivery employee is delayed or does not deliver as planned. This makes it possible to efficiently check in real time whether the plans and actual actions of climbers and delivery employees match, preventing risks and problems before they occur.

[0210] A "mountaineer" is an individual who engages in the activity of climbing mountains in a natural environment.

[0211] A "delivery employee" is an individual who performs food delivery or other delivery duties.

[0212] "Voice data" refers to data that is a digital recording of a human voice.

[0213] "Third-party app URL" means a web link related to application software provided by another company.

[0214] A "paper image" is information written on paper that has been converted into image data using a digital camera or scanner.

[0215] A "mountain climbing plan" is a detailed schedule of a mountain climb that a climber prepares in advance.

[0216] A "delivery plan" is a detailed delivery schedule prepared in advance by delivery employees.

[0217] A "generative AI model" is an artificial intelligence model that learns from large amounts of data and performs specific tasks.

[0218] "Digital format" refers to the form of data that is represented or recorded using digital technology.

[0219] A "mobile terminal" is a portable information device such as a smartphone or tablet.

[0220] "Location information" is data that indicates a geographic location. It is obtained using technologies such as GPS.

[0221] An "alert" is a warning message that notifies the user of an abnormal situation or important notification.

[0222] A mountain hut is a building set up for climbers to stay overnight or take a break.

[0223] "Related parties" refers to any person or organization that plays a significant role in delivery operations.

[0224] This invention is a system that centrally manages and monitors the location information and plans of climbers and delivery employees in real time. This system collects data in multiple formats, such as voice memos, URLs of other companies' apps, and paper images, provided by climbers and delivery employees, and analyzes it using a generative AI model. This data is then converted into a digital format for centralized management, ensuring safety and improving work efficiency.

[0225] Server program processing

[0226] Data collection and analysis

[0227] The server receives voice data, URLs of other companies' apps, and paper images provided by climbers and delivery employees. The collected data is analyzed using a generative AI model and converted into a standard digital format. For example, if a climber sends a voice memo saying, "Tomorrow I'll be climbing Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database as "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[0228] Centralized data management

[0229] The server stores the analyzed climbing plan data and delivery plan data in a database and manages them centrally, allowing the plans and progress of each climber and delivery employee to be visualized in one place and checked in real time.

[0230] Location Tracking

[0231] The server receives location information periodically sent from the mobile devices of climbers and delivery employees. This location information is checked against the climbing plan and delivery plan to confirm consistency. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[0232] Conflict notification and stagnation management

[0233] If there is a discrepancy between the location information and the climbing plan or delivery plan, the server immediately generates an alert and sends a notification to the mobile devices of the climber or delivery employee. Also, if a climber is delayed or does not descend as planned, or if a delivery employee is late, the information is notified to other parties so that they can take prompt action.

[0234] Terminal program processing

[0235] User authentication and plan input

[0236] When the device is first started, the user is prompted to create an account and log in. After that, the user inputs mountain climbing and delivery plans using voice memos, URLs of other companies' apps, and images of paper. This data is sent to the server and analyzed.

[0237] Sending location information

[0238] The device activates its GPS function when starting a climb or delivery, acquires location information at regular intervals, and sends it to the server. This location information is then compared with the plan to confirm consistency.

[0239] Receive notifications

[0240] The device receives instant alert notifications and displays them to the user in a pop-up if it detects a discrepancy between the plan and location information, if a climber is delayed, or if a delivery employee is late.

[0241] User Examples

[0242] Initial setup and planning input

[0243] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by speaking, they can say, "Tomorrow I'm going to climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," or "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan is sent to the server and analyzed.

[0244] Initiating planning and location sharing

[0245] When a user arrives at the trailhead or delivery start point, they press the "Start Climbing" or "Start Delivery" button on the app. The device activates its GPS function and sends location information to the server every 15 minutes. The location information is compared with the plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[0246] Examples of specific examples and prompts

[0247] As a specific example, a user may say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture, and plan to arrive at the mountain hut at 1:00 PM," and the server will analyze and save this. If the user does not arrive at the mountain hut as planned during the climb, the server will detect the discrepancy and immediately generate an alert, notifying the user's device. As an example of a delivery, a delivery employee may say by voice, "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan will be analyzed and saved. If an unplanned route is taken, a discrepancy will be detected and an immediate notification will be sent.

[0248] Prompt Sentence Examples

[0249] text

[0250] When a food delivery worker says "I have 10 deliveries today" to the app, the voice data is sent to the server and converted to "I have 10 deliveries today" using the Google Cloud Speech-to-Text API.

[0251] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0252] Program processing steps

[0253] Step 1:

[0254] The server receives the voice data provided by the climbers and delivery employees, the URL of the third-party app, and the paper image. This input data depends on the format each user sends from the app. For example, a user records a voice memo in the app, which is then sent to the server. The output of this step is the data in the format received by the server.

[0255] Step 2:

[0256] The server inputs the received data into the generative AI model. For audio data, it uses the Google Cloud Speech-to-Text API to convert the audio to text. For URLs from other apps, it sends a single web request and analyzes the response. For paper images, it uses the Google Cloud Vision API to convert handwritten characters to text. The output of this step is the converted text data.

[0257] Step 3:

[0258] The server then compiles the converted text data into a digital format. For example, a voice memo saying "Tomorrow we will climb Mt. Tanzawa. We plan to arrive at the mountain hut at 1:00 PM" is converted into the format "Mt. Tanzawa, 1:00 PM, we plan to arrive at the mountain hut." The output of this step is the climbing plan data and delivery plan data in a unified digital format.

[0259] Step 4:

[0260] The server stores the data in a unified digital format in a database and manages it centrally. This allows the plans and progress of each climber and delivery employee to be visualized in one place. The output of this step is the plan data in a digital format stored in a database.

[0261] Step 5:

[0262] The device activates the GPS function at the start of the climb or delivery and acquires location information at regular intervals. This location information is periodically sent to the server. The output of this step is the location information data that is periodically sent.

[0263] Step 6:

[0264] The server receives the periodically transmitted location information and checks it against the previously stored digitally formatted plan data, for example, to see if the location information is updated according to the plan. The output of this step is the check result.

[0265] Step 7:

[0266] The server generates an alert immediately if there is a discrepancy between the location information and the planned data. The alert is sent to the climber's or delivery employee's device. For example, if the location is not updated after the planned arrival time, an alert is issued. The output of this step is an alert notification sent to the user's device.

[0267] Step 8:

[0268] The terminal displays the alert notification received from the server as a popup to the user, allowing the user to immediately check the abnormality and take necessary action. The output of this step is an alert notification that is displayed in a visible form to the user.

[0269] Examples of specific examples and prompts

[0270] As a specific example, a user might say, "Tomorrow I'm going to climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and saves this. The voice data is converted to text using the Google Cloud Speech-to-Text API and saved as "Mt. Tanzawa, 1:00 PM, expected arrival at the mountain hut." If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending a notification to the user's device. As an example of a delivery, a delivery employee might say, "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan is analyzed and saved. If an unplanned route is taken, the discrepancy is detected and an immediate notification is sent.

[0271] Prompt Sentence Examples

[0272] text

[0273] When a food delivery worker says "I have 10 deliveries today" to the app, the voice data is sent to the server and converted to "I have 10 deliveries today" using the Google Cloud Speech-to-Text API.

[0274] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0275] This invention combines an emotion engine with a system that centrally manages and monitors climbers' location information and climbing plans in real time. This system collects, analyzes, and centrally manages climbing plan data in various formats provided by climbers (audio, URLs of other companies' apps, paper images). It also uses the emotion engine to analyze the climber's emotional state in real time and generates alerts and messages to reduce stress and anxiety as needed. This further ensures climber safety and enables the prevention and rapid response of mountaineering accidents.

[0276] Server program processing

[0277] Data collection and analysis

[0278] The server receives the voice data provided by the climber, the URL of the third-party app, and the paper image. This data is analyzed using the generative AI model and converted into a standard digital format. For example, if a climber says, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[0279] Emotion Engine Analysis

[0280] The server is equipped with an emotion engine that analyzes the climber's emotional state (e.g., stress or anxiety) from the voice data and input data. The analysis results are stored in association with each climber's profile.

[0281] Centralized data management

[0282] The server stores the analyzed digital climbing plans in a database and manages them centrally. The analysis results of the emotion engine are also stored. This allows each climber's plan, progress, and emotional state to be visualized in one place and checked in real time.

[0283] Location Tracking

[0284] The server receives location information periodically sent from the climber's mobile device. This location information is checked against the climbing plan to see if it matches the plan. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[0285] Conflict notification and stagnation management

[0286] If there is a discrepancy between the location information and the climbing plan, the server immediately generates an alert and sends a notification to the climber's mobile device. If the climber is delayed or does not descend as planned, the server will link the analysis results of the emotion engine and send a notification to the mountain hut and other climbers. If the emotion engine detects high stress or anxiety, a particularly urgent notification will be generated.

[0287] Terminal program processing

[0288] User authentication and mountain climbing plan input

[0289] The device requires users to create an account and log in when they first start the device. After that, users can input their hiking plans using voice commands, URLs from other companies' apps, or images of paper. This data is sent to a server for analysis.

[0290] Sending location information

[0291] The device activates its GPS function when the climb begins, acquires location information at regular intervals, and sends it to the server. This location information is then checked against the climbing plan to confirm its consistency.

[0292] Emotion Engine Input

[0293] The device sends the user's emotional state to the emotion engine through voice or text input. For example, if the user says, "I'm feeling a little tired," the information is analyzed and sent to the emotion engine.

[0294] Receive notifications

[0295] If the device detects a discrepancy between the plan and location information, or if the emotion engine analyzes that the user is in a state of high stress or anxiety, the device will quickly receive an alert notification and display it to the user in a pop-up.

[0296] User Examples

[0297] Initial setup and climbing plan input

[0298] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by verbally inputting, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the climbing plan is sent to the server and analyzed.

[0299] Start climbing and share your location

[0300] When a user arrives at the trailhead, they press the "Start Climbing" button on the app. The device activates its GPS function and sends its location information to the server every 15 minutes. The location information is compared with the climbing plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[0301] Emotional state analysis and response

[0302] While climbing, the user inputs their emotional state, such as "I'm anxious," through voice. The device sends this voice to the emotion engine, which interprets it as a high anxiety state. The server then quickly generates an alert and sends a notification to the user's device, as well as to mountain huts and other climbers.

[0303] Specific examples

[0304] The user vocally inputs, "I'm going to climb Mt. Tanzawa tomorrow. I plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and stores this information. If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending a notification to the user's device. If the user also says, "I'm a little scared," the emotion engine analyzes the high level of anxiety and generates a high-urgency notification. The user receives this notification and checks for safety. The system also takes into account the user's emotional state, ensuring the user's safety and responding quickly.

[0305] The processing flow will be explained below.

[0306] User creates an account

[0307] Step 1:

[0308] A user downloads the mobile app and launches it for the first time.

[0309] Step 2:

[0310] The user enters basic information such as name, email address, and password, and clicks the registration button.

[0311] Step 3:

[0312] The terminal transmits the user's registration information to the server.

[0313] Step 4:

[0314] The server stores the received user information in a database.

[0315] Step 5:

[0316] The server sends a confirmation message to the terminal to confirm the completion of registration, and the terminal displays the confirmation message.

[0317] The user inputs a mountain climbing plan

[0318] Step 1:

[0319] The user logs in and selects "Enter Mountain Climbing Plan" from the main menu.

[0320] Step 2:

[0321] The user selects one of the following methods: "Voice input," "URL of a third-party app," or "Image upload."

[0322] Step 3:

[0323] For example, a user might say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 p.m."

[0324] Step 4:

[0325] The device records the audio data and sends it to the server.

[0326] Step 5:

[0327] The server analyzes the voice data using a generative AI model and converts it into text data.

[0328] Step 6:

[0329] The server stores the parsed text data in a database and associates it with the user's account.

[0330] User inputs emotional state

[0331] Step 1:

[0332] The user selects "Voice Input" from the emotion input screen.

[0333] Step 2:

[0334] A user says, "I'm feeling a little anxious today."

[0335] Step 3:

[0336] The device transmits voice data related to emotions to the server.

[0337] Step 4:

[0338] The server analyzes the voice data using an emotion engine to identify the emotional state.

[0339] Step 5:

[0340] The server stores the analyzed emotional state in a database and associates it with the user's account.

[0341] User starts climbing

[0342] Step 1:

[0343] The user arrives at the trailhead and launches the app.

[0344] Step 2:

[0345] The user presses the "Start climbing" button.

[0346] Step 3:

[0347] The device will activate the GPS function and begin acquiring location information.

[0348] Step 4:

[0349] The device periodically (e.g., every 15 minutes) sends its location information to the server.

[0350] Location monitoring and discrepancy notification during mountain climbing

[0351] Step 1:

[0352] The server compares the received location information with the mountain climbing plan.

[0353] Step 2:

[0354] The server compares the climbing plan (e.g., scheduled arrival at the mountain hut at 13:00) with the current location information.

[0355] Step 3:

[0356] If the server has not reached the hut by the scheduled time (13:00), it detects a discrepancy.

[0357] Step 4:

[0358] If the server detects a mismatch, it generates an alert message and sends it to the terminal.

[0359] Step 5:

[0360] The device displays a warning to the user saying, "You have not arrived at the mountain hut by the scheduled time. Please check that your surroundings are safe."

[0361] Notification of stagnation and those who have not yet descended

[0362] Step 1:

[0363] The server periodically compares the location information of all users with their mountain climbing plans.

[0364] Step 2:

[0365] If the server detects stagnation or failure to descend, it generates an alert list.

[0366] Step 3:

[0367] The server sends alerts of stagnation or failure to descend the mountain to the relevant user's device.

[0368] Step 4:

[0369] The device displays a warning to the user saying, "You have been stuck at your current location for an extended period of time. Please make sure you are safe."

[0370] Step 5:

[0371] The server also sends relevant notifications to mountain huts and other climbers, encouraging them to take prompt action.

[0372] Countermeasures and notifications based on emotional state

[0373] Step 1:

[0374] The server periodically checks the emotional state analyzed by the emotion engine.

[0375] Step 2:

[0376] The server generates special alerts if high stress or anxiety states are detected.

[0377] Step 3:

[0378] The server sends a message to the user's device saying, "Your current emotional state is unstable. Please check the safety of your surroundings and seek support if necessary."

[0379] Step 4:

[0380] The server sends an emergency notification of the user to the mountain hut and other nearby climbers, asking them to check for safety.

[0381] Centralized data management

[0382] Step 1:

[0383] The server stores all climbers' climbing plans, location information, emotional state, and discrepancy data in a database.

[0384] Step 2:

[0385] The server updates and displays the location information and mountain climbing plan received in real time.

[0386] Step 3:

[0387] If the server needs search and rescue, it will provide relevant information to the police and local authorities.

[0388] Specific examples

[0389] Initial setup and climbing plan input

[0390] Step 1:

[0391] A user launches the app for the first time and creates an account by entering their name, email address, password, etc.

[0392] Step 2:

[0393] The user says in a voice message, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 p.m."

[0394] Step 3:

[0395] The terminal transmits the voice data to the server, which analyzes it and converts it into text data.

[0396] Step 4:

[0397] The server stores the parsed data in a database and associates it with the user's account.

[0398] Start climbing and share your location

[0399] Step 1:

[0400] The user arrives at the trailhead and presses the "Start Climbing" button on the app.

[0401] Step 2:

[0402] The device will enable the GPS function and send location information to the server every 15 minutes.

[0403] Step 3:

[0404] The server compares the location information with the climbing plan to check for any discrepancies.

[0405] Step 4:

[0406] If the server detects a mismatch, it immediately sends an alert to the user terminal.

[0407] Emotional state analysis and response

[0408] Step 1:

[0409] During the climb, the user says, "I'm getting a little tired."

[0410] Step 2:

[0411] The device sends the voice data to the server, which then analyzes it using an emotion engine.

[0412] Step 3:

[0413] The server detects high levels of fatigue and anxiety and generates special alerts.

[0414] Step 4:

[0415] The server sends the user a message saying, "We've checked your status. Stay safe and rest."

[0416] Step 5:

[0417] The server also notifies nearby mountain huts and other climbers, urging users to check for safety.

[0418] Example 2

[0419] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0420] While mountaineering has become increasingly popular in recent years, the number of mountaineering accidents and emergencies has also increased. To solve this problem, a system is needed that can track climbers' location information in real time, detect discrepancies between their climbing plans and their current location, and respond quickly. It is also necessary to better ensure the safety of climbers by analyzing their emotional states, such as stress and anxiety, during climbing in real time and taking appropriate measures.

[0421] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting a mountain climbing plan from audio data provided by the climber, a URL of a third-party app, or a paper image; means for using a generative AI model to analyze the collected mountain climbing plan and convert it into a digital format; means for using an emotion engine to analyze the climber's emotional state; means for periodically acquiring the climber's location information from the mobile device and comparing it with the collected mountain climbing plan; means for generating an alert and notifying the climber if the mountain climbing plan and current location information do not match; and means for notifying mountain huts and other climbers based on the analysis results of the emotion engine if the climber is delayed or has not descended the mountain as planned. This allows the location information and emotional state of the climber to be tracked in real time, enabling rapid response in the event of a mountaineering accident or emergency.

[0422] "Mountain climbing plan" refers to the mountain climbing route, estimated time of arrival, and other related information planned in advance by a climber.

[0423] A "generative AI model" refers to an algorithm or system that uses machine learning technology to analyze input information such as voice data, images, and URLs and convert them into a digital format.

[0424] The "emotion engine" refers to a system that analyzes voice and text data to evaluate and determine the climber's emotional state (e.g., stress or anxiety).

[0425] "Digital format" refers to a data format in which planned climbing information has been converted into a format that can be stored, processed, and analyzed electronically.

[0426] "Centralized management" refers to consolidating various information on a unified platform and efficiently managing and operating it.

[0427] "Location information" refers to geographic coordinate data (e.g., GPS information) that indicates the climber's current location.

[0428] "Mismatch" refers to a situation in which the climber's pre-planned climbing plan does not match the climber's actual behavior or location information.

[0429] "Alert" means a warning message generated by the system when it detects a discrepancy or emergency.

[0430] "Notification" refers to the transmission of alerts and other important information from the server to climbers and related parties.

[0431] The present invention combines an emotion engine with a system that centrally manages and monitors the location information and mountain climbing plans of climbers in real time. Specific embodiments for carrying out the invention are described below.

[0432] The server collects voice data, URLs of third-party apps, and paper images provided by climbers. This collected data is analyzed using a generative AI model (e.g., Google Cloud Speech-to-Text or OCR technology) and converted into a standard digital format. For example, if a user says, "Tomorrow I'm going to climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this speech into text data and stores it in a database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[0433] Next, the server is equipped with an emotion engine that analyzes the climber's emotional state (stress or anxiety) from the voice and input data. The analysis results are stored in association with each climber's profile.

[0434] The analyzed digital climbing plans and the analysis results of the emotion engine are stored in a database on a server and managed centrally, allowing each climber's plan, progress, and emotional state to be visualized in one place and checked in real time.

[0435] Additionally, the server receives location information periodically sent from the climber's mobile device. This location information is compared with the climbing plan to see if it matches the plan. If a discrepancy is detected, the server immediately generates an alert and sends a notification to the climber's mobile device. If the climber is delayed or does not descend as planned, notifications are sent to the mountain hut and other climbers in conjunction with the emotion engine's analysis results. If the emotion engine detects high stress or anxiety, a particularly urgent notification is generated.

[0436] Meanwhile, on the device side, the user creates an account and logs in when they first start up the app. After that, the user inputs their mountain climbing plan using voice, the URL of another company's app, or an image of paper, and this data is sent to the server. In addition, when the climb begins, the GPS function is enabled, and location information is acquired at regular intervals and sent to the server. When the user records their emotional state through voice input or text input, this information is sent to the emotion engine.

[0437] For example, a user might say, "I'm going to climb Mt. Tanzawa tomorrow. I plan to arrive at the mountain hut at 1:00 PM," and the server would analyze and store this information. If the user does not arrive at the hut as planned during the climb, the server would detect the discrepancy and immediately generate an alert, sending a notification to the user's device. If the user also says, "I'm a little scared," the emotion engine would analyze the high level of anxiety and generate a high-urgency notification. The system takes the user's emotional state into account to ensure their safety and respond quickly.

[0438] An example prompt might be:

[0439] "The following speech data is input into the generative AI model: 'Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM.'"

[0440] In this way, by managing and analyzing climbers' location information and emotional state in real time, it is possible to ensure their safety, prevent accidents, and respond quickly.

[0441] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0442] Step 1: Data collection and reception

[0443] The server receives voice data, URLs of other companies' apps, and paper image data from climbers. For input, the user speaks, "Tomorrow I'm going to climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," and this is sent to the server. For output, the voice data and other data formats are stored in a buffer on the server side. Specifically, the climber inputs voice data into their smartphone, and the voice is uploaded to the server via the device.

[0444] Step 2: Data analysis and transformation

[0445] The server analyzes the received data using a generative AI model and converts it into a digital format. Inputs include the received voice data, URL, and image data. The data is analyzed according to the prompts of the generative AI model (e.g., Google Cloud Speech-to-Text or OCR technology). The output is the analyzed text data or data in a structured digital format. Specifically, the voice data is converted into text data such as "Expected arrival at Tanzawa Mountain Hut at 13:00" and stored in a database.

[0446] Step 3: Analysis by Emotion Engine

[0447] The server uses an emotion engine to analyze the climber's emotional state from voice and text data. The input is voice data and text data converted by a generative AI model. The emotion engine analyzes this to determine the climber's state of stress and anxiety. The output is the evaluation result of the emotional state, which is associated with the climber's profile and saved. Specifically, the emotion engine analyzes the user's voice saying "I'm getting a little tired," and adds the "fatigue" status to the profile.

[0448] Step 4: Save to database and centralize management

[0449] The server stores the analyzed digital data and emotion analysis results in a database and manages them centrally. The input is the digital format data and emotional state data analyzed by the generative AI model and emotion engine. As an output, this data is stored in a centrally managed database. Specifically, "Expected arrival at Tanzawa Mountain Hut at 13:00" and the emotion tag "fatigue" are stored and managed in the database.

[0450] Step 5: Track and match location information

[0451] The server receives GPS location information periodically sent from the climber's mobile device and compares it with the collected climbing plan. The input is the climber's current location information periodically received as GPS data and pre-saved climbing plan data. The output is a determination result of whether the location information matches the climbing plan or not. Specifically, the server receives location information every 15 minutes and compares data such as "12:45 local time, 500m before Mt. Tanzawa" with the climbing plan.

[0452] Step 6: Detect and alert on discrepancies

[0453] If there is a discrepancy between the location information and the climbing plan, the server immediately generates an alert and sends a notification to the climber's mobile device. In addition, if the climber is delayed or does not descend as planned, the server notifies the mountain hut and other climbers in conjunction with the analysis results of the emotion engine. The input is data on discrepancies between the location information and the climbing plan, and emotional state data. The output is an alert and notification message in the event of a discrepancy. In concrete terms, if the climber does not arrive at the mountain hut at the scheduled time of 1:00 PM, the server generates an alert and notifies the climber, "There is a discrepancy in the plan. Please check."

[0454] Step 7: User authentication and mountain climbing plan input

[0455] When the user first starts up the device, they create an account and log in. The user then inputs their mountain climbing plan using voice, the URL of a third-party app, and an image of a piece of paper. This input data is sent to the server. The inputs are the user's name, email address, password, and mountain climbing plan data. The output is the user account authentication information and mountain climbing plan data, which are sent to the server. Specific operations include the user entering information on the account creation screen and inputting the plan using voice.

[0456] Step 8: Send location information

[0457] The device activates its GPS function when starting a climb, and acquires and sends location information to the server at regular intervals. The input is the start climbing button operated by the user and the current location data acquired by the GPS. The output is the acquired location information that is periodically sent to the server. Specifically, the user presses the "start climbing" button, and the device sends location information to the server every 15 minutes.

[0458] Step 9: Enter and send emotional states

[0459] The device records the user's emotional state through voice or text input and sends it to the emotion engine. The input is voice data or text data of the emotional state input by the user. The output is the emotional data to be analyzed and sent to the server. In concrete terms, the user may say "I'm getting a little tired" and the data is sent to the server.

[0460] Step 10: Receiving and Viewing Notifications

[0461] The terminal receives an alert notification from the server and displays a pop-up notification to the user. The input is the alert notification data sent from the server. As an output, the received alert notification is displayed in a pop-up format on the terminal screen. Specifically, a pop-up message saying "There is a discrepancy in the plan. Please check" is displayed on the terminal screen.

[0462] (Application example 2)

[0463] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0464] Current systems that manage climbers' location information and climbing plans have difficulty determining their emotional state in real time and taking appropriate action in emergencies. Furthermore, there is a lack of effective means to monitor the work plans and emotional state of factory workers to improve work efficiency and safety. This increases the risk of accidents and problems during mountain climbing and factory work.

[0465] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting mountain climbing plans from either voice data provided by climbers, URLs of other companies' apps, or paper images, means for using a generative AI model to analyze the collected mountain climbing plans and convert them into a digital format, and means for periodically obtaining climber location information from a mobile device and comparing it with the collected mountain climbing plans. This makes it possible to analyze and manage the plans and location information of climbers and factory workers in real time, monitor their emotional states, and generate appropriate alerts.

[0466] "Audio data" refers to audio information provided by climbers or workers through a microphone or audio input device.

[0467] "URLs from other apps" are link information shared from other applications, and are URLs used to represent mountain climbing plans or work plans.

[0468] "Paper images" are digitalized images of paper documents containing mountain climbing plans and work plans.

[0469] "Mountain climbing plan" means a schedule or route plan that a climber makes before climbing a mountain, and includes digital data, audio data, and image data that are provided.

[0470] A "generative AI model" is an artificial intelligence algorithm used to analyze collected audio and image data and convert it into a standard digital format.

[0471] "Digital format" refers to a format in which analog information has been digitized and is a format that can be processed by a computer.

[0472] "Location information" is data indicating the current location of climbers and workers, obtained using GPS or other location information acquisition means.

[0473] The "emotion engine" is an artificial intelligence technology that analyzes emotional states from voice and text data and identifies emotions such as stress, anxiety, and fatigue.

[0474] An "alert" is a message used to warn or caution climbers and workers.

[0475] A "mountain hut" is a facility in a mountainous area set up for climbers to rest and stay overnight.

[0476] The "database" is a digital storage system for centrally storing and managing collected mountaineering and work plan data.

[0477] A "work plan" is a document that outlines the work processes and procedures that workers in a factory draw up, and includes information provided in the form of audio data, URLs of other companies' apps, paper images, etc.

[0478] This invention is a system that centrally manages the location information and plans of mountain climbers and factory workers, and monitors them in real time by combining an emotion engine. This system improves user safety and efficiency by collecting and analyzing mountain climbing and work plan data provided by users, such as voice data, URLs of other companies' apps, and paper images.

[0479] Data collection and analysis

[0480] The server receives the voice data, URLs of other companies' apps, and paper images provided by the user, and uses the generative AI model to analyze and convert this data into a digital format. For example, if a climber says, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in a database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[0481] Emotion Engine Analysis

[0482] The server is equipped with an emotion engine that analyzes the user's emotional state (e.g., stress or anxiety) from voice data and input data. The analysis results are stored in association with each user's profile. The emotion engine uses the Python library emotion_analysis.

[0483] Centralized data management

[0484] The server stores the analyzed digital climbing and work plans in a database and manages them centrally. The analysis results of the emotion engine are also stored. This allows users' plans, progress, and emotional state to be visualized in one place and checked in real time.

[0485] Location Tracking

[0486] The server receives location information periodically sent from the user's mobile device. This location information is checked against the plan to see if it matches. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[0487] Conflict notification and stagnation management

[0488] If there is a discrepancy between the location information and the plan, the server immediately generates an alert and sends a notification to the user's mobile device. If the user is stagnating or not progressing with the plan as planned, the server will link the analysis results of the emotion engine and send an appropriate notification. If the emotion engine detects high stress or anxiety, a particularly urgent notification will be generated.

[0489] Terminal program processing

[0490] After creating an account and logging in, the user's mobile device has a function that allows them to input plans using voice data, URLs of other companies' apps, and paper images. This data is sent to the server and analyzed. By enabling the GPS function, location information is acquired at regular intervals and sent to the server. In addition, the user's emotional state is sent to the emotion engine via voice or text input. If a discrepancy between the plan and location information is detected, an alert notification is promptly received and displayed as a pop-up.

[0491] Specific examples

[0492] A user may say, "I'm going to climb Mt. Tanzawa tomorrow. I plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and stores this information. If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert and notifies the user's device. If the user also says, "I'm a little scared," the emotion engine analyzes high anxiety and generates a high-urgency notification.

[0493] Prompt Sentence Examples

[0494] Prompt: "Please speak to describe your work schedule and current feelings for sentiment analysis. For example, "I'll take three breaks by the end of today's work" or "I'm feeling a bit tired.""

[0495] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0496] Step 1:

[0497] A user creates an account and logs in. The user enters their name, email address, and password and presses the Create Account button. This information is sent to the server, which stores the new user information in a database. The input data is the user's basic information (name, email address, password), and the output is a message that the new account was successfully created.

[0498] Step 2:

[0499] Users input their mountain climbing and work plans using voice, a URL from another company's app, or a paper image. In the case of voice input, the user speaks to their smartphone something like, "Tomorrow I'll climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM." This data is sent to a server, where a generative AI model converts the speech into text. The input data is voice data, a URL, or image data, and the output is the plan data in text format.

[0500] Step 3:

[0501] The server converts the received plan data into a digital format and stores it in a database. For example, text data such as "Scheduled arrival at Tanzawa Mountain Hut at 13:00" is stored in the database. The input data is the text data converted by the generative AI model, and the output is the plan data stored in the database.

[0502] Step 4:

[0503] When a user starts climbing or working, they press the "Start" button on the device app. The device activates the GPS function, acquires location information at regular intervals (e.g., 15 minutes), and sends it to the server. The input data is the location information from the GPS, and the output is the location information sent to the server.

[0504] Step 5:

[0505] The server periodically checks the received location information against the planned data in the database, for example, to see if the current location is along the planned route. The input data are the current location information and the planned data in the database, and the output is a determination of whether the location information matches the plan.

[0506] Step 6:

[0507] If the location information does not match the plan, the server immediately generates an alert and notifies the user's device. For example, if the location deviates from the planned route, an alert is sent to the user. The input data is the mismatch determination result, and the output is an alert notification.

[0508] Step 7:

[0509] The user inputs their emotional state by voice while climbing a mountain or working. For example, they might say, "I'm feeling a little tired." This data is sent to the server via the device, and the emotion engine analyzes it. The input data is voice data, and the output is the analysis result of the emotional state.

[0510] Step 8:

[0511] The server associates the results of the emotion analysis with the profile and generates an alert if the level of urgency is high. For example, if a "high anxiety state" is detected, an emergency alert is sent to the user's device. The input data is the emotion analysis result, and the output is an emergency alert notification.

[0512] Step 9:

[0513] The server centrally manages the analyzed mountain climbing plan data, work plan data, and emotional state, and stores them in a database. This allows the user's plan, progress, and emotional state to be visualized in real time. The input data are the plan data and the emotion analysis results, and the output is storage in the database and real-time visualization.

[0514] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0515] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0516] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0517] [Second embodiment]

[0518] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0519] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0520] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0521] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0522] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0523] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0524] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0525] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0526] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0527] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0528] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0529] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0530] This invention is a system that centrally manages and monitors climbers' location information and climbing plans in real time. This system collects climbing plan data provided by climbers in various formats (voice, URLs of other companies' apps, paper images), analyzes it using a generative AI model, converts it into a digital format, and centrally manages it. As a result, it ensures the safety of climbers and enables the prevention and rapid response of accidents.

[0531] Server program processing

[0532] Data collection and analysis

[0533] The server receives the voice data provided by the climber, the URL of the third-party app, and the paper image. This data is analyzed using the generative AI model and converted into a standard digital format. For example, if a climber says, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[0534] Centralized data management

[0535] The server stores the analyzed climbing plans in a digital format in a database and manages them centrally, allowing each climber's plan and progress to be visualized in one place and checked in real time.

[0536] Location Tracking

[0537] The server receives location information periodically sent from the climber's mobile device. This location information is checked against the climbing plan to see if it matches the plan. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[0538] Conflict notification and stagnation management

[0539] If there is a discrepancy between the location information and the climbing plan, the server immediately generates an alert and sends a notification to the climber's mobile device. If the climber is delayed or does not descend as planned, notifications are sent to the mountain hut and other climbers so that a prompt response can be taken.

[0540] Terminal program processing

[0541] User authentication and mountain climbing plan input

[0542] The device requires users to create an account and log in when they first start the device. After that, users can input their hiking plans using voice commands, URLs from other companies' apps, or images of paper. This data is sent to a server for analysis.

[0543] Sending location information

[0544] The device activates its GPS function when the climb begins, acquires location information at regular intervals, and sends it to the server. This location information is then checked against the climbing plan to confirm its consistency.

[0545] Receive notifications

[0546] If the device detects a discrepancy between the plan and location information, or if the climber is delayed, it will immediately receive an alert notification and display it to the user in a pop-up.

[0547] User Examples

[0548] Initial setup and climbing plan input

[0549] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by verbally inputting, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the climbing plan is sent to the server and analyzed.

[0550] Start climbing and share your location

[0551] When a user arrives at the trailhead, they press the "Start Climbing" button on the app. The device activates its GPS function and sends its location information to the server every 15 minutes. The location information is compared with the climbing plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[0552] Specific examples

[0553] A user vocally inputs, "Tomorrow I'm going to climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and stores this information. If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending it to the user's device. The user receives the notification and takes appropriate measures to ensure their safety. In this way, the system can prevent accidents and quickly identify stranded individuals.

[0554] The processing flow will be explained below.

[0555] User creates an account

[0556] Step 1:

[0557] A user downloads the mobile app and launches it for the first time.

[0558] Step 2:

[0559] The user enters basic information such as name, email address, and password, and clicks the registration button.

[0560] Step 3:

[0561] The terminal transmits the user's registration information to the server.

[0562] Step 4:

[0563] The server stores the received user information in a database.

[0564] Step 5:

[0565] The server sends a confirmation message to the terminal to confirm the completion of registration, and the terminal displays the confirmation message.

[0566] The user inputs a mountain climbing plan

[0567] Step 1:

[0568] The user logs in and selects "Enter Mountain Climbing Plan" from the main menu.

[0569] Step 2:

[0570] The user selects one of the following methods: "Voice input," "URL of a third-party app," or "Image upload."

[0571] Step 3:

[0572] For example, a user might say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 p.m."

[0573] Step 4:

[0574] The device records the audio data and sends it to the server.

[0575] Step 5:

[0576] The server analyzes the voice data using a generative AI model and converts it into text data.

[0577] Step 6:

[0578] The server stores the parsed text data in a database and associates it with the user's account.

[0579] User starts climbing

[0580] Step 1:

[0581] The user arrives at the trailhead and launches the app.

[0582] Step 2:

[0583] The user presses the "Start climbing" button.

[0584] Step 3:

[0585] The device will activate the GPS function and begin acquiring location information.

[0586] Step 4:

[0587] The device periodically (e.g., every 15 minutes) sends its location information to the server.

[0588] Location monitoring and discrepancy notification during mountain climbing

[0589] Step 1:

[0590] The server compares the received location information with the mountain climbing plan.

[0591] Step 2:

[0592] The server compares the climbing plan (e.g., scheduled arrival at the mountain hut at 13:00) with the current location information.

[0593] Step 3:

[0594] If the server has not reached the hut by the scheduled time (13:00), it detects a discrepancy.

[0595] Step 4:

[0596] If the server detects a mismatch, it generates an alert message and sends it to the terminal.

[0597] Step 5:

[0598] The device displays a warning to the user saying, "You have not arrived at the mountain hut by the scheduled time. Please check that your surroundings are safe."

[0599] Notification of stagnation and those who have not yet descended

[0600] Step 1:

[0601] The server periodically compares the location information of all users with their mountain climbing plans.

[0602] Step 2:

[0603] If the server detects stagnation or failure to descend, it generates an alert list.

[0604] Step 3:

[0605] The server sends alerts of stagnation or failure to descend the mountain to the relevant user's device.

[0606] Step 4:

[0607] The device displays a warning to the user saying, "You have been stuck at your current location for an extended period of time. Please make sure you are safe."

[0608] Step 5:

[0609] The server also sends relevant notifications to mountain huts and other climbers, encouraging them to take prompt action.

[0610] The server centrally manages data

[0611] Step 1:

[0612] The server stores all climbers' climbing plans, location information, and conflict data in a database.

[0613] Step 2:

[0614] The server updates and displays the location information and mountain climbing plan received in real time.

[0615] Step 3:

[0616] If the server needs search and rescue, it will provide relevant information to the police and local authorities.

[0617] Specific examples

[0618] Initial setup and climbing plan input

[0619] Step 1:

[0620] A user launches the app for the first time and creates an account by entering their name, email address, password, etc.

[0621] Step 2:

[0622] The user says in a voice message, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 p.m."

[0623] Step 3:

[0624] The server analyzes the audio and converts it into text data.

[0625] Step 4:

[0626] The server stores the parsed data in a database and associates it with the user's account.

[0627] Start climbing and share your location

[0628] Step 1:

[0629] The user arrives at the trailhead and presses the "Start Climbing" button on the app.

[0630] Step 2:

[0631] The device will enable the GPS function and send location information to the server every 15 minutes.

[0632] Step 3:

[0633] The server compares the location information with the climbing plan to check for any discrepancies.

[0634] Step 4:

[0635] If things aren't going according to plan, the user is immediately alerted.

[0636] Inconsistency warnings and actions

[0637] Step 1:

[0638] The server detects that the user has not yet arrived at the mountain hut even though it is now 13:00.

[0639] Step 2:

[0640] The server detects the discrepancy and generates an alert that is sent to the user terminal.

[0641] Step 3:

[0642] The terminal displays a warning to the user that "You have not arrived at the mountain hut even though the scheduled time has passed."

[0643] Step 4:

[0644] The user checks the notification to check safety and understand the situation.

[0645] Example 1

[0646] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0647] When climbing mountains, ensuring the safety of climbers and responding quickly are essential, but conventional methods have made managing climbing plans and current locations cumbersome. Furthermore, there are insufficient means for responding quickly when climbers are not following plan or when there is a possibility of getting lost. Therefore, a system is needed that efficiently and centrally manages climbers' location information and climbing plans, and monitors them in real time.

[0648] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0649] In this invention, the server includes a means for collecting mountain climbing plans from either voice data provided by climbers, links to other companies' applications, or paper images, a means for using a generative artificial intelligence model to analyze the collected mountain climbing plans and convert them into a digital format, and a means for storing the analyzed mountain climbing plan data in a database for centralized management. This allows for efficient management of climbers' plans and current location information, and makes it possible to quickly detect and respond to any abnormalities.

[0650] A "mountaineer" is a person who goes mountain climbing and refers to a user of this system.

[0651] "Audio data" refers to information input by voice by a climber that has been recorded as digital data.

[0652] "Links to third-party applications" refers to URLs that connect to pages or information within applications provided by other companies.

[0653] "Paper images" refer to image data that has been digitized from paper media containing mountain climbing plans.

[0654] A "mountain climbing plan" is a detailed description of information such as the climber's planned time to reach the summit, route, and destination.

[0655] A "generative artificial intelligence model" refers to a machine learning model that analyzes natural language and converts it into digital data.

[0656] A "digital format" is a unified format for expressing mountain climbing plans as digital data.

[0657] A "database" refers to a system or device for centrally managing and storing analyzed mountain climbing plan data and location information.

[0658] "Mobile devices" refer to electronic devices that climbers can carry with them, such as smartphones or devices with GPS functionality.

[0659] "Location information" refers to data indicating a climber's current location, typically obtained by GPS or other positioning systems.

[0660] "Detecting anomalies" refers to checking for discrepancies when they arise between the mountain climbing plan and the current location information.

[0661] A "notification" is a message sent to the climber or a designated third party regarding detected abnormalities or important information.

[0662] "Designated locations" refers to locations or facilities that have been registered in advance as part of mountain climbing plans or as emergency contact points.

[0663] "Other climbers" refers to other climbers using this system, who may be subject to notification in the event of an emergency.

[0664] This invention is a management system for mountain climbing plans and location information, with the aim of ensuring the safety of climbers and providing prompt response. This system is composed of a server, terminals, and climbers.

[0665] The server has a means for receiving voice data provided by the climber, links to third-party applications, and paper image data. Specifically, it uses a generative AI model (e.g., GPT-4) to analyze this data and convert it into a standard digital format. If the climber verbally inputs, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the server uses the generative AI model to convert the voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut." This analysis uses the prompt, "Please analyze this climber's voice data and convert his / her climbing plan into a digital format."

[0666] The server is equipped with a system for storing analyzed digital climbing plans in a database and managing them centrally. This database aggregates each climber's plans and progress, allowing them to be viewed in real time. The server also has the function of periodically receiving location information from the climber's mobile device. This location information is obtained using a GPS module.

[0667] The server compares the received location information with the climbing plan data to see if it matches the plan. If a discrepancy is detected, the server detects the anomaly and generates an alert to notify the climber's mobile device. If a climber is delayed or does not descend as planned, notifications are sent to the mountain hut and other climbers, allowing for a prompt response.

[0668] The device, on the other hand, has functions for user authentication and input of mountain climbing plans. When the user starts the device for the first time, they create an account by entering information such as their name, email address, and password, and then log in. After that, the user can enter their mountain climbing plan using voice, a link to a third-party application, or an image of paper, and send it to the server.

[0669] The device also activates its GPS function when the climb begins and acquires and transmits location information to the server at regular intervals (for example, every 15 minutes). This location information is compared with the climbing plan to confirm consistency. Furthermore, the device has the ability to instantly receive an alert notification and display it to the user in a pop-up if a discrepancy between the plan and location information is detected, or if the climber is stalled.

[0670] In this way, the present invention efficiently manages climbers' plans and current locations, and quickly detects and responds to abnormalities, ensuring the safety of climbers. It also enables prompt action through a notification function to mountain huts and other climbers.

[0671] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0672] Server program processing

[0673] Step 1: Data collection

[0674] The server receives audio data provided by the climber, links to other companies' applications, and paper image data. This input data is sent to the server through various data collection methods. For example, if a climber records audio data, the recording is uploaded to the server. Specifically, the server receives an HTTP request and saves the data in a local directory.

[0675] Step 2: Data analysis

[0676] The server inputs the received data into a generative AI model (e.g., GPT-4) and parses it into a digital format. The input data is an audio file, and the output is analyzed text data. The prompt "Please analyze this climber's audio data and convert the climbing plan into a digital format" is used and input into the generative AI model. Specifically, the server sends the audio data to a text conversion API and receives the conversion result.

[0677] Step 3: Store in the database

[0678] The server stores the analyzed digitally formatted mountain climbing plan in a database. The input data is the text data of the analysis results, and the output data is registered in the database. Specifically, the server issues an SQL query and performs an insert process into the database.

[0679] Step 4: Receiving and analyzing location information

[0680] The server receives location information periodically sent from the climber's mobile device. The input data is GPS information, and the output is the current location, which is registered in the database. Specifically, the server receives an HTTP request and saves the location information in the database.

[0681] Step 5: Detect and notify discrepancies

[0682] The server compares the received location information with the climbing plan data and detects any discrepancies. The input data is the current location and climbing plan data, and the output generates an alert notification if an abnormality is detected. Specifically, the server uses an SQL query to retrieve the necessary data from the database, compares it with the plan, and calls a notification API if an abnormality is found.

[0683] Step 6: Alert response actions

[0684] The server notifies the relevant parties of the generated alert. The input data is the alert information, and the output is a notification. Specifically, the server notifies the mountain hut and other climbers using the notification API.

[0685] Terminal program processing

[0686] Step 1: Initial setup and user authentication

[0687] When a user starts up a device for the first time, they create an account by entering information such as their name, email address, and password. The input data is user information, and the output is an account registered on the server. Specifically, the device sends the entered data to the server as an HTTP POST request.

[0688] Step 2: Enter and submit your climbing plan

[0689] Users input their mountain climbing plans using voice, links to third-party applications, and paper images. The input data includes voice data and image data, which are sent to the server as output. Specifically, after collecting the data, the device sends it to the server as an HTTP POST request.

[0690] Step 3: Collecting and sending location information

[0691] When the user presses the "Start Climbing" button on the app to begin climbing, the device activates its GPS function. GPS information is the input data, and location information is sent to the server every 15 minutes as output. Specifically, the device starts a timer, periodically acquires GPS data, and sends it to the server as an HTTP POST request.

[0692] Step 4: Receive and view alert notifications

[0693] If a mismatch is detected between the user's location information and the mountain climbing plan, the device immediately receives an alert notification. The input data is the alert information, and the output is a pop-up notification that is displayed to the user. Specifically, the device receives a push notification from the server and displays the notification content on the screen.

[0694] (Application example 1)

[0695] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0696] In modern society, serious risks and problems often arise when plans and actual actions do not match in situations such as mountain climbing and food delivery. For climbers, the risk of getting lost increases if they do not proceed along the mountain path as planned, requiring a quick response. Meanwhile, in food delivery, if delivery employees are unable to deliver as scheduled, customer satisfaction decreases and work efficiency deteriorates. To prevent such situations from occurring, it is necessary to confirm the match between plans and location information in real time. However, current technology does not provide sufficient means to do this efficiently.

[0697] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0698] In this invention, the server includes: means for collecting mountain climbing plans from either voice data, URLs of other companies' apps, or paper images provided by climbers; means for using a generative AI model to analyze the collected mountain climbing plans and convert them into a digital format; means for centrally managing the analyzed mountain climbing plan data; means for periodically obtaining climbers' location information from a mobile device and comparing it with the collected mountain climbing plans; means for generating an alert and notifying the climber if the mountain climbing plan and current location information do not match; means for notifying mountain huts and other climbers if the climber is delayed or does not descend the mountain as planned; means for collecting delivery plans from either voice memos, URLs of other companies' apps, or images of handwritten notes provided by delivery employees; means for using a generative AI model to analyze the collected delivery plans and convert them into a digital format; means for centrally managing the analyzed delivery plan data; means for periodically obtaining delivery employee location information from a mobile device and comparing it with the collected delivery plans; means for generating an alert and notifying the delivery employee if the delivery plan and current location information do not match; and means for notifying other relevant parties if the delivery employee is delayed or does not deliver as planned. This makes it possible to efficiently check in real time whether the plans and actual actions of climbers and delivery employees match, preventing risks and problems before they occur.

[0699] A "mountaineer" is an individual who engages in the activity of climbing mountains in a natural environment.

[0700] A "delivery employee" is an individual who performs food delivery or other delivery duties.

[0701] "Voice data" refers to data that is a digital recording of a human voice.

[0702] "Third-party app URL" means a web link related to application software provided by another company.

[0703] A "paper image" is information written on paper that has been converted into image data using a digital camera or scanner.

[0704] A "mountain climbing plan" is a detailed schedule of a mountain climb that a climber prepares in advance.

[0705] A "delivery plan" is a detailed delivery schedule prepared in advance by delivery employees.

[0706] A "generative AI model" is an artificial intelligence model that learns from large amounts of data and performs specific tasks.

[0707] "Digital format" refers to the form of data that is represented or recorded using digital technology.

[0708] A "mobile terminal" is a portable information device such as a smartphone or tablet.

[0709] "Location information" is data that indicates a geographic location. It is obtained using technologies such as GPS.

[0710] An "alert" is a warning message that notifies the user of an abnormal situation or important notification.

[0711] A mountain hut is a building set up for climbers to stay overnight or take a break.

[0712] "Related parties" refers to any person or organization that plays a significant role in delivery operations.

[0713] This invention is a system that centrally manages and monitors the location information and plans of climbers and delivery employees in real time. This system collects data in multiple formats, such as voice memos, URLs of other companies' apps, and paper images, provided by climbers and delivery employees, and analyzes it using a generative AI model. This data is then converted into a digital format for centralized management, ensuring safety and improving work efficiency.

[0714] Server program processing

[0715] Data collection and analysis

[0716] The server receives voice data, URLs of other companies' apps, and paper images provided by climbers and delivery employees. The collected data is analyzed using a generative AI model and converted into a standard digital format. For example, if a climber sends a voice memo saying, "Tomorrow I'll be climbing Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database as "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[0717] Centralized data management

[0718] The server stores the analyzed climbing plan data and delivery plan data in a database and manages them centrally, allowing the plans and progress of each climber and delivery employee to be visualized in one place and checked in real time.

[0719] Location Tracking

[0720] The server receives location information periodically sent from the mobile devices of climbers and delivery employees. This location information is checked against the climbing plan and delivery plan to confirm consistency. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[0721] Conflict notification and stagnation management

[0722] If there is a discrepancy between the location information and the climbing plan or delivery plan, the server immediately generates an alert and sends a notification to the mobile devices of the climber or delivery employee. Also, if a climber is delayed or does not descend as planned, or if a delivery employee is late, the information is notified to other parties so that they can take prompt action.

[0723] Terminal program processing

[0724] User authentication and plan input

[0725] When the device is first started, the user is prompted to create an account and log in. After that, the user inputs mountain climbing and delivery plans using voice memos, URLs of other companies' apps, and images of paper. This data is sent to the server and analyzed.

[0726] Sending location information

[0727] The device activates its GPS function when starting a climb or delivery, acquires location information at regular intervals, and sends it to the server. This location information is then compared with the plan to confirm consistency.

[0728] Receive notifications

[0729] The device receives instant alert notifications and displays them to the user in a pop-up if it detects a discrepancy between the plan and location information, if a climber is delayed, or if a delivery employee is late.

[0730] User Examples

[0731] Initial setup and planning input

[0732] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by speaking, they can say, "Tomorrow I'm going to climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," or "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan is sent to the server and analyzed.

[0733] Initiating planning and location sharing

[0734] When a user arrives at the trailhead or delivery start point, they press the "Start Climbing" or "Start Delivery" button on the app. The device activates its GPS function and sends location information to the server every 15 minutes. The location information is compared with the plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[0735] Examples of specific examples and prompts

[0736] As a specific example, a user may say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture, and plan to arrive at the mountain hut at 1:00 PM," and the server will analyze and save this. If the user does not arrive at the mountain hut as planned during the climb, the server will detect the discrepancy and immediately generate an alert, notifying the user's device. As an example of a delivery, a delivery employee may say by voice, "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan will be analyzed and saved. If an unplanned route is taken, a discrepancy will be detected and an immediate notification will be sent.

[0737] Prompt Sentence Examples

[0738] text

[0739] When a food delivery worker says "I have 10 deliveries today" to the app, the voice data is sent to the server and converted to "I have 10 deliveries today" using the Google Cloud Speech-to-Text API.

[0740] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0741] Program processing steps

[0742] Step 1:

[0743] The server receives the voice data provided by the climbers and delivery employees, the URL of the third-party app, and the paper image. This input data depends on the format each user sends from the app. For example, a user records a voice memo in the app, which is then sent to the server. The output of this step is the data in the format received by the server.

[0744] Step 2:

[0745] The server inputs the received data into the generative AI model. For audio data, it uses the Google Cloud Speech-to-Text API to convert the audio to text. For URLs from other apps, it sends a single web request and analyzes the response. For paper images, it uses the Google Cloud Vision API to convert handwritten characters to text. The output of this step is the converted text data.

[0746] Step 3:

[0747] The server then compiles the converted text data into a digital format. For example, a voice memo saying "Tomorrow we will climb Mt. Tanzawa. We plan to arrive at the mountain hut at 1:00 PM" is converted into the format "Mt. Tanzawa, 1:00 PM, we plan to arrive at the mountain hut." The output of this step is the climbing plan data and delivery plan data in a unified digital format.

[0748] Step 4:

[0749] The server stores the data in a unified digital format in a database and manages it centrally. This allows the plans and progress of each climber and delivery employee to be visualized in one place. The output of this step is the plan data in a digital format stored in a database.

[0750] Step 5:

[0751] The device activates the GPS function at the start of the climb or delivery and acquires location information at regular intervals. This location information is periodically sent to the server. The output of this step is the location information data that is periodically sent.

[0752] Step 6:

[0753] The server receives the periodically transmitted location information and checks it against the previously stored digitally formatted plan data, for example, to see if the location information is updated according to the plan. The output of this step is the check result.

[0754] Step 7:

[0755] The server generates an alert immediately if there is a discrepancy between the location information and the planned data. The alert is sent to the climber's or delivery employee's device. For example, if the location is not updated after the planned arrival time, an alert is issued. The output of this step is an alert notification sent to the user's device.

[0756] Step 8:

[0757] The terminal displays the alert notification received from the server as a popup to the user, allowing the user to immediately check the abnormality and take necessary action. The output of this step is an alert notification that is displayed in a visible form to the user.

[0758] Examples of specific examples and prompts

[0759] As a specific example, a user might say, "Tomorrow I'm going to climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and saves this. The voice data is converted to text using the Google Cloud Speech-to-Text API and saved as "Mt. Tanzawa, 1:00 PM, expected arrival at the mountain hut." If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending a notification to the user's device. As an example of a delivery, a delivery employee might say, "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan is analyzed and saved. If an unplanned route is taken, the discrepancy is detected and an immediate notification is sent.

[0760] Prompt Sentence Examples

[0761] text

[0762] When a food delivery worker says "I have 10 deliveries today" to the app, the voice data is sent to the server and converted to "I have 10 deliveries today" using the Google Cloud Speech-to-Text API.

[0763] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0764] This invention combines an emotion engine with a system that centrally manages and monitors climbers' location information and climbing plans in real time. This system collects, analyzes, and centrally manages climbing plan data in various formats provided by climbers (audio, URLs of other companies' apps, paper images). It also uses the emotion engine to analyze the climber's emotional state in real time and generates alerts and messages to reduce stress and anxiety as needed. This further ensures climber safety and enables the prevention and rapid response of mountaineering accidents.

[0765] Server program processing

[0766] Data collection and analysis

[0767] The server receives the voice data provided by the climber, the URL of the third-party app, and the paper image. This data is analyzed using the generative AI model and converted into a standard digital format. For example, if a climber says, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[0768] Emotion Engine Analysis

[0769] The server is equipped with an emotion engine that analyzes the climber's emotional state (e.g., stress or anxiety) from the voice data and input data. The analysis results are stored in association with each climber's profile.

[0770] Centralized data management

[0771] The server stores the analyzed digital climbing plans in a database and manages them centrally. The analysis results of the emotion engine are also stored. This allows each climber's plan, progress, and emotional state to be visualized in one place and checked in real time.

[0772] Location Tracking

[0773] The server receives location information periodically sent from the climber's mobile device. This location information is checked against the climbing plan to see if it matches the plan. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[0774] Conflict notification and stagnation management

[0775] If there is a discrepancy between the location information and the climbing plan, the server immediately generates an alert and sends a notification to the climber's mobile device. If the climber is delayed or does not descend as planned, the server will link the analysis results of the emotion engine and send a notification to the mountain hut and other climbers. If the emotion engine detects high stress or anxiety, a particularly urgent notification will be generated.

[0776] Terminal program processing

[0777] User authentication and mountain climbing plan input

[0778] The device requires users to create an account and log in when they first start the device. After that, users can input their hiking plans using voice commands, URLs from other companies' apps, or images of paper. This data is sent to a server for analysis.

[0779] Sending location information

[0780] The device activates its GPS function when the climb begins, acquires location information at regular intervals, and sends it to the server. This location information is then checked against the climbing plan to confirm its consistency.

[0781] Emotion Engine Input

[0782] The device sends the user's emotional state to the emotion engine through voice or text input. For example, if the user says, "I'm feeling a little tired," the information is analyzed and sent to the emotion engine.

[0783] Receive notifications

[0784] If the device detects a discrepancy between the plan and location information, or if the emotion engine analyzes that the user is in a state of high stress or anxiety, the device will quickly receive an alert notification and display it to the user in a pop-up.

[0785] User Examples

[0786] Initial setup and climbing plan input

[0787] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by verbally inputting, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the climbing plan is sent to the server and analyzed.

[0788] Start climbing and share your location

[0789] When a user arrives at the trailhead, they press the "Start Climbing" button on the app. The device activates its GPS function and sends its location information to the server every 15 minutes. The location information is compared with the climbing plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[0790] Emotional state analysis and response

[0791] While climbing, the user inputs their emotional state, such as "I'm anxious," through voice. The device sends this voice to the emotion engine, which interprets it as a high anxiety state. The server then quickly generates an alert and sends a notification to the user's device, as well as to mountain huts and other climbers.

[0792] Specific examples

[0793] The user vocally inputs, "I'm going to climb Mt. Tanzawa tomorrow. I plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and stores this information. If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending a notification to the user's device. If the user also says, "I'm a little scared," the emotion engine analyzes the high level of anxiety and generates a high-urgency notification. The user receives this notification and checks for safety. The system also takes into account the user's emotional state, ensuring the user's safety and responding quickly.

[0794] The processing flow will be explained below.

[0795] User creates an account

[0796] Step 1:

[0797] A user downloads the mobile app and launches it for the first time.

[0798] Step 2:

[0799] The user enters basic information such as name, email address, and password, and clicks the registration button.

[0800] Step 3:

[0801] The terminal transmits the user's registration information to the server.

[0802] Step 4:

[0803] The server stores the received user information in a database.

[0804] Step 5:

[0805] The server sends a confirmation message to the terminal to confirm the completion of registration, and the terminal displays the confirmation message.

[0806] The user inputs a mountain climbing plan

[0807] Step 1:

[0808] The user logs in and selects "Enter Mountain Climbing Plan" from the main menu.

[0809] Step 2:

[0810] The user selects one of the following methods: "Voice input," "URL of a third-party app," or "Image upload."

[0811] Step 3:

[0812] For example, a user might say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 p.m."

[0813] Step 4:

[0814] The device records the audio data and sends it to the server.

[0815] Step 5:

[0816] The server analyzes the voice data using a generative AI model and converts it into text data.

[0817] Step 6:

[0818] The server stores the parsed text data in a database and associates it with the user's account.

[0819] User inputs emotional state

[0820] Step 1:

[0821] The user selects "Voice Input" from the emotion input screen.

[0822] Step 2:

[0823] A user says, "I'm feeling a little anxious today."

[0824] Step 3:

[0825] The device transmits voice data related to emotions to the server.

[0826] Step 4:

[0827] The server analyzes the voice data using an emotion engine to identify the emotional state.

[0828] Step 5:

[0829] The server stores the analyzed emotional state in a database and associates it with the user's account.

[0830] User starts climbing

[0831] Step 1:

[0832] The user arrives at the trailhead and launches the app.

[0833] Step 2:

[0834] The user presses the "Start climbing" button.

[0835] Step 3:

[0836] The device will activate the GPS function and begin acquiring location information.

[0837] Step 4:

[0838] The device periodically (e.g., every 15 minutes) sends its location information to the server.

[0839] Location monitoring and discrepancy notification during mountain climbing

[0840] Step 1:

[0841] The server compares the received location information with the mountain climbing plan.

[0842] Step 2:

[0843] The server compares the climbing plan (e.g., scheduled arrival at the mountain hut at 13:00) with the current location information.

[0844] Step 3:

[0845] If the server has not reached the hut by the scheduled time (13:00), it detects a discrepancy.

[0846] Step 4:

[0847] If the server detects a mismatch, it generates an alert message and sends it to the terminal.

[0848] Step 5:

[0849] The device displays a warning to the user saying, "You have not arrived at the mountain hut by the scheduled time. Please check that your surroundings are safe."

[0850] Notification of stagnation and those who have not yet descended

[0851] Step 1:

[0852] The server periodically compares the location information of all users with their mountain climbing plans.

[0853] Step 2:

[0854] If the server detects stagnation or failure to descend, it generates an alert list.

[0855] Step 3:

[0856] The server sends alerts of stagnation or failure to descend the mountain to the relevant user's device.

[0857] Step 4:

[0858] The device displays a warning to the user saying, "You have been stuck at your current location for an extended period of time. Please make sure you are safe."

[0859] Step 5:

[0860] The server also sends relevant notifications to mountain huts and other climbers, encouraging them to take prompt action.

[0861] Countermeasures and notifications based on emotional state

[0862] Step 1:

[0863] The server periodically checks the emotional state analyzed by the emotion engine.

[0864] Step 2:

[0865] The server generates special alerts if high stress or anxiety states are detected.

[0866] Step 3:

[0867] The server sends a message to the user's device saying, "Your current emotional state is unstable. Please check the safety of your surroundings and seek support if necessary."

[0868] Step 4:

[0869] The server sends an emergency notification of the user to the mountain hut and other nearby climbers, asking them to check for safety.

[0870] Centralized data management

[0871] Step 1:

[0872] The server stores all climbers' climbing plans, location information, emotional state, and discrepancy data in a database.

[0873] Step 2:

[0874] The server updates and displays the location information and mountain climbing plan received in real time.

[0875] Step 3:

[0876] If the server needs search and rescue, it will provide relevant information to the police and local authorities.

[0877] Specific examples

[0878] Initial setup and climbing plan input

[0879] Step 1:

[0880] A user launches the app for the first time and creates an account by entering their name, email address, password, etc.

[0881] Step 2:

[0882] The user says in a voice message, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 p.m."

[0883] Step 3:

[0884] The terminal transmits the voice data to the server, which analyzes it and converts it into text data.

[0885] Step 4:

[0886] The server stores the parsed data in a database and associates it with the user's account.

[0887] Start climbing and share your location

[0888] Step 1:

[0889] The user arrives at the trailhead and presses the "Start Climbing" button on the app.

[0890] Step 2:

[0891] The device will enable the GPS function and send location information to the server every 15 minutes.

[0892] Step 3:

[0893] The server compares the location information with the climbing plan to check for any discrepancies.

[0894] Step 4:

[0895] If the server detects a mismatch, it immediately sends an alert to the user terminal.

[0896] Emotional state analysis and response

[0897] Step 1:

[0898] During the climb, the user says, "I'm getting a little tired."

[0899] Step 2:

[0900] The device sends the voice data to the server, which then analyzes it using an emotion engine.

[0901] Step 3:

[0902] The server detects high levels of fatigue and anxiety and generates special alerts.

[0903] Step 4:

[0904] The server sends the user a message saying, "We've checked your status. Stay safe and rest."

[0905] Step 5:

[0906] The server also notifies nearby mountain huts and other climbers, urging users to check for safety.

[0907] Example 2

[0908] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0909] While mountaineering has become increasingly popular in recent years, the number of mountaineering accidents and emergencies has also increased. To solve this problem, a system is needed that can track climbers' location information in real time, detect discrepancies between their climbing plans and their current location, and respond quickly. It is also necessary to better ensure the safety of climbers by analyzing their emotional states, such as stress and anxiety, during climbing in real time and taking appropriate measures.

[0910] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting a mountain climbing plan from audio data provided by the climber, a URL of a third-party app, or a paper image; means for using a generative AI model to analyze the collected mountain climbing plan and convert it into a digital format; means for using an emotion engine to analyze the climber's emotional state; means for periodically acquiring the climber's location information from the mobile device and comparing it with the collected mountain climbing plan; means for generating an alert and notifying the climber if the mountain climbing plan and current location information do not match; and means for notifying mountain huts and other climbers based on the analysis results of the emotion engine if the climber is delayed or has not descended the mountain as planned. This allows the location information and emotional state of the climber to be tracked in real time, enabling rapid response in the event of a mountaineering accident or emergency.

[0911] "Mountain climbing plan" refers to the mountain climbing route, estimated time of arrival, and other related information planned in advance by a climber.

[0912] A "generative AI model" refers to an algorithm or system that uses machine learning technology to analyze input information such as voice data, images, and URLs and convert them into a digital format.

[0913] The "emotion engine" refers to a system that analyzes voice and text data to evaluate and determine the climber's emotional state (e.g., stress or anxiety).

[0914] "Digital format" refers to a data format in which planned climbing information has been converted into a format that can be stored, processed, and analyzed electronically.

[0915] "Centralized management" refers to consolidating various information on a unified platform and efficiently managing and operating it.

[0916] "Location information" refers to geographic coordinate data (e.g., GPS information) that indicates the climber's current location.

[0917] "Mismatch" refers to a situation in which the climber's pre-planned climbing plan does not match the climber's actual behavior or location information.

[0918] "Alert" means a warning message generated by the system when it detects a discrepancy or emergency.

[0919] "Notification" refers to the transmission of alerts and other important information from the server to climbers and related parties.

[0920] The present invention combines an emotion engine with a system that centrally manages and monitors the location information and mountain climbing plans of climbers in real time. Specific embodiments for carrying out the invention are described below.

[0921] The server collects voice data, URLs of third-party apps, and paper images provided by climbers. This collected data is analyzed using a generative AI model (e.g., Google Cloud Speech-to-Text or OCR technology) and converted into a standard digital format. For example, if a user says, "Tomorrow I'm going to climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this speech into text data and stores it in a database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[0922] Next, the server is equipped with an emotion engine that analyzes the climber's emotional state (stress or anxiety) from the voice and input data. The analysis results are stored in association with each climber's profile.

[0923] The analyzed digital climbing plans and the analysis results of the emotion engine are stored in a database on a server and managed centrally, allowing each climber's plan, progress, and emotional state to be visualized in one place and checked in real time.

[0924] Additionally, the server receives location information periodically sent from the climber's mobile device. This location information is compared with the climbing plan to see if it matches the plan. If a discrepancy is detected, the server immediately generates an alert and sends a notification to the climber's mobile device. If the climber is delayed or does not descend as planned, notifications are sent to the mountain hut and other climbers in conjunction with the emotion engine's analysis results. If the emotion engine detects high stress or anxiety, a particularly urgent notification is generated.

[0925] Meanwhile, on the device side, the user creates an account and logs in when they first start up the app. After that, the user inputs their mountain climbing plan using voice, the URL of another company's app, or an image of paper, and this data is sent to the server. In addition, when the climb begins, the GPS function is enabled, and location information is acquired at regular intervals and sent to the server. When the user records their emotional state through voice input or text input, this information is sent to the emotion engine.

[0926] For example, a user might say, "I'm going to climb Mt. Tanzawa tomorrow. I plan to arrive at the mountain hut at 1:00 PM," and the server would analyze and store this information. If the user does not arrive at the hut as planned during the climb, the server would detect the discrepancy and immediately generate an alert, sending a notification to the user's device. If the user also says, "I'm a little scared," the emotion engine would analyze the high level of anxiety and generate a high-urgency notification. The system takes the user's emotional state into account to ensure their safety and respond quickly.

[0927] An example prompt might be:

[0928] "The following speech data is input into the generative AI model: 'Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM.'"

[0929] In this way, by managing and analyzing climbers' location information and emotional state in real time, it is possible to ensure their safety, prevent accidents, and respond quickly.

[0930] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0931] Step 1: Data collection and reception

[0932] The server receives voice data, URLs of other companies' apps, and paper image data from climbers. For input, the user speaks, "Tomorrow I'm going to climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," and this is sent to the server. For output, the voice data and other data formats are stored in a buffer on the server side. Specifically, the climber inputs voice data into their smartphone, and the voice is uploaded to the server via the device.

[0933] Step 2: Data analysis and transformation

[0934] The server analyzes the received data using a generative AI model and converts it into a digital format. Inputs include the received voice data, URL, and image data. The data is analyzed according to the prompts of the generative AI model (e.g., Google Cloud Speech-to-Text or OCR technology). The output is the analyzed text data or data in a structured digital format. Specifically, the voice data is converted into text data such as "Expected arrival at Tanzawa Mountain Hut at 13:00" and stored in a database.

[0935] Step 3: Analysis by Emotion Engine

[0936] The server uses an emotion engine to analyze the climber's emotional state from voice and text data. The input is voice data and text data converted by a generative AI model. The emotion engine analyzes this to determine the climber's state of stress and anxiety. The output is the evaluation result of the emotional state, which is associated with the climber's profile and saved. Specifically, the emotion engine analyzes the user's voice saying "I'm getting a little tired," and adds the "fatigue" status to the profile.

[0937] Step 4: Save to database and centralize management

[0938] The server stores the analyzed digital data and emotion analysis results in a database and manages them centrally. The input is the digital format data and emotional state data analyzed by the generative AI model and emotion engine. As an output, this data is stored in a centrally managed database. Specifically, "Expected arrival at Tanzawa Mountain Hut at 13:00" and the emotion tag "fatigue" are stored and managed in the database.

[0939] Step 5: Track and match location information

[0940] The server receives GPS location information periodically sent from the climber's mobile device and compares it with the collected climbing plan. The input is the climber's current location information periodically received as GPS data and pre-saved climbing plan data. The output is a determination result of whether the location information matches the climbing plan or not. Specifically, the server receives location information every 15 minutes and compares data such as "12:45 local time, 500m before Mt. Tanzawa" with the climbing plan.

[0941] Step 6: Detect and alert on discrepancies

[0942] If there is a discrepancy between the location information and the climbing plan, the server immediately generates an alert and sends a notification to the climber's mobile device. In addition, if the climber is delayed or does not descend as planned, the server notifies the mountain hut and other climbers in conjunction with the analysis results of the emotion engine. The input is data on discrepancies between the location information and the climbing plan, and emotional state data. The output is an alert and notification message in the event of a discrepancy. In concrete terms, if the climber does not arrive at the mountain hut at the scheduled time of 1:00 PM, the server generates an alert and notifies the climber, "There is a discrepancy in the plan. Please check."

[0943] Step 7: User authentication and mountain climbing plan input

[0944] When the user first starts up the device, they create an account and log in. The user then inputs their mountain climbing plan using voice, the URL of a third-party app, and an image of a piece of paper. This input data is sent to the server. The inputs are the user's name, email address, password, and mountain climbing plan data. The output is the user account authentication information and mountain climbing plan data, which are sent to the server. Specific operations include the user entering information on the account creation screen and inputting the plan using voice.

[0945] Step 8: Send location information

[0946] The device activates its GPS function when starting a climb, and acquires and sends location information to the server at regular intervals. The input is the start climbing button operated by the user and the current location data acquired by the GPS. The output is the acquired location information that is periodically sent to the server. Specifically, the user presses the "start climbing" button, and the device sends location information to the server every 15 minutes.

[0947] Step 9: Enter and send emotional states

[0948] The device records the user's emotional state through voice or text input and sends it to the emotion engine. The input is voice data or text data of the emotional state input by the user. The output is the emotional data to be analyzed and sent to the server. In concrete terms, the user may say "I'm getting a little tired" and the data is sent to the server.

[0949] Step 10: Receiving and Viewing Notifications

[0950] The terminal receives an alert notification from the server and displays a pop-up notification to the user. The input is the alert notification data sent from the server. As an output, the received alert notification is displayed in a pop-up format on the terminal screen. Specifically, a pop-up message saying "There is a discrepancy in the plan. Please check" is displayed on the terminal screen.

[0951] (Application example 2)

[0952] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0953] Current systems that manage climbers' location information and climbing plans have difficulty determining their emotional state in real time and taking appropriate action in emergencies. Furthermore, there is a lack of effective means to monitor the work plans and emotional state of factory workers to improve work efficiency and safety. This increases the risk of accidents and problems during mountain climbing and factory work.

[0954] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting mountain climbing plans from either voice data provided by climbers, URLs of other companies' apps, or paper images, means for using a generative AI model to analyze the collected mountain climbing plans and convert them into a digital format, and means for periodically obtaining climber location information from a mobile device and comparing it with the collected mountain climbing plans. This makes it possible to analyze and manage the plans and location information of climbers and factory workers in real time, monitor their emotional states, and generate appropriate alerts.

[0955] "Audio data" refers to audio information provided by climbers or workers through a microphone or audio input device.

[0956] "URLs from other apps" are link information shared from other applications, and are URLs used to represent mountain climbing plans or work plans.

[0957] "Paper images" are digitalized images of paper documents containing mountain climbing plans and work plans.

[0958] "Mountain climbing plan" means a schedule or route plan that a climber makes before climbing a mountain, and includes digital data, audio data, and image data that are provided.

[0959] A "generative AI model" is an artificial intelligence algorithm used to analyze collected audio and image data and convert it into a standard digital format.

[0960] "Digital format" refers to a format in which analog information has been digitized and is a format that can be processed by a computer.

[0961] "Location information" is data indicating the current location of climbers and workers, obtained using GPS or other location information acquisition means.

[0962] The "emotion engine" is an artificial intelligence technology that analyzes emotional states from voice and text data and identifies emotions such as stress, anxiety, and fatigue.

[0963] An "alert" is a message used to warn or caution climbers and workers.

[0964] A "mountain hut" is a facility in a mountainous area set up for climbers to rest and stay overnight.

[0965] The "database" is a digital storage system for centrally storing and managing collected mountaineering and work plan data.

[0966] A "work plan" is a document that outlines the work processes and procedures that workers in a factory draw up, and includes information provided in the form of audio data, URLs of other companies' apps, paper images, etc.

[0967] This invention is a system that centrally manages the location information and plans of mountain climbers and factory workers, and monitors them in real time by combining an emotion engine. This system improves user safety and efficiency by collecting and analyzing mountain climbing and work plan data provided by users, such as voice data, URLs of other companies' apps, and paper images.

[0968] Data collection and analysis

[0969] The server receives the voice data, URLs of other companies' apps, and paper images provided by the user, and uses the generative AI model to analyze and convert this data into a digital format. For example, if a climber says, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in a database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[0970] Emotion Engine Analysis

[0971] The server is equipped with an emotion engine that analyzes the user's emotional state (e.g., stress or anxiety) from voice data and input data. The analysis results are stored in association with each user's profile. The emotion engine uses the Python library emotion_analysis.

[0972] Centralized data management

[0973] The server stores the analyzed digital climbing and work plans in a database and manages them centrally. The analysis results of the emotion engine are also stored. This allows users' plans, progress, and emotional state to be visualized in one place and checked in real time.

[0974] Location Tracking

[0975] The server receives location information periodically sent from the user's mobile device. This location information is checked against the plan to see if it matches. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[0976] Conflict notification and stagnation management

[0977] If there is a discrepancy between the location information and the plan, the server immediately generates an alert and sends a notification to the user's mobile device. If the user is stagnating or not progressing with the plan as planned, the server will link the analysis results of the emotion engine and send an appropriate notification. If the emotion engine detects high stress or anxiety, a particularly urgent notification will be generated.

[0978] Terminal program processing

[0979] After creating an account and logging in, the user's mobile device has a function that allows them to input plans using voice data, URLs of other companies' apps, and paper images. This data is sent to the server and analyzed. By enabling the GPS function, location information is acquired at regular intervals and sent to the server. In addition, the user's emotional state is sent to the emotion engine via voice or text input. If a discrepancy between the plan and location information is detected, an alert notification is promptly received and displayed as a pop-up.

[0980] Specific examples

[0981] A user may say, "I'm going to climb Mt. Tanzawa tomorrow. I plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and stores this information. If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert and notifies the user's device. If the user also says, "I'm a little scared," the emotion engine analyzes high anxiety and generates a high-urgency notification.

[0982] Prompt Sentence Examples

[0983] Prompt: "Please speak to describe your work schedule and current feelings for sentiment analysis. For example, "I'll take three breaks by the end of today's work" or "I'm feeling a bit tired.""

[0984] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0985] Step 1:

[0986] A user creates an account and logs in. The user enters their name, email address, and password and presses the Create Account button. This information is sent to the server, which stores the new user information in a database. The input data is the user's basic information (name, email address, password), and the output is a message that the new account was successfully created.

[0987] Step 2:

[0988] Users input their mountain climbing and work plans using voice, a URL from another company's app, or a paper image. In the case of voice input, the user speaks to their smartphone something like, "Tomorrow I'll climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM." This data is sent to a server, where a generative AI model converts the speech into text. The input data is voice data, a URL, or image data, and the output is the plan data in text format.

[0989] Step 3:

[0990] The server converts the received plan data into a digital format and stores it in a database. For example, text data such as "Scheduled arrival at Tanzawa Mountain Hut at 13:00" is stored in the database. The input data is the text data converted by the generative AI model, and the output is the plan data stored in the database.

[0991] Step 4:

[0992] When a user starts climbing or working, they press the "Start" button on the device app. The device activates the GPS function, acquires location information at regular intervals (e.g., 15 minutes), and sends it to the server. The input data is the location information from the GPS, and the output is the location information sent to the server.

[0993] Step 5:

[0994] The server periodically checks the received location information against the planned data in the database, for example, to see if the current location is along the planned route. The input data are the current location information and the planned data in the database, and the output is a determination of whether the location information matches the plan.

[0995] Step 6:

[0996] If the location information does not match the plan, the server immediately generates an alert and notifies the user's device. For example, if the location deviates from the planned route, an alert is sent to the user. The input data is the mismatch determination result, and the output is an alert notification.

[0997] Step 7:

[0998] The user inputs their emotional state by voice while climbing a mountain or working. For example, they might say, "I'm feeling a little tired." This data is sent to the server via the device, and the emotion engine analyzes it. The input data is voice data, and the output is the analysis result of the emotional state.

[0999] Step 8:

[1000] The server associates the results of the emotion analysis with the profile and generates an alert if the level of urgency is high. For example, if a "high anxiety state" is detected, an emergency alert is sent to the user's device. The input data is the emotion analysis result, and the output is an emergency alert notification.

[1001] Step 9:

[1002] The server centrally manages the analyzed mountain climbing plan data, work plan data, and emotional state, and stores them in a database. This allows the user's plan, progress, and emotional state to be visualized in real time. The input data are the plan data and the emotion analysis results, and the output is storage in the database and real-time visualization.

[1003] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1004] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1005] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1006] [Third embodiment]

[1007] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1008] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1009] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1010] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1011] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1012] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1013] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1014] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1015] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1016] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1017] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1018] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1019] This invention is a system that centrally manages and monitors climbers' location information and climbing plans in real time. This system collects climbing plan data provided by climbers in various formats (voice, URLs of other companies' apps, paper images), analyzes it using a generative AI model, converts it into a digital format, and centrally manages it. As a result, it ensures the safety of climbers and enables the prevention and rapid response of accidents.

[1020] Server program processing

[1021] Data collection and analysis

[1022] The server receives the voice data provided by the climber, the URL of the third-party app, and the paper image. This data is analyzed using the generative AI model and converted into a standard digital format. For example, if a climber says, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[1023] Centralized data management

[1024] The server stores the analyzed climbing plans in a digital format in a database and manages them centrally, allowing each climber's plan and progress to be visualized in one place and checked in real time.

[1025] Location Tracking

[1026] The server receives location information periodically sent from the climber's mobile device. This location information is checked against the climbing plan to see if it matches the plan. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[1027] Conflict notification and stagnation management

[1028] If there is a discrepancy between the location information and the climbing plan, the server immediately generates an alert and sends a notification to the climber's mobile device. If the climber is delayed or does not descend as planned, notifications are sent to the mountain hut and other climbers so that a prompt response can be taken.

[1029] Terminal program processing

[1030] User authentication and mountain climbing plan input

[1031] The device requires users to create an account and log in when they first start the device. After that, users can input their hiking plans using voice commands, URLs from other companies' apps, or images of paper. This data is sent to a server for analysis.

[1032] Sending location information

[1033] The device activates its GPS function when the climb begins, acquires location information at regular intervals, and sends it to the server. This location information is then checked against the climbing plan to confirm its consistency.

[1034] Receive notifications

[1035] If the device detects a discrepancy between the plan and location information, or if the climber is delayed, it will immediately receive an alert notification and display it to the user in a pop-up.

[1036] User Examples

[1037] Initial setup and climbing plan input

[1038] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by verbally inputting, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the climbing plan is sent to the server and analyzed.

[1039] Start climbing and share your location

[1040] When a user arrives at the trailhead, they press the "Start Climbing" button on the app. The device activates its GPS function and sends its location information to the server every 15 minutes. The location information is compared with the climbing plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[1041] Specific examples

[1042] A user vocally inputs, "Tomorrow I'm going to climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and stores this information. If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending it to the user's device. The user receives the notification and takes appropriate measures to ensure their safety. In this way, the system can prevent accidents and quickly identify stranded individuals.

[1043] The processing flow will be explained below.

[1044] User creates an account

[1045] Step 1:

[1046] A user downloads the mobile app and launches it for the first time.

[1047] Step 2:

[1048] The user enters basic information such as name, email address, and password, and clicks the registration button.

[1049] Step 3:

[1050] The terminal transmits the user's registration information to the server.

[1051] Step 4:

[1052] The server stores the received user information in a database.

[1053] Step 5:

[1054] The server sends a confirmation message to the terminal indicating that registration has been completed, and the terminal displays the confirmation message.

[1055] The user inputs a mountain climbing plan

[1056] Step 1:

[1057] The user logs in and selects "Enter Mountain Climbing Plan" from the main menu.

[1058] Step 2:

[1059] The user selects one of the following methods: "Voice input," "URL of a third-party app," or "Image upload."

[1060] Step 3:

[1061] For example, a user might say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 p.m."

[1062] Step 4:

[1063] The device records the audio data and sends it to the server.

[1064] Step 5:

[1065] The server analyzes the voice data using a generative AI model and converts it into text data.

[1066] Step 6:

[1067] The server stores the parsed text data in a database and associates it with the user's account.

[1068] User starts climbing

[1069] Step 1:

[1070] The user arrives at the trailhead and launches the app.

[1071] Step 2:

[1072] The user presses the "Start climbing" button.

[1073] Step 3:

[1074] The device will activate the GPS function and begin acquiring location information.

[1075] Step 4:

[1076] The device periodically (e.g., every 15 minutes) sends its location information to the server.

[1077] Location monitoring and discrepancy notification during mountain climbing

[1078] Step 1:

[1079] The server compares the received location information with the mountain climbing plan.

[1080] Step 2:

[1081] The server compares the climbing plan (e.g., scheduled arrival at the mountain hut at 13:00) with the current location information.

[1082] Step 3:

[1083] If the server has not reached the hut by the scheduled time (13:00), it detects a discrepancy.

[1084] Step 4:

[1085] If the server detects a mismatch, it generates an alert message and sends it to the terminal.

[1086] Step 5:

[1087] The device displays a warning to the user saying, "You have not arrived at the mountain hut by the scheduled time. Please check that your surroundings are safe."

[1088] Notification of stagnation and those who have not yet descended

[1089] Step 1:

[1090] The server periodically compares the location information of all users with their mountain climbing plans.

[1091] Step 2:

[1092] If the server detects stagnation or failure to descend, it generates an alert list.

[1093] Step 3:

[1094] The server sends alerts of stagnation or failure to descend the mountain to the relevant user's device.

[1095] Step 4:

[1096] The device will warn the user that "You have been stuck at your current location for an extended period of time. Please make sure you are safe."

[1097] Step 5:

[1098] The server also sends relevant notifications to mountain huts and other climbers, encouraging them to take prompt action.

[1099] The server centrally manages data

[1100] Step 1:

[1101] The server stores all climbers' climbing plans, location information, and conflict data in a database.

[1102] Step 2:

[1103] The server updates and displays the location information and mountain climbing plan received in real time.

[1104] Step 3:

[1105] If the server needs search and rescue, it will provide relevant information to the police and local authorities.

[1106] Specific examples

[1107] Initial setup and climbing plan input

[1108] Step 1:

[1109] A user launches the app for the first time and creates an account by entering their name, email address, password, etc.

[1110] Step 2:

[1111] The user says in a voice message, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 p.m."

[1112] Step 3:

[1113] The server analyzes the audio and converts it into text data.

[1114] Step 4:

[1115] The server stores the parsed data in a database and associates it with the user's account.

[1116] Start climbing and share your location

[1117] Step 1:

[1118] The user arrives at the trailhead and presses the "Start Climbing" button on the app.

[1119] Step 2:

[1120] The device will enable the GPS function and send location information to the server every 15 minutes.

[1121] Step 3:

[1122] The server compares the location information with the climbing plan to check for any discrepancies.

[1123] Step 4:

[1124] If things aren't going according to plan, the user is immediately alerted.

[1125] Inconsistency warnings and actions

[1126] Step 1:

[1127] The server detects that the user has not yet arrived at the mountain hut even though it is now 13:00.

[1128] Step 2:

[1129] The server detects the discrepancy and generates an alert that is sent to the user terminal.

[1130] Step 3:

[1131] The terminal displays a warning to the user that "You have not arrived at the mountain hut even though the scheduled time has passed."

[1132] Step 4:

[1133] The user checks the notification to check safety and understand the situation.

[1134] Example 1

[1135] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1136] When climbing mountains, ensuring the safety of climbers and responding quickly are essential, but conventional methods have made managing climbing plans and current locations cumbersome. Furthermore, there are insufficient means for responding quickly when climbers are not following plan or when there is a possibility of getting lost. Therefore, a system is needed that efficiently and centrally manages climbers' location information and climbing plans, and monitors them in real time.

[1137] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1138] In this invention, the server includes a means for collecting mountain climbing plans from either voice data provided by climbers, links to other companies' applications, or paper images, a means for using a generative artificial intelligence model to analyze the collected mountain climbing plans and convert them into a digital format, and a means for storing the analyzed mountain climbing plan data in a database for centralized management. This allows for efficient management of climbers' plans and current location information, and makes it possible to quickly detect and respond to any abnormalities.

[1139] A "mountaineer" is a person who goes mountain climbing and refers to a user of this system.

[1140] "Audio data" refers to information input by voice by a climber that has been recorded as digital data.

[1141] "Links to third-party applications" refers to URLs that connect to pages or information within applications provided by other companies.

[1142] "Paper images" refer to image data that has been digitized from paper media containing mountain climbing plans.

[1143] A "mountain climbing plan" is a detailed description of information such as the climber's planned time to reach the summit, route, and destination.

[1144] A "generative artificial intelligence model" refers to a machine learning model that analyzes natural language and converts it into digital data.

[1145] A "digital format" is a unified format for expressing mountain climbing plans as digital data.

[1146] A "database" refers to a system or device for centrally managing and storing analyzed mountain climbing plan data and location information.

[1147] "Mobile devices" refer to electronic devices that climbers can carry with them, such as smartphones or devices with GPS functionality.

[1148] "Location information" refers to data indicating a climber's current location, typically obtained by GPS or other positioning systems.

[1149] "Detecting anomalies" refers to checking for discrepancies when they arise between the mountain climbing plan and the current location information.

[1150] A "notification" is a message sent to the climber or a designated third party regarding detected abnormalities or important information.

[1151] "Designated locations" refers to locations and facilities that have been registered in advance as part of mountain climbing plans or as emergency contact points.

[1152] "Other climbers" refers to other climbers using this system, who may be subject to notification in the event of an emergency.

[1153] This invention is a management system for mountain climbing plans and location information, with the aim of ensuring the safety of climbers and providing prompt response. This system is composed of a server, terminals, and climbers.

[1154] The server has a means for receiving voice data provided by the climber, links to third-party applications, and paper image data. Specifically, it uses a generative AI model (e.g., GPT-4) to analyze this data and convert it into a standard digital format. If the climber verbally inputs, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the server uses the generative AI model to convert the voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut." This analysis uses the prompt, "Please analyze this climber's voice data and convert his / her climbing plan into a digital format."

[1155] The server is equipped with a system for storing analyzed digital climbing plans in a database and managing them centrally. This database aggregates each climber's plans and progress, allowing them to be viewed in real time. The server also has the function of periodically receiving location information from the climber's mobile device. This location information is obtained using a GPS module.

[1156] The server compares the received location information with the climbing plan data to see if it matches the plan. If a discrepancy is detected, the server detects the anomaly and generates an alert to notify the climber's mobile device. If a climber is delayed or does not descend as planned, notifications are sent to the mountain hut and other climbers, allowing for a prompt response.

[1157] The device, on the other hand, has functions for user authentication and input of mountain climbing plans. When the user starts the device for the first time, they create an account by entering information such as their name, email address, and password, and then log in. After that, the user can enter their mountain climbing plan using voice, a link to a third-party application, or an image of paper, and send it to the server.

[1158] The device also activates its GPS function when the climb begins and acquires and transmits location information to the server at regular intervals (for example, every 15 minutes). This location information is compared with the climbing plan to confirm consistency. Furthermore, the device has the ability to instantly receive an alert notification and display it to the user in a pop-up if a discrepancy between the plan and location information is detected, or if the climber is stalled.

[1159] In this way, the present invention efficiently manages climbers' plans and current locations, and quickly detects and responds to abnormalities, ensuring the safety of climbers. It also enables prompt action through a notification function to mountain huts and other climbers.

[1160] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1161] Server program processing

[1162] Step 1: Data collection

[1163] The server receives audio data provided by the climber, links to other companies' applications, and paper image data. This input data is sent to the server through various data collection methods. For example, if a climber records audio data, the recording is uploaded to the server. Specifically, the server receives an HTTP request and saves the data in a local directory.

[1164] Step 2: Data analysis

[1165] The server inputs the received data into a generative AI model (e.g., GPT-4) and parses it into a digital format. The input data is an audio file, and the output is analyzed text data. The prompt "Please analyze this climber's audio data and convert the climbing plan into a digital format" is used and input into the generative AI model. Specifically, the server sends the audio data to a text conversion API and receives the conversion result.

[1166] Step 3: Store in the database

[1167] The server stores the analyzed digitally formatted mountain climbing plan in a database. The input data is the text data of the analysis results, and the output data is registered in the database. Specifically, the server issues an SQL query and performs an insert process into the database.

[1168] Step 4: Receiving and analyzing location information

[1169] The server receives location information periodically sent from the climber's mobile device. The input data is GPS information, and the output is the current location, which is registered in the database. Specifically, the server receives an HTTP request and saves the location information in the database.

[1170] Step 5: Detect and notify discrepancies

[1171] The server compares the received location information with the climbing plan data and detects any discrepancies. The input data is the current location and climbing plan data, and the output generates an alert notification if an abnormality is detected. Specifically, the server uses an SQL query to retrieve the necessary data from the database, compares it with the plan, and calls a notification API if an abnormality is found.

[1172] Step 6: Alert response actions

[1173] The server notifies the relevant parties of the generated alert. The input data is the alert information, and the output is a notification. Specifically, the server notifies the mountain hut and other climbers using the notification API.

[1174] Terminal program processing

[1175] Step 1: Initial setup and user authentication

[1176] When a user starts up a device for the first time, they create an account by entering information such as their name, email address, and password. The input data is user information, and the output is an account registered on the server. Specifically, the device sends the entered data to the server as an HTTP POST request.

[1177] Step 2: Enter and submit your climbing plan

[1178] Users input their mountain climbing plans using voice, links to third-party applications, and paper images. The input data includes voice data and image data, which are sent to the server as output. Specifically, after collecting the data, the device sends it to the server as an HTTP POST request.

[1179] Step 3: Collecting and sending location information

[1180] When the user presses the "Start Climbing" button on the app to begin climbing, the device activates its GPS function. GPS information is the input data, and location information is sent to the server every 15 minutes as output. Specifically, the device starts a timer, periodically acquires GPS data, and sends it to the server as an HTTP POST request.

[1181] Step 4: Receive and view alert notifications

[1182] If a mismatch is detected between the user's location information and the mountain climbing plan, the device immediately receives an alert notification. The input data is the alert information, and the output is a pop-up notification that is displayed to the user. Specifically, the device receives a push notification from the server and displays the notification content on the screen.

[1183] (Application example 1)

[1184] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1185] In modern society, serious risks and problems often arise when plans and actual actions do not match in situations such as mountain climbing and food delivery. For climbers, the risk of getting lost increases if they do not proceed along the mountain path as planned, requiring a quick response. Meanwhile, in food delivery, if delivery employees are unable to deliver as scheduled, customer satisfaction decreases and work efficiency deteriorates. To prevent such situations from occurring, it is necessary to confirm the match between plans and location information in real time. However, current technology does not provide sufficient means to do this efficiently.

[1186] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1187] In this invention, the server includes: means for collecting mountain climbing plans from either voice data, URLs of other companies' apps, or paper images provided by climbers; means for using a generative AI model to analyze the collected mountain climbing plans and convert them into a digital format; means for centrally managing the analyzed mountain climbing plan data; means for periodically obtaining climbers' location information from a mobile device and comparing it with the collected mountain climbing plans; means for generating an alert and notifying the climber if the mountain climbing plan and current location information do not match; means for notifying mountain huts and other climbers if the climber is delayed or does not descend the mountain as planned; means for collecting delivery plans from either voice memos, URLs of other companies' apps, or images of handwritten notes provided by delivery employees; means for using a generative AI model to analyze the collected delivery plans and convert them into a digital format; means for centrally managing the analyzed delivery plan data; means for periodically obtaining delivery employee location information from a mobile device and comparing it with the collected delivery plans; means for generating an alert and notifying the delivery employee if the delivery plan and current location information do not match; and means for notifying other relevant parties if the delivery employee is delayed or does not deliver as planned. This makes it possible to efficiently check in real time whether the plans and actual actions of climbers and delivery employees match, preventing risks and problems before they occur.

[1188] A "mountaineer" is an individual who engages in the activity of climbing mountains in a natural environment.

[1189] A "delivery employee" is an individual who performs food delivery or other delivery duties.

[1190] "Voice data" refers to data that is a digital recording of a human voice.

[1191] "Third-party app URL" means a web link related to application software provided by another company.

[1192] A "paper image" is information written on paper that has been converted into image data using a digital camera or scanner.

[1193] A "mountain climbing plan" is a detailed schedule of a mountain climb that a climber prepares in advance.

[1194] A "delivery plan" is a detailed delivery schedule prepared in advance by delivery employees.

[1195] A "generative AI model" is an artificial intelligence model that learns from large amounts of data and performs specific tasks.

[1196] "Digital format" refers to the form of data that is represented or recorded using digital technology.

[1197] A "mobile terminal" is a portable information device such as a smartphone or tablet.

[1198] "Location information" is data that indicates a geographic location. It is obtained using technologies such as GPS.

[1199] An "alert" is a warning message that notifies the user of an abnormal situation or important notification.

[1200] A mountain hut is a building set up for climbers to stay overnight or take a break.

[1201] "Related parties" refers to any person or organization that plays a significant role in delivery operations.

[1202] This invention is a system that centrally manages and monitors the location information and plans of climbers and delivery employees in real time. This system collects data in multiple formats, such as voice memos, URLs of other companies' apps, and paper images, provided by climbers and delivery employees, and analyzes it using a generative AI model. This data is then converted into a digital format for centralized management, ensuring safety and improving work efficiency.

[1203] Server program processing

[1204] Data collection and analysis

[1205] The server receives voice data, URLs of other companies' apps, and paper images provided by climbers and delivery employees. The collected data is analyzed using a generative AI model and converted into a standard digital format. For example, if a climber sends a voice memo saying, "Tomorrow I'll be climbing Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database as "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[1206] Centralized data management

[1207] The server stores the analyzed climbing plan data and delivery plan data in a database and manages them centrally, allowing the plans and progress of each climber and delivery employee to be visualized in one place and checked in real time.

[1208] Location Tracking

[1209] The server receives location information periodically sent from the mobile devices of climbers and delivery employees. This location information is checked against the climbing plan and delivery plan to confirm consistency. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[1210] Conflict notification and stagnation management

[1211] If there is a discrepancy between the location information and the climbing plan or delivery plan, the server immediately generates an alert and sends a notification to the mobile devices of the climber or delivery employee. Also, if a climber is delayed or does not descend as planned, or if a delivery employee is late, the information is notified to other parties so that they can take prompt action.

[1212] Terminal program processing

[1213] User authentication and plan input

[1214] When the device is first started, the user is prompted to create an account and log in. After that, the user inputs mountain climbing and delivery plans using voice memos, URLs of other companies' apps, and images of paper. This data is sent to the server and analyzed.

[1215] Sending location information

[1216] The device activates its GPS function when starting a climb or delivery, acquires location information at regular intervals, and sends it to the server. This location information is then compared with the plan to confirm consistency.

[1217] Receive notifications

[1218] The device receives instant alert notifications and displays them to the user in a pop-up if it detects a discrepancy between the plan and location information, if a climber is delayed, or if a delivery employee is late.

[1219] User Examples

[1220] Initial setup and planning input

[1221] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by speaking, they can say, "Tomorrow I'm going to climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," or "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan is sent to the server and analyzed.

[1222] Initiating planning and location sharing

[1223] When a user arrives at the trailhead or delivery start point, they press the "Start Climbing" or "Start Delivery" button on the app. The device activates its GPS function and sends location information to the server every 15 minutes. The location information is compared with the plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[1224] Examples of concrete examples and prompts

[1225] As a specific example, a user may say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture, and plan to arrive at the mountain hut at 1:00 PM," and the server will analyze and save this. If the user does not arrive at the mountain hut as planned during the climb, the server will detect the discrepancy and immediately generate an alert, notifying the user's device. As an example of a delivery, a delivery employee may say by voice, "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan will be analyzed and saved. If an unplanned route is taken, a discrepancy will be detected and an immediate notification will be sent.

[1226] Prompt Sentence Examples

[1227] text

[1228] When a food delivery worker says "I have 10 deliveries today" to the app, the voice data is sent to the server and converted to "I have 10 deliveries today" using the Google Cloud Speech-to-Text API.

[1229] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1230] Program processing steps

[1231] Step 1:

[1232] The server receives the voice data provided by the climbers and delivery employees, the URL of the third-party app, and the paper image. This input data depends on the format each user sends from the app. For example, a user records a voice memo in the app, which is then sent to the server. The output of this step is the data in the format received by the server.

[1233] Step 2:

[1234] The server inputs the received data into the generative AI model. For audio data, it uses the Google Cloud Speech-to-Text API to convert the audio to text. For URLs from other apps, it sends a single web request and analyzes the response. For paper images, it uses the Google Cloud Vision API to convert handwritten characters to text. The output of this step is the converted text data.

[1235] Step 3:

[1236] The server then compiles the converted text data into a digital format. For example, a voice memo saying "Tomorrow we will climb Mt. Tanzawa. We plan to arrive at the mountain hut at 1:00 PM" is converted into the format "Mt. Tanzawa, 1:00 PM, we plan to arrive at the mountain hut." The output of this step is the climbing plan data and delivery plan data in a unified digital format.

[1237] Step 4:

[1238] The server stores the data in a unified digital format in a database and manages it centrally. This allows the plans and progress of each climber and delivery employee to be visualized in one place. The output of this step is the plan data in a digital format stored in a database.

[1239] Step 5:

[1240] The device activates the GPS function at the start of the climb or delivery and acquires location information at regular intervals. This location information is periodically sent to the server. The output of this step is the location information data that is periodically sent.

[1241] Step 6:

[1242] The server receives the periodically transmitted location information and checks it against the previously stored digitally formatted plan data, for example, to see if the location information is updated according to the plan. The output of this step is the check result.

[1243] Step 7:

[1244] The server generates an alert immediately if there is a discrepancy between the location information and the planned data. The alert is sent to the climber's or delivery employee's device. For example, if the location is not updated after the planned arrival time, an alert is issued. The output of this step is an alert notification sent to the user's device.

[1245] Step 8:

[1246] The terminal displays the alert notification received from the server to the user as a popup, allowing the user to immediately check the abnormality and take necessary action. The output of this step is an alert notification that is displayed in a visible form to the user.

[1247] Examples of concrete examples and prompts

[1248] As a specific example, a user might say, "Tomorrow I'm going to climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and saves this. The voice data is converted to text using the Google Cloud Speech-to-Text API and saved as "Mt. Tanzawa, 1:00 PM, expected arrival at the mountain hut." If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending a notification to the user's device. As an example of a delivery, a delivery employee might say, "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan is analyzed and saved. If an unplanned route is taken, the discrepancy is detected and an immediate notification is sent.

[1249] Prompt Sentence Examples

[1250] text

[1251] When a food delivery worker says "I have 10 deliveries today" to the app, the voice data is sent to the server and converted to "I have 10 deliveries today" using the Google Cloud Speech-to-Text API.

[1252] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1253] This invention combines an emotion engine with a system that centrally manages and monitors climbers' location information and climbing plans in real time. This system collects, analyzes, and centrally manages climbing plan data in various formats provided by climbers (audio, URLs of other companies' apps, paper images). It also uses the emotion engine to analyze the climber's emotional state in real time and generates alerts and messages to reduce stress and anxiety as needed. This further ensures climber safety and enables the prevention and rapid response of mountaineering accidents.

[1254] Server program processing

[1255] Data collection and analysis

[1256] The server receives the voice data provided by the climber, the URL of the third-party app, and the paper image. This data is analyzed using the generative AI model and converted into a standard digital format. For example, if a climber says, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[1257] Emotion Engine Analysis

[1258] The server is equipped with an emotion engine that analyzes the climber's emotional state (e.g., stress or anxiety) from the voice data and input data. The analysis results are stored in association with each climber's profile.

[1259] Centralized data management

[1260] The server stores the analyzed digital climbing plans in a database and manages them centrally. The analysis results of the emotion engine are also stored. This allows each climber's plan, progress, and emotional state to be visualized in one place and checked in real time.

[1261] Location Tracking

[1262] The server receives location information periodically sent from the climber's mobile device. This location information is checked against the climbing plan to see if it matches the plan. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[1263] Conflict notification and stagnation management

[1264] If there is a discrepancy between the location information and the climbing plan, the server immediately generates an alert and sends a notification to the climber's mobile device. If the climber is delayed or does not descend as planned, the server will link the analysis results of the emotion engine and send a notification to the mountain hut and other climbers. If the emotion engine detects high stress or anxiety, a particularly urgent notification will be generated.

[1265] Terminal program processing

[1266] User authentication and mountain climbing plan input

[1267] The device requires users to create an account and log in when they first start the device. After that, users can input their hiking plans using voice commands, URLs from other companies' apps, or images of paper. This data is sent to a server for analysis.

[1268] Sending location information

[1269] The device activates its GPS function when the climb begins, acquires location information at regular intervals, and sends it to the server. This location information is then checked against the climbing plan to confirm its consistency.

[1270] Emotion Engine Input

[1271] The device sends the user's emotional state to the emotion engine through voice or text input. For example, if the user says, "I'm feeling a little tired," the information is analyzed and sent to the emotion engine.

[1272] Receive notifications

[1273] If the device detects a discrepancy between the plan and location information, or if the emotion engine analyzes that the user is in a state of high stress or anxiety, the device will quickly receive an alert notification and display it to the user in a pop-up.

[1274] User Examples

[1275] Initial setup and climbing plan input

[1276] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by verbally inputting, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the climbing plan is sent to the server and analyzed.

[1277] Start climbing and share your location

[1278] When a user arrives at the trailhead, they press the "Start Climbing" button on the app. The device activates its GPS function and sends its location information to the server every 15 minutes. The location information is compared with the climbing plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[1279] Emotional state analysis and response

[1280] While climbing, the user inputs their emotional state, such as "I'm anxious," through voice. The device sends this voice to the emotion engine, which interprets it as a high anxiety state. The server then quickly generates an alert and sends a notification to the user's device, as well as to mountain huts and other climbers.

[1281] Specific examples

[1282] The user vocally inputs, "I'm going to climb Mt. Tanzawa tomorrow. I plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and stores this information. If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending a notification to the user's device. If the user also says, "I'm a little scared," the emotion engine analyzes the high level of anxiety and generates a high-urgency notification. The user receives this notification and checks for safety. The system also takes into account the user's emotional state, ensuring the user's safety and responding quickly.

[1283] The processing flow will be explained below.

[1284] User creates an account

[1285] Step 1:

[1286] A user downloads the mobile app and launches it for the first time.

[1287] Step 2:

[1288] The user enters basic information such as name, email address, and password, and clicks the registration button.

[1289] Step 3:

[1290] The terminal transmits the user's registration information to the server.

[1291] Step 4:

[1292] The server stores the received user information in a database.

[1293] Step 5:

[1294] The server sends a confirmation message to the terminal indicating that registration has been completed, and the terminal displays the confirmation message.

[1295] The user inputs a mountain climbing plan

[1296] Step 1:

[1297] The user logs in and selects "Enter Mountain Climbing Plan" from the main menu.

[1298] Step 2:

[1299] The user selects one of the following methods: "Voice input," "URL of a third-party app," or "Image upload."

[1300] Step 3:

[1301] For example, a user might say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 p.m."

[1302] Step 4:

[1303] The device records the audio data and sends it to the server.

[1304] Step 5:

[1305] The server analyzes the voice data using a generative AI model and converts it into text data.

[1306] Step 6:

[1307] The server stores the parsed text data in a database and associates it with the user's account.

[1308] User inputs emotional state

[1309] Step 1:

[1310] The user selects "Voice Input" from the emotion input screen.

[1311] Step 2:

[1312] A user says, "I'm feeling a little anxious today."

[1313] Step 3:

[1314] The device transmits voice data related to emotions to the server.

[1315] Step 4:

[1316] The server analyzes the voice data using an emotion engine to identify the emotional state.

[1317] Step 5:

[1318] The server stores the analyzed emotional state in a database and associates it with the user's account.

[1319] User starts climbing

[1320] Step 1:

[1321] The user arrives at the trailhead and launches the app.

[1322] Step 2:

[1323] The user presses the "Start climbing" button.

[1324] Step 3:

[1325] The device will activate the GPS function and begin acquiring location information.

[1326] Step 4:

[1327] The device periodically (e.g., every 15 minutes) sends its location information to the server.

[1328] Location monitoring and discrepancy notification during mountain climbing

[1329] Step 1:

[1330] The server compares the received location information with the mountain climbing plan.

[1331] Step 2:

[1332] The server compares the climbing plan (e.g., scheduled arrival at the mountain hut at 13:00) with the current location information.

[1333] Step 3:

[1334] If the server has not reached the hut by the scheduled time (13:00), it detects a discrepancy.

[1335] Step 4:

[1336] If the server detects a mismatch, it generates an alert message and sends it to the terminal.

[1337] Step 5:

[1338] The device displays a warning to the user saying, "You have not arrived at the mountain hut by the scheduled time. Please check that your surroundings are safe."

[1339] Notification of stagnation and those who have not yet descended

[1340] Step 1:

[1341] The server periodically compares the location information of all users with their mountain climbing plans.

[1342] Step 2:

[1343] If the server detects stagnation or failure to descend, it generates an alert list.

[1344] Step 3:

[1345] The server sends alerts of stagnation or failure to descend the mountain to the relevant user's device.

[1346] Step 4:

[1347] The device displays a warning to the user saying, "You have been stuck at your current location for an extended period of time. Please make sure you are safe."

[1348] Step 5:

[1349] The server also sends relevant notifications to mountain huts and other climbers, encouraging them to take prompt action.

[1350] Countermeasures and notifications based on emotional state

[1351] Step 1:

[1352] The server periodically checks the emotional state analyzed by the emotion engine.

[1353] Step 2:

[1354] The server generates special alerts if high stress or anxiety states are detected.

[1355] Step 3:

[1356] The server sends a message to the user's device saying, "Your current emotional state is unstable. Please check the safety of your surroundings and seek support if necessary."

[1357] Step 4:

[1358] The server sends an emergency notification of the user to the mountain hut and other nearby climbers, asking them to check for safety.

[1359] Centralized data management

[1360] Step 1:

[1361] The server stores all climbers' climbing plans, location information, emotional state, and discrepancy data in a database.

[1362] Step 2:

[1363] The server updates and displays the location information and mountain climbing plan received in real time.

[1364] Step 3:

[1365] If the server needs search and rescue, it will provide relevant information to the police and local authorities.

[1366] Specific examples

[1367] Initial setup and climbing plan input

[1368] Step 1:

[1369] A user launches the app for the first time and creates an account by entering their name, email address, password, etc.

[1370] Step 2:

[1371] The user says in a voice message, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 p.m."

[1372] Step 3:

[1373] The terminal transmits the voice data to the server, which analyzes it and converts it into text data.

[1374] Step 4:

[1375] The server stores the parsed data in a database and associates it with the user's account.

[1376] Start climbing and share your location

[1377] Step 1:

[1378] The user arrives at the trailhead and presses the "Start Climbing" button on the app.

[1379] Step 2:

[1380] The device will enable the GPS function and send location information to the server every 15 minutes.

[1381] Step 3:

[1382] The server compares the location information with the climbing plan to check for any discrepancies.

[1383] Step 4:

[1384] If the server detects a mismatch, it immediately sends an alert to the user terminal.

[1385] Emotional state analysis and response

[1386] Step 1:

[1387] During the climb, the user says, "I'm getting a little tired."

[1388] Step 2:

[1389] The device sends the voice data to the server, which then analyzes it using an emotion engine.

[1390] Step 3:

[1391] The server detects high levels of fatigue and anxiety and generates special alerts.

[1392] Step 4:

[1393] The server sends the user a message saying, "We've checked your status. Stay safe and rest."

[1394] Step 5:

[1395] The server also notifies nearby mountain huts and other climbers, urging users to check for safety.

[1396] Example 2

[1397] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1398] While mountaineering has become increasingly popular in recent years, the number of mountaineering accidents and emergencies has also increased. To solve this problem, a system is needed that can track climbers' location information in real time, detect discrepancies between their climbing plans and their current location, and respond quickly. It is also necessary to better ensure the safety of climbers by analyzing their emotional states, such as stress and anxiety, during climbing in real time and taking appropriate measures.

[1399] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting a mountain climbing plan from audio data provided by the climber, a URL of a third-party app, or a paper image; means for using a generative AI model to analyze the collected mountain climbing plan and convert it into a digital format; means for using an emotion engine to analyze the climber's emotional state; means for periodically acquiring the climber's location information from the mobile device and comparing it with the collected mountain climbing plan; means for generating an alert and notifying the climber if the mountain climbing plan and current location information do not match; and means for notifying mountain huts and other climbers based on the analysis results of the emotion engine if the climber is delayed or has not descended the mountain as planned. This allows the location information and emotional state of the climber to be tracked in real time, enabling rapid response in the event of a mountaineering accident or emergency.

[1400] "Mountain climbing plan" refers to the mountain climbing route, estimated time of arrival, and other related information planned in advance by a climber.

[1401] A "generative AI model" refers to an algorithm or system that uses machine learning technology to analyze input information such as voice data, images, and URLs and convert them into a digital format.

[1402] The "emotion engine" refers to a system that analyzes voice and text data to evaluate and determine the climber's emotional state (e.g., stress or anxiety).

[1403] "Digital format" refers to a data format in which planned climbing information has been converted into a format that can be stored, processed, and analyzed electronically.

[1404] "Centralized management" refers to consolidating various information on a unified platform and efficiently managing and operating it.

[1405] "Location information" refers to geographic coordinate data (e.g., GPS information) that indicates the climber's current location.

[1406] "Mismatch" refers to a situation in which the climber's pre-planned climbing plan does not match the climber's actual behavior or location information.

[1407] "Alert" means a warning message generated by the system when it detects a discrepancy or emergency.

[1408] "Notification" refers to the transmission of alerts and other important information from the server to climbers and related parties.

[1409] The present invention combines an emotion engine with a system that centrally manages and monitors the location information and mountain climbing plans of climbers in real time. Specific embodiments for carrying out the invention are described below.

[1410] The server collects voice data, URLs of third-party apps, and paper images provided by climbers. This collected data is analyzed using a generative AI model (e.g., Google Cloud Speech-to-Text or OCR technology) and converted into a standard digital format. For example, if a user says, "Tomorrow I'm going to climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this speech into text data and stores it in a database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[1411] Next, the server is equipped with an emotion engine that analyzes the climber's emotional state (stress or anxiety) from the voice and input data. The analysis results are stored in association with each climber's profile.

[1412] The analyzed digital climbing plans and the analysis results of the emotion engine are stored in a database on a server and managed centrally, allowing each climber's plan, progress, and emotional state to be visualized in one place and checked in real time.

[1413] Additionally, the server receives location information periodically sent from the climber's mobile device. This location information is compared with the climbing plan to see if it matches the plan. If a discrepancy is detected, the server immediately generates an alert and sends a notification to the climber's mobile device. If the climber is delayed or does not descend as planned, notifications are sent to the mountain hut and other climbers in conjunction with the emotion engine's analysis results. If the emotion engine detects high stress or anxiety, a particularly urgent notification is generated.

[1414] Meanwhile, on the device side, the user creates an account and logs in when they first start up the app. After that, the user inputs their mountain climbing plan using voice, the URL of another company's app, or an image of paper, and this data is sent to the server. In addition, when the climb begins, the GPS function is enabled, and location information is acquired at regular intervals and sent to the server. When the user records their emotional state through voice input or text input, this information is sent to the emotion engine.

[1415] For example, a user might say, "I'm going to climb Mt. Tanzawa tomorrow. I plan to arrive at the mountain hut at 1:00 PM," and the server would analyze and store this information. If the user does not arrive at the hut as planned during the climb, the server would detect the discrepancy and immediately generate an alert, sending a notification to the user's device. If the user also says, "I'm a little scared," the emotion engine would analyze the high level of anxiety and generate a high-urgency notification. The system takes the user's emotional state into account to ensure their safety and respond quickly.

[1416] An example prompt might be:

[1417] "The following speech data is input into the generative AI model: 'Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM.'"

[1418] In this way, by managing and analyzing climbers' location information and emotional state in real time, it is possible to ensure their safety, prevent accidents, and respond quickly.

[1419] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1420] Step 1: Data collection and reception

[1421] The server receives voice data, URLs of other companies' apps, and paper image data from climbers. For input, the user speaks, "Tomorrow I'm going to climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," and this is sent to the server. For output, the voice data and other data formats are stored in a buffer on the server side. Specifically, the climber inputs voice data into their smartphone, and the voice is uploaded to the server via the device.

[1422] Step 2: Data analysis and transformation

[1423] The server analyzes the received data using a generative AI model and converts it into a digital format. Inputs include the received voice data, URL, and image data. The data is analyzed according to the prompts of the generative AI model (e.g., Google Cloud Speech-to-Text or OCR technology). The output is the analyzed text data or data in a structured digital format. Specifically, the voice data is converted into text data such as "Expected arrival at Tanzawa Mountain Hut at 13:00" and stored in a database.

[1424] Step 3: Analysis by Emotion Engine

[1425] The server uses an emotion engine to analyze the climber's emotional state from voice and text data. The input is voice data and text data converted by a generative AI model. The emotion engine analyzes this to determine the climber's state of stress and anxiety. The output is the evaluation result of the emotional state, which is associated with the climber's profile and saved. Specifically, the emotion engine analyzes the user's voice saying "I'm getting a little tired," and adds the "fatigue" status to the profile.

[1426] Step 4: Save to database and centralize management

[1427] The server stores the analyzed digital data and emotion analysis results in a database and manages them centrally. The input is the digital format data and emotional state data analyzed by the generative AI model and emotion engine. As an output, this data is stored in a centrally managed database. Specifically, "Expected arrival at Tanzawa Mountain Hut at 13:00" and the emotion tag "fatigue" are stored and managed in the database.

[1428] Step 5: Track and match location information

[1429] The server receives GPS location information periodically sent from the climber's mobile device and compares it with the collected climbing plan. The input is the climber's current location information periodically received as GPS data and pre-saved climbing plan data. The output is a determination result of whether the location information matches the climbing plan or not. Specifically, the server receives location information every 15 minutes and compares data such as "12:45 local time, 500m before Mt. Tanzawa" with the climbing plan.

[1430] Step 6: Detect and alert on discrepancies

[1431] If there is a discrepancy between the location information and the climbing plan, the server immediately generates an alert and sends a notification to the climber's mobile device. In addition, if the climber is delayed or does not descend as planned, the server notifies the mountain hut and other climbers in conjunction with the analysis results of the emotion engine. The input is data on discrepancies between the location information and the climbing plan, and emotional state data. The output is an alert and notification message in the event of a discrepancy. In concrete terms, if the climber does not arrive at the mountain hut at the scheduled time of 1:00 PM, the server generates an alert and notifies the climber, "There is a discrepancy in the plan. Please check."

[1432] Step 7: User authentication and mountain climbing plan input

[1433] When the user first starts up the device, they create an account and log in. The user then inputs their mountain climbing plan using voice, the URL of a third-party app, and an image of a piece of paper. This input data is sent to the server. The inputs are the user's name, email address, password, and mountain climbing plan data. The output is the user account authentication information and mountain climbing plan data, which are sent to the server. Specific operations include the user entering information on the account creation screen and inputting the plan using voice.

[1434] Step 8: Send location information

[1435] The device activates its GPS function when starting a climb, and acquires and sends location information to the server at regular intervals. The input is the start climbing button operated by the user and the current location data acquired by the GPS. The output is the acquired location information that is periodically sent to the server. Specifically, the user presses the "start climbing" button, and the device sends location information to the server every 15 minutes.

[1436] Step 9: Enter and send emotional states

[1437] The device records the user's emotional state through voice or text input and sends it to the emotion engine. The input is voice data or text data of the emotional state input by the user. The output is the emotional data to be analyzed and sent to the server. In concrete terms, the user may say "I'm getting a little tired" and the data is sent to the server.

[1438] Step 10: Receiving and Viewing Notifications

[1439] The terminal receives an alert notification from the server and displays a pop-up notification to the user. The input is the alert notification data sent from the server. As an output, the received alert notification is displayed in a pop-up format on the terminal screen. Specifically, a pop-up message saying "There is a discrepancy in the plan. Please check" is displayed on the terminal screen.

[1440] (Application example 2)

[1441] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1442] Current systems that manage climbers' location information and climbing plans have difficulty determining their emotional state in real time and taking appropriate action in emergencies. Furthermore, there is a lack of effective means to monitor the work plans and emotional state of factory workers to improve work efficiency and safety. This increases the risk of accidents and problems during mountain climbing and factory work.

[1443] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting mountain climbing plans from either voice data provided by climbers, URLs of other companies' apps, or paper images, means for using a generative AI model to analyze the collected mountain climbing plans and convert them into a digital format, and means for periodically obtaining climber location information from a mobile device and comparing it with the collected mountain climbing plans. This makes it possible to analyze and manage the plans and location information of climbers and factory workers in real time, monitor their emotional states, and generate appropriate alerts.

[1444] "Audio data" refers to audio information provided by climbers or workers through a microphone or audio input device.

[1445] "URLs from other apps" are link information shared from other applications, and are URLs used to represent mountain climbing plans or work plans.

[1446] "Paper images" are digitalized images of paper documents containing mountain climbing plans and work plans.

[1447] "Mountain climbing plan" means a schedule or route plan that a climber makes before climbing a mountain, and includes digital data, audio data, and image data that are provided.

[1448] A "generative AI model" is an artificial intelligence algorithm used to analyze collected audio and image data and convert it into a standard digital format.

[1449] "Digital format" refers to a format in which analog information has been digitized and is a format that can be processed by a computer.

[1450] "Location information" is data indicating the current location of climbers and workers, obtained using GPS or other location information acquisition means.

[1451] The "emotion engine" is an artificial intelligence technology that analyzes emotional states from voice and text data and identifies emotions such as stress, anxiety, and fatigue.

[1452] An "alert" is a message used to warn or caution climbers and workers.

[1453] A "mountain hut" is a facility in a mountainous area set up for climbers to rest and stay overnight.

[1454] The "database" is a digital storage system for centrally storing and managing collected mountaineering and work plan data.

[1455] A "work plan" is a document that outlines the work processes and procedures that workers in a factory draw up, and includes information provided in the form of audio data, URLs of other companies' apps, paper images, etc.

[1456] This invention is a system that centrally manages the location information and plans of mountain climbers and factory workers, and monitors them in real time by combining an emotion engine. This system improves user safety and efficiency by collecting and analyzing mountain climbing and work plan data provided by users, such as voice data, URLs of other companies' apps, and paper images.

[1457] Data collection and analysis

[1458] The server receives the voice data, URLs of other companies' apps, and paper images provided by the user, and uses the generative AI model to analyze and convert this data into a digital format. For example, if a climber says, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in a database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[1459] Emotion Engine Analysis

[1460] The server is equipped with an emotion engine that analyzes the user's emotional state (e.g., stress or anxiety) from voice data and input data. The analysis results are stored in association with each user's profile. The emotion engine uses the Python library emotion_analysis.

[1461] Centralized data management

[1462] The server stores the analyzed digital climbing and work plans in a database and manages them centrally. The analysis results of the emotion engine are also stored. This allows users' plans, progress, and emotional state to be visualized in one place and checked in real time.

[1463] Location Tracking

[1464] The server receives location information periodically sent from the user's mobile device. This location information is checked against the plan to see if it matches. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[1465] Conflict notification and stagnation management

[1466] If there is a discrepancy between the location information and the plan, the server immediately generates an alert and sends a notification to the user's mobile device. If the user is stagnating or not progressing with the plan as planned, the server will link the analysis results of the emotion engine and send an appropriate notification. If the emotion engine detects high stress or anxiety, a particularly urgent notification will be generated.

[1467] Terminal program processing

[1468] After creating an account and logging in, the user's mobile device has a function that allows them to input plans using voice data, URLs of other companies' apps, and paper images. This data is sent to the server and analyzed. By enabling the GPS function, location information is acquired at regular intervals and sent to the server. In addition, the user's emotional state is sent to the emotion engine via voice or text input. If a discrepancy between the plan and location information is detected, an alert notification is promptly received and displayed as a pop-up.

[1469] Specific examples

[1470] A user may say, "I'm going to climb Mt. Tanzawa tomorrow. I plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and stores this information. If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert and notifies the user's device. If the user also says, "I'm a little scared," the emotion engine analyzes high anxiety and generates a high-urgency notification.

[1471] Prompt Sentence Examples

[1472] Prompt: "Please speak to describe your work schedule and current feelings for sentiment analysis. For example, "I'll take three breaks by the end of today's work" or "I'm feeling a bit tired.""

[1473] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1474] Step 1:

[1475] A user creates an account and logs in. The user enters their name, email address, and password and presses the Create Account button. This information is sent to the server, which stores the new user information in a database. The input data is the user's basic information (name, email address, password), and the output is a message that the new account was successfully created.

[1476] Step 2:

[1477] Users input their mountain climbing and work plans using voice, a URL from another company's app, or a paper image. In the case of voice input, the user speaks to their smartphone something like, "Tomorrow I'll climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM." This data is sent to a server, where a generative AI model converts the speech into text. The input data is voice data, a URL, or image data, and the output is the plan data in text format.

[1478] Step 3:

[1479] The server converts the received plan data into a digital format and stores it in a database. For example, text data such as "Scheduled arrival at Tanzawa Mountain Hut at 13:00" is stored in the database. The input data is the text data converted by the generative AI model, and the output is the plan data stored in the database.

[1480] Step 4:

[1481] When a user starts climbing or working, they press the "Start" button on the device app. The device activates the GPS function, acquires location information at regular intervals (e.g., 15 minutes), and sends it to the server. The input data is the location information from the GPS, and the output is the location information sent to the server.

[1482] Step 5:

[1483] The server periodically checks the received location information against the planned data in the database, for example, to see if the current location is along the planned route. The input data are the current location information and the planned data in the database, and the output is a determination of whether the location information matches the plan.

[1484] Step 6:

[1485] If the location information does not match the plan, the server immediately generates an alert and notifies the user's device. For example, if the location deviates from the planned route, an alert is sent to the user. The input data is the mismatch determination result, and the output is an alert notification.

[1486] Step 7:

[1487] The user inputs their emotional state by voice while climbing a mountain or working. For example, they might say, "I'm feeling a little tired." This data is sent to the server via the device, and the emotion engine analyzes it. The input data is voice data, and the output is the analysis result of the emotional state.

[1488] Step 8:

[1489] The server associates the results of the emotion analysis with the profile and generates an alert if the level of urgency is high. For example, if a "high anxiety state" is detected, an emergency alert is sent to the user's device. The input data is the emotion analysis result, and the output is an emergency alert notification.

[1490] Step 9:

[1491] The server centrally manages the analyzed mountain climbing plan data, work plan data, and emotional state, and stores them in a database. This allows the user's plan, progress, and emotional state to be visualized in real time. The input data are the plan data and the emotion analysis results, and the output is storage in the database and real-time visualization.

[1492] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1493] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1494] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1495] [Fourth embodiment]

[1496] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1497] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1498] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1499] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1500] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1501] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1502] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1503] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1504] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1505] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1506] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1507] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1508] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1509] This invention is a system that centrally manages and monitors climbers' location information and climbing plans in real time. This system collects climbing plan data provided by climbers in various formats (voice, URLs of other companies' apps, paper images), analyzes it using a generative AI model, converts it into a digital format, and centrally manages it. As a result, it ensures the safety of climbers and enables the prevention and rapid response of accidents.

[1510] Server program processing

[1511] Data collection and analysis

[1512] The server receives the voice data provided by the climber, the URL of the third-party app, and the paper image. This data is analyzed using the generative AI model and converted into a standard digital format. For example, if a climber says, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[1513] Centralized data management

[1514] The server stores the analyzed climbing plans in a digital format in a database and manages them centrally, allowing each climber's plan and progress to be visualized in one place and checked in real time.

[1515] Location Tracking

[1516] The server receives location information periodically sent from the climber's mobile device. This location information is checked against the climbing plan to see if it matches the plan. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[1517] Conflict notification and stagnation management

[1518] If there is a discrepancy between the location information and the climbing plan, the server immediately generates an alert and sends a notification to the climber's mobile device. If the climber is delayed or does not descend as planned, notifications are sent to the mountain hut and other climbers so that a prompt response can be taken.

[1519] Terminal program processing

[1520] User authentication and mountain climbing plan input

[1521] The device requires users to create an account and log in when they first start the device. After that, users can input their hiking plans using voice commands, URLs from other companies' apps, or images of paper. This data is sent to a server for analysis.

[1522] Sending location information

[1523] The device activates its GPS function when the climb begins, acquires location information at regular intervals, and sends it to the server. This location information is then checked against the climbing plan to confirm its consistency.

[1524] Receive notifications

[1525] If the device detects a discrepancy between the plan and location information, or if the climber is delayed, it will immediately receive an alert notification and display it to the user in a pop-up.

[1526] User Examples

[1527] Initial setup and climbing plan input

[1528] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by verbally inputting, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the climbing plan is sent to the server and analyzed.

[1529] Start climbing and share your location

[1530] When a user arrives at the trailhead, they press the "Start Climbing" button on the app. The device activates its GPS function and sends its location information to the server every 15 minutes. The location information is compared with the climbing plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[1531] Specific examples

[1532] A user vocally inputs, "Tomorrow I'm going to climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and stores this information. If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending it to the user's device. The user receives the notification and takes appropriate measures to ensure their safety. In this way, the system can prevent accidents and quickly identify stranded individuals.

[1533] The processing flow will be explained below.

[1534] User creates an account

[1535] Step 1:

[1536] A user downloads the mobile app and launches it for the first time.

[1537] Step 2:

[1538] The user enters basic information such as name, email address, and password, and clicks the registration button.

[1539] Step 3:

[1540] The terminal transmits the user's registration information to the server.

[1541] Step 4:

[1542] The server stores the received user information in a database.

[1543] Step 5:

[1544] The server sends a confirmation message to the terminal indicating that registration has been completed, and the terminal displays the confirmation message.

[1545] The user inputs a mountain climbing plan

[1546] Step 1:

[1547] The user logs in and selects "Enter Mountain Climbing Plan" from the main menu.

[1548] Step 2:

[1549] The user selects one of the following methods: "Voice input," "URL of a third-party app," or "Image upload."

[1550] Step 3:

[1551] For example, a user might say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 p.m."

[1552] Step 4:

[1553] The device records the audio data and sends it to the server.

[1554] Step 5:

[1555] The server analyzes the voice data using a generative AI model and converts it into text data.

[1556] Step 6:

[1557] The server stores the parsed text data in a database and associates it with the user's account.

[1558] User starts climbing

[1559] Step 1:

[1560] The user arrives at the trailhead and launches the app.

[1561] Step 2:

[1562] The user presses the "Start climbing" button.

[1563] Step 3:

[1564] The device will activate the GPS function and begin acquiring location information.

[1565] Step 4:

[1566] The device periodically (e.g., every 15 minutes) sends its location information to the server.

[1567] Location monitoring and discrepancy notification during mountain climbing

[1568] Step 1:

[1569] The server compares the received location information with the mountain climbing plan.

[1570] Step 2:

[1571] The server compares the climbing plan (e.g., scheduled arrival at the mountain hut at 13:00) with the current location information.

[1572] Step 3:

[1573] If the server has not reached the hut by the scheduled time (13:00), it detects a discrepancy.

[1574] Step 4:

[1575] If the server detects a mismatch, it generates an alert message and sends it to the terminal.

[1576] Step 5:

[1577] The device displays a warning to the user saying, "You have not arrived at the mountain hut by the scheduled time. Please check that your surroundings are safe."

[1578] Notification of stagnation and those who have not yet descended

[1579] Step 1:

[1580] The server periodically compares the location information of all users with their mountain climbing plans.

[1581] Step 2:

[1582] If the server detects stagnation or failure to descend, it generates an alert list.

[1583] Step 3:

[1584] The server sends alerts of stagnation or failure to descend the mountain to the relevant user's device.

[1585] Step 4:

[1586] The device will warn the user that "You have been stuck at your current location for an extended period of time. Please make sure you are safe."

[1587] Step 5:

[1588] The server also sends relevant notifications to mountain huts and other climbers, encouraging them to take prompt action.

[1589] The server centrally manages data

[1590] Step 1:

[1591] The server stores all climbers' climbing plans, location information, and conflict data in a database.

[1592] Step 2:

[1593] The server updates and displays the location information and mountain climbing plan received in real time.

[1594] Step 3:

[1595] If the server needs search and rescue, it will provide relevant information to the police and local authorities.

[1596] Specific examples

[1597] Initial setup and climbing plan input

[1598] Step 1:

[1599] A user launches the app for the first time and creates an account by entering their name, email address, password, etc.

[1600] Step 2:

[1601] The user says in a voice message, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 p.m."

[1602] Step 3:

[1603] The server analyzes the audio and converts it into text data.

[1604] Step 4:

[1605] The server stores the parsed data in a database and associates it with the user's account.

[1606] Start climbing and share your location

[1607] Step 1:

[1608] The user arrives at the trailhead and presses the "Start Climbing" button on the app.

[1609] Step 2:

[1610] The device will enable the GPS function and send location information to the server every 15 minutes.

[1611] Step 3:

[1612] The server compares the location information with the climbing plan to check for any discrepancies.

[1613] Step 4:

[1614] If things aren't going according to plan, the user is immediately alerted.

[1615] Inconsistency warnings and actions

[1616] Step 1:

[1617] The server detects that the user has not yet arrived at the mountain hut even though it is now 13:00.

[1618] Step 2:

[1619] The server detects the discrepancy and generates an alert that is sent to the user terminal.

[1620] Step 3:

[1621] The terminal displays a warning to the user that "You have not arrived at the mountain hut even though the scheduled time has passed."

[1622] Step 4:

[1623] The user checks the notification to check safety and understand the situation.

[1624] Example 1

[1625] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1626] When climbing mountains, ensuring the safety of climbers and responding quickly are essential, but conventional methods have made managing climbing plans and current locations cumbersome. Furthermore, there are insufficient means for responding quickly when climbers are not following plan or when there is a possibility of getting lost. Therefore, a system is needed that efficiently and centrally manages climbers' location information and climbing plans, and monitors them in real time.

[1627] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1628] In this invention, the server includes a means for collecting mountain climbing plans from either voice data provided by climbers, links to other companies' applications, or paper images, a means for using a generative artificial intelligence model to analyze the collected mountain climbing plans and convert them into a digital format, and a means for storing the analyzed mountain climbing plan data in a database for centralized management. This allows for efficient management of climbers' plans and current location information, and makes it possible to quickly detect and respond to any abnormalities.

[1629] A "mountaineer" is a person who goes mountain climbing and refers to a user of this system.

[1630] "Audio data" refers to information input by voice by a climber that has been recorded as digital data.

[1631] "Links to third-party applications" refers to URLs that connect to pages or information within applications provided by other companies.

[1632] "Paper images" refer to image data that has been digitized from paper media containing mountain climbing plans.

[1633] A "mountain climbing plan" is a detailed description of information such as the climber's planned time to reach the summit, route, and destination.

[1634] A "generative artificial intelligence model" refers to a machine learning model that analyzes natural language and converts it into digital data.

[1635] A "digital format" is a unified format for expressing mountain climbing plans as digital data.

[1636] A "database" refers to a system or device for centrally managing and storing analyzed mountain climbing plan data and location information.

[1637] "Mobile devices" refer to electronic devices that climbers can carry with them, such as smartphones or devices with GPS functionality.

[1638] "Location information" refers to data indicating a climber's current location, typically obtained by GPS or other positioning systems.

[1639] "Detecting anomalies" refers to checking for discrepancies when they arise between the mountain climbing plan and the current location information.

[1640] A "notification" is a message sent to the climber or a designated third party regarding detected abnormalities or important information.

[1641] "Designated locations" refers to locations and facilities that have been registered in advance as part of mountain climbing plans or as emergency contact points.

[1642] "Other climbers" refers to other climbers using this system, who may be subject to notification in the event of an emergency.

[1643] This invention is a management system for mountain climbing plans and location information, with the aim of ensuring the safety of climbers and providing prompt response. This system is composed of a server, terminals, and climbers.

[1644] The server has a means for receiving voice data provided by the climber, links to third-party applications, and paper image data. Specifically, it uses a generative AI model (e.g., GPT-4) to analyze this data and convert it into a standard digital format. If the climber verbally inputs, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the server uses the generative AI model to convert the voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut." This analysis uses the prompt, "Please analyze this climber's voice data and convert his / her climbing plan into a digital format."

[1645] The server is equipped with a system for storing analyzed digital climbing plans in a database and managing them centrally. This database aggregates each climber's plans and progress, allowing them to be viewed in real time. The server also has the function of periodically receiving location information from the climber's mobile device. This location information is obtained using a GPS module.

[1646] The server compares the received location information with the climbing plan data to see if it matches the plan. If a discrepancy is detected, the server detects the anomaly and generates an alert to notify the climber's mobile device. If a climber is delayed or does not descend as planned, notifications are sent to the mountain hut and other climbers, allowing for a prompt response.

[1647] The device, on the other hand, has functions for user authentication and input of mountain climbing plans. When the user starts the device for the first time, they create an account by entering information such as their name, email address, and password, and then log in. After that, the user can enter their mountain climbing plan using voice, a link to a third-party application, or an image of paper, and send it to the server.

[1648] The device also activates its GPS function when the climb begins and acquires and transmits location information to the server at regular intervals (for example, every 15 minutes). This location information is compared with the climbing plan to confirm consistency. Furthermore, the device has the ability to instantly receive an alert notification and display it to the user in a pop-up if a discrepancy between the plan and location information is detected, or if the climber is stalled.

[1649] In this way, the present invention efficiently manages climbers' plans and current locations, and quickly detects and responds to abnormalities, ensuring the safety of climbers. It also enables prompt action through a notification function to mountain huts and other climbers.

[1650] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1651] Server program processing

[1652] Step 1: Data collection

[1653] The server receives audio data provided by the climber, links to other companies' applications, and paper image data. This input data is sent to the server through various data collection methods. For example, if a climber records audio data, the recording is uploaded to the server. Specifically, the server receives an HTTP request and saves the data in a local directory.

[1654] Step 2: Data analysis

[1655] The server inputs the received data into a generative AI model (e.g., GPT-4) and parses it into a digital format. The input data is an audio file, and the output is analyzed text data. The prompt "Please analyze this climber's audio data and convert the climbing plan into a digital format" is used and input into the generative AI model. Specifically, the server sends the audio data to a text conversion API and receives the conversion result.

[1656] Step 3: Store in the database

[1657] The server stores the analyzed digitally formatted mountain climbing plan in a database. The input data is the text data of the analysis results, and the output data is registered in the database. Specifically, the server issues an SQL query and performs an insert process into the database.

[1658] Step 4: Receiving and analyzing location information

[1659] The server receives location information periodically sent from the climber's mobile device. The input data is GPS information, and the output is the current location, which is registered in the database. Specifically, the server receives an HTTP request and saves the location information in the database.

[1660] Step 5: Detect and notify discrepancies

[1661] The server compares the received location information with the climbing plan data and detects any discrepancies. The input data is the current location and climbing plan data, and the output generates an alert notification if an abnormality is detected. Specifically, the server uses an SQL query to retrieve the necessary data from the database, compares it with the plan, and calls a notification API if an abnormality is found.

[1662] Step 6: Alert response actions

[1663] The server notifies the relevant parties of the generated alert. The input data is the alert information, and the output is a notification. Specifically, the server notifies the mountain hut and other climbers using the notification API.

[1664] Terminal program processing

[1665] Step 1: Initial setup and user authentication

[1666] When a user starts up a device for the first time, they create an account by entering information such as their name, email address, and password. The input data is user information, and the output is an account registered on the server. Specifically, the device sends the entered data to the server as an HTTP POST request.

[1667] Step 2: Enter and submit your climbing plan

[1668] Users input their mountain climbing plans using voice, links to third-party applications, and paper images. The input data includes voice data and image data, which are sent to the server as output. Specifically, after collecting the data, the device sends it to the server as an HTTP POST request.

[1669] Step 3: Collecting and sending location information

[1670] When the user presses the "Start Climbing" button on the app to begin climbing, the device activates its GPS function. GPS information is the input data, and location information is sent to the server every 15 minutes as output. Specifically, the device starts a timer, periodically acquires GPS data, and sends it to the server as an HTTP POST request.

[1671] Step 4: Receive and view alert notifications

[1672] If a mismatch is detected between the user's location information and the mountain climbing plan, the device immediately receives an alert notification. The input data is the alert information, and the output is a pop-up notification that is displayed to the user. Specifically, the device receives a push notification from the server and displays the notification content on the screen.

[1673] (Application example 1)

[1674] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1675] In modern society, serious risks and problems often arise when plans and actual actions do not match in situations such as mountain climbing and food delivery. For climbers, the risk of getting lost increases if they do not proceed along the mountain path as planned, requiring a quick response. Meanwhile, in food delivery, if delivery employees are unable to deliver as scheduled, customer satisfaction decreases and work efficiency deteriorates. To prevent such situations from occurring, it is necessary to confirm the match between plans and location information in real time. However, current technology does not provide sufficient means to do this efficiently.

[1676] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1677] In this invention, the server includes: means for collecting mountain climbing plans from either voice data, URLs of other companies' apps, or paper images provided by climbers; means for using a generative AI model to analyze the collected mountain climbing plans and convert them into a digital format; means for centrally managing the analyzed mountain climbing plan data; means for periodically obtaining climbers' location information from a mobile device and comparing it with the collected mountain climbing plans; means for generating an alert and notifying the climber if the mountain climbing plan and current location information do not match; means for notifying mountain huts and other climbers if the climber is delayed or does not descend the mountain as planned; means for collecting delivery plans from either voice memos, URLs of other companies' apps, or images of handwritten notes provided by delivery employees; means for using a generative AI model to analyze the collected delivery plans and convert them into a digital format; means for centrally managing the analyzed delivery plan data; means for periodically obtaining delivery employee location information from a mobile device and comparing it with the collected delivery plans; means for generating an alert and notifying the delivery employee if the delivery plan and current location information do not match; and means for notifying other relevant parties if the delivery employee is delayed or does not deliver as planned. This makes it possible to efficiently check in real time whether the plans and actual actions of climbers and delivery employees match, preventing risks and problems before they occur.

[1678] A "mountaineer" is an individual who engages in the activity of climbing mountains in a natural environment.

[1679] A "delivery employee" is an individual who performs food delivery or other delivery duties.

[1680] "Voice data" refers to data that is a digital recording of a human voice.

[1681] "Third-party app URL" means a web link related to application software provided by another company.

[1682] A "paper image" is information written on paper that has been converted into image data using a digital camera or scanner.

[1683] A "mountain climbing plan" is a detailed schedule of a mountain climb that a climber prepares in advance.

[1684] A "delivery plan" is a detailed delivery schedule prepared in advance by delivery employees.

[1685] A "generative AI model" is an artificial intelligence model that learns from large amounts of data and performs specific tasks.

[1686] "Digital format" refers to the form of data that is represented or recorded using digital technology.

[1687] A "mobile terminal" is a portable information device such as a smartphone or tablet.

[1688] "Location information" is data that indicates a geographic location. It is obtained using technologies such as GPS.

[1689] An "alert" is a warning message that notifies the user of an abnormal situation or important notification.

[1690] A mountain hut is a building set up for climbers to stay overnight or take a break.

[1691] "Related parties" refers to any person or organization that plays a significant role in delivery operations.

[1692] This invention is a system that centrally manages and monitors the location information and plans of climbers and delivery employees in real time. This system collects data in multiple formats, such as voice memos, URLs of other companies' apps, and paper images, provided by climbers and delivery employees, and analyzes it using a generative AI model. This data is then converted into a digital format for centralized management, ensuring safety and improving work efficiency.

[1693] Server program processing

[1694] Data collection and analysis

[1695] The server receives voice data, URLs of other companies' apps, and paper images provided by climbers and delivery employees. The collected data is analyzed using a generative AI model and converted into a standard digital format. For example, if a climber sends a voice memo saying, "Tomorrow I'll be climbing Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database as "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[1696] Centralized data management

[1697] The server stores the analyzed climbing plan data and delivery plan data in a database and manages them centrally, allowing the plans and progress of each climber and delivery employee to be visualized in one place and checked in real time.

[1698] Location Tracking

[1699] The server receives location information periodically sent from the mobile devices of climbers and delivery employees. This location information is checked against the climbing plan and delivery plan to confirm consistency. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[1700] Conflict notification and stagnation management

[1701] If there is a discrepancy between the location information and the climbing plan or delivery plan, the server immediately generates an alert and sends a notification to the mobile devices of the climber or delivery employee. Also, if a climber is delayed or does not descend as planned, or if a delivery employee is late, the information is notified to other parties so that they can take prompt action.

[1702] Terminal program processing

[1703] User authentication and plan input

[1704] When the device is first started, the user is prompted to create an account and log in. After that, the user inputs mountain climbing and delivery plans using voice memos, URLs of other companies' apps, and images of paper. This data is sent to the server and analyzed.

[1705] Sending location information

[1706] The device activates its GPS function when starting a climb or delivery, acquires location information at regular intervals, and sends it to the server. This location information is then compared with the plan to confirm consistency.

[1707] Receive notifications

[1708] The device receives instant alert notifications and displays them to the user in a pop-up if it detects a discrepancy between the plan and location information, if a climber is delayed, or if a delivery employee is late.

[1709] User Examples

[1710] Initial setup and planning input

[1711] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by speaking, they can say, "Tomorrow I'm going to climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," or "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan is sent to the server and analyzed.

[1712] Initiating planning and location sharing

[1713] When a user arrives at the trailhead or delivery start point, they press the "Start Climbing" or "Start Delivery" button on the app. The device activates its GPS function and sends location information to the server every 15 minutes. The location information is compared with the plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[1714] Examples of concrete examples and prompts

[1715] As a specific example, a user may say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture, and plan to arrive at the mountain hut at 1:00 PM," and the server will analyze and save this. If the user does not arrive at the mountain hut as planned during the climb, the server will detect the discrepancy and immediately generate an alert, notifying the user's device. As an example of a delivery, a delivery employee may say by voice, "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan will be analyzed and saved. If an unplanned route is taken, a discrepancy will be detected and an immediate notification will be sent.

[1716] Prompt Sentence Examples

[1717] text

[1718] When a food delivery worker says "I have 10 deliveries today" to the app, the voice data is sent to the server and converted to "I have 10 deliveries today" using the Google Cloud Speech-to-Text API.

[1719] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1720] Program processing steps

[1721] Step 1:

[1722] The server receives the voice data provided by the climbers and delivery employees, the URL of the third-party app, and the paper image. This input data depends on the format each user sends from the app. For example, a user records a voice memo in the app, which is then sent to the server. The output of this step is the data in the format received by the server.

[1723] Step 2:

[1724] The server inputs the received data into the generative AI model. For audio data, it uses the Google Cloud Speech-to-Text API to convert the audio to text. For URLs from other apps, it sends a single web request and analyzes the response. For paper images, it uses the Google Cloud Vision API to convert handwritten characters to text. The output of this step is the converted text data.

[1725] Step 3:

[1726] The server then compiles the converted text data into a digital format. For example, a voice memo saying "Tomorrow we will climb Mt. Tanzawa. We plan to arrive at the mountain hut at 1:00 PM" is converted into the format "Mt. Tanzawa, 1:00 PM, we plan to arrive at the mountain hut." The output of this step is the climbing plan data and delivery plan data in a unified digital format.

[1727] Step 4:

[1728] The server stores the data in a unified digital format in a database and manages it centrally. This allows the plans and progress of each climber and delivery employee to be visualized in one place. The output of this step is the plan data in a digital format stored in a database.

[1729] Step 5:

[1730] The device activates the GPS function at the start of the climb or delivery and acquires location information at regular intervals. This location information is periodically sent to the server. The output of this step is the location information data that is periodically sent.

[1731] Step 6:

[1732] The server receives the periodically transmitted location information and checks it against the previously stored digitally formatted plan data, for example, to see if the location information is updated according to the plan. The output of this step is the check result.

[1733] Step 7:

[1734] The server generates an alert immediately if there is a discrepancy between the location information and the planned data. The alert is sent to the climber's or delivery employee's device. For example, if the location is not updated after the planned arrival time, an alert is issued. The output of this step is an alert notification sent to the user's device.

[1735] Step 8:

[1736] The terminal displays the alert notification received from the server to the user as a popup, allowing the user to immediately check the abnormality and take necessary action. The output of this step is an alert notification that is displayed in a visible form to the user.

[1737] Examples of concrete examples and prompts

[1738] As a specific example, a user might say, "Tomorrow I'm going to climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and saves this. The voice data is converted to text using the Google Cloud Speech-to-Text API and saved as "Mt. Tanzawa, 1:00 PM, expected arrival at the mountain hut." If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending a notification to the user's device. As an example of a delivery, a delivery employee might say, "My next delivery destination is Nishi-Shinjuku, Shinjuku Ward, and I plan to arrive in 10 minutes," and the plan is analyzed and saved. If an unplanned route is taken, the discrepancy is detected and an immediate notification is sent.

[1739] Prompt Sentence Examples

[1740] text

[1741] When a food delivery worker says "I have 10 deliveries today" to the app, the voice data is sent to the server and converted to "I have 10 deliveries today" using the Google Cloud Speech-to-Text API.

[1742] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1743] This invention combines an emotion engine with a system that centrally manages and monitors climbers' location information and climbing plans in real time. This system collects, analyzes, and centrally manages climbing plan data in various formats provided by climbers (audio, URLs of other companies' apps, paper images). It also uses the emotion engine to analyze the climber's emotional state in real time and generates alerts and messages to reduce stress and anxiety as needed. This further ensures climber safety and enables the prevention and rapid response of mountaineering accidents.

[1744] Server program processing

[1745] Data collection and analysis

[1746] The server receives the voice data provided by the climber, the URL of the third-party app, and the paper image. This data is analyzed using the generative AI model and converted into a standard digital format. For example, if a climber says, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the generative AI model converts this voice data into text data and stores it in the database in the format "Mt. Tanzawa, 1:00 PM, I plan to arrive at the mountain hut."

[1747] Emotion Engine Analysis

[1748] The server is equipped with an emotion engine that analyzes the climber's emotional state (e.g., stress or anxiety) from the voice data and input data. The analysis results are stored in association with each climber's profile.

[1749] Centralized data management

[1750] The server stores the analyzed digital climbing plans in a database and manages them centrally. The analysis results of the emotion engine are also stored. This allows each climber's plan, progress, and emotional state to be visualized in one place and checked in real time.

[1751] Location Tracking

[1752] The server receives location information periodically sent from the climber's mobile device. This location information is checked against the climbing plan to see if it matches the plan. If a discrepancy is detected, an alert is generated and a prompt response is taken.

[1753] Conflict notification and stagnation management

[1754] If there is a discrepancy between the location information and the climbing plan, the server immediately generates an alert and sends a notification to the climber's mobile device. If the climber is delayed or does not descend as planned, the server will link the analysis results of the emotion engine and send a notification to the mountain hut and other climbers. If the emotion engine detects high stress or anxiety, a particularly urgent notification will be generated.

[1755] Terminal program processing

[1756] User authentication and mountain climbing plan input

[1757] The device requires users to create an account and log in when they first start the device. After that, users can input their hiking plans using voice commands, URLs from other companies' apps, or images of paper. This data is sent to a server for analysis.

[1758] Sending location information

[1759] The device activates its GPS function when the climb begins, acquires location information at regular intervals, and sends it to the server. This location information is then checked against the climbing plan to confirm its consistency.

[1760] Emotion Engine Input

[1761] The device sends the user's emotional state to the emotion engine through voice or text input. For example, if the user says, "I'm feeling a little tired," the information is analyzed and sent to the emotion engine.

[1762] Receive notifications

[1763] If the device detects a discrepancy between the plan and location information, or if the emotion engine analyzes that the user is in a state of high stress or anxiety, the device will quickly receive an alert notification and display it to the user in a pop-up.

[1764] User Examples

[1765] Initial setup and climbing plan input

[1766] When a user starts the app for the first time, they create an account by entering information such as their name, email address, and password. Then, by verbally inputting, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 PM," the climbing plan is sent to the server and analyzed.

[1767] Start climbing and share your location

[1768] When a user arrives at the trailhead, they press the "Start Climbing" button on the app. The device activates its GPS function and sends its location information to the server every 15 minutes. The location information is compared with the climbing plan, and if any discrepancies or delays are detected, an alert is sent immediately.

[1769] Emotional state analysis and response

[1770] While climbing, the user inputs their emotional state, such as "I'm anxious," through voice. The device sends this voice to the emotion engine, which interprets it as a high anxiety state. The server then quickly generates an alert and sends a notification to the user's device, as well as to mountain huts and other climbers.

[1771] Specific examples

[1772] The user vocally inputs, "I'm going to climb Mt. Tanzawa tomorrow. I plan to arrive at the mountain hut at 1:00 PM," and the server analyzes and stores this information. If the user does not arrive at the mountain hut as planned during the climb, the server detects the discrepancy and immediately generates an alert, sending a notification to the user's device. If the user also says, "I'm a little scared," the emotion engine analyzes the high level of anxiety and generates a high-urgency notification. The user receives this notification and checks for safety. The system also takes into account the user's emotional state, ensuring the user's safety and responding quickly.

[1773] The processing flow will be explained below.

[1774] User creates an account

[1775] Step 1:

[1776] A user downloads the mobile app and launches it for the first time.

[1777] Step 2:

[1778] The user enters basic information such as name, email address, and password, and clicks the registration button.

[1779] Step 3:

[1780] The terminal transmits the user's registration information to the server.

[1781] Step 4:

[1782] The server stores the received user information in a database.

[1783] Step 5:

[1784] The server sends a confirmation message to the terminal indicating that registration has been completed, and the terminal displays the confirmation message.

[1785] The user inputs a mountain climbing plan

[1786] Step 1:

[1787] The user logs in and selects "Enter Mountain Climbing Plan" from the main menu.

[1788] Step 2:

[1789] The user selects one of the following methods: "Voice input," "URL of a third-party app," or "Image upload."

[1790] Step 3:

[1791] For example, a user might say by voice, "Tomorrow I will climb Mt. Tanzawa in Yamanashi Prefecture and plan to arrive at the mountain hut at 1:00 p.m."

[1792] Step 4:

[1793] The device records the audio data and sends it to the server.

[1794] Step 5:

[1795] The server analyzes the voice data using a generative AI model and converts it into text data.

[1796] Step 6:

[1797] The server stores the parsed text data in a database and associates it with the user's account.

[1798] User inputs emotional state

[1799] Step 1:

[1800] The user selects "Voice Input" from the emotion input screen.

[1801] Step 2:

[1802] A user says, "I'm feeling a little anxious today."

[1803] Step 3:

[1804] The device transmits voice data related to emotions to the server.

[1805] Step 4:

[1806] The server analyzes the voice data using an emotion engine to identify the emotional state.

[1807] Step 5:

[1808] The server stores the analyzed emotional state in a database and associates it with the user's account.

[1809] User starts climbing

[1810] Step 1:

[1811] The user arrives at the trailhead and launches the app.

[1812] Step 2:

[1813] The user presses the "Start climbing" button.

[1814] Step 3:

[1815] The device will activate the GPS function and begin acquiring location information.

[1816] Step 4:

[1817] The device periodically (e.g., every 15 minutes) sends its location information to the server.

[1818] Location monitoring and discrepancy notification during mountain climbing

[1819] Step 1:

[1820] The server compares the received location information with the mountain climbing plan.

[1821] Step 2:

[1822] The server compares the climbing plan (e.g., scheduled arrival at the mountain hut at 13:00) with the current location information.

[1823] Step 3:

[1824] If the server has not reached the hut by the scheduled time (13:00), it detects a discrepancy.

[1825] Step 4:

[1826] If the server detects a mismatch, it generates an alert message and sends it to the terminal.

[1827] Step 5:

[1828] The device displays a warning to the user saying, "You haven't arrived at the mountain hut by the scheduled time. Please check that your surroundings are safe."

[1829] Notification of stagnation and those who have not yet descended

[1830] Step 1:

[1831] The server periodically compares the location information of all users with their mountain climbing plans.

[1832] Step 2:

[1833] If the server detects stagnation or failure to descend, it generates an alert list.

[1834] Step 3:

[1835] The server sends alerts of stagnation or failure to descend the mountain to the relevant user's device.

[1836] Step 4:

[1837] The device displays a warning to the user saying, "You have been stuck at your current location for an extended period of time. Please make sure you are safe."

[1838] Step 5:

[1839] The server also sends relevant notifications to mountain huts and other climbers, encouraging them to take prompt action.

[1840] Countermeasures and notifications based on emotional state

[1841] Step 1:

[1842] The server periodically checks the emotional state analyzed by the emotion engine.

[1843] Step 2:

[1844] The server generates special alerts if high stress or anxiety states are detected.

[1845] Step 3:

[1846] The server sends a message to the user's device saying, "Your current emotional state is unstable. Please check the safety of your surroundings and seek support if necessary."

[1847] Step 4:

[1848] The server sends an emergency notification of the user to the mountain hut and other nearby climbers, asking them to check for safety.

[1849] Centralize your data

[1850] Step 1:

[1851] The server stores all climbers' climbing plans, location information, emotional state, and discrepancy data in a database.

[1852] Step 2:

[1853] The server updates and displays the location information and mountain climbing plan received in real time.

[1854] Step 3:

[1855] If the server needs search and rescue, it will provide relevant information to the police and local authorities.

[1856] Specific examples

[1857] Initial setup and climbing plan input

[1858] Step 1:

[1859] A user launches the app for the first time and creates an account by entering their name, email address, password, etc.

[1860] Step 2:

[1861] The user says in a voice message, "Tomorrow I will climb Mt. Tanzawa. I plan to arrive at the mountain hut at 1:00 p.m."

[1862] Step 3:

[1863] The terminal transmits the voice data to the server, which analyzes it and converts it into text data.

[1864] Step 4:

[1865] The server stores the parsed data in a database and associates it with the user's account.

[1866] Start climbing and share your location

[1867] Step 1:

[1868] The user arrives at the trailhead and presses the "Start Climbing" button on the app.

[1869] Step 2:

[1870] The device will enable the GPS function and send location information to the server every 15 minutes.

[1871] Step 3:

[1872] The server compares the location information with the climbing plan to check for any discrepancies.

[1873] Step 4:

[1874] If the server detects a mismatch, it immediately sends an alert to the user terminal.

[1875] Emotional state analysis and response

[1876] Step 1:

[1877] During the climb, the user says, "I'm getting a little tired."

[1878] Step 2:

[1879] The device sends the voice data to the server, which then analyzes it using an emotion engine.

[1880] Step 3:

[1881] The server detects high levels of fatigue and anxiety and generates special alerts.

[1882] Step 4:

[1883] The server sends the user a message saying, "We've checked your status. Stay safe and rest."

[1884] Step 5:

[1885] The server also notifies nearby mountain huts and other climbers, urging users to check for safety.

[1886] Example 2

[1887] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1888] While mountaineering has become increasingly popular in recent years, the number of mountaineering accidents and emergencies has also increased. To solve this problem, a system is needed that can track climbers' location information in real time, detect discrepancies between their climbing plans and their current location, and respond quickly. It is also necessary to better ensure the safety of climbers by analyzing their emotional states, such as stress and anxiety, during climbing in real time and taking appropriate measures.

[1889] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for collecting a mountain climbing plan from audio data provided by the climber, a URL of a third-party app, or a paper image; means for using a generative AI model to analyze the collected mountain climbing plan and convert it into a digital format; means for using an emotion engine to analyze the climber's emotional state; means for periodically acquiring the climber's location information from the mobile device and comparing it with the collected mountain climbing plan; means for generating an alert and notifying the climber if the mountain climbing plan and current location information do not match; and means for notifying mountain huts and other climbers based on the analysis results of the emotion engine if the climber is delayed or has not descended the mountain as planned. This allows the location information and emotional state of the climber to be tracked in real time, enabling rapid response in the event of a mountaineering accident or emergency.

[1890] "Mountain climbing plan" refers to the mountain climbing route, estimated time of arrival, and other related information...

Claims

1. A means of collecting climbing plans from either audio data provided by climbers or URLs of other companies' apps or paper images; using a generative AI model to analyze the collected climbing plans and convert them into a digital format; A means for centrally managing the analyzed mountain climbing plan data; A means for periodically acquiring location information of climbers from the mobile device and comparing it with the collected climbing plan; A means for generating an alert and notifying the climber when there is a discrepancy between the climbing plan and the current location information; A way to notify mountain huts and other climbers if a climber is delayed or not descending as planned, and A system including:

2. 10. The system of claim 1, wherein said system tracks the location information of climbers in real time.

3. 2. The system according to claim 1, further comprising means for storing the analyzed mountain climbing plan data in association with a database.

Images