system

Abstract

We provide the system. [Solution] Means of collecting snow depth information, Means for generating a snow removal plan based on the aforementioned snow accumulation information, Means for remotely operating snow removal equipment in accordance with the aforementioned snow removal plan, A means for monitoring the progress of the snow removal device and confirming the completion of the snow removal work, A means for notifying the user of the completion of the snow removal work, A means for notifying the progress of the snow removal plan, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to reduce the burden associated with snow removal work in front of the store or in the parking lot that store operators face before opening the store in snowy areas. In particular, in small-scale privately-owned stores, snow removal work is heavy labor, and if appropriate measures cannot be taken, it directly affects business activities. Furthermore, the risk of accidents due to working without ensuring safety is also an issue.

Means for Solving the Problems

[0005] This invention provides a system that collects snow accumulation information and generates a snow removal plan based on it. This system has the function of remotely operating snow removal equipment according to the generated snow removal plan. Furthermore, by monitoring the progress of the snow removal equipment, confirming the completion of snow removal work, and notifying the user of the results, it realizes an environment in which store managers can prepare based on appropriate information before opening.

[0006] "Snowfall information" refers to information about snow conditions in a specific region, including snowfall amount, snow cover status, and predicted snowfall amount.

[0007] A "snow removal plan" is a plan for efficiently removing snow based on collected snow accumulation information, and includes the start time of snow removal, the snow removal equipment to be used, and the snow removal route.

[0008] A "snow removal device" is a machine or device used to remove accumulated snow, which operates manually or automatically and has the function of removing snow from a designated area.

[0009] "Remote control" refers to a method of controlling a device located at a physically distant location via communication means.

[0010] "User" refers to a store owner or individual who uses this system to receive snow removal services.

[0011] "Monitoring" refers to the act of continuously observing the operating status of a device or system and checking for abnormalities or progress.

[0012] "Notification" refers to the act of transmitting specific information to a recipient, and in this invention, it refers to informing the user of the progress or completion of snow removal work. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is a system that efficiently and safely handles snow removal work, which is a burden for store managers, and is implemented primarily through interaction between a server, terminals, and users.

[0035] The server collects snow accumulation information in real time from weather data providers. This allows the server to understand the exact snow conditions in the area where the store is located. Store owners, as users, use smartphones or computers to identify the store's location and the areas requiring snow removal, and then send snow removal requests.

[0036] The received request is processed by the server. The server uses an AI algorithm based on the collected snow depth information to generate an efficient snow removal plan. This plan includes the placement of snow removal equipment, the timing of operation, and the route to be taken. The server sends the generated snow removal plan to the terminal, which then remotely operates the snow removal equipment based on it.

[0037] The terminal activates the snow removal equipment based on instructions from the server and automatically performs snow removal along a planned route. During snow removal, the terminal constantly sends progress information to the server, which monitors it.

[0038] Once snow removal is complete, the server verifies the information and sends a completion notification to the user. The user receives this notification on their smartphone or computer and confirms that the snow removal has been successfully completed. This entire process ensures that the parking lot and the area in front of the store are safely cleared of snow by opening time, allowing the business owner to focus on their work.

[0039] As a concrete example, consider a case where the owner of a hair salon requests snow removal in preparation for opening the next morning. During the night when snowfall is expected, the owner sends a request from their smartphone at home. The server plans for the snow removal equipment to be activated at the appropriate time the next morning and operates the snow removal equipment remotely to clear the parking lot. In this way, the owner can concentrate on their normal business without having to spend their busy morning hours on snow removal.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The server periodically collects snow information, such as predicted snowfall and current snow depth, using the weather data provider's API. This ensures that the system is always up-to-date on the latest weather conditions.

[0043] Step 2:

[0044] Users enter information such as the store's location and desired snow removal times via a smartphone or PC application and submit a snow removal request. This request reaches the server and is stored in the database.

[0045] Step 3:

[0046] The server determines the need for snow removal based on user requests and the latest snow depth information. If deemed necessary, it uses an AI algorithm to generate an optimal snow removal plan. This plan includes snow removal machine usage time, operating routes, and expected work time.

[0047] Step 4:

[0048] The server distributes the generated snow removal plan to the terminal. The terminal then prepares to activate the snow removal equipment at the specified time based on this plan.

[0049] Step 5:

[0050] The terminal remotely controls the snow removal equipment based on instructions received from the server and begins snow removal work along the designated route. The operating status and progress information of the snow removal equipment are transmitted to the server in real time.

[0051] Step 6:

[0052] The server constantly monitors whether snow removal is progressing normally based on progress information from the terminals. When the work is completed, it records the resources used and the work time, and updates the database.

[0053] Step 7:

[0054] After confirming the completion of snow removal, the server sends a completion notification to the user. Upon receiving this notification, the user can confirm that snow removal has been successfully completed and proceed with opening preparations.

[0055] (Example 1)

[0056] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0057] Currently, store owners in snowy regions face a significant burden of clearing snow from parking lots and storefronts before opening. This task is time-consuming and labor-intensive, causing considerable stress, especially during the busy pre-opening hours. Furthermore, improper snow removal can compromise safety. To address these issues, there is a need for the development of an automated system that performs efficient and safe snow removal.

[0058] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0059] In this invention, the server includes means for collecting snow accumulation information, means for receiving location information and area information where snow removal is required, provided by users, and means for formulating an efficient snow removal process using an artificial intelligence algorithm. This reduces the burden on store managers and enables safe and efficient snow removal work within a given timeframe.

[0060] "Snowfall information" refers to data on snowfall amount and snow depth provided by weather data providers, and is fundamental information for snow removal planning.

[0061] A "snow removal plan" refers to a plan that includes details such as the timing of operation and routes for snow removal equipment necessary for efficient snow removal work.

[0062] A "snow removal device" refers to a machine or mechanism that physically removes accumulated snow and is controlled by remote operation.

[0063] "Remote control" refers to the act of controlling a machine or system using electronic means from a physically distant location.

[0064] "Progress status" refers to data representing the current execution status and progress of snow removal work, which is continuously monitored by the server.

[0065] "Completion confirmed" refers to the process in which the server confirms that all scheduled snow removal operations have been carried out as planned and that the designated area has been properly cleared of snow.

[0066] "Notification" refers to the act of informing users about the completion or progress of snow removal work, and is mainly done via email or push notifications on mobile apps.

[0067] "Location information" refers to digital data that indicates a specific point on Earth, and is often expressed using a geographic coordinate system.

[0068] "Area information" refers to data about specific areas where snow removal is required, and serves as a basis for formulating snow removal plans.

[0069] "Machine learning technology" refers to techniques that enable computers to autonomously recognize patterns using large amounts of data and generate predictive models.

[0070] An "artificial intelligence algorithm" is a type of computational procedure executed by a computer, designed to solve specific problems while mimicking human knowledge and reasoning.

[0071] This invention is an automated system for reducing the burden on store managers and efficiently performing snow removal. The system is implemented through the interaction of a server, terminals, and users.

[0072] The server plays a central role in collecting snow depth information and generating effective snow removal plans. Specifically, the server retrieves snow depth information in real time from weather data providers. For example, it periodically collects snowfall and weather forecast data from providers using APIs. This data is essential for the server to efficiently create snow removal plans. The server also maintains a database to manage user request information and past snow removal history.

[0073] Users submit snow removal requests using devices such as smartphones or computers. A dedicated application allows users to easily specify the store's location and the area to be cleared. Users can also request snow removal at the optimal time by entering their desired time slot. After the snow removal is complete, users can check the status through the application.

[0074] The terminal remotely controls the snow removal equipment based on the snow removal plan transmitted from the server. The terminal uses wireless communication to send instructions directly to the snow removal equipment on site. Based on the instructions received by the terminal, the snow removal equipment automatically follows the planned route and removes snow from the designated area. The terminal feeds back the progress of the snow removal work to the server during the operation, and the server uses this information to monitor the work in real time.

[0075] The generative AI model is the heart of the system, generating efficient snow removal plans on the server. This algorithm uses machine learning techniques to consider weather conditions and terrain data to determine the optimal route and timing for snow removal.

[0076] A concrete example is when a hair salon owner requests snow removal in preparation for opening the next morning. The owner uses their smartphone to enter a prompt and send a request saying, "Please efficiently and automatically remove snow from the store's parking lot in preparation for opening the next morning. Generate an optimal snow removal plan based on the current snow depth information and remotely manage the snow removal equipment." Based on these instructions, the server develops an appropriate plan and arranges for the snow removal work to be carried out reliably.

[0077] In this way, the system provides flexible snow removal services tailored to the needs of each store, creating an environment where business owners can focus on their core business.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The server collects snow depth information from weather data providers via APIs. Inputs include the API key from the connected provider and the store's location coordinates. Based on these inputs, the server retrieves real-time snowfall and weather forecast data, and generates the latest snow depth data as output. This data is stored as information necessary for subsequent snow removal planning.

[0081] Step 2:

[0082] Users submit snow removal requests using a smartphone or computer application. Input includes the store's location, the area requiring snow removal, and the desired snow removal time. The user's input is sent to the server via the application, generating request data which is stored in the server's database. This data is then used by the server to develop a snow removal plan in the next step.

[0083] Step 3:

[0084] The server generates a snow removal plan based on received requests and acquired snow depth data using a generative AI model. Inputs include user request data and collected snow depth information. The server uses machine learning algorithms to calculate the optimal placement of snow removal equipment, start time, and route. As output, detailed snow removal plan data is constructed and sent to the terminal in JSON format.

[0085] Step 4:

[0086] The terminal remotely controls the snow removal equipment based on the snow removal plan received from the server. The input is snow removal plan data provided by the server. The terminal sends instructions to the snow removal equipment via wireless communication, and the equipment automatically performs snow removal according to the planned route. Real-time work progress data is generated as output and fed back to the server.

[0087] Step 5:

[0088] The server monitors the progress transmitted from the terminal and confirms the completion of the snow removal work. The input is progress data continuously provided by the terminal. The server analyzes this data and confirms that the snow removal work has been completed as planned. As output, a completion notification is generated and sent to the user to inform them of the completion of the work.

[0089] Step 6:

[0090] Users receive completion notifications on their smartphones or computers to confirm the completion of snow removal work. The input is a completion notification from the server. Based on this, users can check if the snow removal was performed as desired and make additional requests if necessary. The output is that, having confirmed completion, users are ready to proceed with their work with peace of mind.

[0091] (Application Example 1)

[0092] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0093] In areas with heavy snowfall, a challenge for store owners is efficiently clearing snow from parking lots and in front of stores before opening. In particular, obstruction of access to stores due to snow directly impacts business operations, requiring swift and reliable snow removal. Furthermore, a system is needed that allows store owners to access snow-related information in a timely manner and take appropriate measures.

[0094] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0095] In this invention, the server includes means for collecting snow accumulation information, means for notifying progress information on snow removal plans, and means for receiving snow removal requests that include location information and information on areas requiring snow removal. This enables store managers to obtain snow accumulation information in real time and efficiently manage snow removal operations.

[0096] "Means for collecting snow accumulation information" refers to technologies for obtaining real-time local snow accumulation conditions from weather data providers.

[0097] "Methods for generating snow removal plans" refers to a technology that uses artificial intelligence algorithms based on collected snow accumulation information to determine the most efficient snow removal method.

[0098] "Means for remotely operating snow removal equipment" refers to a technology that allows snow removal equipment to be operated remotely and automatically performed according to a specified plan.

[0099] "A means of monitoring the progress of snow removal equipment and confirming the completion of snow removal work" refers to a technology that tracks whether snow removal equipment is proceeding according to plan and evaluates whether the work has been completed.

[0100] "Means of notifying users of the completion of snow removal work" refers to technology used to inform store managers that snow removal work has been completed.

[0101] "Means for notifying users of the progress of snow removal plans" refers to technologies that provide users with the status of ongoing snow removal work in real time.

[0102] "Means for receiving snow removal requests that include location information and information on areas requiring snow removal" refers to technology that allows users to specify the location and area they wish to have snow removed from, and the system to receive requests based on that information.

[0103] An "artificial intelligence algorithm" is a program used to analyze collected data and determine the optimal snow removal route and timing.

[0104] The system implementing this invention is designed to allow store managers to efficiently manage snow removal operations. At its core are a server that processes data in real time and terminals used by users. The server collects snow depth information from weather data providers (e.g., OpenWeatherMap API) and uses an artificial intelligence algorithm to create an efficient snow removal plan based on this information. This algorithm is implemented using technologies such as Python and Tensorflow®. The generated plan is sent to the terminal via WebSocket communication, and the terminal remotely controls the snow removal equipment.

[0105] The device is a smartphone app developed using React Native, through which users send snow removal requests. These requests include location information and information about the area requiring snow removal, which is then sent to the server. The server adjusts the plan based on this information and notifies the user of the progress in real time. A MongoDB database records the progress and plan details, allowing for reference as needed.

[0106] As a concrete example, a hair salon owner checks the overnight snow forecast in preparation for opening the next morning and sends a snow removal request via the app. Based on this request, the server activates the snow removal equipment at the appropriate time the next morning and begins clearing snow from the parking lot. Progress is notified in real time via the app, allowing the owner to stay informed and focus on their normal business operations.

[0107] An example prompt for the generated AI model is, "Develop an AI model that plans efficient snow removal operations based on snow conditions." This is intended to show how the server's AI algorithm should operate.

[0108] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0109] Step 1:

[0110] The server retrieves snow depth information from a weather data provider. This information is obtained in JSON format. The server requests real-time data via an API and stores the response in a database. The input is the API request, and the output is snow depth data.

[0111] Step 2:

[0112] Users submit snow removal requests using a smartphone app. They specify their location and the area requiring snow removal. The device collects this information and sends it to the server. The input is the user's location and area information, and the output is the request data sent to the server.

[0113] Step 3:

[0114] The server generates a snow removal plan based on the received request. Artificial intelligence algorithms are used to analyze collected snow depth data and determine the optimal snow removal route and timing. The AI ​​model receives real-time environmental data as input and outputs snow removal instructions through calculations.

[0115] Step 4:

[0116] The server sends the generated snow removal plan to the terminal, which remotely controls the snow removal equipment. The terminal receives the plan and activates the snow removal equipment based on the presence or absence of snow and the route to be taken. The input is the snow removal plan, and the output is the operation action of the snow removal equipment.

[0117] Step 5:

[0118] The terminal sends the progress of the snow removal work to the server. The progress information is updated in real time and used to evaluate whether the work is progressing according to plan. The input is progress data, and the output is updated information sent to the server.

[0119] Step 6:

[0120] The server confirms the completion of the task and sends a notification to the user. The user's terminal displays a notification that the task is complete, allowing the user to confirm the completion of the snow removal work. The input is the task completion flag, and the output is the notification information sent to the user.

[0121] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0122] This invention enhances the user experience by combining an emotion engine with a system for efficiently performing snow removal work. The system consists of a scheme that includes a server, a terminal, and an engine that analyzes the user's emotions.

[0123] First, the server collects snow depth information from weather data providers and determines the need for snow removal based on that information. Based on this information, it uses a machine learning algorithm to generate an optimal snow removal plan. Users can also send requests using their smartphones or PCs, which include store location information and information about the areas that need snow removal. The server receives these requests and further refines the snow removal plan.

[0124] The generated snow removal plan is sent to a terminal, which then executes the plan by remotely controlling the snow removal equipment. The server constantly monitors progress information from the terminal and verifies whether the snow removal work has been completed.

[0125] The emotion engine, a key feature of this invention, has the function of recognizing the user's emotional state in real time and customizing the content of notifications based on that. This emotion analysis is performed using data such as the user's voice, text input, or facial expressions. For example, if the user is feeling stressed, comforting or encouraging words will be added to the notification message.

[0126] As a concrete example, consider a case where the owner of a shop requests snow removal. The user sends a snow removal request using a smartphone app the night before. The server immediately analyzes the snow accumulation information and sets the optimal snow removal plan by the next morning. At this time, the emotion engine detects that the user has entered a message expressing busyness or fatigue, and the snow removal completion notification includes a special message such as, "Thank you as always. We wish you the best in your preparations today."

[0127] This system allows users to not only check the progress of their work, but also receive support that is tailored to their emotional needs. This makes it possible to increase user satisfaction and provide a better service experience.

[0128] The following describes the processing flow.

[0129] Step 1:

[0130] The server obtains snow depth information in real time from weather data providers to understand current and projected snowfall amounts. This allows for an accurate assessment of snow conditions in each region.

[0131] Step 2:

[0132] Users use their smartphones or PCs to input their store's location information and the time of day when snow removal is needed, and then submit a snow removal request. The request may also include information about their mood on that day.

[0133] Step 3:

[0134] The server receives snow removal requests from users and combines them with snow depth information to generate an optimal snow removal plan. Machine learning algorithms are used to determine the most efficient routes and timing.

[0135] Step 4:

[0136] The emotion engine analyzes user input and past data to recognize the user's current emotional state. Based on this information, it determines which elements should be reflected in notifications and plans.

[0137] Step 5:

[0138] The server sends the generated snow removal plan to the terminal and instructs it to start the snow removal equipment at the specified time. The terminal remotely controls the snow removal equipment to perform the work according to the plan received from the server.

[0139] Step 6:

[0140] The terminal reports the progress of the snow removal equipment to the server in real time. Based on this, the server monitors the progress of the work and confirms that it is proceeding as planned.

[0141] Step 7:

[0142] After the server confirms that snow removal is complete, it sends a message customized by the emotion engine to the user. The user receives this notification and confirms that the snow removal is finished. This notification includes a special message tailored to the user's emotion, reflecting consideration for the user.

[0143] (Example 2)

[0144] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0145] The present invention aims to provide a system that can improve the emotional satisfaction of users while efficiently carrying out snow removal work. Conventional snow removal systems have focused so much on efficiency that notifications to users have become uniform, and appropriate responses have not been made according to the emotions and circumstances of individual users. Therefore, the challenge has been to satisfy both the progress of snow removal work and the emotional needs of users at the same time.

[0146] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0147] In this invention, the server includes means for acquiring and analyzing weather information, means for evaluating the necessity of snow removal work based on the weather information and generating a plan, means for monitoring the progress of snow removal work and confirming its completion, means for analyzing the user's status, and means for adjusting notification content according to the analysis results. This makes it possible to carry out efficient snow removal work while providing notifications that take the user's feelings into consideration.

[0148] "Weather information" refers to data related to weather conditions, including data such as snowfall, temperature, and wind speed.

[0149] A "snow removal plan" is a plan created based on collected weather information, which includes the specific procedures and schedule for snow removal operations.

[0150] A "snow removal device" is a machine or automated system for physically removing accumulated snow.

[0151] "User" refers to an individual or organization that operates a snow removal system or receives information about it.

[0152] "Analysis means" refers to a method or system for analyzing data to extract specific information or trends.

[0153] "Notification content" refers to messages that inform users of the progress and completion status of snow removal work.

[0154] This invention is a system for efficiently carrying out snow removal work and improving the user experience. The specific forms for implementing this system are described below.

[0155] The server first obtains weather information. It uses a common weather forecast API, such as the OpenWeatherMap API, to receive information from weather data providers. The server uses this API to collect information such as snowfall, temperature, and forecast weather, and uses this to assess the need for snow removal. Furthermore, it uses a machine learning library like Scikit-learn to generate a snow removal plan from the collected data. This plan includes optimal routes and work times.

[0156] Users send snow removal requests using their smartphones or PCs. The application uses the Google Maps API and other tools to obtain the user's current location and the area requiring snow removal, and sends this information to the server. The server then uses this information to create a plan and sends instructions to the user's device to control the snow removal equipment.

[0157] The terminal remotely controls the snow removal equipment using information received from the server. A computer such as a Raspberry Pi is used to control motors and sensors to perform the actual snow removal work. The terminal's control program reports the progress to the server in real time.

[0158] Systems equipped with an emotion engine are also used. The server can analyze the user's emotions in real time using the Google Cloud Natural Language API or the Microsoft® Azure® Emotion API. Based on the analysis results, the content of the notification message is customized and sent to the user. This allows for flexible responses tailored to the user's emotions, improving satisfaction.

[0159] As a concrete example, consider a scenario where a user enters the message "I'm tired" using a smartphone app. The server detects this and sends a notification upon completion of snow removal, including a special message such as "Thank you as always. We wish you the best in your preparations today." An example of a prompt for the generative AI model would be, "If the user feels tired, please suggest what kind of kind message to add to the notification."

[0160] In this way, the system can provide a convenient and emotionally resonant service to the user.

[0161] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0162] Step 1:

[0163] The server obtains weather information through the weather data provider's API. It uses the API key and request parameters as input and receives weather information (e.g., snowfall, temperature) in JSON format as output. Based on this information, it analyzes snowfall and creates basic data to determine the necessity of snow removal.

[0164] Step 2:

[0165] The server uses Scikit-learn's machine learning algorithms to generate an optimal snow removal plan from acquired weather information. The input is analyzed weather data, and the output is a snow removal plan with optimal routes and work times. In this process, the model is trained using historical weather data and snow removal history, and the plan is adjusted in real time.

[0166] Step 3:

[0167] Users submit snow removal requests from smartphone or PC applications. The input includes the user's location and the required snow removal area, and the output is request data sent to the server. The application utilizes the Google Maps API to accurately obtain the user's location.

[0168] Step 4:

[0169] The server receives requests from users and adjusts the snow removal plan to suit the specific situation. The input is the request data and the generated snow removal plan, and the output is the adjusted plan data. This operation updates the snow removal route while considering priorities.

[0170] Step 5:

[0171] The terminal remotely controls the snow removal equipment using a Raspberry Pi, following a snow removal plan sent from the server. The input is the adjusted snow removal plan, and the output is the actual operation of the equipment. At this stage, motor control and progress monitoring are performed.

[0172] Step 6:

[0173] The server uses an emotion engine to analyze user data in real time. Input consists of user text messages and voice data, and output is information about the user's emotional state. This analysis is performed using the Google Cloud Natural Language API.

[0174] Step 7:

[0175] When the server sends notification messages to users, it customizes the content based on analyzed sentiment information. The input is the sentiment analysis result, and the output is the customized message sent to the user. This process generates sentiment-sensitive messages, such as "Thank you as always. We wish you the best in your preparations today."

[0176] (Application Example 2)

[0177] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0178] While conventional snow removal systems were capable of efficiently generating snow removal plans based on snow conditions and managing progress, they lacked individualized support that considered user emotions, resulting in a failure to improve the user experience. Therefore, there is a need for methods that enhance user satisfaction and provide snow removal notifications in a more personalized way.

[0179] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0180] In this invention, the server includes means for acquiring snow depth data, means for generating a snow removal plan based on the snow depth data, and means for analyzing the user's emotional state and customizing notification content. This enables personalized notifications that respond to the user's emotions.

[0181] "Snow accumulation data" refers to data obtained from weather information providers regarding the amount of snow accumulated in a specific area.

[0182] A "snow removal plan" is a work plan that includes the optimal snow removal method, route, and time, based on collected snow depth data.

[0183] "Snow removal equipment" refers to mechanical devices used for snow removal work that can be remotely controlled.

[0184] "Users" refers to individuals or organizations that use the snow removal system and have the right to receive notifications regarding snow accumulation and the progress of snow removal.

[0185] "Emotional state" refers to the user's psychological or emotional condition, which is analyzed from voice, text, facial expressions, etc.

[0186] "Notification content" refers to information provided to users, including reports on the progress of snow removal and reports sent after the completion of the plan.

[0187] "Means of customization" refers to methods for adjusting notification content based on the user's emotional state and providing personalized information tailored to the user.

[0188] The specific system for implementing this invention is implemented with the following configuration: The server acquires snow depth data from a weather information service and generates a snow removal plan based on this data. The snow removal plan is generated using an algorithm that has learned past snow depth patterns to determine the optimal snow removal route and timing. TensorFlow is used in this process. The generated plan is sent to the terminal.

[0189] The terminal remotely controls snow removal equipment placed around the house. A commercially available IoT device controller is used for control, and sensor data is collected to monitor the work status in real time. Progress information is sent to the server as the snow removal work progresses, and the overall system status is updated when the work is completed.

[0190] Furthermore, to take into account the user's emotional state, the server analyzes voice and text input. Google Cloud Speech-to-Text and AWS® Rekognition are used for the analysis. If a message indicating stress or fatigue is detected, the sentiment analysis engine sends a personalized notification to the user. For example, upon completion of snow removal, the user receives a relaxing message such as, "Thank you for your hard work today. Snow removal has been completed successfully."

[0191] As a concrete example, consider a scenario where a snow removal robot is operated at a home. The user checks the amount of snow on their smartphone and requests snow removal for the day. The server creates an optimal plan and notifies the user of the start time. Sentiment analysis begins during the operation, and after completion, a message based on the results is sent. In this way, it is possible to provide a service that is sensitive to the user's feelings.

[0192] Example prompt: "Please tell me the snow conditions this morning. Show me how to create a message that takes the user's mood into consideration."

[0193] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0194] Step 1:

[0195] The server retrieves snow depth data from a weather information service. This data is obtained using an HTTP request, receiving data in JSON format from the API. The input is regional information, and the output includes data on the current snow depth in that region.

[0196] Step 2:

[0197] The server generates a snow removal plan based on acquired snow depth data. This plan utilizes a machine learning model (built with TensorFlow) based on historical snowfall data to calculate the optimal snow removal route and time. Input consists of snow depth data in JSON format and model parameters; output is the optimized snow removal plan.

[0198] Step 3:

[0199] The generated snow removal plan is sent to the terminal. The terminal remotely controls the snow removal equipment based on the received plan. It issues instructions to the equipment via an IoT controller, enabling real-time control. The input is the snow removal plan, and the output is the operating status of the snow removal equipment.

[0200] Step 4:

[0201] The terminal uses sensor data acquired from the equipment to monitor the progress of the work in progress. The progress is reported to the server in real time. The input is sensor data, and the output is the work progress status.

[0202] Step 5:

[0203] The server confirms the completion of the task based on the work progress. It then generates notification data indicating that the task is complete. The input is the work progress, and the output is the completion notification data.

[0204] Step 6:

[0205] After the task is completed, the server analyzes the user's emotional state. It uses Google Cloud Speech-to-Text and AWS Rekognition to process voice input and facial expression data and perform sentiment analysis. Input is the user's voice or image data, and output is the analyzed emotional state.

[0206] Step 7:

[0207] Based on the analyzed emotional state, the server customizes the notification content. A generative AI model is used to create a relaxing message tailored to the user. The input is emotional state data, and the output is a personalized notification message.

[0208] Step 8:

[0209] Customized notification messages are sent to the user. The messages are displayed on the user's device, providing an emotionally resonant service experience. The input is the notification message, and the output is the user's reception status.

[0210] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0211] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0212] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0213] [Second Embodiment]

[0214] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0215] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0216] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0217] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0218] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0219] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0220] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0221] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0222] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0223] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0224] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0225] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0226] This invention is a system that efficiently and safely handles snow removal work, which is a burden for store managers, and is implemented primarily through interaction between a server, terminals, and users.

[0227] The server collects snow accumulation information in real time from weather data providers. This allows the server to understand the exact snow conditions in the area where the store is located. Store owners, as users, use smartphones or computers to identify the store's location and the areas requiring snow removal, and then send snow removal requests.

[0228] The received request is processed by the server. The server uses an AI algorithm based on the collected snow depth information to generate an efficient snow removal plan. This plan includes the placement of snow removal equipment, the timing of operation, and the route to be taken. The server sends the generated snow removal plan to the terminal, which then remotely operates the snow removal equipment based on it.

[0229] The terminal activates the snow removal equipment based on instructions from the server and automatically performs snow removal along a planned route. During snow removal, the terminal constantly sends progress information to the server, which monitors it.

[0230] Once snow removal is complete, the server verifies the information and sends a completion notification to the user. The user receives this notification on their smartphone or computer and confirms that the snow removal has been successfully completed. This entire process ensures that the parking lot and the area in front of the store are safely cleared of snow by opening time, allowing the business owner to focus on their work.

[0231] As a concrete example, consider a case where the owner of a hair salon requests snow removal in preparation for opening the next morning. During the night when snowfall is expected, the owner sends a request from their smartphone at home. The server plans for the snow removal equipment to be activated at the appropriate time the next morning and operates the snow removal equipment remotely to clear the parking lot. In this way, the owner can concentrate on their normal business without having to spend their busy morning hours on snow removal.

[0232] The following describes the processing flow.

[0233] Step 1:

[0234] The server periodically collects snow information, such as predicted snowfall and current snow depth, using the weather data provider's API. This ensures that the system is always up-to-date on the latest weather conditions.

[0235] Step 2:

[0236] Users enter information such as the store's location and desired snow removal times via a smartphone or PC application and submit a snow removal request. This request reaches the server and is stored in the database.

[0237] Step 3:

[0238] The server determines the need for snow removal based on user requests and the latest snow depth information. If deemed necessary, it uses an AI algorithm to generate an optimal snow removal plan. This plan includes snow removal machine usage time, operating routes, and expected work time.

[0239] Step 4:

[0240] The server distributes the generated snow removal plan to the terminal. The terminal then prepares to activate the snow removal equipment at the specified time based on this plan.

[0241] Step 5:

[0242] The terminal remotely controls the snow removal equipment based on instructions received from the server and begins snow removal work along the designated route. The operating status and progress information of the snow removal equipment are transmitted to the server in real time.

[0243] Step 6:

[0244] The server constantly monitors whether snow removal is progressing normally based on progress information from the terminals. When the work is completed, it records the resources used and the work time, and updates the database.

[0245] Step 7:

[0246] After confirming the completion of snow removal, the server sends a completion notification to the user. Upon receiving this notification, the user can confirm that snow removal has been successfully completed and proceed with opening preparations.

[0247] (Example 1)

[0248] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0249] Currently, store owners in snowy regions face a significant burden of clearing snow from parking lots and storefronts before opening. This task is time-consuming and labor-intensive, causing considerable stress, especially during the busy pre-opening hours. Furthermore, improper snow removal can compromise safety. To address these issues, there is a need for the development of an automated system that performs efficient and safe snow removal.

[0250] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0251] In this invention, the server includes means for collecting snow accumulation information, means for receiving location information and area information where snow removal is required, provided by users, and means for formulating an efficient snow removal process using an artificial intelligence algorithm. This reduces the burden on store managers and enables safe and efficient snow removal work within a given timeframe.

[0252] "Snowfall information" refers to data on snowfall amount and snow depth provided by weather data providers, and is fundamental information for snow removal planning.

[0253] A "snow removal plan" refers to a plan that includes details such as the timing of operation and routes for snow removal equipment necessary for efficient snow removal work.

[0254] A "snow removal device" refers to a machine or mechanism that physically removes accumulated snow and is controlled by remote operation.

[0255] "Remote control" refers to the act of controlling a machine or system using electronic means from a physically distant location.

[0256] "Progress status" refers to data representing the current execution status and progress of snow removal work, which is continuously monitored by the server.

[0257] "Completion confirmed" refers to the process in which the server confirms that all scheduled snow removal operations have been carried out as planned and that the designated area has been properly cleared of snow.

[0258] "Notification" refers to the act of informing users about the completion or progress of snow removal work, and is mainly done via email or push notifications on mobile apps.

[0259] "Location information" refers to digital data that indicates a specific point on Earth, and is often expressed using a geographic coordinate system.

[0260] "Area information" refers to data about specific areas where snow removal is required, and serves as a basis for formulating snow removal plans.

[0261] "Machine learning technology" refers to techniques that enable computers to autonomously recognize patterns using large amounts of data and generate predictive models.

[0262] An "artificial intelligence algorithm" is a type of computational procedure executed by a computer, designed to solve specific problems while mimicking human knowledge and reasoning.

[0263] This invention is an automated system for reducing the burden on store managers and efficiently performing snow removal. The system is implemented through the interaction of a server, terminals, and users.

[0264] The server plays a central role in collecting snow depth information and generating effective snow removal plans. Specifically, the server retrieves snow depth information in real time from weather data providers. For example, it periodically collects snowfall and weather forecast data from providers using APIs. This data is essential for the server to efficiently create snow removal plans. The server also maintains a database to manage user request information and past snow removal history.

[0265] Users submit snow removal requests using devices such as smartphones or computers. A dedicated application allows users to easily specify the store's location and the area to be cleared. Users can also request snow removal at the optimal time by entering their desired time slot. After the snow removal is complete, users can check the status through the application.

[0266] The terminal remotely controls the snow removal equipment based on the snow removal plan transmitted from the server. The terminal uses wireless communication to send instructions directly to the snow removal equipment on site. Based on the instructions received by the terminal, the snow removal equipment automatically follows the planned route and removes snow from the designated area. The terminal feeds back the progress of the snow removal work to the server during the operation, and the server uses this information to monitor the work in real time.

[0267] The generative AI model is the heart of the system, generating efficient snow removal plans on the server. This algorithm uses machine learning techniques to consider weather conditions and terrain data to determine the optimal route and timing for snow removal.

[0268] A concrete example is when a hair salon owner requests snow removal in preparation for opening the next morning. The owner uses their smartphone to enter a prompt and send a request saying, "Please efficiently and automatically remove snow from the store's parking lot in preparation for opening the next morning. Generate an optimal snow removal plan based on the current snow depth information and remotely manage the snow removal equipment." Based on these instructions, the server develops an appropriate plan and arranges for the snow removal work to be carried out reliably.

[0269] In this way, the system provides flexible snow removal services tailored to the needs of each store, creating an environment where business owners can focus on their core business.

[0270] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0271] Step 1:

[0272] The server collects snow depth information from weather data providers via APIs. Inputs include the API key from the connected provider and the store's location coordinates. Based on these inputs, the server retrieves real-time snowfall and weather forecast data, and generates the latest snow depth data as output. This data is stored as information necessary for subsequent snow removal planning.

[0273] Step 2:

[0274] Users submit snow removal requests using a smartphone or computer application. Input includes the store's location, the area requiring snow removal, and the desired snow removal time. The user's input is sent to the server via the application, generating request data which is stored in the server's database. This data is then used by the server to develop a snow removal plan in the next step.

[0275] Step 3:

[0276] The server generates a snow removal plan based on received requests and acquired snow depth data using a generative AI model. Inputs include user request data and collected snow depth information. The server uses machine learning algorithms to calculate the optimal placement of snow removal equipment, start time, and route. As output, detailed snow removal plan data is constructed and sent to the terminal in JSON format.

[0277] Step 4:

[0278] Based on the snow removal plan received from the server, the terminal remotely operates the snow removal device. The input includes the snow removal plan data provided by the server. The terminal sends instructions to the snow removal device via wireless communication, and the device automatically performs snow removal according to the planned route. As output, real-time work progress data is generated and fed back to the server.

[0279] Step 5:

[0280] The server monitors the progress sent from the terminal and confirms the completion of the snow removal operation. The input includes the progress data continuously provided by the terminal. The server analyzes this data and confirms that the snow removal operation has ended as planned. As output, a completion notification is generated and sent to the user to inform them of the completion of the operation.

[0281] Step 6:

[0282] The user receives the completion notification on a smartphone or computer and confirms the completion of the snow removal operation. The input includes the completion notification from the server. Based on this, the user can confirm whether the snow removal has been carried out as desired and make additional requests if necessary. As output, the user is ready to enter the business with confidence after the confirmation is completed.

[0283] (Application Example 1)

[0284] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0285] In an area with snow accumulation, it is an issue for store operators to efficiently perform snow removal operations in the parking lot and in front of the store before opening. In particular, the obstruction of access to the store due to snow accumulation directly affects business operations, so a quick and reliable snow removal operation is required. Also, a system is needed that allows store operators to timely grasp information related to snow removal and take countermeasures.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0287] In this invention, the server includes means for collecting snow accumulation information, means for notifying the progress information of the snow removal plan, and means for receiving a snow removal request including location information and information on areas where snow removal is required. As a result, the store manager can obtain snow accumulation information in real time and manage the snow removal work efficiently.

[0288] The "means for collecting snow accumulation information" is a technology for obtaining the snow accumulation situation in a region in real time from a meteorological data provider.

[0289] The "means for generating a snow removal plan" is a technology for determining an efficient snow removal method using an artificial intelligence algorithm based on the collected snow accumulation information.

[0290] The "means for remotely operating a snow removal device" is a technology for operating snow removal equipment from a remote location and automatically performing snow removal according to a specified plan.

[0291] The "means for monitoring the progress of the snow removal device and confirming the completion of the snow removal work" is a technology for tracking whether the snow removal device is working as planned and evaluating whether the work has been completed.

[0292] The "means for notifying the user of the completion of the snow removal work" is a technology for notifying the store manager of the end of the snow removal work.

[0293] The "means for notifying the progress information of the snow removal plan" is a technology for providing the user with the status of the ongoing snow removal work in real time.

[0294] The "means for receiving a snow removal request including location information and information on areas where snow removal is required" is a technology for the system to receive a request based on identifying the location and scope where the user desires snow removal.

[0295] An "artificial intelligence algorithm" is a program used to analyze collected data and determine the optimal snow removal route and timing.

[0296] The system implementing this invention is designed to allow store managers to efficiently manage snow removal operations. At its core are a server that processes data in real time and terminals used by users. The server collects snow depth information from weather data providers (e.g., OpenWeatherMap API) and uses an artificial intelligence algorithm to create an efficient snow removal plan based on this information. This algorithm is implemented using technologies such as Python and TensorFlow. The generated plan is sent to the terminal via WebSocket communication, and the terminal remotely controls the snow removal equipment.

[0297] The device is a smartphone app developed using React Native, through which users send snow removal requests. These requests include location information and information about the area requiring snow removal, which is then sent to the server. The server adjusts the plan based on this information and notifies the user of the progress in real time. A MongoDB database records the progress and plan details, allowing for reference as needed.

[0298] As a concrete example, a hair salon owner checks the overnight snow forecast in preparation for opening the next morning and sends a snow removal request via the app. Based on this request, the server activates the snow removal equipment at the appropriate time the next morning and begins clearing snow from the parking lot. Progress is notified in real time via the app, allowing the owner to stay informed and focus on their normal business operations.

[0299] An example prompt for the generated AI model is, "Develop an AI model that plans efficient snow removal operations based on snow conditions." This is intended to show how the server's AI algorithm should operate.

[0300] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0301] Step 1:

[0302] The server acquires snow accumulation information from a weather data provider. This information is acquired in JSON format. The server requests real-time data through an API and stores the response in a database. The input is the API request, and the output is the snow accumulation status data.

[0303] Step 2:

[0304] The user sends a snow removal request using a smartphone app. The user specifies the location information and the area where snow removal is required. The terminal collects this information and sends it to the server. The input is the input of the location information and area information by the user, and the output is the request data to the server.

[0305] Step 3:

[0306] The server generates a snow removal plan based on the received request. Here, an artificial intelligence algorithm is used to analyze the collected snow accumulation data and determine the optimal snow removal route and timing. The environmental data obtained in real time is input into the AI model, and a snow removal instruction is output by calculation.

[0307] Step 4:

[0308] The server sends the generated snow removal plan to the terminal and remotely operates the snow removal device. The terminal receives the plan and activates the snow removal device based on the presence or absence of snow accumulation and the progress route. The input is the snow removal plan, and the output is the operation action of the snow removal device.

[0309] Step 5:

[0310] The terminal sends the progress of the snow removal work to the server. The progress information is updated in real time and used to evaluate whether the work is progressing according to plan. The input is progress data, and the output is updated information sent to the server.

[0311] Step 6:

[0312] The server confirms the completion of the task and sends a notification to the user. The user's terminal displays a notification that the task is complete, allowing the user to confirm the completion of the snow removal work. The input is the task completion flag, and the output is the notification information sent to the user.

[0313] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0314] This invention enhances the user experience by combining an emotion engine with a system for efficiently performing snow removal work. The system consists of a scheme that includes a server, a terminal, and an engine that analyzes the user's emotions.

[0315] First, the server collects snow depth information from weather data providers and determines the need for snow removal based on that information. Based on this information, it uses a machine learning algorithm to generate an optimal snow removal plan. Users can also send requests using their smartphones or PCs, which include store location information and information about the areas that need snow removal. The server receives these requests and further refines the snow removal plan.

[0316] The generated snow removal plan is sent to a terminal, which then executes the plan by remotely controlling the snow removal equipment. The server constantly monitors progress information from the terminal and verifies whether the snow removal work has been completed.

[0317] The emotion engine, a key feature of this invention, has the function of recognizing the user's emotional state in real time and customizing the content of notifications based on that. This emotion analysis is performed using data such as the user's voice, text input, or facial expressions. For example, if the user is feeling stressed, comforting or encouraging words will be added to the notification message.

[0318] As a concrete example, consider a case where the owner of a shop requests snow removal. The user sends a snow removal request using a smartphone app the night before. The server immediately analyzes the snow accumulation information and sets the optimal snow removal plan by the next morning. At this time, the emotion engine detects that the user has entered a message expressing busyness or fatigue, and the snow removal completion notification includes a special message such as, "Thank you as always. We wish you the best in your preparations today."

[0319] This system allows users to not only check the progress of their work, but also receive support that is tailored to their emotional needs. This makes it possible to increase user satisfaction and provide a better service experience.

[0320] The following describes the processing flow.

[0321] Step 1:

[0322] The server obtains snow depth information in real time from weather data providers to understand current and projected snowfall amounts. This allows for an accurate assessment of snow conditions in each region.

[0323] Step 2:

[0324] Users use their smartphones or PCs to input their store's location information and the time of day when snow removal is needed, and then submit a snow removal request. The request may also include information about their mood on that day.

[0325] Step 3:

[0326] The server receives snow removal requests from users and combines them with snow depth information to generate an optimal snow removal plan. Machine learning algorithms are used to determine the most efficient routes and timing.

[0327] Step 4:

[0328] The emotion engine analyzes user input and past data to recognize the user's current emotional state. Based on this information, it determines which elements should be reflected in notifications and plans.

[0329] Step 5:

[0330] The server sends the generated snow removal plan to the terminal and instructs it to start the snow removal equipment at the specified time. The terminal remotely controls the snow removal equipment to perform the work according to the plan received from the server.

[0331] Step 6:

[0332] The terminal reports the progress of the snow removal equipment to the server in real time. Based on this, the server monitors the progress of the work and confirms that it is proceeding as planned.

[0333] Step 7:

[0334] After the server confirms that snow removal is complete, it sends a message customized by the emotion engine to the user. The user receives this notification and confirms that the snow removal is finished. This notification includes a special message tailored to the user's emotion, reflecting consideration for the user.

[0335] (Example 2)

[0336] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0337] The present invention aims to provide a system that can improve the emotional satisfaction of users while efficiently carrying out snow removal work. Conventional snow removal systems have focused so much on efficiency that notifications to users have become uniform, and appropriate responses have not been made according to the emotions and circumstances of individual users. Therefore, the challenge has been to satisfy both the progress of snow removal work and the emotional needs of users at the same time.

[0338] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0339] In this invention, the server includes means for acquiring and analyzing weather information, means for evaluating the necessity of snow removal work based on the weather information and generating a plan, means for monitoring the progress of snow removal work and confirming its completion, means for analyzing the user's status, and means for adjusting notification content according to the analysis results. This makes it possible to carry out efficient snow removal work while providing notifications that take the user's feelings into consideration.

[0340] "Weather information" refers to data related to weather conditions, including data such as snowfall, temperature, and wind speed.

[0341] A "snow removal plan" is a plan created based on collected weather information, which includes the specific procedures and schedule for snow removal operations.

[0342] A "snow removal device" is a machine or automated system for physically removing accumulated snow.

[0343] "User" refers to an individual or organization that operates a snow removal system or receives information about it.

[0344] "Analysis means" refers to a method or system for analyzing data to extract specific information or trends.

[0345] "Notification content" refers to messages that inform users of the progress and completion status of snow removal work.

[0346] This invention is a system for efficiently carrying out snow removal work and improving the user experience. The specific forms for implementing this system are described below.

[0347] The server first obtains weather information. It uses a common weather forecast API, such as the OpenWeatherMap API, to receive information from weather data providers. The server uses this API to collect information such as snowfall, temperature, and forecast weather, and uses this to assess the need for snow removal. Furthermore, it uses a machine learning library like Scikit-learn to generate a snow removal plan from the collected data. This plan includes optimal routes and work times.

[0348] Users send snow removal requests using their smartphones or PCs. The application uses the Google Maps API and other tools to obtain the user's current location and the area requiring snow removal, and sends this information to the server. The server then uses this information to create a plan and sends instructions to the user's device to control the snow removal equipment.

[0349] The terminal remotely controls the snow removal equipment using information received from the server. A computer such as a Raspberry Pi is used to control motors and sensors to perform the actual snow removal work. The terminal's control program reports the progress to the server in real time.

[0350] Systems equipped with emotion engines are also used. The server can analyze user emotions in real time using Google Cloud Natural Language API or Microsoft Azure Emotion API. Based on the analysis results, the content of notification messages is customized and sent to the user. This allows for flexible responses tailored to the user's emotions, improving satisfaction.

[0351] As a concrete example, consider a scenario where a user enters the message "I'm tired" using a smartphone app. The server detects this and sends a notification upon completion of snow removal, including a special message such as "Thank you as always. We wish you the best in your preparations today." An example of a prompt for the generative AI model would be, "If the user feels tired, please suggest what kind of kind message to add to the notification."

[0352] In this way, the system can provide a convenient and emotionally resonant service to the user.

[0353] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0354] Step 1:

[0355] The server obtains weather information through the weather data provider's API. It uses the API key and request parameters as input and receives weather information (e.g., snowfall, temperature) in JSON format as output. Based on this information, it analyzes snowfall and creates basic data to determine the necessity of snow removal.

[0356] Step 2:

[0357] The server uses Scikit-learn's machine learning algorithms to generate an optimal snow removal plan from acquired weather information. The input is analyzed weather data, and the output is a snow removal plan with optimal routes and work times. In this process, the model is trained using historical weather data and snow removal history, and the plan is adjusted in real time.

[0358] Step 3:

[0359] Users submit snow removal requests from smartphone or PC applications. The input includes the user's location and the required snow removal area, and the output is request data sent to the server. The application utilizes the Google Maps API to accurately obtain the user's location.

[0360] Step 4:

[0361] The server receives requests from users and adjusts the snow removal plan to suit the specific situation. The input is the request data and the generated snow removal plan, and the output is the adjusted plan data. This operation updates the snow removal route while considering priorities.

[0362] Step 5:

[0363] The terminal remotely controls the snow removal equipment using a Raspberry Pi, following a snow removal plan sent from the server. The input is the adjusted snow removal plan, and the output is the actual operation of the equipment. At this stage, motor control and progress monitoring are performed.

[0364] Step 6:

[0365] The server uses an emotion engine to analyze user data in real time. Input consists of user text messages and voice data, and output is information about the user's emotional state. This analysis is performed using the Google Cloud Natural Language API.

[0366] Step 7:

[0367] When the server sends notification messages to users, it customizes the content based on analyzed sentiment information. The input is the sentiment analysis result, and the output is the customized message sent to the user. This process generates sentiment-sensitive messages, such as "Thank you as always. We wish you the best in your preparations today."

[0368] (Application Example 2)

[0369] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0370] While conventional snow removal systems were capable of efficiently generating snow removal plans based on snow conditions and managing progress, they lacked individualized support that considered user emotions, resulting in a failure to improve the user experience. Therefore, there is a need for methods that enhance user satisfaction and provide snow removal notifications in a more personalized way.

[0371] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0372] In this invention, the server includes means for acquiring snow depth data, means for generating a snow removal plan based on the snow depth data, and means for analyzing the user's emotional state and customizing notification content. This enables personalized notifications that respond to the user's emotions.

[0373] "Snow accumulation data" refers to data obtained from weather information providers regarding the amount of snow accumulated in a specific area.

[0374] A "snow removal plan" is a work plan that includes the optimal snow removal method, route, and time, based on collected snow depth data.

[0375] "Snow removal equipment" refers to mechanical devices used for snow removal work that can be remotely controlled.

[0376] "Users" refers to individuals or organizations that use the snow removal system and have the right to receive notifications regarding snow accumulation and the progress of snow removal.

[0377] "Emotional state" refers to the user's psychological or emotional condition, which is analyzed from voice, text, facial expressions, etc.

[0378] "Notification content" refers to information provided to users, including reports on the progress of snow removal and reports sent after the completion of the plan.

[0379] "Means of customization" refers to methods for adjusting notification content based on the user's emotional state and providing personalized information tailored to the user.

[0380] The specific system for implementing this invention is implemented with the following configuration: The server acquires snow depth data from a weather information service and generates a snow removal plan based on this data. The snow removal plan is generated using an algorithm that has learned past snow depth patterns to determine the optimal snow removal route and timing. TensorFlow is used in this process. The generated plan is sent to the terminal.

[0381] The terminal remotely controls snow removal equipment placed around the house. A commercially available IoT device controller is used for control, and sensor data is collected to monitor the work status in real time. Progress information is sent to the server as the snow removal work progresses, and the overall system status is updated when the work is completed.

[0382] Furthermore, to take into account the user's emotional state, the server analyzes voice and text input. Google Cloud Speech-to-Text and AWS Rekognition are used for the analysis. If a message is detected indicating the user is stressed or feeling fatigued, the sentiment analysis engine sends a personalized notification to the user. For example, upon completion of snow removal, the user receives a relaxing message such as, "Thank you for your hard work today. Snow removal has been completed successfully."

[0383] As a concrete example, consider a scenario where a snow removal robot is operated at a home. The user checks the amount of snow on their smartphone and requests snow removal for the day. The server creates an optimal plan and notifies the user of the start time. Sentiment analysis begins during the operation, and after completion, a message based on the results is sent. In this way, it is possible to provide a service that is sensitive to the user's feelings.

[0384] Example prompt: "Please tell me the snow conditions this morning. Show me how to create a message that takes the user's mood into consideration."

[0385] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0386] Step 1:

[0387] The server retrieves snow depth data from a weather information service. This data is obtained using an HTTP request, receiving data in JSON format from the API. The input is regional information, and the output includes data on the current snow depth in that region.

[0388] Step 2:

[0389] The server generates a snow removal plan based on acquired snow depth data. This plan utilizes a machine learning model (built with TensorFlow) based on historical snowfall data to calculate the optimal snow removal route and time. Input consists of snow depth data in JSON format and model parameters; output is the optimized snow removal plan.

[0390] Step 3:

[0391] The generated snow removal plan is sent to the terminal. The terminal remotely controls the snow removal equipment based on the received plan. It issues instructions to the equipment via an IoT controller, enabling real-time control. The input is the snow removal plan, and the output is the operating status of the snow removal equipment.

[0392] Step 4:

[0393] The terminal uses sensor data acquired from the equipment to monitor the progress of the work in progress. The progress is reported to the server in real time. The input is sensor data, and the output is the work progress status.

[0394] Step 5:

[0395] The server confirms the completion of the task based on the work progress. It then generates notification data indicating that the task is complete. The input is the work progress, and the output is the completion notification data.

[0396] Step 6:

[0397] After the task is completed, the server analyzes the user's emotional state. It uses Google Cloud Speech-to-Text and AWS Rekognition to process voice input and facial expression data and perform sentiment analysis. Input is the user's voice or image data, and output is the analyzed emotional state.

[0398] Step 7:

[0399] Based on the analyzed emotional state, the server customizes the notification content. A generative AI model is used to create a relaxing message tailored to the user. The input is emotional state data, and the output is a personalized notification message.

[0400] Step 8:

[0401] Customized notification messages are sent to the user. The messages are displayed on the user's device, providing an emotionally resonant service experience. The input is the notification message, and the output is the user's reception status.

[0402] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0403] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0404] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0405] [Third Embodiment]

[0406] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0407] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0408] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0409] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0410] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0411] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0412] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0413] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0414] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0415] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0416] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0417] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0418] This invention is a system that efficiently and safely handles snow removal work, which is a burden for store managers, and is implemented primarily through interaction between a server, terminals, and users.

[0419] The server collects snow accumulation information in real time from weather data providers. This allows the server to understand the exact snow conditions in the area where the store is located. Store owners, as users, use smartphones or computers to identify the store's location and the areas requiring snow removal, and then send snow removal requests.

[0420] The received request is processed by the server. The server uses an AI algorithm based on the collected snow depth information to generate an efficient snow removal plan. This plan includes the placement of snow removal equipment, the timing of operation, and the route to be taken. The server sends the generated snow removal plan to the terminal, which then remotely operates the snow removal equipment based on it.

[0421] The terminal activates the snow removal equipment based on instructions from the server and automatically performs snow removal along a planned route. During snow removal, the terminal constantly sends progress information to the server, which monitors it.

[0422] Once snow removal is complete, the server verifies the information and sends a completion notification to the user. The user receives this notification on their smartphone or computer and confirms that the snow removal has been successfully completed. This entire process ensures that the parking lot and the area in front of the store are safely cleared of snow by opening time, allowing the business owner to focus on their work.

[0423] As a concrete example, consider a case where the owner of a hair salon requests snow removal in preparation for opening the next morning. During the night when snowfall is expected, the owner sends a request from their smartphone at home. The server plans for the snow removal equipment to be activated at the appropriate time the next morning and operates the snow removal equipment remotely to clear the parking lot. In this way, the owner can concentrate on their normal business without having to spend their busy morning hours on snow removal.

[0424] The following describes the processing flow.

[0425] Step 1:

[0426] The server periodically collects snow information, such as predicted snowfall and current snow depth, using the weather data provider's API. This ensures that the system is always up-to-date on the latest weather conditions.

[0427] Step 2:

[0428] Users enter information such as the store's location and desired snow removal times via a smartphone or PC application and submit a snow removal request. This request reaches the server and is stored in the database.

[0429] Step 3:

[0430] The server determines the need for snow removal based on user requests and the latest snow depth information. If deemed necessary, it uses an AI algorithm to generate an optimal snow removal plan. This plan includes snow removal machine usage time, operating routes, and expected work time.

[0431] Step 4:

[0432] The server distributes the generated snow removal plan to the terminal. The terminal then prepares to activate the snow removal equipment at the specified time based on this plan.

[0433] Step 5:

[0434] The terminal remotely controls the snow removal equipment based on instructions received from the server and begins snow removal work along the designated route. The operating status and progress information of the snow removal equipment are transmitted to the server in real time.

[0435] Step 6:

[0436] The server constantly monitors whether snow removal is progressing normally based on progress information from the terminals. When the work is completed, it records the resources used and the work time, and updates the database.

[0437] Step 7:

[0438] After confirming the completion of snow removal, the server sends a completion notification to the user. Upon receiving this notification, the user can confirm that snow removal has been successfully completed and proceed with opening preparations.

[0439] (Example 1)

[0440] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0441] Currently, store owners in snowy regions face a significant burden of clearing snow from parking lots and storefronts before opening. This task is time-consuming and labor-intensive, causing considerable stress, especially during the busy pre-opening hours. Furthermore, improper snow removal can compromise safety. To address these issues, there is a need for the development of an automated system that performs efficient and safe snow removal.

[0442] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0443] In this invention, the server includes means for collecting snow accumulation information, means for receiving location information and area information where snow removal is required, provided by users, and means for formulating an efficient snow removal process using an artificial intelligence algorithm. This reduces the burden on store managers and enables safe and efficient snow removal work within a given timeframe.

[0444] "Snowfall information" refers to data on snowfall amount and snow depth provided by weather data providers, and is fundamental information for snow removal planning.

[0445] A "snow removal plan" refers to a plan that includes details such as the timing of operation and routes for snow removal equipment necessary for efficient snow removal work.

[0446] A "snow removal device" refers to a machine or mechanism that physically removes accumulated snow and is controlled by remote operation.

[0447] "Remote control" refers to the act of controlling a machine or system using electronic means from a physically distant location.

[0448] "Progress status" refers to data representing the current execution status and progress of snow removal work, which is continuously monitored by the server.

[0449] "Completion confirmed" refers to the process in which the server confirms that all scheduled snow removal operations have been carried out as planned and that the designated area has been properly cleared of snow.

[0450] "Notification" refers to the act of informing users about the completion or progress of snow removal work, and is mainly done via email or push notifications on mobile apps.

[0451] "Location information" refers to digital data that indicates a specific point on Earth, and is often expressed using a geographic coordinate system.

[0452] "Area information" refers to data about specific areas where snow removal is required, and serves as a basis for formulating snow removal plans.

[0453] "Machine learning technology" refers to techniques that enable computers to autonomously recognize patterns using large amounts of data and generate predictive models.

[0454] An "artificial intelligence algorithm" is a type of computational procedure executed by a computer, designed to solve specific problems while mimicking human knowledge and reasoning.

[0455] This invention is an automated system for reducing the burden on store managers and efficiently performing snow removal. The system is implemented through the interaction of a server, terminals, and users.

[0456] The server plays a central role in collecting snow depth information and generating effective snow removal plans. Specifically, the server retrieves snow depth information in real time from weather data providers. For example, it periodically collects snowfall and weather forecast data from providers using APIs. This data is essential for the server to efficiently create snow removal plans. The server also maintains a database to manage user request information and past snow removal history.

[0457] Users submit snow removal requests using devices such as smartphones or computers. A dedicated application allows users to easily specify the store's location and the area to be cleared. Users can also request snow removal at the optimal time by entering their desired time slot. After the snow removal is complete, users can check the status through the application.

[0458] The terminal remotely controls the snow removal equipment based on the snow removal plan transmitted from the server. The terminal uses wireless communication to send instructions directly to the snow removal equipment on site. Based on the instructions received by the terminal, the snow removal equipment automatically follows the planned route and removes snow from the designated area. The terminal feeds back the progress of the snow removal work to the server during the operation, and the server uses this information to monitor the work in real time.

[0459] The generative AI model is the heart of the system, generating efficient snow removal plans on the server. This algorithm uses machine learning techniques to consider weather conditions and terrain data to determine the optimal route and timing for snow removal.

[0460] A concrete example is when a hair salon owner requests snow removal in preparation for opening the next morning. The owner uses their smartphone to enter a prompt and send a request saying, "Please efficiently and automatically remove snow from the store's parking lot in preparation for opening the next morning. Generate an optimal snow removal plan based on the current snow depth information and remotely manage the snow removal equipment." Based on these instructions, the server develops an appropriate plan and arranges for the snow removal work to be carried out reliably.

[0461] In this way, the system provides flexible snow removal services tailored to the needs of each store, creating an environment where business owners can focus on their core business.

[0462] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0463] Step 1:

[0464] The server collects snow depth information from weather data providers via APIs. Inputs include the API key from the connected provider and the store's location coordinates. Based on these inputs, the server retrieves real-time snowfall and weather forecast data, and generates the latest snow depth data as output. This data is stored as information necessary for subsequent snow removal planning.

[0465] Step 2:

[0466] Users submit snow removal requests using a smartphone or computer application. Input includes the store's location, the area requiring snow removal, and the desired snow removal time. The user's input is sent to the server via the application, generating request data which is stored in the server's database. This data is then used by the server to develop a snow removal plan in the next step.

[0467] Step 3:

[0468] The server generates a snow removal plan based on received requests and acquired snow depth data using a generative AI model. Inputs include user request data and collected snow depth information. The server uses machine learning algorithms to calculate the optimal placement of snow removal equipment, start time, and route. As output, detailed snow removal plan data is constructed and sent to the terminal in JSON format.

[0469] Step 4:

[0470] The terminal remotely controls the snow removal equipment based on the snow removal plan received from the server. The input is snow removal plan data provided by the server. The terminal sends instructions to the snow removal equipment via wireless communication, and the equipment automatically performs snow removal according to the planned route. Real-time work progress data is generated as output and fed back to the server.

[0471] Step 5:

[0472] The server monitors the progress transmitted from the terminal and confirms the completion of the snow removal work. The input is progress data continuously provided by the terminal. The server analyzes this data and confirms that the snow removal work has been completed as planned. As output, a completion notification is generated and sent to the user to inform them of the completion of the work.

[0473] Step 6:

[0474] Users receive completion notifications on their smartphones or computers to confirm the completion of snow removal work. The input is a completion notification from the server. Based on this, users can check if the snow removal was performed as desired and make additional requests if necessary. The output is that, having confirmed completion, users are ready to proceed with their work with peace of mind.

[0475] (Application Example 1)

[0476] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0477] In areas with heavy snowfall, a challenge for store owners is efficiently clearing snow from parking lots and in front of stores before opening. In particular, obstruction of access to stores due to snow directly impacts business operations, requiring swift and reliable snow removal. Furthermore, a system is needed that allows store owners to access snow-related information in a timely manner and take appropriate measures.

[0478] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0479] In this invention, the server includes means for collecting snow accumulation information, means for notifying progress information on snow removal plans, and means for receiving snow removal requests that include location information and information on areas requiring snow removal. This enables store managers to obtain snow accumulation information in real time and efficiently manage snow removal operations.

[0480] "Means for collecting snow accumulation information" refers to technologies for obtaining real-time local snow accumulation conditions from weather data providers.

[0481] "Methods for generating snow removal plans" refers to a technology that uses artificial intelligence algorithms based on collected snow accumulation information to determine the most efficient snow removal method.

[0482] "Means for remotely operating snow removal equipment" refers to a technology that allows snow removal equipment to be operated remotely and automatically performed according to a specified plan.

[0483] "A means of monitoring the progress of snow removal equipment and confirming the completion of snow removal work" refers to a technology that tracks whether snow removal equipment is proceeding according to plan and evaluates whether the work has been completed.

[0484] "Means of notifying users of the completion of snow removal work" refers to technology used to inform store managers that snow removal work has been completed.

[0485] "Means for notifying users of the progress of snow removal plans" refers to technologies that provide users with the status of ongoing snow removal work in real time.

[0486] "Means for receiving snow removal requests that include location information and information on areas requiring snow removal" refers to technology that allows users to specify the location and area they wish to have snow removed from, and the system to receive requests based on that information.

[0487] An "artificial intelligence algorithm" is a program used to analyze collected data and determine the optimal snow removal route and timing.

[0488] The system implementing this invention is designed to allow store managers to efficiently manage snow removal operations. At its core are a server that processes data in real time and terminals used by users. The server collects snow depth information from weather data providers (e.g., OpenWeatherMap API) and uses an artificial intelligence algorithm to create an efficient snow removal plan based on this information. This algorithm is implemented using technologies such as Python and TensorFlow. The generated plan is sent to the terminal via WebSocket communication, and the terminal remotely controls the snow removal equipment.

[0489] The device is a smartphone app developed using React Native, through which users send snow removal requests. These requests include location information and information about the area requiring snow removal, which is then sent to the server. The server adjusts the plan based on this information and notifies the user of the progress in real time. A MongoDB database records the progress and plan details, allowing for reference as needed.

[0490] As a concrete example, a hair salon owner checks the overnight snow forecast in preparation for opening the next morning and sends a snow removal request via the app. Based on this request, the server activates the snow removal equipment at the appropriate time the next morning and begins clearing snow from the parking lot. Progress is notified in real time via the app, allowing the owner to stay informed and focus on their normal business operations.

[0491] An example prompt for the generated AI model is, "Develop an AI model that plans efficient snow removal operations based on snow conditions." This is intended to show how the server's AI algorithm should operate.

[0492] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0493] Step 1:

[0494] The server retrieves snow depth information from a weather data provider. This information is obtained in JSON format. The server requests real-time data via an API and stores the response in a database. The input is the API request, and the output is snow depth data.

[0495] Step 2:

[0496] Users submit snow removal requests using a smartphone app. They specify their location and the area requiring snow removal. The device collects this information and sends it to the server. The input is the user's location and area information, and the output is the request data sent to the server.

[0497] Step 3:

[0498] The server generates a snow removal plan based on the received request. Artificial intelligence algorithms are used to analyze collected snow depth data and determine the optimal snow removal route and timing. The AI ​​model receives real-time environmental data as input and outputs snow removal instructions through calculations.

[0499] Step 4:

[0500] The server sends the generated snow removal plan to the terminal, which remotely controls the snow removal equipment. The terminal receives the plan and activates the snow removal equipment based on the presence or absence of snow and the route to be taken. The input is the snow removal plan, and the output is the operation action of the snow removal equipment.

[0501] Step 5:

[0502] The terminal sends the progress of the snow removal work to the server. The progress information is updated in real time and used to evaluate whether the work is progressing according to plan. The input is progress data, and the output is updated information sent to the server.

[0503] Step 6:

[0504] The server confirms the completion of the task and sends a notification to the user. The user's terminal displays a notification that the task is complete, allowing the user to confirm the completion of the snow removal work. The input is the task completion flag, and the output is the notification information sent to the user.

[0505] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0506] This invention enhances the user experience by combining an emotion engine with a system for efficiently performing snow removal work. The system consists of a scheme that includes a server, a terminal, and an engine that analyzes the user's emotions.

[0507] First, the server collects snow depth information from weather data providers and determines the need for snow removal based on that information. Based on this information, it uses a machine learning algorithm to generate an optimal snow removal plan. Users can also send requests using their smartphones or PCs, which include store location information and information about the areas that need snow removal. The server receives these requests and further refines the snow removal plan.

[0508] The generated snow removal plan is sent to a terminal, which then executes the plan by remotely controlling the snow removal equipment. The server constantly monitors progress information from the terminal and verifies whether the snow removal work has been completed.

[0509] The emotion engine, a key feature of this invention, has the function of recognizing the user's emotional state in real time and customizing the content of notifications based on that. This emotion analysis is performed using data such as the user's voice, text input, or facial expressions. For example, if the user is feeling stressed, comforting or encouraging words will be added to the notification message.

[0510] As a concrete example, consider a case where the owner of a shop requests snow removal. The user sends a snow removal request using a smartphone app the night before. The server immediately analyzes the snow accumulation information and sets the optimal snow removal plan by the next morning. At this time, the emotion engine detects that the user has entered a message expressing busyness or fatigue, and the snow removal completion notification includes a special message such as, "Thank you as always. We wish you the best in your preparations today."

[0511] This system allows users to not only check the progress of their work, but also receive support that is tailored to their emotional needs. This makes it possible to increase user satisfaction and provide a better service experience.

[0512] The following describes the processing flow.

[0513] Step 1:

[0514] The server obtains snow depth information in real time from weather data providers to understand current and projected snowfall amounts. This allows for an accurate assessment of snow conditions in each region.

[0515] Step 2:

[0516] Users use their smartphones or PCs to input their store's location information and the time of day when snow removal is needed, and then submit a snow removal request. The request may also include information about their mood on that day.

[0517] Step 3:

[0518] The server receives snow removal requests from users and combines them with snow depth information to generate an optimal snow removal plan. Machine learning algorithms are used to determine the most efficient routes and timing.

[0519] Step 4:

[0520] The emotion engine analyzes user input and past data to recognize the user's current emotional state. Based on this information, it determines which elements should be reflected in notifications and plans.

[0521] Step 5:

[0522] The server sends the generated snow removal plan to the terminal and instructs it to start the snow removal equipment at the specified time. The terminal remotely controls the snow removal equipment to perform the work according to the plan received from the server.

[0523] Step 6:

[0524] The terminal reports the progress of the snow removal equipment to the server in real time. Based on this, the server monitors the progress of the work and confirms that it is proceeding as planned.

[0525] Step 7:

[0526] After the server confirms that snow removal is complete, it sends a message customized by the emotion engine to the user. The user receives this notification and confirms that the snow removal is finished. This notification includes a special message tailored to the user's emotion, reflecting consideration for the user.

[0527] (Example 2)

[0528] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0529] The present invention aims to provide a system that can improve the emotional satisfaction of users while efficiently carrying out snow removal work. Conventional snow removal systems have focused so much on efficiency that notifications to users have become uniform, and appropriate responses have not been made according to the emotions and circumstances of individual users. Therefore, the challenge has been to satisfy both the progress of snow removal work and the emotional needs of users at the same time.

[0530] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0531] In this invention, the server includes means for acquiring and analyzing weather information, means for evaluating the necessity of snow removal work based on the weather information and generating a plan, means for monitoring the progress of snow removal work and confirming its completion, means for analyzing the user's status, and means for adjusting notification content according to the analysis results. This makes it possible to carry out efficient snow removal work while providing notifications that take the user's feelings into consideration.

[0532] "Weather information" refers to data related to weather conditions, including data such as snowfall, temperature, and wind speed.

[0533] A "snow removal plan" is a plan created based on collected weather information, which includes the specific procedures and schedule for snow removal operations.

[0534] A "snow removal device" is a machine or automated system for physically removing accumulated snow.

[0535] "User" refers to an individual or organization that operates a snow removal system or receives information about it.

[0536] "Analysis means" refers to a method or system for analyzing data to extract specific information or trends.

[0537] "Notification content" refers to messages that inform users of the progress and completion status of snow removal work.

[0538] This invention is a system for efficiently carrying out snow removal work and improving the user experience. The specific forms for implementing this system are described below.

[0539] The server first obtains weather information. It uses a common weather forecast API, such as the OpenWeatherMap API, to receive information from weather data providers. The server uses this API to collect information such as snowfall, temperature, and forecast weather, and uses this to assess the need for snow removal. Furthermore, it uses a machine learning library like Scikit-learn to generate a snow removal plan from the collected data. This plan includes optimal routes and work times.

[0540] Users send snow removal requests using their smartphones or PCs. The application uses the Google Maps API and other tools to obtain the user's current location and the area requiring snow removal, and sends this information to the server. The server then uses this information to create a plan and sends instructions to the user's device to control the snow removal equipment.

[0541] The terminal remotely controls the snow removal equipment using information received from the server. A computer such as a Raspberry Pi is used to control motors and sensors to perform the actual snow removal work. The terminal's control program reports the progress to the server in real time.

[0542] Systems equipped with emotion engines are also used. The server can analyze user emotions in real time using Google Cloud Natural Language API or Microsoft Azure Emotion API. Based on the analysis results, the content of notification messages is customized and sent to the user. This allows for flexible responses tailored to the user's emotions, improving satisfaction.

[0543] As a concrete example, consider a scenario where a user enters the message "I'm tired" using a smartphone app. The server detects this and sends a notification upon completion of snow removal, including a special message such as "Thank you as always. We wish you the best in your preparations today." An example of a prompt for the generative AI model would be, "If the user feels tired, please suggest what kind of kind message to add to the notification."

[0544] In this way, the system can provide a convenient and emotionally resonant service to the user.

[0545] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0546] Step 1:

[0547] The server obtains weather information through the weather data provider's API. It uses the API key and request parameters as input and receives weather information (e.g., snowfall, temperature) in JSON format as output. Based on this information, it analyzes snowfall and creates basic data to determine the necessity of snow removal.

[0548] Step 2:

[0549] The server uses Scikit-learn's machine learning algorithms to generate an optimal snow removal plan from acquired weather information. The input is analyzed weather data, and the output is a snow removal plan with optimal routes and work times. In this process, the model is trained using historical weather data and snow removal history, and the plan is adjusted in real time.

[0550] Step 3:

[0551] Users submit snow removal requests from smartphone or PC applications. The input includes the user's location and the required snow removal area, and the output is request data sent to the server. The application utilizes the Google Maps API to accurately obtain the user's location.

[0552] Step 4:

[0553] The server receives requests from users and adjusts the snow removal plan to suit the specific situation. The input is the request data and the generated snow removal plan, and the output is the adjusted plan data. This operation updates the snow removal route while considering priorities.

[0554] Step 5:

[0555] The terminal remotely controls the snow removal equipment using a Raspberry Pi, following a snow removal plan sent from the server. The input is the adjusted snow removal plan, and the output is the actual operation of the equipment. At this stage, motor control and progress monitoring are performed.

[0556] Step 6:

[0557] The server uses an emotion engine to analyze user data in real time. Input consists of user text messages and voice data, and output is information about the user's emotional state. This analysis is performed using the Google Cloud Natural Language API.

[0558] Step 7:

[0559] When the server sends notification messages to users, it customizes the content based on analyzed sentiment information. The input is the sentiment analysis result, and the output is the customized message sent to the user. This process generates sentiment-sensitive messages, such as "Thank you as always. We wish you the best in your preparations today."

[0560] (Application Example 2)

[0561] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0562] While conventional snow removal systems were capable of efficiently generating snow removal plans based on snow conditions and managing progress, they lacked individualized support that considered user emotions, resulting in a failure to improve the user experience. Therefore, there is a need for methods that enhance user satisfaction and provide snow removal notifications in a more personalized way.

[0563] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0564] In this invention, the server includes means for acquiring snow depth data, means for generating a snow removal plan based on the snow depth data, and means for analyzing the user's emotional state and customizing notification content. This enables personalized notifications that respond to the user's emotions.

[0565] "Snow accumulation data" refers to data obtained from weather information providers regarding the amount of snow accumulated in a specific area.

[0566] A "snow removal plan" is a work plan that includes the optimal snow removal method, route, and time, based on collected snow depth data.

[0567] "Snow removal equipment" refers to mechanical devices used for snow removal work that can be remotely controlled.

[0568] "Users" refers to individuals or organizations that use the snow removal system and have the right to receive notifications regarding snow accumulation and the progress of snow removal.

[0569] "Emotional state" refers to the user's psychological or emotional condition, which is analyzed from voice, text, facial expressions, etc.

[0570] "Notification content" refers to information provided to users, including reports on the progress of snow removal and reports sent after the completion of the plan.

[0571] "Means of customization" refers to methods for adjusting notification content based on the user's emotional state and providing personalized information tailored to the user.

[0572] The specific system for implementing this invention is implemented with the following configuration: The server acquires snow depth data from a weather information service and generates a snow removal plan based on this data. The snow removal plan is generated using an algorithm that has learned past snow depth patterns to determine the optimal snow removal route and timing. TensorFlow is used in this process. The generated plan is sent to the terminal.

[0573] The terminal remotely controls snow removal equipment placed around the house. A commercially available IoT device controller is used for control, and sensor data is collected to monitor the work status in real time. Progress information is sent to the server as the snow removal work progresses, and the overall system status is updated when the work is completed.

[0574] Furthermore, to take into account the user's emotional state, the server analyzes voice and text input. Google Cloud Speech-to-Text and AWS Rekognition are used for the analysis. If a message is detected indicating the user is stressed or feeling fatigued, the sentiment analysis engine sends a personalized notification to the user. For example, upon completion of snow removal, the user receives a relaxing message such as, "Thank you for your hard work today. Snow removal has been completed successfully."

[0575] As a concrete example, consider a scenario where a snow removal robot is operated at a home. The user checks the amount of snow on their smartphone and requests snow removal for the day. The server creates an optimal plan and notifies the user of the start time. Sentiment analysis begins during the operation, and after completion, a message based on the results is sent. In this way, it is possible to provide a service that is sensitive to the user's feelings.

[0576] Example prompt: "Please tell me the snow conditions this morning. Show me how to create a message that takes the user's mood into consideration."

[0577] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0578] Step 1:

[0579] The server retrieves snow depth data from a weather information service. This data is obtained using an HTTP request, receiving data in JSON format from the API. The input is regional information, and the output includes data on the current snow depth in that region.

[0580] Step 2:

[0581] The server generates a snow removal plan based on acquired snow depth data. This plan utilizes a machine learning model (built with TensorFlow) based on historical snowfall data to calculate the optimal snow removal route and time. Input consists of snow depth data in JSON format and model parameters; output is the optimized snow removal plan.

[0582] Step 3:

[0583] The generated snow removal plan is sent to the terminal. The terminal remotely controls the snow removal equipment based on the received plan. It issues instructions to the equipment via an IoT controller, enabling real-time control. The input is the snow removal plan, and the output is the operating status of the snow removal equipment.

[0584] Step 4:

[0585] The terminal uses sensor data acquired from the equipment to monitor the progress of the work in progress. The progress is reported to the server in real time. The input is sensor data, and the output is the work progress status.

[0586] Step 5:

[0587] The server confirms the completion of the task based on the work progress. It then generates notification data indicating that the task is complete. The input is the work progress, and the output is the completion notification data.

[0588] Step 6:

[0589] After the task is completed, the server analyzes the user's emotional state. It uses Google Cloud Speech-to-Text and AWS Rekognition to process voice input and facial expression data and perform sentiment analysis. Input is the user's voice or image data, and output is the analyzed emotional state.

[0590] Step 7:

[0591] Based on the analyzed emotional state, the server customizes the notification content. A generative AI model is used to create a relaxing message tailored to the user. The input is emotional state data, and the output is a personalized notification message.

[0592] Step 8:

[0593] Customized notification messages are sent to the user. The messages are displayed on the user's device, providing an emotionally resonant service experience. The input is the notification message, and the output is the user's reception status.

[0594] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0595] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0596] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0597] [Fourth Embodiment]

[0598] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0599] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0600] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0601] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0602] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0603] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0604] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0605] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0606] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0607] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0608] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0609] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0610] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0611] This invention is a system that efficiently and safely handles snow removal work, which is a burden for store managers, and is implemented primarily through interaction between a server, terminals, and users.

[0612] The server collects snow accumulation information in real time from weather data providers. This allows the server to understand the exact snow conditions in the area where the store is located. Store owners, as users, use smartphones or computers to identify the store's location and the areas requiring snow removal, and then send snow removal requests.

[0613] The received request is processed by the server. The server uses an AI algorithm based on the collected snow depth information to generate an efficient snow removal plan. This plan includes the placement of snow removal equipment, the timing of operation, and the route to be taken. The server sends the generated snow removal plan to the terminal, which then remotely operates the snow removal equipment based on it.

[0614] The terminal activates the snow removal equipment based on instructions from the server and automatically performs snow removal along a planned route. During snow removal, the terminal constantly sends progress information to the server, which monitors it.

[0615] Once snow removal is complete, the server verifies the information and sends a completion notification to the user. The user receives this notification on their smartphone or computer and confirms that the snow removal has been successfully completed. This entire process ensures that the parking lot and the area in front of the store are safely cleared of snow by opening time, allowing the business owner to focus on their work.

[0616] As a concrete example, consider a case where the owner of a hair salon requests snow removal in preparation for opening the next morning. During the night when snowfall is expected, the owner sends a request from their smartphone at home. The server plans for the snow removal equipment to be activated at the appropriate time the next morning and operates the snow removal equipment remotely to clear the parking lot. In this way, the owner can concentrate on their normal business without having to spend their busy morning hours on snow removal.

[0617] The following describes the processing flow.

[0618] Step 1:

[0619] The server periodically collects snow information, such as predicted snowfall and current snow depth, using the weather data provider's API. This ensures that the system is always up-to-date on the latest weather conditions.

[0620] Step 2:

[0621] Users enter information such as the store's location and desired snow removal times via a smartphone or PC application and submit a snow removal request. This request reaches the server and is stored in the database.

[0622] Step 3:

[0623] The server determines the need for snow removal based on user requests and the latest snow depth information. If deemed necessary, it uses an AI algorithm to generate an optimal snow removal plan. This plan includes snow removal machine usage time, operating routes, and expected work time.

[0624] Step 4:

[0625] The server distributes the generated snow removal plan to the terminal. The terminal then prepares to activate the snow removal equipment at the specified time based on this plan.

[0626] Step 5:

[0627] The terminal remotely controls the snow removal equipment based on instructions received from the server and begins snow removal work along the designated route. The operating status and progress information of the snow removal equipment are transmitted to the server in real time.

[0628] Step 6:

[0629] The server constantly monitors whether snow removal is progressing normally based on progress information from the terminals. When the work is completed, it records the resources used and the work time, and updates the database.

[0630] Step 7:

[0631] After confirming the completion of snow removal, the server sends a completion notification to the user. Upon receiving this notification, the user can confirm that snow removal has been successfully completed and proceed with opening preparations.

[0632] (Example 1)

[0633] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0634] Currently, store owners in snowy regions face a significant burden of clearing snow from parking lots and storefronts before opening. This task is time-consuming and labor-intensive, causing considerable stress, especially during the busy pre-opening hours. Furthermore, improper snow removal can compromise safety. To address these issues, there is a need for the development of an automated system that performs efficient and safe snow removal.

[0635] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0636] In this invention, the server includes means for collecting snow accumulation information, means for receiving location information and area information where snow removal is required, provided by users, and means for formulating an efficient snow removal process using an artificial intelligence algorithm. This reduces the burden on store managers and enables safe and efficient snow removal work within a given timeframe.

[0637] "Snowfall information" refers to data on snowfall amount and snow depth provided by weather data providers, and is fundamental information for snow removal planning.

[0638] A "snow removal plan" refers to a plan that includes details such as the timing of operation and routes for snow removal equipment necessary for efficient snow removal work.

[0639] A "snow removal device" refers to a machine or mechanism that physically removes accumulated snow and is controlled by remote operation.

[0640] "Remote control" refers to the act of controlling a machine or system using electronic means from a physically distant location.

[0641] "Progress status" refers to data representing the current execution status and progress of snow removal work, which is continuously monitored by the server.

[0642] "Completion confirmed" refers to the process in which the server confirms that all scheduled snow removal operations have been carried out as planned and that the designated area has been properly cleared of snow.

[0643] "Notification" refers to the act of informing users about the completion or progress of snow removal work, and is mainly done via email or push notifications on mobile apps.

[0644] "Location information" refers to digital data that indicates a specific point on Earth, and is often expressed using a geographic coordinate system.

[0645] "Area information" refers to data about specific areas where snow removal is required, and serves as a basis for formulating snow removal plans.

[0646] "Machine learning technology" refers to techniques that enable computers to autonomously recognize patterns using large amounts of data and generate predictive models.

[0647] An "artificial intelligence algorithm" is a type of computational procedure executed by a computer, designed to solve specific problems while mimicking human knowledge and reasoning.

[0648] This invention is an automated system for reducing the burden on store managers and efficiently performing snow removal. The system is implemented through the interaction of a server, terminals, and users.

[0649] The server plays a central role in collecting snow depth information and generating effective snow removal plans. Specifically, the server retrieves snow depth information in real time from weather data providers. For example, it periodically collects snowfall and weather forecast data from providers using APIs. This data is essential for the server to efficiently create snow removal plans. The server also maintains a database to manage user request information and past snow removal history.

[0650] Users submit snow removal requests using devices such as smartphones or computers. A dedicated application allows users to easily specify the store's location and the area to be cleared. Users can also request snow removal at the optimal time by entering their desired time slot. After the snow removal is complete, users can check the status through the application.

[0651] The terminal remotely controls the snow removal equipment based on the snow removal plan transmitted from the server. The terminal uses wireless communication to send instructions directly to the snow removal equipment on site. Based on the instructions received by the terminal, the snow removal equipment automatically follows the planned route and removes snow from the designated area. The terminal feeds back the progress of the snow removal work to the server during the operation, and the server uses this information to monitor the work in real time.

[0652] The generative AI model is the heart of the system, generating efficient snow removal plans on the server. This algorithm uses machine learning techniques to consider weather conditions and terrain data to determine the optimal route and timing for snow removal.

[0653] A concrete example is when a hair salon owner requests snow removal in preparation for opening the next morning. The owner uses their smartphone to enter a prompt and send a request saying, "Please efficiently and automatically remove snow from the store's parking lot in preparation for opening the next morning. Generate an optimal snow removal plan based on the current snow depth information and remotely manage the snow removal equipment." Based on these instructions, the server develops an appropriate plan and arranges for the snow removal work to be carried out reliably.

[0654] In this way, the system provides flexible snow removal services tailored to the needs of each store, creating an environment where business owners can focus on their core business.

[0655] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0656] Step 1:

[0657] The server collects snow depth information from weather data providers via APIs. Inputs include the API key from the connected provider and the store's location coordinates. Based on these inputs, the server retrieves real-time snowfall and weather forecast data, and generates the latest snow depth data as output. This data is stored as information necessary for subsequent snow removal planning.

[0658] Step 2:

[0659] Users submit snow removal requests using a smartphone or computer application. Input includes the store's location, the area requiring snow removal, and the desired snow removal time. The user's input is sent to the server via the application, generating request data which is stored in the server's database. This data is then used by the server to develop a snow removal plan in the next step.

[0660] Step 3:

[0661] The server generates a snow removal plan based on received requests and acquired snow depth data using a generative AI model. Inputs include user request data and collected snow depth information. The server uses machine learning algorithms to calculate the optimal placement of snow removal equipment, start time, and route. As output, detailed snow removal plan data is constructed and sent to the terminal in JSON format.

[0662] Step 4:

[0663] The terminal remotely controls the snow removal equipment based on the snow removal plan received from the server. The input is snow removal plan data provided by the server. The terminal sends instructions to the snow removal equipment via wireless communication, and the equipment automatically performs snow removal according to the planned route. Real-time work progress data is generated as output and fed back to the server.

[0664] Step 5:

[0665] The server monitors the progress transmitted from the terminal and confirms the completion of the snow removal work. The input is progress data continuously provided by the terminal. The server analyzes this data and confirms that the snow removal work has been completed as planned. As output, a completion notification is generated and sent to the user to inform them of the completion of the work.

[0666] Step 6:

[0667] Users receive completion notifications on their smartphones or computers to confirm the completion of snow removal work. The input is a completion notification from the server. Based on this, users can check if the snow removal was performed as desired and make additional requests if necessary. The output is that, having confirmed completion, users are ready to proceed with their work with peace of mind.

[0668] (Application Example 1)

[0669] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0670] In areas with heavy snowfall, a challenge for store owners is efficiently clearing snow from parking lots and in front of stores before opening. In particular, obstruction of access to stores due to snow directly impacts business operations, requiring swift and reliable snow removal. Furthermore, a system is needed that allows store owners to access snow-related information in a timely manner and take appropriate measures.

[0671] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0672] In this invention, the server includes means for collecting snow accumulation information, means for notifying progress information on snow removal plans, and means for receiving snow removal requests that include location information and information on areas requiring snow removal. This enables store managers to obtain snow accumulation information in real time and efficiently manage snow removal operations.

[0673] "Means for collecting snow accumulation information" refers to technologies for obtaining real-time local snow accumulation conditions from weather data providers.

[0674] "Methods for generating snow removal plans" refers to a technology that uses artificial intelligence algorithms based on collected snow accumulation information to determine the most efficient snow removal method.

[0675] "Means for remotely operating snow removal equipment" refers to a technology that allows snow removal equipment to be operated remotely and automatically performed according to a specified plan.

[0676] "A means of monitoring the progress of snow removal equipment and confirming the completion of snow removal work" refers to a technology that tracks whether snow removal equipment is proceeding according to plan and evaluates whether the work has been completed.

[0677] "Means of notifying users of the completion of snow removal work" refers to technology used to inform store managers that snow removal work has been completed.

[0678] "Means for notifying users of the progress of snow removal plans" refers to technologies that provide users with the status of ongoing snow removal work in real time.

[0679] "Means for receiving snow removal requests that include location information and information on areas requiring snow removal" refers to technology that allows users to specify the location and area they wish to have snow removed from, and the system to receive requests based on that information.

[0680] An "artificial intelligence algorithm" is a program used to analyze collected data and determine the optimal snow removal route and timing.

[0681] The system implementing this invention is designed to allow store managers to efficiently manage snow removal operations. At its core are a server that processes data in real time and terminals used by users. The server collects snow depth information from weather data providers (e.g., OpenWeatherMap API) and uses an artificial intelligence algorithm to create an efficient snow removal plan based on this information. This algorithm is implemented using technologies such as Python and TensorFlow. The generated plan is sent to the terminal via WebSocket communication, and the terminal remotely controls the snow removal equipment.

[0682] The device is a smartphone app developed using React Native, through which users send snow removal requests. These requests include location information and information about the area requiring snow removal, which is then sent to the server. The server adjusts the plan based on this information and notifies the user of the progress in real time. A MongoDB database records the progress and plan details, allowing for reference as needed.

[0683] As a concrete example, a hair salon owner checks the overnight snow forecast in preparation for opening the next morning and sends a snow removal request via the app. Based on this request, the server activates the snow removal equipment at the appropriate time the next morning and begins clearing snow from the parking lot. Progress is notified in real time via the app, allowing the owner to stay informed and focus on their normal business operations.

[0684] An example prompt for the generated AI model is, "Develop an AI model that plans efficient snow removal operations based on snow conditions." This is intended to show how the server's AI algorithm should operate.

[0685] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0686] Step 1:

[0687] The server retrieves snow depth information from a weather data provider. This information is obtained in JSON format. The server requests real-time data via an API and stores the response in a database. The input is the API request, and the output is snow depth data.

[0688] Step 2:

[0689] Users submit snow removal requests using a smartphone app. They specify their location and the area requiring snow removal. The device collects this information and sends it to the server. The input is the user's location and area information, and the output is the request data sent to the server.

[0690] Step 3:

[0691] The server generates a snow removal plan based on the received request. Artificial intelligence algorithms are used to analyze collected snow depth data and determine the optimal snow removal route and timing. The AI ​​model receives real-time environmental data as input and outputs snow removal instructions through calculations.

[0692] Step 4:

[0693] The server sends the generated snow removal plan to the terminal, which remotely controls the snow removal equipment. The terminal receives the plan and activates the snow removal equipment based on the presence or absence of snow and the route to be taken. The input is the snow removal plan, and the output is the operation action of the snow removal equipment.

[0694] Step 5:

[0695] The terminal sends the progress of the snow removal work to the server. The progress information is updated in real time and used to evaluate whether the work is progressing according to plan. The input is progress data, and the output is updated information sent to the server.

[0696] Step 6:

[0697] The server confirms the completion of the task and sends a notification to the user. The user's terminal displays a notification that the task is complete, allowing the user to confirm the completion of the snow removal work. The input is the task completion flag, and the output is the notification information sent to the user.

[0698] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0699] This invention enhances the user experience by combining an emotion engine with a system for efficiently performing snow removal work. The system consists of a scheme that includes a server, a terminal, and an engine that analyzes the user's emotions.

[0700] First, the server collects snow depth information from weather data providers and determines the need for snow removal based on that information. Based on this information, it uses a machine learning algorithm to generate an optimal snow removal plan. Users can also send requests using their smartphones or PCs, which include store location information and information about the areas that need snow removal. The server receives these requests and further refines the snow removal plan.

[0701] The generated snow removal plan is sent to a terminal, which then executes the plan by remotely controlling the snow removal equipment. The server constantly monitors progress information from the terminal and verifies whether the snow removal work has been completed.

[0702] The emotion engine, a key feature of this invention, has the function of recognizing the user's emotional state in real time and customizing the content of notifications based on that. This emotion analysis is performed using data such as the user's voice, text input, or facial expressions. For example, if the user is feeling stressed, comforting or encouraging words will be added to the notification message.

[0703] As a concrete example, consider a case where the owner of a shop requests snow removal. The user sends a snow removal request using a smartphone app the night before. The server immediately analyzes the snow accumulation information and sets the optimal snow removal plan by the next morning. At this time, the emotion engine detects that the user has entered a message expressing busyness or fatigue, and the snow removal completion notification includes a special message such as, "Thank you as always. We wish you the best in your preparations today."

[0704] This system allows users to not only check the progress of their work, but also receive support that is tailored to their emotional needs. This makes it possible to increase user satisfaction and provide a better service experience.

[0705] The following describes the processing flow.

[0706] Step 1:

[0707] The server obtains snow depth information in real time from weather data providers to understand current and projected snowfall amounts. This allows for an accurate assessment of snow conditions in each region.

[0708] Step 2:

[0709] Users use their smartphones or PCs to input their store's location information and the time of day when snow removal is needed, and then submit a snow removal request. The request may also include information about their mood on that day.

[0710] Step 3:

[0711] The server receives snow removal requests from users and combines them with snow depth information to generate an optimal snow removal plan. Machine learning algorithms are used to determine the most efficient routes and timing.

[0712] Step 4:

[0713] The emotion engine analyzes user input and past data to recognize the user's current emotional state. Based on this information, it determines which elements should be reflected in notifications and plans.

[0714] Step 5:

[0715] The server sends the generated snow removal plan to the terminal and instructs it to start the snow removal equipment at the specified time. The terminal remotely controls the snow removal equipment to perform the work according to the plan received from the server.

[0716] Step 6:

[0717] The terminal reports the progress of the snow removal equipment to the server in real time. Based on this, the server monitors the progress of the work and confirms that it is proceeding as planned.

[0718] Step 7:

[0719] After the server confirms that snow removal is complete, it sends a message customized by the emotion engine to the user. The user receives this notification and confirms that the snow removal is finished. This notification includes a special message tailored to the user's emotion, reflecting consideration for the user.

[0720] (Example 2)

[0721] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0722] The present invention aims to provide a system that can improve the emotional satisfaction of users while efficiently carrying out snow removal work. Conventional snow removal systems have focused so much on efficiency that notifications to users have become uniform, and appropriate responses have not been made according to the emotions and circumstances of individual users. Therefore, the challenge has been to satisfy both the progress of snow removal work and the emotional needs of users at the same time.

[0723] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0724] In this invention, the server includes means for acquiring and analyzing weather information, means for evaluating the necessity of snow removal work based on the weather information and generating a plan, means for monitoring the progress of snow removal work and confirming its completion, means for analyzing the user's status, and means for adjusting notification content according to the analysis results. This makes it possible to carry out efficient snow removal work while providing notifications that take the user's feelings into consideration.

[0725] "Weather information" refers to data related to weather conditions, including data such as snowfall, temperature, and wind speed.

[0726] A "snow removal plan" is a plan created based on collected weather information, which includes the specific procedures and schedule for snow removal operations.

[0727] A "snow removal device" is a machine or automated system for physically removing accumulated snow.

[0728] "User" refers to an individual or organization that operates a snow removal system or receives information about it.

[0729] "Analysis means" refers to a method or system for analyzing data to extract specific information or trends.

[0730] "Notification content" refers to messages that inform users of the progress and completion status of snow removal work.

[0731] This invention is a system for efficiently carrying out snow removal work and improving the user experience. The specific forms for implementing this system are described below.

[0732] The server first obtains weather information. It uses a common weather forecast API, such as the OpenWeatherMap API, to receive information from weather data providers. The server uses this API to collect information such as snowfall, temperature, and forecast weather, and uses this to assess the need for snow removal. Furthermore, it uses a machine learning library like Scikit-learn to generate a snow removal plan from the collected data. This plan includes optimal routes and work times.

[0733] Users send snow removal requests using their smartphones or PCs. The application uses the Google Maps API and other tools to obtain the user's current location and the area requiring snow removal, and sends this information to the server. The server then uses this information to create a plan and sends instructions to the user's device to control the snow removal equipment.

[0734] The terminal remotely controls the snow removal equipment using information received from the server. A computer such as a Raspberry Pi is used to control motors and sensors to perform the actual snow removal work. The terminal's control program reports the progress to the server in real time.

[0735] Systems equipped with emotion engines are also used. The server can analyze user emotions in real time using Google Cloud Natural Language API or Microsoft Azure Emotion API. Based on the analysis results, the content of notification messages is customized and sent to the user. This allows for flexible responses tailored to the user's emotions, improving satisfaction.

[0736] As a concrete example, consider a scenario where a user enters the message "I'm tired" using a smartphone app. The server detects this and sends a notification upon completion of snow removal, including a special message such as "Thank you as always. We wish you the best in your preparations today." An example of a prompt for the generative AI model would be, "If the user feels tired, please suggest what kind of kind message to add to the notification."

[0737] In this way, the system can provide a convenient and emotionally resonant service to the user.

[0738] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0739] Step 1:

[0740] The server obtains weather information through the weather data provider's API. It uses the API key and request parameters as input and receives weather information (e.g., snowfall, temperature) in JSON format as output. Based on this information, it analyzes snowfall and creates basic data to determine the necessity of snow removal.

[0741] Step 2:

[0742] The server uses Scikit-learn's machine learning algorithms to generate an optimal snow removal plan from acquired weather information. The input is analyzed weather data, and the output is a snow removal plan with optimal routes and work times. In this process, the model is trained using historical weather data and snow removal history, and the plan is adjusted in real time.

[0743] Step 3:

[0744] Users submit snow removal requests from smartphone or PC applications. The input includes the user's location and the required snow removal area, and the output is request data sent to the server. The application utilizes the Google Maps API to accurately obtain the user's location.

[0745] Step 4:

[0746] The server receives requests from users and adjusts the snow removal plan to suit the specific situation. The input is the request data and the generated snow removal plan, and the output is the adjusted plan data. This operation updates the snow removal route while considering priorities.

[0747] Step 5:

[0748] The terminal remotely controls the snow removal equipment using a Raspberry Pi, following a snow removal plan sent from the server. The input is the adjusted snow removal plan, and the output is the actual operation of the equipment. At this stage, motor control and progress monitoring are performed.

[0749] Step 6:

[0750] The server uses an emotion engine to analyze user data in real time. Input consists of user text messages and voice data, and output is information about the user's emotional state. This analysis is performed using the Google Cloud Natural Language API.

[0751] Step 7:

[0752] When the server sends notification messages to users, it customizes the content based on analyzed sentiment information. The input is the sentiment analysis result, and the output is the customized message sent to the user. This process generates sentiment-sensitive messages, such as "Thank you as always. We wish you the best in your preparations today."

[0753] (Application Example 2)

[0754] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0755] While conventional snow removal systems were capable of efficiently generating snow removal plans based on snow conditions and managing progress, they lacked individualized support that considered user emotions, resulting in a failure to improve the user experience. Therefore, there is a need for methods that enhance user satisfaction and provide snow removal notifications in a more personalized way.

[0756] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0757] In this invention, the server includes means for acquiring snow depth data, means for generating a snow removal plan based on the snow depth data, and means for analyzing the user's emotional state and customizing notification content. This enables personalized notifications that respond to the user's emotions.

[0758] "Snow accumulation data" refers to data obtained from weather information providers regarding the amount of snow accumulated in a specific area.

[0759] A "snow removal plan" is a work plan that includes the optimal snow removal method, route, and time, based on collected snow depth data.

[0760] "Snow removal equipment" refers to mechanical devices used for snow removal work that can be remotely controlled.

[0761] "Users" refers to individuals or organizations that use the snow removal system and have the right to receive notifications regarding snow accumulation and the progress of snow removal.

[0762] "Emotional state" refers to the user's psychological or emotional condition, which is analyzed from voice, text, facial expressions, etc.

[0763] "Notification content" refers to information provided to users, including reports on the progress of snow removal and reports sent after the completion of the plan.

[0764] "Means of customization" refers to methods for adjusting notification content based on the user's emotional state and providing personalized information tailored to the user.

[0765] The specific system for implementing this invention is implemented with the following configuration: The server acquires snow depth data from a weather information service and generates a snow removal plan based on this data. The snow removal plan is generated using an algorithm that has learned past snow depth patterns to determine the optimal snow removal route and timing. TensorFlow is used in this process. The generated plan is sent to the terminal.

[0766] The terminal remotely controls snow removal equipment placed around the house. A commercially available IoT device controller is used for control, and sensor data is collected to monitor the work status in real time. Progress information is sent to the server as the snow removal work progresses, and the overall system status is updated when the work is completed.

[0767] Furthermore, to take into account the user's emotional state, the server analyzes voice and text input. Google Cloud Speech-to-Text and AWS Rekognition are used for the analysis. If a message is detected indicating the user is stressed or feeling fatigued, the sentiment analysis engine sends a personalized notification to the user. For example, upon completion of snow removal, the user receives a relaxing message such as, "Thank you for your hard work today. Snow removal has been completed successfully."

[0768] As a concrete example, consider a scenario where a snow removal robot is operated at a home. The user checks the amount of snow on their smartphone and requests snow removal for the day. The server creates an optimal plan and notifies the user of the start time. Sentiment analysis begins during the operation, and after completion, a message based on the results is sent. In this way, it is possible to provide a service that is sensitive to the user's feelings.

[0769] Example prompt: "Please tell me the snow conditions this morning. Show me how to create a message that takes the user's mood into consideration."

[0770] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0771] Step 1:

[0772] The server retrieves snow depth data from a weather information service. This data is obtained using an HTTP request, receiving data in JSON format from the API. The input is regional information, and the output includes data on the current snow depth in that region.

[0773] Step 2:

[0774] The server generates a snow removal plan based on acquired snow depth data. This plan utilizes a machine learning model (built with TensorFlow) based on historical snowfall data to calculate the optimal snow removal route and time. Input consists of snow depth data in JSON format and model parameters; output is the optimized snow removal plan.

[0775] Step 3:

[0776] The generated snow removal plan is sent to the terminal. The terminal remotely controls the snow removal equipment based on the received plan. It issues instructions to the equipment via an IoT controller, enabling real-time control. The input is the snow removal plan, and the output is the operating status of the snow removal equipment.

[0777] Step 4:

[0778] The terminal uses sensor data acquired from the equipment to monitor the progress of the work in progress. The progress is reported to the server in real time. The input is sensor data, and the output is the work progress status.

[0779] Step 5:

[0780] The server confirms the completion of the task based on the work progress. It then generates notification data indicating that the task is complete. The input is the work progress, and the output is the completion notification data.

[0781] Step 6:

[0782] After the task is completed, the server analyzes the user's emotional state. It uses Google Cloud Speech-to-Text and AWS Rekognition to process voice input and facial expression data and perform sentiment analysis. Input is the user's voice or image data, and output is the analyzed emotional state.

[0783] Step 7:

[0784] Based on the analyzed emotional state, the server customizes the notification content. A generative AI model is used to create a relaxing message tailored to the user. The input is emotional state data, and the output is a personalized notification message.

[0785] Step 8:

[0786] Customized notification messages are sent to the user. The messages are displayed on the user's device, providing an emotionally resonant service experience. The input is the notification message, and the output is the user's reception status.

[0787] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0788] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0789] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0790] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0791] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0792] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0793] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0794] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0795] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0796] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0797] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0798] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0799] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0800] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0801] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0802] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0803] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0804] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0805] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0806] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0807] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0808] The following is further disclosed regarding the embodiments described above.

[0809] (Claim 1)

[0810] Means of collecting snow depth information,

[0811] Means for generating a snow removal plan based on the aforementioned snow accumulation information,

[0812] Means for remotely operating snow removal equipment in accordance with the aforementioned snow removal plan,

[0813] A means for monitoring the progress of the snow removal device and confirming the completion of the snow removal work,

[0814] A means for notifying the user of the completion of the snow removal work,

[0815] A system that includes this.

[0816] (Claim 2)

[0817] The system according to claim 1, further comprising means for receiving snow removal requests that include store location information and information on areas requiring snow removal.

[0818] (Claim 3)

[0819] The system according to claim 1, wherein the means for generating the snow removal plan determines the optimal route and timing using a machine learning algorithm.

[0820] "Example 1"

[0821] (Claim 1)

[0822] Means of collecting snow depth information,

[0823] Means for generating a snow removal plan based on the aforementioned snow accumulation information,

[0824] Means for remotely operating snow removal equipment in accordance with the aforementioned snow removal plan,

[0825] A means for monitoring the progress of the snow removal device and confirming the completion of the snow removal work,

[0826] A means for notifying the user of the completion of the snow removal work,

[0827] A means for receiving location information and area information where snow removal is required, provided by the user,

[0828] A means of formulating an efficient snow removal process using machine learning technology,

[0829] A system that includes this.

[0830] (Claim 2)

[0831] The system according to claim 1, further comprising means for inputting and transmitting snow removal requests through user operation.

[0832] (Claim 3)

[0833] The system according to claim 1, characterized in that the means for generating a snow removal plan determines an optimized route and working time using an artificial intelligence algorithm.

[0834] "Application Example 1"

[0835] (Claim 1)

[0836] Means of collecting snow depth information,

[0837] Means for generating a snow removal plan based on the aforementioned snow accumulation information,

[0838] Means for remotely operating snow removal equipment in accordance with the aforementioned snow removal plan,

[0839] A means for monitoring the progress of the snow removal device and confirming the completion of the snow removal work,

[0840] A means for notifying the user of the completion of the snow removal work,

[0841] A means for notifying the progress of the snow removal plan,

[0842] A system that includes this.

[0843] (Claim 2)

[0844] The system according to claim 1, further comprising means for receiving snow removal requests including location information and information on areas requiring snow removal.

[0845] (Claim 3)

[0846] The system according to claim 1, wherein the means for generating the snow removal plan determines the optimal route and timing using an artificial intelligence algorithm.

[0847] "Example 2 of combining an emotion engine"

[0848] (Claim 1)

[0849] Means for acquiring and analyzing weather information,

[0850] A means for evaluating the necessity of snow removal work based on the aforementioned weather information and generating a plan,

[0851] A means for controlling the snow removal equipment based on the aforementioned snow removal plan,

[0852] A means for monitoring the progress of the snow removal work and confirming its completion,

[0853] A means of analyzing the user's condition,

[0854] Means for adjusting the notification content according to the analysis results,

[0855] Means for sending the aforementioned notification to the user,

[0856] A system that includes this.

[0857] (Claim 2)

[0858] The system according to claim 1, further comprising means for receiving store or local information.

[0859] (Claim 3)

[0860] The system according to claim 1, characterized in that the means for generating the plan determines the optimal route and work time using a learning algorithm.

[0861] "Application example 2 when combining with an emotional engine"

[0862] (Claim 1)

[0863] Methods for acquiring snow depth data,

[0864] A means for generating a snow removal plan based on the aforementioned snow depth data,

[0865] Means for remotely controlling snow removal equipment in accordance with the aforementioned snow removal plan,

[0866] A means for monitoring the progress of the snow removal equipment and confirming the completion of the snow removal work,

[0867] A means for notifying the user of the completion of the snow removal work,

[0868] A means of analyzing the user's emotional state and customizing notification content,

[0869] A system that includes this.

[0870] (Claim 2)

[0871] The system according to claim 1, further comprising means for receiving snow removal requests including location information of the activity base and information of the area requiring snow removal.

[0872] (Claim 3)

[0873] The system according to claim 1, wherein the means for generating the snow removal plan determines the optimal route and timing using a learning algorithm. [Explanation of Symbols]

[0874] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of collecting snow depth information, Means for generating a snow removal plan based on the aforementioned snow accumulation information, Means for remotely operating snow removal equipment in accordance with the aforementioned snow removal plan, A means for monitoring the progress of the snow removal device and confirming the completion of the snow removal work, A means for notifying the user of the completion of the snow removal work, A means for notifying the progress of the snow removal plan, A system that includes this.

2. The system according to claim 1, further comprising means for receiving snow removal requests including location information and information on areas requiring snow removal.

3. The system according to claim 1, wherein the means for generating the snow removal plan determines the optimal route and timing using an artificial intelligence algorithm.

Images