system

Abstract

This system provides comprehensive data-driven solutions to challenges in urban areas, such as frequent traffic congestion, excessive energy consumption, and a lack of efficient waste management. [Solution] A system comprising means for acquiring location information and search data from a location information service provider, means for acquiring payment data from an electronic payment service provider, means for integrating the acquired location information, search data, and payment data and performing data cleaning, means for applying a machine learning algorithm to predict pedestrian flow based on the integrated data, means for proposing optimization of transportation operations based on pedestrian flow predictions, means for performing demand forecasting in commercial facilities and adjusting energy-saving measures, means for collecting wide-area data using a high-altitude platform and evaluating disaster risk, and means for providing personalized action guidelines to users.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to provide a comprehensive solution using data for problems such as frequent traffic jams in urban areas, excessive consumption of energy resources, and lack of efficient waste management. In addition, it is necessary to improve the accuracy of demand prediction for public transportation, grasp customer needs in commercial facilities, effectively promote energy-saving measures, and strengthen disaster response capabilities. Furthermore, in order to address the problem of lack of guidance on specific environmental protection actions for citizens, the present invention provides means for solving these problems.

Means for Solving the Problems

[0005] The present invention provides a system comprising means for acquiring location information from a location information service provider, means for acquiring search data from an information retrieval service provider, means for acquiring payment data from an electronic payment service provider, and means for integrating these data and performing data cleaning. Furthermore, it includes means for applying machine learning algorithms based on the integrated data to predict pedestrian flow, and means for proposing optimization of transportation operations based on pedestrian flow predictions. By including means for coordinating demand forecasting and energy conservation measures in commercial facilities, disaster risk assessment through the collection of wide-area data using a high-altitude platform, and providing personalized action guidelines to users, it is possible to optimize urban functions and improve sustainability.

[0006] A "location information service provider" is a general term for organizations and systems that provide data related to a user's location.

[0007] An "information retrieval service provider" is a general term for organizations and systems that provide users with information retrieval functions and collect data such as search history.

[0008] An "electronic payment service provider" is a general term for organizations and systems that provide digital payment and settlement services and collect transaction data.

[0009] "Data cleaning" is the process of removing inaccurate or incomplete information from collected data and organizing it into a standardized format.

[0010] A "machine learning algorithm" is a set of rules and procedures that a computer uses to recognize and identify patterns based on empirical data.

[0011] "Human flow forecasting" is a method aimed at predicting and analyzing the movement and gatherings of people.

[0012] "Optimizing public transportation operations" refers to the process of efficiently and effectively adjusting the schedules and routes of public transportation based on predictions.

[0013] A "high-altitude platform" is a device or system, such as an unmanned aerial vehicle or balloon, that is located at a high altitude above the ground and is capable of collecting data over a wide area.

[0014] "Personalized behavioral guidelines" refer to recommended actions and suggestions provided individually based on the specific conditions and needs of each user. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the language used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention provides a data-centric smart city solution system aimed at optimizing urban functions. This system acquires necessary data from various service providers and analyzes it in an integrated format, enabling improved transportation efficiency, demand forecasting for commercial facilities, measurement of the effectiveness of energy-saving measures, and disaster response based on wide-area data.

[0037] Data acquisition and integration

[0038] The server periodically collects data from location service providers, information retrieval service providers, and electronic payment service providers. Data collection is performed using APIs, and the obtained information is stored in a database on the server in real time. Location data includes longitude, latitude, and time data, while retrieval data includes search queries and related time zones. Payment data includes transaction amounts and time information.

[0039] Data analysis and forecasting

[0040] The server integrates the collected data and removes inaccurate data through data cleaning. This improves data accuracy and converts it into a format that can be input into the pedestrian flow prediction model. Pedestrian flow prediction is performed using machine learning algorithms to predict future human movement based on historical data.

[0041] Optimization of public transport operations

[0042] The server utilizes the pedestrian flow prediction results to optimize transportation schedules. For example, it suggests to the transportation management system that the number of buses should be increased during predicted peak hours. Furthermore, if congestion is expected in a specific area, it provides feedback to subway and train service information to improve passenger convenience.

[0043] Demand forecasting and energy conservation for commercial facilities

[0044] The terminal acquires information from sensors installed in commercial facilities and combines it with payment data to predict demand. Based on the demand forecast, measures to reduce energy consumption during business hours are automatically applied. For example, temperature control is appropriately managed during peak hours to reduce power consumption.

[0045] Disaster response and environmental monitoring

[0046] The server analyzes environmental data by integrating wide-area data collected by high-altitude platforms with data from ground-based sensors. This allows it to issue warnings before disasters such as floods and heavy rains occur, prompting local governments and related organizations to take countermeasures.

[0047] Providing services to users

[0048] Users receive personalized behavioral guidelines based on their individual data through a provided smartphone app. This app notifies users of suggestions for appropriate travel times and energy-saving behaviors, improving their quality of life. For example, it displays suggestions for travel routes that avoid congestion and specific instructions to help save energy.

[0049] The system of this invention will help improve the efficiency of critical infrastructure in cities and contribute to the realization of a sustainable urban environment.

[0050] The following describes the processing flow.

[0051] Step 1:

[0052] The server retrieves data in real time via APIs from location information service providers, information retrieval service providers, and electronic payment service providers. This allows a wide range of data, such as longitude, latitude, search queries, and transaction amounts, to be stored in the server's database.

[0053] Step 2:

[0054] The server integrates the collected data and removes inaccurate and duplicate data through a data cleaning process. This step also standardizes the format and eliminates outliers, preparing the data for analysis.

[0055] Step 3:

[0056] The server uses clean data and applies machine learning algorithms to predict future pedestrian traffic. It analyzes past trends, builds predictive models based on people's movement patterns, and forecasts pedestrian traffic for the next 24 hours and week.

[0057] Step 4:

[0058] The server uses the prediction results to make suggestions for optimizing transportation schedules. These suggestions are sent to the traffic management system, which then develops operational plans to provide additional traffic during peak hours.

[0059] Step 5:

[0060] The terminal collects real-time data from sensors installed in commercial facilities and combines it with payment data to predict demand. Based on the demand forecast, the facility's air conditioning and lighting are adjusted to improve energy efficiency.

[0061] Step 6:

[0062] The server acquires wide-area data through a high-altitude platform and integrates it with ground sensor data to conduct environmental risk assessments. By combining this with meteorological data, it enables the early issuance of warnings for potential disasters.

[0063] Step 7:

[0064] Users receive personalized suggestions based on predicted pedestrian flow data and energy consumption through a smartphone app. The app provides users with optimal travel routes and specific guidance for energy saving.

[0065] (Example 1)

[0066] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0067] The increasing complexity of urban functions and population concentration have made it difficult to efficiently operate transportation infrastructure and commercial facilities. In addition, there are challenges such as the need for early prediction of disaster risks and optimal energy consumption. Conventional technologies have not provided an efficient system to comprehensively analyze this data and translate it into appropriate actions.

[0068] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0069] In this invention, the server includes means for acquiring location information from location information data providers, means for acquiring search information from data search providers, and means for acquiring transaction information from electronic transaction information providers. This enables the integrated analysis of urban information, optimization of transportation infrastructure, demand forecasting for commercial facilities, disaster risk assessment, and provision of personalized action plans.

[0070] A "location data provider" refers to an entity that provides location information services, typically responsible for accurately measuring the location of a target and providing the results to an external system.

[0071] A "data search provider" is an entity that makes information and data searchable and provides relevant information based on the user's search query.

[0072] An "electronic transaction information provider" is an entity that manages and provides information on electronically conducted transactions, recording details such as transaction amounts and dates, and sharing them with third parties.

[0073] "People movement" refers to the patterns and flows of people's movements in a specific region or time period, and is analyzed based on observational data.

[0074] "Optimization of transportation infrastructure operations" refers to the act of optimizing the operation plan in order to maximize the efficiency of the transportation system, and mainly includes adjusting operations in response to user demand.

[0075] "Demand forecasting for commercial facilities" refers to predicting the demand necessary for future sales activities and service provision at commercial facilities, and is analyzed based on past data.

[0076] "Resource consumption reduction measures" refer to measures and methods aimed at reducing the use of energy and materials, and are implemented with the goal of efficient operation.

[0077] A "high-altitude observation system" typically refers to a system that collects data over a wide area from an advanced position, such as an aircraft or satellite, enabling observations over a wider area than those near the Earth's surface.

[0078] "Assessing the risk of disaster" means quantitatively or qualitatively analyzing the risk of natural disasters occurring and predicting their impact.

[0079] An "individualized action plan" refers to suggestions and instructions for actions optimized according to the specific needs and circumstances of a particular user, with the aim of providing personalized information.

[0080] To implement this invention, the server first collects necessary data from location data providers, data retrieval providers, and electronic transaction information providers. Specifically, it periodically sends HTTP requests via an API, parses the responses received, and stores them in a database. This process is typically carried out using Python or Java® libraries. The server then integrates this data, cleans it using the Pandas library, and ensures its accuracy.

[0081] Next, the server uses the collected clean data to run a machine learning algorithm equipped with a generative AI model. In predicting people's movement, the server utilizes the scikit-learn library to create a predictive model based on historical data. This model enables optimization of transportation infrastructure operations, demand forecasting in commercial facilities, and reduction of resource consumption.

[0082] For example, prompts such as "Please tell me the best route to get to the office at 10 AM next Monday" or "Please suggest ways to reduce energy consumption at commercial facilities this weekend" are presented. Based on these prompts, the user can receive optimized information provided by the server.

[0083] Furthermore, the terminal acquires environmental information from IoT sensors installed in commercial facilities and performs real-time data analysis. Using hardware such as Raspberry Pi, it collects data and then optimizes the control of air conditioning and lighting on the terminal. Users can receive personalized behavioral guidelines through a smartphone app, which is expected to improve their quality of life. By providing users with optimal guidance tailored to specific times and situations, this system can contribute to urban sustainability.

[0084] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0085] Step 1: Data Collection

[0086] The server retrieves data from location data providers, data search providers, and electronic transaction information providers. As input, the server sends HTTP requests through each provider's API. As output, location information, search information, and transaction information are retrieved from these providers in JSON format and stored in the database. The server uses the Python Requests library to process requests and analyze the obtained data.

[0087] Step 2: Data Integration and Cleaning

[0088] The server integrates the collected data and removes inaccurate data. Raw data from multiple data sources is provided as input. The output is a clean, integrated dataset. Specifically, the Pandas library is used to filter out unnecessary data and impute missing values ​​to format the data.

[0089] Step 3: Conduct a pedestrian flow forecast.

[0090] The server executes a machine learning algorithm using a generative AI model based on the integrated data. The clean data from the previous step is used as input. The output generates predictions of people flow. Specifically, it builds a prediction model using the scikit-learn library and trains the data to predict future fluctuations in people flow.

[0091] Step 4: Proposal for optimizing transportation infrastructure

[0092] The server proposes optimized traffic infrastructure operations based on pedestrian flow prediction results. The input is the pedestrian flow prediction results. The output is a specific operational proposal to the traffic infrastructure management system. The server generates an operational plan based on the prediction and distributes the proposal via an API.

[0093] Step 5: Demand forecasting and energy-saving measures in commercial facilities

[0094] The terminal acquires sensor information installed in commercial facilities and combines it with payment data. Inputs include sensor and electronic transaction information. Outputs include demand forecasting and energy-saving measures for the facility. A Raspberry Pi is used to collect data from sensors and optimize temperature control as needed.

[0095] Step 6: Disaster Risk Assessment

[0096] The server analyzes data from high-altitude observation systems and ground sensors to assess disaster risk. Observation data and sensor data are used as input. The output is the issuance of warnings to local governments and related organizations. Specifically, it assesses risk based on collected data and automatically sends notifications if a certain threshold is exceeded.

[0097] Step 7: Notifying users of the action plan

[0098] Users receive personalized behavioral guidelines through a smartphone app. Inputs include information on pedestrian flow forecasts and energy-saving measures. Outputs include optimal travel routes and energy-saving instructions sent to the user. The app uses push notifications and map display functions to provide suggestions for efficient actions.

[0099] (Application Example 1)

[0100] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0101] In modern cities, there is a need to respond quickly and efficiently to traffic congestion and fluctuations in demand for commercial facilities. Furthermore, environmental considerations necessitate the optimization of energy-saving measures. Meanwhile, in logistics, the use of real-time data is required to enable efficient operations. However, conventional methods have struggled to address these multifaceted challenges in a unified manner. Therefore, the challenge lies in integrating and analyzing data obtained from multiple sources to achieve efficient urban management and optimized logistics.

[0102] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0103] In this invention, the server includes means for acquiring location data from a location information source, means for acquiring search data from an information search source, and means for acquiring transaction data from an electronic transaction source. This makes it possible to provide a system that integrates data, makes predictions based on a learning algorithm, and enables the efficiency of transportation, demand forecasting for commercial facilities, adjustment of energy-saving measures, and optimization of logistics operations.

[0104] A "location information source" is an information source that provides location data and is used to obtain information about geographical location.

[0105] An "information search source" is an information source that provides search data, acquiring data based on users' search queries and actions.

[0106] An "electronic transaction source" is a source of information that provides transaction data, allowing access to transaction details and timing.

[0107] "Data preparation" is the process of correcting acquired data to make it accurate and consistent, and converting it into a state that can be analyzed.

[0108] A "learning algorithm" is a computational method used to make predictions and classifications based on data, and is used in building models in machine learning.

[0109] A "transportation system" is a system of transportation that provides a means of moving goods and people.

[0110] "Energy conservation measures" are policies aimed at efficiently managing and reducing energy consumption.

[0111] A "high-altitude platform" is a foundation for collecting wide-area data from the air and is used for monitoring the environment, disasters, and other phenomena.

[0112] A "smart device" is a computing device that has data communication capabilities and can run various applications.

[0113] "Operational support" refers to the provision of information and assistance to support the efficiency and smooth operation of business processes.

[0114] "Route suggestion" aims to reduce time and costs by proposing the optimal travel route.

[0115] To implement this invention, it is necessary to construct a system that effectively collects data from various data sources and performs analysis based on that data. Specifically, the server acquires corresponding data from location information sources, information search sources, and electronic transaction sources. Location information provides data on geographical location, search data indicates user interests and behavior, and transaction data shows details of commercial activities. This data is integrated and organized on the server, and machine learning algorithms are used to predict human flow and optimize supply and demand.

[0116] The hardware used includes servers for computational processing, utilizing cloud platforms (such as AWS®). For software, machine learning libraries such as TENSORFLOW® are used for data analysis.

[0117] The terminal uses a smart device to run an application that provides real-time operational support. As part of this operational support, the terminal provides optimal route suggestions for logistics operations.

[0118] For example, in operations at a logistics center in Tokyo, this data can be used to calculate efficient delivery routes for the following day, thus avoiding traffic congestion. This improves logistics efficiency and reduces the burden on drivers. Such proposals are made through generative AI models, and improved prediction accuracy is expected.

[0119] An example of a prompt message is, "Analyze logistics data within Tokyo and propose the optimal delivery route for tomorrow." This prompt message enables concrete operational support.

[0120] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0121] Step 1:

[0122] The server acquires data from location sources, information search sources, and electronic transaction sources. This data acquisition is performed via APIs and sent to the server. The inputs are location data, search data, and transaction data, and the output is a dataset of these data.

[0123] Step 2:

[0124] The server classifies and organizes the acquired data and stores it in a database. This prepares the information necessary for generating prompt statements. Data preparation includes imputing missing data and removing outliers. The input is a raw dataset, and the output is an integrated database.

[0125] Step 3:

[0126] The server utilizes a generative AI model with well-organized data to perform supply and demand forecasting. It applies machine learning algorithms and analyzes past trends to predict future pedestrian and commercial demand. The input is integrated data, and the output is the forecast result.

[0127] Step 4:

[0128] The terminal receives forecast results from the server and calculates a route plan for optimal operation. Specifically, it optimizes vehicle dispatch and delivery routes based on predicted congestion and demand. The input is the supply and demand forecast result, and the output is the optimized route suggestion.

[0129] Step 5:

[0130] Users receive optimized route suggestions via smart devices and perform operations based on them. This enables effective work execution, such as avoiding traffic congestion. The input is optimized route information, and the output is improved efficiency in actual work.

[0131] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0132] This invention expands the functionality of a smart city solution system, originally developed to improve the efficiency of urban functions, by incorporating an emotion engine that recognizes user emotions. This system covers everything from data collection and analysis to prediction and the provision of interactive services to users.

[0133] Data collection and analysis

[0134] The server periodically collects location information, search data, and payment data from each provider via APIs. The emotion engine also collects emotion-related data from the user's digital interactions, and all of this data is integrated into a database on the server.

[0135] Application of the emotion engine

[0136] The server processes the integrated data through an emotion engine to recognize the user's emotional state. This process extracts emotions from sources such as voice tone, text analysis, and payment patterns, and records them in a database.

[0137] pedestrian flow prediction and personalization

[0138] The server leverages integrated data and recognized sentiment data to apply machine learning algorithms and predict pedestrian traffic and urban demand. The compiled dataset is used to predict people's behavioral trends and analyze congestion levels in specific areas.

[0139] Optimization of transportation operations and commercial activities

[0140] The server generates suggestions to optimize transportation operations based on predictions. For example, if there is a high level of negative emotion, it adjusts the transportation plan to alleviate traffic flow in that area. In commercial facilities, it can also change promotions and service offerings based on user emotions.

[0141] User Feedback

[0142] Users receive personalized suggestions based on their emotional state through a smartphone app. For example, if they are experiencing prolonged stress, they may receive notifications via the app suggesting relaxation spots or routes to avoid crowds.

[0143] The system of this invention can improve the quality of urban life by enabling the provision of situation-appropriate services that take into account the user's emotions. This is expected to contribute to improved convenience in urban areas and the realization of a sustainable society.

[0144] The following describes the processing flow.

[0145] Step 1:

[0146] The server periodically collects data from location service providers, information retrieval service providers, and electronic payment service providers via APIs. The collected data includes longitude and latitude, search queries, and transaction amounts.

[0147] Step 2:

[0148] The server uses an emotion engine to analyze emotional data from the user's digital interactions (voice, text, payment patterns, etc.). The emotion engine infers emotions from voice tone and text content and adds emotional states such as positive and negative to the database.

[0149] Step 3:

[0150] The server uses machine learning algorithms to predict pedestrian flow based on all the integrated data. This prediction combines location information and sentiment data to accurately predict congestion levels in specific areas.

[0151] Step 4:

[0152] The server generates suggestions for optimizing transportation operations based on pedestrian flow predictions and sentiment data. For example, if there is a high level of negative sentiment in a particular area, it will suggest adjusting public transportation services to increase those services in that area.

[0153] Step 5:

[0154] The terminal uses sensor and payment data from commercial facilities to forecast demand. It leverages data from an emotion engine to generate suggestions for promotions and service adjustments tailored to the user's emotional state.

[0155] Step 6:

[0156] Users receive personalized recommendations based on emotional data through a smartphone app. If the app determines that a user is experiencing excessive stress, it will notify them of relaxing spots or routes that avoid crowds.

[0157] Step 7:

[0158] The server collects data from a wide area and integrates it with information from ground sensors to assess environmental risks. Especially when disaster risk is high, it proposes crisis management plans to local governments that also take emotional data into consideration, encouraging a swift response.

[0159] (Example 2)

[0160] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0161] Modern cities, facing population growth and rapid urbanization, demand efficient urban management. In particular, predicting traffic congestion and demand for commercial facilities are critical issues that significantly impact residents' quality of life. Furthermore, considering people's emotional states is essential for providing more optimized services. However, conventional systems struggle to integrate various data, perform sentiment analysis, and provide appropriate predictions and feedback.

[0162] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0163] In this invention, the server includes means for acquiring location information from a location information service provider, means for acquiring search data from an information retrieval service provider, and means for acquiring transaction data from an electronic transaction service provider. This enables the efficient operation of urban functions and the optimization of public transportation, taking into account the emotional state of users, as well as improved accuracy in demand forecasting at commercial facilities.

[0164] A "location information service provider" is an entity that is responsible for acquiring and providing location information.

[0165] An "information retrieval service provider" is an entity that operates a system that collects and provides relevant information based on search queries from users.

[0166] An "electronic transaction service provider" is an entity that provides online commercial transactions, including digital payments.

[0167] "Data correction" refers to the process of processing and formatting acquired data to improve its consistency and accuracy.

[0168] A "machine learning algorithm" is a mathematical method used to learn patterns and rules from data and perform predictions and classifications.

[0169] "Population movement forecasting" is the act of analyzing location information and other data to predict people's movement patterns and trends.

[0170] "Sentiment analysis" is a technology that identifies and quantifies a user's emotional state based on their text and actions.

[0171] "Optimizing public transport operations" refers to adjusting operating schedules and routes with the aim of achieving efficient traffic flow and service delivery.

[0172] A "commercial facility" is a facility established for the purpose of providing goods or services.

[0173] "Personalized behavioral guidelines" are specific and appropriate instructions provided based on the circumstances and conditions of a particular user.

[0174] This invention is a system aimed at the efficient operation of urban functions and improving the quality of life for residents. The server collects location information, search data, and transaction data, performs sentiment analysis, and aims to optimize traffic and enhance services at commercial facilities.

[0175] The server first acquires data from location service providers, information retrieval service providers, and electronic transaction service providers via APIs. The server uses Google® Maps API, online search engine API, and electronic payment API. The acquired data is integrated into a database. After data correction, this integrated data is used to predict population movement using machine learning algorithms. By using open-source platforms such as TensorFlow and PyTorch, predictions can be made with high accuracy.

[0176] Simultaneously, the server processes the user's social media posts and voice interactions through an emotion analysis engine, extracting emotional data using NLP (Neuro-Linguistic Programming) technology. By utilizing Google Cloud Natural Language or similar services, the user's emotional state is quantified and recorded in a database.

[0177] Through the user's device, the server provides personalized suggestions based on the analysis results. The device is the user's smartphone app, which is developed using Flutter® or React Native. The optimal action plan generated by the server is sent to the device, and suggestions based on the user's emotional state are provided via push notifications.

[0178] For example, if a user searches for "tired" several times, the system will provide information about relaxing parks and cafes. An example of a prompt message would be text like this: "Please suggest ways to improve the quality of urban life. The user's current emotional state is 'stressed'. Available resources are parks and cafes."

[0179] It is expected that this system will improve the quality of urban life and create a convenient and sustainable environment for residents.

[0180] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0181] Step 1:

[0182] The server collects data. It receives real-time data as input from location service providers, information retrieval service providers, and electronic transaction service providers. The server retrieves this data via APIs and stores it in a database. Specifically, it sends requests to API endpoints every minute, organizes the retrieved data, and stores it in MongoDB or MySQL®.

[0183] Step 2:

[0184] The server collects and analyzes emotional data in parallel. Inputs include user social media posts and voice interactions; this data is sent to an emotional analysis engine for processing. The server utilizes NLP and voice analysis technologies to generate emotional scores and record them in a database. Specifically, it uses the Google Cloud Natural Language API to quantify emotional states and adds the results to the dataset.

[0185] Step 3:

[0186] The server integrates and corrects the data. The inputs are location information, search data, transaction data, and sentiment data obtained in steps 1 and 2. The server combines this data and formats it into a consistent format through a data correction process. Specifically, it stores the corrected data in a new table and prepares a data mart dedicated to analysis.

[0187] Step 4:

[0188] The server performs analysis by applying machine learning algorithms. The input is integrated and corrected data, and the server uses TensorFlow to build a population flow prediction model. The output is a forecast of pedestrian flow trends and congestion in a specific area. Specifically, batch processing is performed daily to update the latest population flow model.

[0189] Step 5:

[0190] The server proposes an action plan based on the analysis results. Here, it uses the pedestrian flow prediction data and emotional state data from Step 4 as input. The server adjusts transportation schedules and generates promotional strategies for commercial facilities. Specifically, it sends proposals to business systems via email notifications and APIs, and receives feedback as it progresses.

[0191] Step 6:

[0192] Users receive personalized insights via their devices. Input consists of suggestions received from the server, which users then review using a smartphone app. Output includes information on parks for stress relief and optimal travel routes. In terms of specific actions, users receive push notifications with "recommended actions appropriate to their current state."

[0193] (Application Example 2)

[0194] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0195] In recent years, with the increase in urban populations and the hosting of events, maintaining public safety and providing a comfortable urban environment have become crucial issues. However, currently, it is difficult to adequately address these issues, and there is a particular need for enhanced security in places where large numbers of people gather and smooth management of pedestrian flow. This invention aims to achieve more effective and rapid assurance of public safety and improve the efficiency of urban management by integrating technology that visualizes users' emotional states into urban management systems.

[0196] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0197] In this invention, the server includes means for acquiring spatial data from a location information providing device, means for acquiring search information from an information retrieval providing device, and means for acquiring transaction information from a payment processing providing device. This makes it possible to sense the emotional state of surrounding individuals in real time and provide personalized behavioral guidelines.

[0198] A "location information provider" is a device that collects location-related data and provides it to other devices or systems.

[0199] "Spatial data" refers to data that represents information about geographical location or space.

[0200] An "information retrieval and provision device" is a device that acquires and provides relevant information in response to a user's search request.

[0201] "Search information" refers to data related to keywords and results obtained when a user performs a search.

[0202] A "payment processing and provision device" is a device that collects payment data in transactions and provides it to other devices or systems.

[0203] "Transaction information" refers to data related to purchasing activities conducted through electronic payments.

[0204] "Data cleansing" is a method of processing collected data to improve its quality, such as imputing missing values ​​and correcting outliers.

[0205] A "learning algorithm" is a method that learns patterns based on past data and uses that knowledge to make predictions and classifications about future data.

[0206] A "transportation system" is a system or mechanism that provides means of transporting people and goods.

[0207] "Personalization" refers to providing information and services that are optimized according to each user's characteristics and circumstances.

[0208] A "visual device" is a device that presents external information visually to the wearer and provides it to them.

[0209] The system used to realize this application example takes the form of a server that acquires and analyzes diverse data and provides information to various devices. The server acquires spatial data from a location information provider, search information from an information retrieval provider, and transaction information from a payment processing provider. This data is then integrated and subjected to data cleansing.

[0210] Based on a comprehensive dataset, the server uses learning algorithms to predict pedestrian flow and emotional states. This process utilizes libraries such as Python and TensorFlow to train machine learning models. The server further processes real-time data acquired through sensors mounted on visual devices using an emotion engine. This emotion engine leverages natural language processing technologies such as IBM Watson® to analyze speech tone and text.

[0211] Users can monitor the emotional state of their surroundings in real time through a visual device. This device consists of smart glasses such as Google Glass®, and the recognized information is presented to the user as a simplified heatmap. This enables rapid responses to public safety and congestion levels. For example, security staff can be effectively deployed to areas where anxiety levels are high during holiday events.

[0212] An example of a prompt to the generating AI model would be, "Identify the area with the most emotionally unstable situation and quickly notify security staff of the details." This would allow security staff to focus their efforts on the area where the situation is most severe.

[0213] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0214] Step 1:

[0215] The server acquires spatial data from a location information provider. The input is real-time latitude and longitude data transmitted from the location information provider, and the output is the accurate storage of that data on the server. The server records this data in a database and prepares it for later analysis.

[0216] Step 2:

[0217] The server retrieves search information from the information retrieval device. The input is the search query entered by the user, and the output is data related to the search results. The server records this information as search trends and uses it in predictive models.

[0218] Step 3:

[0219] The server retrieves transaction information from the payment processing device. The input is detailed data about the user's transactions, and the output is an organized list of that data. The server analyzes transaction patterns and models which time periods have the most purchasing activity.

[0220] Step 4:

[0221] The server performs data cleansing based on the integrated data. The input is all location, search, and transaction data, and the output is a clean dataset with noise removed. The server improves data accuracy by imputing missing values ​​and removing outliers.

[0222] Step 5:

[0223] The server applies a learning algorithm using a clean dataset to predict pedestrian flow and emotional states. The input is a clean dataset, and the output is the prediction result. Python and TensorFlow are used to train the learning model and calculate future pedestrian flow patterns.

[0224] Step 6:

[0225] The server incorporates an emotion engine into the visual device and analyzes data acquired in real time. The input is video and audio data acquired by the visual device, and the output is emotional states obtained through text analysis and speech tone analysis. Natural language processing technologies, such as IBM Watson, are used to visualize these emotions.

[0226] Step 7:

[0227] Users can view the emotional state of their surroundings in real time through a visual device. The input is emotion visualization data transmitted by the server, and the output is a heat map presented to the user. Based on this information, users can quickly respond to the situation in a specific area.

[0228] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0229] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0230] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0231] [Second Embodiment]

[0232] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0233] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0234] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0235] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0236] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0237] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0238] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0239] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0240] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0241] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0242] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0243] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0244] This invention provides a data-centric smart city solution system aimed at optimizing urban functions. This system acquires necessary data from various service providers and analyzes it in an integrated format, enabling improved transportation efficiency, demand forecasting for commercial facilities, measurement of the effectiveness of energy-saving measures, and disaster response based on wide-area data.

[0245] Data acquisition and integration

[0246] The server periodically collects data from location service providers, information retrieval service providers, and electronic payment service providers. Data collection is performed using APIs, and the obtained information is stored in a database on the server in real time. Location data includes longitude, latitude, and time data, while retrieval data includes search queries and related time zones. Payment data includes transaction amounts and time information.

[0247] Data analysis and forecasting

[0248] The server integrates the collected data and removes inaccurate data through data cleaning. This improves data accuracy and converts it into a format that can be input into the pedestrian flow prediction model. Pedestrian flow prediction is performed using machine learning algorithms to predict future human movement based on historical data.

[0249] Optimization of public transport operations

[0250] The server utilizes the pedestrian flow prediction results to optimize transportation schedules. For example, it suggests to the transportation management system that the number of buses should be increased during predicted peak hours. Furthermore, if congestion is expected in a specific area, it provides feedback to subway and train service information to improve passenger convenience.

[0251] Demand forecasting and energy conservation for commercial facilities

[0252] The terminal acquires information from sensors installed in commercial facilities and combines it with payment data to predict demand. Based on the demand forecast, measures to reduce energy consumption during business hours are automatically applied. For example, temperature control is appropriately managed during peak hours to reduce power consumption.

[0253] Disaster response and environmental monitoring

[0254] The server analyzes environmental data by integrating wide-area data collected by high-altitude platforms with data from ground-based sensors. This allows it to issue warnings before disasters such as floods and heavy rains occur, prompting local governments and related organizations to take countermeasures.

[0255] Providing services to users

[0256] Users receive personalized behavioral guidelines based on their individual data through a provided smartphone app. This app notifies users of suggestions for appropriate travel times and energy-saving behaviors, improving their quality of life. For example, it displays suggestions for travel routes that avoid congestion and specific instructions to help save energy.

[0257] The system of this invention will help improve the efficiency of critical infrastructure in cities and contribute to the realization of a sustainable urban environment.

[0258] The following describes the processing flow.

[0259] Step 1:

[0260] The server retrieves data in real time via APIs from location information service providers, information retrieval service providers, and electronic payment service providers. This allows a wide range of data, such as longitude, latitude, search queries, and transaction amounts, to be stored in the server's database.

[0261] Step 2:

[0262] The server integrates the collected data and removes inaccurate and duplicate data through a data cleaning process. This step also standardizes the format and eliminates outliers, preparing the data for analysis.

[0263] Step 3:

[0264] The server uses clean data and applies machine learning algorithms to predict future pedestrian traffic. It analyzes past trends, builds predictive models based on people's movement patterns, and forecasts pedestrian traffic for the next 24 hours and week.

[0265] Step 4:

[0266] The server uses the prediction results to make suggestions for optimizing transportation schedules. These suggestions are sent to the traffic management system, which then develops operational plans to provide additional traffic during peak hours.

[0267] Step 5:

[0268] The terminal collects real-time data from sensors installed in commercial facilities and combines it with payment data to predict demand. Based on the demand forecast, the facility's air conditioning and lighting are adjusted to improve energy efficiency.

[0269] Step 6:

[0270] The server acquires wide-area data through a high-altitude platform and integrates it with ground sensor data to conduct environmental risk assessments. By combining this with meteorological data, it enables the early issuance of warnings for potential disasters.

[0271] Step 7:

[0272] Users receive personalized suggestions based on predicted pedestrian flow data and energy consumption through a smartphone app. The app provides users with optimal travel routes and specific guidance for energy saving.

[0273] (Example 1)

[0274] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0275] The increasing complexity of urban functions and population concentration have made it difficult to efficiently operate transportation infrastructure and commercial facilities. In addition, there are challenges such as the need for early prediction of disaster risks and optimal energy consumption. Conventional technologies have not provided an efficient system to comprehensively analyze this data and translate it into appropriate actions.

[0276] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0277] In this invention, the server includes means for acquiring location information from location information data providers, means for acquiring search information from data search providers, and means for acquiring transaction information from electronic transaction information providers. This enables the integrated analysis of urban information, optimization of transportation infrastructure, demand forecasting for commercial facilities, disaster risk assessment, and provision of personalized action plans.

[0278] A "location data provider" refers to an entity that provides location information services, typically responsible for accurately measuring the location of a target and providing the results to an external system.

[0279] A "data search provider" is an entity that makes information and data searchable and provides relevant information based on the user's search query.

[0280] An "electronic transaction information provider" is an entity that manages and provides information on electronically conducted transactions, recording details such as transaction amounts and dates, and sharing them with third parties.

[0281] "People movement" refers to the patterns and flows of people's movements in a specific region or time period, and is analyzed based on observational data.

[0282] "Optimization of transportation infrastructure operations" refers to the act of optimizing the operation plan in order to maximize the efficiency of the transportation system, and mainly includes adjusting operations in response to user demand.

[0283] "Demand forecasting for commercial facilities" refers to predicting the demand required for future sales activities and service provision in commercial facilities, which is analyzed based on past data.

[0284] "Resource consumption reduction measures" refer to measures and methods for suppressing the use of energy and materials, which are implemented aiming at efficient operation.

[0285] "High-altitude observation system" usually means a system that collects a wide range of data from high positions such as aircraft and satellites, and can observe a wider range than the area near the ground surface.

[0286] "Evaluating the risk of disasters" means quantitatively or qualitatively analyzing the risk of natural disasters occurring and predicting their impacts.

[0287] "Individualized action plan" refers to proposals and instructions for actions optimized according to the needs and situations of specific users, aiming to provide personalized information.

[0288] To implement this invention, first, the server collects necessary data from location information data providers, data search providers, and e-commerce transaction information providers. Specifically, it periodically sends HTTP requests through APIs, analyzes the obtained responses, and saves them in the database. This process is generally carried out using libraries in Python or Java. The server integrates this data, performs data cleaning using the Pandas library, and ensures its accuracy.

[0289] Next, the server executes a machine learning algorithm equipped with a generative AI model using the collected clean data. In predicting human mobility, the server utilizes the scikit-learn library to create a prediction model based on past data. This model enables the optimization of the operation of transportation infrastructure, demand forecasting in commercial facilities, and the suppression of resource consumption.

[0290] For example, prompts such as "Please tell me the best route to get to the office at 10 AM next Monday" or "Please suggest ways to reduce energy consumption at commercial facilities this weekend" are presented. Based on these prompts, the user can receive optimized information provided by the server.

[0291] Furthermore, the terminal acquires environmental information from IoT sensors installed in commercial facilities and performs real-time data analysis. Using hardware such as Raspberry Pi, it collects data and then optimizes the control of air conditioning and lighting on the terminal. Users can receive personalized behavioral guidelines through a smartphone app, which is expected to improve their quality of life. By providing users with optimal guidance tailored to specific times and situations, this system can contribute to urban sustainability.

[0292] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0293] Step 1: Data Collection

[0294] The server retrieves data from location data providers, data search providers, and electronic transaction information providers. As input, the server sends HTTP requests through each provider's API. As output, location information, search information, and transaction information are retrieved from these providers in JSON format and stored in the database. The server uses the Python Requests library to process requests and analyze the obtained data.

[0295] Step 2: Data Integration and Cleaning

[0296] The server integrates the collected data and removes inaccurate data. Raw data from multiple data sources is provided as input. The output is a clean, integrated dataset. Specifically, the Pandas library is used to filter out unnecessary data and impute missing values ​​to format the data.

[0297] Step 3: Conduct a pedestrian flow forecast.

[0298] The server executes a machine learning algorithm using a generative AI model based on the integrated data. The clean data from the previous step is used as input. The output generates predictions of people flow. Specifically, it builds a prediction model using the scikit-learn library and trains the data to predict future fluctuations in people flow.

[0299] Step 4: Proposal for optimizing transportation infrastructure

[0300] The server proposes optimized traffic infrastructure operations based on pedestrian flow prediction results. The input is the pedestrian flow prediction results. The output is a specific operational proposal to the traffic infrastructure management system. The server generates an operational plan based on the prediction and distributes the proposal via an API.

[0301] Step 5: Demand forecasting and energy-saving measures in commercial facilities

[0302] The terminal acquires sensor information installed in commercial facilities and combines it with payment data. Inputs include sensor and electronic transaction information. Outputs include demand forecasting and energy-saving measures for the facility. A Raspberry Pi is used to collect data from sensors and optimize temperature control as needed.

[0303] Step 6: Disaster Risk Assessment

[0304] The server analyzes data from high-altitude observation systems and ground sensors to evaluate disaster risks. Observation data and sensor data are used as inputs. Alerts are sent to local governments and relevant agencies as outputs. In a specific operation, risks are evaluated based on the collected data, and notifications are automatically sent when specific thresholds are exceeded.

[0305] Step 7: Notification of the action plan to the user

[0306] The user receives personalized action guidelines through a smartphone app. Information such as crowd flow prediction and energy-saving measures is utilized as inputs. Optimal movement routes and energy-saving instructions are sent to the user as outputs. The app uses push notifications and map display functions to perform the operation of receiving proposals for efficient actions.

[0307] (Application Example 1)

[0308] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0309] In modern cities, it is required to respond quickly and efficiently to traffic congestion in transportation and demand fluctuations in commercial facilities. Also, due to environmental considerations, optimization of energy-saving measures is necessary. On the other hand, in logistics, real-time data utilization to enable efficient operations is required. However, it has been difficult to address these multifaceted issues comprehensively with conventional methods. Therefore, the challenge is to integrally analyze data obtained from multiple information sources and realize efficient urban operation and optimization of logistics.

[0310] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0311] In this invention, the server includes means for acquiring location data from a location information source, means for acquiring search data from an information search source, and means for acquiring transaction data from an electronic transaction source. This makes it possible to provide a system that integrates data, makes predictions based on a learning algorithm, and enables the efficiency of transportation, demand forecasting for commercial facilities, adjustment of energy-saving measures, and optimization of logistics operations.

[0312] A "location information source" is an information source that provides location data and is used to obtain information about geographical location.

[0313] An "information search source" is an information source that provides search data, acquiring data based on users' search queries and actions.

[0314] An "electronic transaction source" is a source of information that provides transaction data, allowing access to transaction details and timing.

[0315] "Data preparation" is the process of correcting acquired data to make it accurate and consistent, and converting it into a state that can be analyzed.

[0316] A "learning algorithm" is a computational method used to make predictions and classifications based on data, and is used in building models in machine learning.

[0317] A "transportation system" is a system of transportation that provides a means of moving goods and people.

[0318] "Energy conservation measures" are policies aimed at efficiently managing and reducing energy consumption.

[0319] A "high-altitude platform" is a foundation for collecting wide-area data from the air and is used for monitoring the environment, disasters, and other phenomena.

[0320] A "smart device" is a computing device that has data communication capabilities and can run various applications.

[0321] "Operational support" refers to the provision of information and assistance to support the efficiency and smooth operation of business processes.

[0322] "Route suggestion" aims to reduce time and costs by proposing the optimal travel route.

[0323] To implement this invention, it is necessary to construct a system that effectively collects data from various data sources and performs analysis based on that data. Specifically, the server acquires corresponding data from location information sources, information search sources, and electronic transaction sources. Location information provides data on geographical location, search data indicates user interests and behavior, and transaction data shows details of commercial activities. This data is integrated and organized on the server, and machine learning algorithms are used to predict human flow and optimize supply and demand.

[0324] The hardware used includes servers for computational processing, utilizing cloud platforms (such as AWS). For software, machine learning libraries like TensorFlow are used for data analysis.

[0325] The terminal uses a smart device to run an application that provides real-time operational support. As part of this operational support, the terminal provides optimal route suggestions for logistics operations.

[0326] For example, in operations at a logistics center in Tokyo, this data can be used to calculate efficient delivery routes for the following day, thus avoiding traffic congestion. This improves logistics efficiency and reduces the burden on drivers. Such proposals are made through generative AI models, and improved prediction accuracy is expected.

[0327] An example of a prompt message is, "Analyze logistics data within Tokyo and propose the optimal delivery route for tomorrow." This prompt message enables concrete operational support.

[0328] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0329] Step 1:

[0330] The server acquires data from location sources, information search sources, and electronic transaction sources. This data acquisition is performed via APIs and sent to the server. The inputs are location data, search data, and transaction data, and the output is a dataset of these data.

[0331] Step 2:

[0332] The server classifies and organizes the acquired data and stores it in a database. This prepares the information necessary for generating prompt statements. Data preparation includes imputing missing data and removing outliers. The input is a raw dataset, and the output is an integrated database.

[0333] Step 3:

[0334] The server utilizes a generative AI model with well-organized data to perform supply and demand forecasting. It applies machine learning algorithms and analyzes past trends to predict future pedestrian and commercial demand. The input is integrated data, and the output is the forecast result.

[0335] Step 4:

[0336] The terminal receives forecast results from the server and calculates a route plan for optimal operation. Specifically, it optimizes vehicle dispatch and delivery routes based on predicted congestion and demand. The input is the supply and demand forecast result, and the output is the optimized route suggestion.

[0337] Step 5:

[0338] Users receive optimized route suggestions via smart devices and perform operations based on them. This enables effective work execution, such as avoiding traffic congestion. The input is optimized route information, and the output is improved efficiency in actual work.

[0339] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0340] This invention expands the functionality of a smart city solution system, originally developed to improve the efficiency of urban functions, by incorporating an emotion engine that recognizes user emotions. This system covers everything from data collection and analysis to prediction and the provision of interactive services to users.

[0341] Data collection and analysis

[0342] The server periodically collects location information, search data, and payment data from each provider via APIs. The emotion engine also collects emotion-related data from the user's digital interactions, and all of this data is integrated into a database on the server.

[0343] Application of the emotion engine

[0344] The server processes the integrated data through an emotion engine to recognize the user's emotional state. This process extracts emotions from sources such as voice tone, text analysis, and payment patterns, and records them in a database.

[0345] pedestrian flow prediction and personalization

[0346] The server leverages integrated data and recognized sentiment data to apply machine learning algorithms and predict pedestrian traffic and urban demand. The compiled dataset is used to predict people's behavioral trends and analyze congestion levels in specific areas.

[0347] Optimization of transportation operations and commercial activities

[0348] The server generates suggestions to optimize transportation operations based on predictions. For example, if there is a high level of negative emotion, it adjusts the transportation plan to alleviate traffic flow in that area. In commercial facilities, it can also change promotions and service offerings based on user emotions.

[0349] User Feedback

[0350] Users receive personalized suggestions based on their emotional state through a smartphone app. For example, if they are experiencing prolonged stress, they may receive notifications via the app suggesting relaxation spots or routes to avoid crowds.

[0351] The system of this invention can improve the quality of urban life by enabling the provision of situation-appropriate services that take into account the user's emotions. This is expected to contribute to improved convenience in urban areas and the realization of a sustainable society.

[0352] The following describes the processing flow.

[0353] Step 1:

[0354] The server periodically collects data from location service providers, information retrieval service providers, and electronic payment service providers via APIs. The collected data includes longitude and latitude, search queries, and transaction amounts.

[0355] Step 2:

[0356] The server uses an emotion engine to analyze emotional data from the user's digital interactions (voice, text, payment patterns, etc.). The emotion engine infers emotions from voice tone and text content and adds emotional states such as positive and negative to the database.

[0357] Step 3:

[0358] The server uses machine learning algorithms to predict pedestrian flow based on all the integrated data. This prediction combines location information and sentiment data to accurately predict congestion levels in specific areas.

[0359] Step 4:

[0360] The server generates suggestions for optimizing transportation operations based on pedestrian flow predictions and sentiment data. For example, if there is a high level of negative sentiment in a particular area, it will suggest adjusting public transportation services to increase those services in that area.

[0361] Step 5:

[0362] The terminal uses sensor and payment data from commercial facilities to forecast demand. It leverages data from an emotion engine to generate suggestions for promotions and service adjustments tailored to the user's emotional state.

[0363] Step 6:

[0364] Users receive personalized recommendations based on emotional data through a smartphone app. If the app determines that a user is experiencing excessive stress, it will notify them of relaxing spots or routes that avoid crowds.

[0365] Step 7:

[0366] The server collects data from a wide area and integrates it with information from ground sensors to assess environmental risks. Especially when disaster risk is high, it proposes crisis management plans to local governments that also take emotional data into consideration, encouraging a swift response.

[0367] (Example 2)

[0368] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0369] Modern cities, facing population growth and rapid urbanization, demand efficient urban management. In particular, predicting traffic congestion and demand for commercial facilities are critical issues that significantly impact residents' quality of life. Furthermore, considering people's emotional states is essential for providing more optimized services. However, conventional systems struggle to integrate various data, perform sentiment analysis, and provide appropriate predictions and feedback.

[0370] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0371] In this invention, the server includes means for acquiring location information from a location information service provider, means for acquiring search data from an information retrieval service provider, and means for acquiring transaction data from an electronic transaction service provider. This enables the efficient operation of urban functions and the optimization of public transportation, taking into account the emotional state of users, as well as improved accuracy in demand forecasting at commercial facilities.

[0372] A "location information service provider" is an entity that is responsible for acquiring and providing location information.

[0373] An "information retrieval service provider" is an entity that operates a system that collects and provides relevant information based on search queries from users.

[0374] An "electronic transaction service provider" is an entity that provides online commercial transactions, including digital payments.

[0375] "Data correction" refers to the process of processing and formatting acquired data to improve its consistency and accuracy.

[0376] A "machine learning algorithm" is a mathematical method used to learn patterns and rules from data and perform predictions and classifications.

[0377] "Population movement forecasting" is the act of analyzing location information and other data to predict people's movement patterns and trends.

[0378] "Sentiment analysis" is a technology that identifies and quantifies a user's emotional state based on their text and actions.

[0379] "Optimizing public transport operations" refers to adjusting operating schedules and routes with the aim of achieving efficient traffic flow and service delivery.

[0380] A "commercial facility" is a facility established for the purpose of providing goods or services.

[0381] "Personalized behavioral guidelines" are specific and appropriate instructions provided based on the circumstances and conditions of a particular user.

[0382] This invention is a system aimed at the efficient operation of urban functions and improving the quality of life for residents. The server collects location information, search data, and transaction data, performs sentiment analysis, and aims to optimize traffic and enhance services at commercial facilities.

[0383] The server first acquires data from location service providers, information retrieval service providers, and electronic transaction service providers via APIs. The server uses Google Maps API, online search engine API, and electronic payment API. The acquired data is integrated into a database. After data correction, this integrated data is used to predict population movement using machine learning algorithms. By using open-source platforms such as TensorFlow and PyTorch, predictions with high accuracy can be achieved.

[0384] Simultaneously, the server processes the user's social media posts and voice interactions through an emotion analysis engine, extracting emotional data using NLP (Neuro-Linguistic Programming) technology. By utilizing Google Cloud Natural Language or similar services, the user's emotional state is quantified and recorded in a database.

[0385] Through the user's device, the server provides personalized suggestions based on the analysis results. The device is the user's smartphone app, which is developed using Flutter or React Native. The optimal action plan generated by the server is sent to the device, and suggestions based on the user's emotional state are delivered via push notifications.

[0386] For example, if a user searches for "tired" several times, the system will provide information about relaxing parks and cafes. An example of a prompt message would be text like this: "Please suggest ways to improve the quality of urban life. The user's current emotional state is 'stressed'. Available resources are parks and cafes."

[0387] It is expected that this system will improve the quality of urban life and create a convenient and sustainable environment for residents.

[0388] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0389] Step 1:

[0390] The server collects data. It receives real-time data as input from location service providers, information retrieval service providers, and e-commerce service providers. The server retrieves this data via APIs and stores it in a database. Specifically, it sends requests to API endpoints every minute, organizes the retrieved data, and stores it in MongoDB or MySQL.

[0391] Step 2:

[0392] The server collects and analyzes emotional data in parallel. Inputs include user social media posts and voice interactions; this data is sent to an emotional analysis engine for processing. The server utilizes NLP and voice analysis technologies to generate emotional scores and record them in a database. Specifically, it uses the Google Cloud Natural Language API to quantify emotional states and adds the results to the dataset.

[0393] Step 3:

[0394] The server integrates and corrects the data. The inputs are location information, search data, transaction data, and sentiment data obtained in steps 1 and 2. The server combines this data and formats it into a consistent format through a data correction process. Specifically, it stores the corrected data in a new table and prepares a data mart dedicated to analysis.

[0395] Step 4:

[0396] The server performs analysis by applying machine learning algorithms. The input is integrated and corrected data, and the server uses TensorFlow to build a population flow prediction model. The output is a forecast of pedestrian flow trends and congestion in a specific area. Specifically, batch processing is performed daily to update the latest population flow model.

[0397] Step 5:

[0398] The server proposes an action plan based on the analysis results. Here, it uses the pedestrian flow prediction data and emotional state data from Step 4 as input. The server adjusts transportation schedules and generates promotional strategies for commercial facilities. Specifically, it sends proposals to business systems via email notifications and APIs, and receives feedback as it progresses.

[0399] Step 6:

[0400] Users receive personalized insights via their devices. Input consists of suggestions received from the server, which users then review using a smartphone app. Output includes information on parks for stress relief and optimal travel routes. In terms of specific actions, users receive push notifications with "recommended actions appropriate to their current state."

[0401] (Application Example 2)

[0402] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0403] In recent years, with the increase in urban populations and the hosting of events, maintaining public safety and providing a comfortable urban environment have become crucial issues. However, currently, it is difficult to adequately address these issues, and there is a particular need for enhanced security in places where large numbers of people gather and smooth management of pedestrian flow. This invention aims to achieve more effective and rapid assurance of public safety and improve the efficiency of urban management by integrating technology that visualizes users' emotional states into urban management systems.

[0404] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0405] In this invention, the server includes means for acquiring spatial data from a location information providing device, means for acquiring search information from an information retrieval providing device, and means for acquiring transaction information from a payment processing providing device. This makes it possible to sense the emotional state of surrounding individuals in real time and provide personalized behavioral guidelines.

[0406] A "location information provider" is a device that collects location-related data and provides it to other devices or systems.

[0407] "Spatial data" refers to data that represents information about geographical location or space.

[0408] An "information retrieval and provision device" is a device that acquires and provides relevant information in response to a user's search request.

[0409] "Search information" refers to data related to keywords and results obtained when a user performs a search.

[0410] A "payment processing and provision device" is a device that collects payment data in transactions and provides it to other devices or systems.

[0411] "Transaction information" refers to data related to purchasing activities conducted through electronic payments.

[0412] "Data cleansing" is a method of processing collected data to improve its quality, such as imputing missing values ​​and correcting outliers.

[0413] A "learning algorithm" is a method that learns patterns based on past data and uses that knowledge to make predictions and classifications about future data.

[0414] A "transportation system" is a system or mechanism that provides means of transporting people and goods.

[0415] "Personalization" refers to providing information and services that are optimized according to each user's characteristics and circumstances.

[0416] A "visual device" is a device that presents external information visually to the wearer and provides it to them.

[0417] The system used to realize this application example takes the form of a server that acquires and analyzes diverse data and provides information to various devices. The server acquires spatial data from a location information provider, search information from an information retrieval provider, and transaction information from a payment processing provider. This data is then integrated and subjected to data cleansing.

[0418] Based on a comprehensive dataset, the server uses learning algorithms to predict pedestrian flow and emotional states. This process utilizes libraries such as Python and TensorFlow to train machine learning models. The server further processes real-time data acquired through sensors mounted on visual devices using an emotion engine. This emotion engine leverages natural language processing technologies like IBM Watson to analyze speech tone and text.

[0419] Users can monitor the emotional state of their surroundings in real time through a visual device. This device consists of smart glasses, such as Google Glass, and the recognized information is presented to the user as a simplified heatmap. This enables rapid responses to public safety and congestion levels. For example, security staff can be effectively deployed to areas where anxiety levels are high during holiday events.

[0420] An example of a prompt to the generating AI model would be, "Identify the area with the most emotionally unstable situation and quickly notify security staff of the details." This would allow security staff to focus their efforts on the area where the situation is most severe.

[0421] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0422] Step 1:

[0423] The server acquires spatial data from a location information provider. The input is real-time latitude and longitude data transmitted from the location information provider, and the output is the accurate storage of that data on the server. The server records this data in a database and prepares it for later analysis.

[0424] Step 2:

[0425] The server retrieves search information from the information retrieval device. The input is the search query entered by the user, and the output is data related to the search results. The server records this information as search trends and uses it in predictive models.

[0426] Step 3:

[0427] The server retrieves transaction information from the payment processing device. The input is detailed data about the user's transactions, and the output is an organized list of that data. The server analyzes transaction patterns and models which time periods have the most purchasing activity.

[0428] Step 4:

[0429] The server performs data cleansing based on the integrated data. The input is all location, search, and transaction data, and the output is a clean dataset with noise removed. The server improves data accuracy by imputing missing values ​​and removing outliers.

[0430] Step 5:

[0431] The server applies a learning algorithm using a clean dataset to predict pedestrian flow and emotional states. The input is a clean dataset, and the output is the prediction result. Python and TensorFlow are used to train the learning model and calculate future pedestrian flow patterns.

[0432] Step 6:

[0433] The server incorporates an emotion engine into the visual device and analyzes data acquired in real time. The input is video and audio data acquired by the visual device, and the output is emotional states obtained through text analysis and speech tone analysis. Natural language processing technologies, such as IBM Watson, are used to visualize these emotions.

[0434] Step 7:

[0435] Users can view the emotional state of their surroundings in real time through a visual device. The input is emotion visualization data transmitted by the server, and the output is a heat map presented to the user. Based on this information, users can quickly respond to the situation in a specific area.

[0436] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0437] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0438] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0439] [Third Embodiment]

[0440] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0441] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0442] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0443] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0444] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0445] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0446] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0447] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0448] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0449] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0450] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0451] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0452] This invention provides a data-centric smart city solution system aimed at optimizing urban functions. This system acquires necessary data from various service providers and analyzes it in an integrated format, enabling improved transportation efficiency, demand forecasting for commercial facilities, measurement of the effectiveness of energy-saving measures, and disaster response based on wide-area data.

[0453] Data acquisition and integration

[0454] The server periodically collects data from location service providers, information retrieval service providers, and electronic payment service providers. Data collection is performed using APIs, and the obtained information is stored in a database on the server in real time. Location data includes longitude, latitude, and time data, while retrieval data includes search queries and related time zones. Payment data includes transaction amounts and time information.

[0455] Data analysis and forecasting

[0456] The server integrates the collected data and removes inaccurate data through data cleaning. This improves data accuracy and converts it into a format that can be input into the pedestrian flow prediction model. Pedestrian flow prediction is performed using machine learning algorithms to predict future human movement based on historical data.

[0457] Optimization of public transport operations

[0458] The server utilizes the pedestrian flow prediction results to optimize transportation schedules. For example, it suggests to the transportation management system that the number of buses should be increased during predicted peak hours. Furthermore, if congestion is expected in a specific area, it provides feedback to subway and train service information to improve passenger convenience.

[0459] Demand forecasting and energy conservation for commercial facilities

[0460] The terminal acquires information from sensors installed in commercial facilities and combines it with payment data to predict demand. Based on the demand forecast, measures to reduce energy consumption during business hours are automatically applied. For example, temperature control is appropriately managed during peak hours to reduce power consumption.

[0461] Disaster response and environmental monitoring

[0462] The server analyzes environmental data by integrating wide-area data collected by high-altitude platforms with data from ground-based sensors. This allows it to issue warnings before disasters such as floods and heavy rains occur, prompting local governments and related organizations to take countermeasures.

[0463] Providing services to users

[0464] Users receive personalized behavioral guidelines based on their individual data through a provided smartphone app. This app notifies users of suggestions for appropriate travel times and energy-saving behaviors, improving their quality of life. For example, it displays suggestions for travel routes that avoid congestion and specific instructions to help save energy.

[0465] The system of this invention will help improve the efficiency of critical infrastructure in cities and contribute to the realization of a sustainable urban environment.

[0466] The following describes the processing flow.

[0467] Step 1:

[0468] The server retrieves data in real time via APIs from location information service providers, information retrieval service providers, and electronic payment service providers. This allows a wide range of data, such as longitude, latitude, search queries, and transaction amounts, to be stored in the server's database.

[0469] Step 2:

[0470] The server integrates the collected data and removes inaccurate and duplicate data through a data cleaning process. This step also standardizes the format and eliminates outliers, preparing the data for analysis.

[0471] Step 3:

[0472] The server uses clean data and applies machine learning algorithms to predict future pedestrian traffic. It analyzes past trends, builds predictive models based on people's movement patterns, and forecasts pedestrian traffic for the next 24 hours and week.

[0473] Step 4:

[0474] The server uses the prediction results to make suggestions for optimizing transportation schedules. These suggestions are sent to the traffic management system, which then develops operational plans to provide additional traffic during peak hours.

[0475] Step 5:

[0476] The terminal collects real-time data from sensors installed in commercial facilities and combines it with payment data to predict demand. Based on the demand forecast, the facility's air conditioning and lighting are adjusted to improve energy efficiency.

[0477] Step 6:

[0478] The server acquires wide-area data through a high-altitude platform and integrates it with ground sensor data to conduct environmental risk assessments. By combining this with meteorological data, it enables the early issuance of warnings for potential disasters.

[0479] Step 7:

[0480] Users receive personalized suggestions based on predicted pedestrian flow data and energy consumption through a smartphone app. The app provides users with optimal travel routes and specific guidance for energy saving.

[0481] (Example 1)

[0482] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0483] The increasing complexity of urban functions and population concentration have made it difficult to efficiently operate transportation infrastructure and commercial facilities. In addition, there are challenges such as the need for early prediction of disaster risks and optimal energy consumption. Conventional technologies have not provided an efficient system to comprehensively analyze this data and translate it into appropriate actions.

[0484] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0485] In this invention, the server includes means for acquiring location information from location information data providers, means for acquiring search information from data search providers, and means for acquiring transaction information from electronic transaction information providers. This enables the integrated analysis of urban information, optimization of transportation infrastructure, demand forecasting for commercial facilities, disaster risk assessment, and provision of personalized action plans.

[0486] A "location data provider" refers to an entity that provides location information services, typically responsible for accurately measuring the location of a target and providing the results to an external system.

[0487] A "data search provider" is an entity that makes information and data searchable and provides relevant information based on the user's search query.

[0488] An "electronic transaction information provider" is an entity that manages and provides information on electronically conducted transactions, recording details such as transaction amounts and dates, and sharing them with third parties.

[0489] "People movement" refers to the patterns and flows of people's movements in a specific region or time period, and is analyzed based on observational data.

[0490] "Optimization of transportation infrastructure operations" refers to the act of optimizing the operation plan in order to maximize the efficiency of the transportation system, and mainly includes adjusting operations in response to user demand.

[0491] "Demand forecasting for commercial facilities" refers to predicting the demand necessary for future sales activities and service provision at commercial facilities, and is analyzed based on past data.

[0492] "Resource consumption reduction measures" refer to measures and methods aimed at reducing the use of energy and materials, and are implemented with the goal of efficient operation.

[0493] A "high-altitude observation system" typically refers to a system that collects data over a wide area from an advanced position, such as an aircraft or satellite, enabling observations over a wider area than those near the Earth's surface.

[0494] "Assessing the risk of disaster" means quantitatively or qualitatively analyzing the risk of natural disasters occurring and predicting their impact.

[0495] An "individualized action plan" refers to suggestions and instructions for actions optimized according to the specific needs and circumstances of a particular user, with the aim of providing personalized information.

[0496] To implement this invention, the server first collects necessary data from location data providers, data retrieval providers, and electronic transaction information providers. Specifically, it periodically sends HTTP requests via APIs, parses the responses received, and stores them in a database. This process is typically carried out using Python or Java libraries. The server then integrates this data, cleans it using the Pandas library, and ensures its accuracy.

[0497] Next, the server uses the collected clean data to run a machine learning algorithm equipped with a generative AI model. In predicting people's movement, the server utilizes the scikit-learn library to create a predictive model based on historical data. This model enables optimization of transportation infrastructure operations, demand forecasting in commercial facilities, and reduction of resource consumption.

[0498] For example, prompts such as "Please tell me the best route to get to the office at 10 AM next Monday" or "Please suggest ways to reduce energy consumption at commercial facilities this weekend" are presented. Based on these prompts, the user can receive optimized information provided by the server.

[0499] Furthermore, the terminal acquires environmental information from IoT sensors installed in commercial facilities and performs real-time data analysis. Using hardware such as Raspberry Pi, it collects data and then optimizes the control of air conditioning and lighting on the terminal. Users can receive personalized behavioral guidelines through a smartphone app, which is expected to improve their quality of life. By providing users with optimal guidance tailored to specific times and situations, this system can contribute to urban sustainability.

[0500] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0501] Step 1: Data Collection

[0502] The server retrieves data from location data providers, data search providers, and electronic transaction information providers. As input, the server sends HTTP requests through each provider's API. As output, location information, search information, and transaction information are retrieved from these providers in JSON format and stored in the database. The server uses the Python Requests library to process requests and analyze the obtained data.

[0503] Step 2: Data Integration and Cleaning

[0504] The server integrates the collected data and removes inaccurate data. Raw data from multiple data sources is provided as input. The output is a clean, integrated dataset. Specifically, the Pandas library is used to filter out unnecessary data and impute missing values ​​to format the data.

[0505] Step 3: Conduct a pedestrian flow forecast.

[0506] The server executes a machine learning algorithm using a generative AI model based on the integrated data. The clean data from the previous step is used as input. The output generates predictions of people flow. Specifically, it builds a prediction model using the scikit-learn library and trains the data to predict future fluctuations in people flow.

[0507] Step 4: Proposal for optimizing transportation infrastructure

[0508] The server proposes optimized traffic infrastructure operations based on pedestrian flow prediction results. The input is the pedestrian flow prediction results. The output is a specific operational proposal to the traffic infrastructure management system. The server generates an operational plan based on the prediction and distributes the proposal via an API.

[0509] Step 5: Demand forecasting and energy-saving measures in commercial facilities

[0510] The terminal acquires sensor information installed in commercial facilities and combines it with payment data. Inputs include sensor and electronic transaction information. Outputs include demand forecasting and energy-saving measures for the facility. A Raspberry Pi is used to collect data from sensors and optimize temperature control as needed.

[0511] Step 6: Disaster Risk Assessment

[0512] The server analyzes data from high-altitude observation systems and ground sensors to assess disaster risk. Observation data and sensor data are used as input. The output is the issuance of warnings to local governments and related organizations. Specifically, it assesses risk based on collected data and automatically sends notifications if a certain threshold is exceeded.

[0513] Step 7: Notifying users of the action plan

[0514] Users receive personalized behavioral guidelines through a smartphone app. Inputs include information on pedestrian flow forecasts and energy-saving measures. Outputs include optimal travel routes and energy-saving instructions sent to the user. The app uses push notifications and map display functions to provide suggestions for efficient actions.

[0515] (Application Example 1)

[0516] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0517] In modern cities, there is a need to respond quickly and efficiently to traffic congestion and fluctuations in demand for commercial facilities. Furthermore, environmental considerations necessitate the optimization of energy-saving measures. Meanwhile, in logistics, the use of real-time data is required to enable efficient operations. However, conventional methods have struggled to address these multifaceted challenges in a unified manner. Therefore, the challenge lies in integrating and analyzing data obtained from multiple sources to achieve efficient urban management and optimized logistics.

[0518] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0519] In this invention, the server includes means for acquiring location data from a location information source, means for acquiring search data from an information search source, and means for acquiring transaction data from an electronic transaction source. This makes it possible to provide a system that integrates data, makes predictions based on a learning algorithm, and enables the efficiency of transportation, demand forecasting for commercial facilities, adjustment of energy-saving measures, and optimization of logistics operations.

[0520] A "location information source" is an information source that provides location data and is used to obtain information about geographical location.

[0521] An "information search source" is an information source that provides search data, acquiring data based on users' search queries and actions.

[0522] An "electronic transaction source" is a source of information that provides transaction data, allowing access to transaction details and timing.

[0523] "Data preparation" is the process of correcting acquired data to make it accurate and consistent, and converting it into a state that can be analyzed.

[0524] A "learning algorithm" is a computational method used to make predictions and classifications based on data, and is used in building models in machine learning.

[0525] A "transportation system" is a system of transportation that provides a means of moving goods and people.

[0526] "Energy conservation measures" are policies aimed at efficiently managing and reducing energy consumption.

[0527] A "high-altitude platform" is a foundation for collecting wide-area data from the air and is used for monitoring the environment, disasters, and other phenomena.

[0528] A "smart device" is a computing device that has data communication capabilities and can run various applications.

[0529] "Operational support" refers to the provision of information and assistance to support the efficiency and smooth operation of business processes.

[0530] "Route suggestion" aims to reduce time and costs by proposing the optimal travel route.

[0531] To implement this invention, it is necessary to construct a system that effectively collects data from various data sources and performs analysis based on that data. Specifically, the server acquires corresponding data from location information sources, information search sources, and electronic transaction sources. Location information provides data on geographical location, search data indicates user interests and behavior, and transaction data shows details of commercial activities. This data is integrated and organized on the server, and machine learning algorithms are used to predict human flow and optimize supply and demand.

[0532] The hardware used includes servers for computational processing, utilizing cloud platforms (such as AWS). For software, machine learning libraries like TensorFlow are used for data analysis.

[0533] The terminal uses a smart device to run an application that provides real-time operational support. As part of this operational support, the terminal provides optimal route suggestions for logistics operations.

[0534] For example, in operations at a logistics center in Tokyo, this data can be used to calculate efficient delivery routes for the following day, thus avoiding traffic congestion. This improves logistics efficiency and reduces the burden on drivers. Such proposals are made through generative AI models, and improved prediction accuracy is expected.

[0535] An example of a prompt message is, "Analyze logistics data within Tokyo and propose the optimal delivery route for tomorrow." This prompt message enables concrete operational support.

[0536] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0537] Step 1:

[0538] The server acquires data from location sources, information search sources, and electronic transaction sources. This data acquisition is performed via APIs and sent to the server. The inputs are location data, search data, and transaction data, and the output is a dataset of these data.

[0539] Step 2:

[0540] The server classifies and organizes the acquired data and stores it in a database. This prepares the information necessary for generating prompt statements. Data preparation includes imputing missing data and removing outliers. The input is a raw dataset, and the output is an integrated database.

[0541] Step 3:

[0542] The server utilizes a generative AI model with well-organized data to perform supply and demand forecasting. It applies machine learning algorithms and analyzes past trends to predict future pedestrian and commercial demand. The input is integrated data, and the output is the forecast result.

[0543] Step 4:

[0544] The terminal receives forecast results from the server and calculates a route plan for optimal operation. Specifically, it optimizes vehicle dispatch and delivery routes based on predicted congestion and demand. The input is the supply and demand forecast result, and the output is the optimized route suggestion.

[0545] Step 5:

[0546] Users receive optimized route suggestions via smart devices and perform operations based on them. This enables effective work execution, such as avoiding traffic congestion. The input is optimized route information, and the output is improved efficiency in actual work.

[0547] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0548] This invention expands the functionality of a smart city solution system, originally developed to improve the efficiency of urban functions, by incorporating an emotion engine that recognizes user emotions. This system covers everything from data collection and analysis to prediction and the provision of interactive services to users.

[0549] Data collection and analysis

[0550] The server periodically collects location information, search data, and payment data from each provider via APIs. The emotion engine also collects emotion-related data from the user's digital interactions, and all of this data is integrated into a database on the server.

[0551] Application of the emotion engine

[0552] The server processes the integrated data through an emotion engine to recognize the user's emotional state. This process extracts emotions from sources such as voice tone, text analysis, and payment patterns, and records them in a database.

[0553] pedestrian flow prediction and personalization

[0554] The server leverages integrated data and recognized sentiment data to apply machine learning algorithms and predict pedestrian traffic and urban demand. The compiled dataset is used to predict people's behavioral trends and analyze congestion levels in specific areas.

[0555] Optimization of transportation operations and commercial activities

[0556] The server generates suggestions to optimize transportation operations based on predictions. For example, if there is a high level of negative emotion, it adjusts the transportation plan to alleviate traffic flow in that area. In commercial facilities, it can also change promotions and service offerings based on user emotions.

[0557] User Feedback

[0558] Users receive personalized suggestions based on their emotional state through a smartphone app. For example, if they are experiencing prolonged stress, they may receive notifications via the app suggesting relaxation spots or routes to avoid crowds.

[0559] The system of this invention can improve the quality of urban life by enabling the provision of situation-appropriate services that take into account the user's emotions. This is expected to contribute to improved convenience in urban areas and the realization of a sustainable society.

[0560] The following describes the processing flow.

[0561] Step 1:

[0562] The server periodically collects data from location service providers, information retrieval service providers, and electronic payment service providers via APIs. The collected data includes longitude and latitude, search queries, and transaction amounts.

[0563] Step 2:

[0564] The server uses an emotion engine to analyze emotional data from the user's digital interactions (voice, text, payment patterns, etc.). The emotion engine infers emotions from voice tone and text content and adds emotional states such as positive and negative to the database.

[0565] Step 3:

[0566] The server uses machine learning algorithms to predict pedestrian flow based on all the integrated data. This prediction combines location information and sentiment data to accurately predict congestion levels in specific areas.

[0567] Step 4:

[0568] The server generates suggestions for optimizing transportation operations based on pedestrian flow predictions and sentiment data. For example, if there is a high level of negative sentiment in a particular area, it will suggest adjusting public transportation services to increase those services in that area.

[0569] Step 5:

[0570] The terminal uses sensor and payment data from commercial facilities to forecast demand. It leverages data from an emotion engine to generate suggestions for promotions and service adjustments tailored to the user's emotional state.

[0571] Step 6:

[0572] Users receive personalized recommendations based on emotional data through a smartphone app. If the app determines that a user is experiencing excessive stress, it will notify them of relaxing spots or routes that avoid crowds.

[0573] Step 7:

[0574] The server collects data from a wide area and integrates it with information from ground sensors to assess environmental risks. Especially when disaster risk is high, it proposes crisis management plans to local governments that also take emotional data into consideration, encouraging a swift response.

[0575] (Example 2)

[0576] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0577] Modern cities, facing population growth and rapid urbanization, demand efficient urban management. In particular, predicting traffic congestion and demand for commercial facilities are critical issues that significantly impact residents' quality of life. Furthermore, considering people's emotional states is essential for providing more optimized services. However, conventional systems struggle to integrate various data, perform sentiment analysis, and provide appropriate predictions and feedback.

[0578] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0579] In this invention, the server includes means for acquiring location information from a location information service provider, means for acquiring search data from an information retrieval service provider, and means for acquiring transaction data from an electronic transaction service provider. This enables the efficient operation of urban functions and the optimization of public transportation, taking into account the emotional state of users, as well as improved accuracy in demand forecasting at commercial facilities.

[0580] A "location information service provider" is an entity that is responsible for acquiring and providing location information.

[0581] An "information retrieval service provider" is an entity that operates a system that collects and provides relevant information based on search queries from users.

[0582] An "electronic transaction service provider" is an entity that provides online commercial transactions, including digital payments.

[0583] "Data correction" refers to the process of processing and formatting acquired data to improve its consistency and accuracy.

[0584] A "machine learning algorithm" is a mathematical method used to learn patterns and rules from data and perform predictions and classifications.

[0585] "Population movement forecasting" is the act of analyzing location information and other data to predict people's movement patterns and trends.

[0586] "Sentiment analysis" is a technology that identifies and quantifies a user's emotional state based on their text and actions.

[0587] "Optimizing public transport operations" refers to adjusting operating schedules and routes with the aim of achieving efficient traffic flow and service delivery.

[0588] A "commercial facility" is a facility established for the purpose of providing goods or services.

[0589] "Personalized behavioral guidelines" are specific and appropriate instructions provided based on the circumstances and conditions of a particular user.

[0590] This invention is a system aimed at the efficient operation of urban functions and improving the quality of life for residents. The server collects location information, search data, and transaction data, performs sentiment analysis, and aims to optimize traffic and enhance services at commercial facilities.

[0591] The server first acquires data from location service providers, information retrieval service providers, and electronic transaction service providers via APIs. The server uses Google Maps API, online search engine API, and electronic payment API. The acquired data is integrated into a database. After data correction, this integrated data is used to predict population movement using machine learning algorithms. By using open-source platforms such as TensorFlow and PyTorch, predictions with high accuracy can be achieved.

[0592] Simultaneously, the server processes the user's social media posts and voice interactions through an emotion analysis engine, extracting emotional data using NLP (Neuro-Linguistic Programming) technology. By utilizing Google Cloud Natural Language or similar services, the user's emotional state is quantified and recorded in a database.

[0593] Through the user's device, the server provides personalized suggestions based on the analysis results. The device is the user's smartphone app, which is developed using Flutter or React Native. The optimal action plan generated by the server is sent to the device, and suggestions based on the user's emotional state are delivered via push notifications.

[0594] For example, if a user searches for "tired" several times, the system will provide information about relaxing parks and cafes. An example of a prompt message would be text like this: "Please suggest ways to improve the quality of urban life. The user's current emotional state is 'stressed'. Available resources are parks and cafes."

[0595] It is expected that this system will improve the quality of urban life and create a convenient and sustainable environment for residents.

[0596] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0597] Step 1:

[0598] The server collects data. It receives real-time data as input from location service providers, information retrieval service providers, and e-commerce service providers. The server retrieves this data via APIs and stores it in a database. Specifically, it sends requests to API endpoints every minute, organizes the retrieved data, and stores it in MongoDB or MySQL.

[0599] Step 2:

[0600] The server collects and analyzes emotional data in parallel. Inputs include user social media posts and voice interactions; this data is sent to an emotional analysis engine for processing. The server utilizes NLP and voice analysis technologies to generate emotional scores and record them in a database. Specifically, it uses the Google Cloud Natural Language API to quantify emotional states and adds the results to the dataset.

[0601] Step 3:

[0602] The server integrates and corrects the data. The inputs are location information, search data, transaction data, and sentiment data obtained in steps 1 and 2. The server combines this data and formats it into a consistent format through a data correction process. Specifically, it stores the corrected data in a new table and prepares a data mart dedicated to analysis.

[0603] Step 4:

[0604] The server performs analysis by applying machine learning algorithms. The input is integrated and corrected data, and the server uses TensorFlow to build a population flow prediction model. The output is a forecast of pedestrian flow trends and congestion in a specific area. Specifically, batch processing is performed daily to update the latest population flow model.

[0605] Step 5:

[0606] The server proposes an action plan based on the analysis results. Here, it uses the pedestrian flow prediction data and emotional state data from Step 4 as input. The server adjusts transportation schedules and generates promotional strategies for commercial facilities. Specifically, it sends proposals to business systems via email notifications and APIs, and receives feedback as it progresses.

[0607] Step 6:

[0608] Users receive personalized insights via their devices. Input consists of suggestions received from the server, which users then review using a smartphone app. Output includes information on parks for stress relief and optimal travel routes. In terms of specific actions, users receive push notifications with "recommended actions appropriate to their current state."

[0609] (Application Example 2)

[0610] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0611] In recent years, with the increase in urban populations and the hosting of events, maintaining public safety and providing a comfortable urban environment have become crucial issues. However, currently, it is difficult to adequately address these issues, and there is a particular need for enhanced security in places where large numbers of people gather and smooth management of pedestrian flow. This invention aims to achieve more effective and rapid assurance of public safety and improve the efficiency of urban management by integrating technology that visualizes users' emotional states into urban management systems.

[0612] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0613] In this invention, the server includes means for acquiring spatial data from a location information providing device, means for acquiring search information from an information retrieval providing device, and means for acquiring transaction information from a payment processing providing device. This makes it possible to sense the emotional state of surrounding individuals in real time and provide personalized behavioral guidelines.

[0614] A "location information provider" is a device that collects location-related data and provides it to other devices or systems.

[0615] "Spatial data" refers to data that represents information about geographical location or space.

[0616] An "information retrieval and provision device" is a device that acquires and provides relevant information in response to a user's search request.

[0617] "Search information" refers to data related to keywords and results obtained when a user performs a search.

[0618] A "payment processing and provision device" is a device that collects payment data in transactions and provides it to other devices or systems.

[0619] "Transaction information" refers to data related to purchasing activities conducted through electronic payments.

[0620] "Data cleansing" is a method of processing collected data to improve its quality, such as imputing missing values ​​and correcting outliers.

[0621] A "learning algorithm" is a method that learns patterns based on past data and uses that knowledge to make predictions and classifications about future data.

[0622] A "transportation system" is a system or mechanism that provides means of transporting people and goods.

[0623] "Personalization" refers to providing information and services that are optimized according to each user's characteristics and circumstances.

[0624] A "visual device" is a device that presents external information visually to the wearer and provides it to them.

[0625] The system used to realize this application example takes the form of a server that acquires and analyzes diverse data and provides information to various devices. The server acquires spatial data from a location information provider, search information from an information retrieval provider, and transaction information from a payment processing provider. This data is then integrated and subjected to data cleansing.

[0626] Based on a comprehensive dataset, the server uses learning algorithms to predict pedestrian flow and emotional states. This process utilizes libraries such as Python and TensorFlow to train machine learning models. The server further processes real-time data acquired through sensors mounted on visual devices using an emotion engine. This emotion engine leverages natural language processing technologies like IBM Watson to analyze speech tone and text.

[0627] Users can monitor the emotional state of their surroundings in real time through a visual device. This device consists of smart glasses, such as Google Glass, and the recognized information is presented to the user as a simplified heatmap. This enables rapid responses to public safety and congestion levels. For example, security staff can be effectively deployed to areas where anxiety levels are high during holiday events.

[0628] An example of a prompt to the generating AI model would be, "Identify the area with the most emotionally unstable situation and quickly notify security staff of the details." This would allow security staff to focus their efforts on the area where the situation is most severe.

[0629] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0630] Step 1:

[0631] The server acquires spatial data from a location information provider. The input is real-time latitude and longitude data transmitted from the location information provider, and the output is the accurate storage of that data on the server. The server records this data in a database and prepares it for later analysis.

[0632] Step 2:

[0633] The server retrieves search information from the information retrieval device. The input is the search query entered by the user, and the output is data related to the search results. The server records this information as search trends and uses it in predictive models.

[0634] Step 3:

[0635] The server retrieves transaction information from the payment processing device. The input is detailed data about the user's transactions, and the output is an organized list of that data. The server analyzes transaction patterns and models which time periods have the most purchasing activity.

[0636] Step 4:

[0637] The server performs data cleansing based on the integrated data. The input is all location, search, and transaction data, and the output is a clean dataset with noise removed. The server improves data accuracy by imputing missing values ​​and removing outliers.

[0638] Step 5:

[0639] The server applies a learning algorithm using a clean dataset to predict pedestrian flow and emotional states. The input is a clean dataset, and the output is the prediction result. Python and TensorFlow are used to train the learning model and calculate future pedestrian flow patterns.

[0640] Step 6:

[0641] The server incorporates an emotion engine into the visual device and analyzes data acquired in real time. The input is video and audio data acquired by the visual device, and the output is emotional states obtained through text analysis and speech tone analysis. Natural language processing technologies, such as IBM Watson, are used to visualize these emotions.

[0642] Step 7:

[0643] Users can view the emotional state of their surroundings in real time through a visual device. The input is emotion visualization data transmitted by the server, and the output is a heat map presented to the user. Based on this information, users can quickly respond to the situation in a specific area.

[0644] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0645] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0646] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0647] [Fourth Embodiment]

[0648] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0649] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0650] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0651] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0652] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0653] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0654] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0655] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0656] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0657] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0658] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0659] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0660] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0661] This invention provides a data-centric smart city solution system aimed at optimizing urban functions. This system acquires necessary data from various service providers and analyzes it in an integrated format, enabling improved transportation efficiency, demand forecasting for commercial facilities, measurement of the effectiveness of energy-saving measures, and disaster response based on wide-area data.

[0662] Data acquisition and integration

[0663] The server periodically collects data from location service providers, information retrieval service providers, and electronic payment service providers. Data collection is performed using APIs, and the obtained information is stored in a database on the server in real time. Location data includes longitude, latitude, and time data, while retrieval data includes search queries and related time zones. Payment data includes transaction amounts and time information.

[0664] Data analysis and forecasting

[0665] The server integrates the collected data and removes inaccurate data through data cleaning. This improves data accuracy and converts it into a format that can be input into the pedestrian flow prediction model. Pedestrian flow prediction is performed using machine learning algorithms to predict future human movement based on historical data.

[0666] Optimization of public transport operations

[0667] The server utilizes the pedestrian flow prediction results to optimize transportation schedules. For example, it suggests to the transportation management system that the number of buses should be increased during predicted peak hours. Furthermore, if congestion is expected in a specific area, it provides feedback to subway and train service information to improve passenger convenience.

[0668] Demand forecasting and energy conservation for commercial facilities

[0669] The terminal acquires information from sensors installed in commercial facilities and combines it with payment data to predict demand. Based on the demand forecast, measures to reduce energy consumption during business hours are automatically applied. For example, temperature control is appropriately managed during peak hours to reduce power consumption.

[0670] Disaster response and environmental monitoring

[0671] The server analyzes environmental data by integrating wide-area data collected by high-altitude platforms with data from ground-based sensors. This allows it to issue warnings before disasters such as floods and heavy rains occur, prompting local governments and related organizations to take countermeasures.

[0672] Providing services to users

[0673] Users receive personalized behavioral guidelines based on their individual data through a provided smartphone app. This app notifies users of suggestions for appropriate travel times and energy-saving behaviors, improving their quality of life. For example, it displays suggestions for travel routes that avoid congestion and specific instructions to help save energy.

[0674] The system of this invention will help improve the efficiency of critical infrastructure in cities and contribute to the realization of a sustainable urban environment.

[0675] The following describes the processing flow.

[0676] Step 1:

[0677] The server retrieves data in real time via APIs from location information service providers, information retrieval service providers, and electronic payment service providers. This allows a wide range of data, such as longitude, latitude, search queries, and transaction amounts, to be stored in the server's database.

[0678] Step 2:

[0679] The server integrates the collected data and removes inaccurate and duplicate data through a data cleaning process. This step also standardizes the format and eliminates outliers, preparing the data for analysis.

[0680] Step 3:

[0681] The server uses clean data and applies machine learning algorithms to predict future pedestrian traffic. It analyzes past trends, builds predictive models based on people's movement patterns, and forecasts pedestrian traffic for the next 24 hours and week.

[0682] Step 4:

[0683] The server uses the prediction results to make suggestions for optimizing transportation schedules. These suggestions are sent to the traffic management system, which then develops operational plans to provide additional traffic during peak hours.

[0684] Step 5:

[0685] The terminal collects real-time data from sensors installed in commercial facilities and combines it with payment data to predict demand. Based on the demand forecast, the facility's air conditioning and lighting are adjusted to improve energy efficiency.

[0686] Step 6:

[0687] The server acquires wide-area data through a high-altitude platform and integrates it with ground sensor data to conduct environmental risk assessments. By combining this with meteorological data, it enables the early issuance of warnings for potential disasters.

[0688] Step 7:

[0689] Users receive personalized suggestions based on predicted pedestrian flow data and energy consumption through a smartphone app. The app provides users with optimal travel routes and specific guidance for energy saving.

[0690] (Example 1)

[0691] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0692] The increasing complexity of urban functions and population concentration have made it difficult to efficiently operate transportation infrastructure and commercial facilities. In addition, there are challenges such as the need for early prediction of disaster risks and optimal energy consumption. Conventional technologies have not provided an efficient system to comprehensively analyze this data and translate it into appropriate actions.

[0693] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0694] In this invention, the server includes means for acquiring location information from location information data providers, means for acquiring search information from data search providers, and means for acquiring transaction information from electronic transaction information providers. This enables the integrated analysis of urban information, optimization of transportation infrastructure, demand forecasting for commercial facilities, disaster risk assessment, and provision of personalized action plans.

[0695] A "location data provider" refers to an entity that provides location information services, typically responsible for accurately measuring the location of a target and providing the results to an external system.

[0696] A "data search provider" is an entity that makes information and data searchable and provides relevant information based on the user's search query.

[0697] An "electronic transaction information provider" is an entity that manages and provides information on electronically conducted transactions, recording details such as transaction amounts and dates, and sharing them with third parties.

[0698] "People movement" refers to the patterns and flows of people's movements in a specific region or time period, and is analyzed based on observational data.

[0699] "Optimization of transportation infrastructure operations" refers to the act of optimizing the operation plan in order to maximize the efficiency of the transportation system, and mainly includes adjusting operations in response to user demand.

[0700] "Demand forecasting for commercial facilities" refers to predicting the demand necessary for future sales activities and service provision at commercial facilities, and is analyzed based on past data.

[0701] "Resource consumption reduction measures" refer to measures and methods aimed at reducing the use of energy and materials, and are implemented with the goal of efficient operation.

[0702] A "high-altitude observation system" typically refers to a system that collects data over a wide area from an advanced position, such as an aircraft or satellite, enabling observations over a wider area than those near the Earth's surface.

[0703] "Assessing the risk of disaster" means quantitatively or qualitatively analyzing the risk of natural disasters occurring and predicting their impact.

[0704] An "individualized action plan" refers to suggestions and instructions for actions optimized according to the specific needs and circumstances of a particular user, with the aim of providing personalized information.

[0705] To implement this invention, the server first collects necessary data from location data providers, data retrieval providers, and electronic transaction information providers. Specifically, it periodically sends HTTP requests via APIs, parses the responses received, and stores them in a database. This process is typically carried out using Python or Java libraries. The server then integrates this data, cleans it using the Pandas library, and ensures its accuracy.

[0706] Next, the server uses the collected clean data to run a machine learning algorithm equipped with a generative AI model. In predicting people's movement, the server utilizes the scikit-learn library to create a predictive model based on historical data. This model enables optimization of transportation infrastructure operations, demand forecasting in commercial facilities, and reduction of resource consumption.

[0707] For example, prompts such as "Please tell me the best route to get to the office at 10 AM next Monday" or "Please suggest ways to reduce energy consumption at commercial facilities this weekend" are presented. Based on these prompts, the user can receive optimized information provided by the server.

[0708] Furthermore, the terminal acquires environmental information from IoT sensors installed in commercial facilities and performs real-time data analysis. Using hardware such as Raspberry Pi, it collects data and then optimizes the control of air conditioning and lighting on the terminal. Users can receive personalized behavioral guidelines through a smartphone app, which is expected to improve their quality of life. By providing users with optimal guidance tailored to specific times and situations, this system can contribute to urban sustainability.

[0709] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0710] Step 1: Data Collection

[0711] The server retrieves data from location data providers, data search providers, and electronic transaction information providers. As input, the server sends HTTP requests through each provider's API. As output, location information, search information, and transaction information are retrieved from these providers in JSON format and stored in the database. The server uses the Python Requests library to process requests and analyze the obtained data.

[0712] Step 2: Data Integration and Cleaning

[0713] The server integrates the collected data and removes inaccurate data. Raw data from multiple data sources is provided as input. The output is a clean, integrated dataset. Specifically, the Pandas library is used to filter out unnecessary data and impute missing values ​​to format the data.

[0714] Step 3: Conduct a pedestrian flow forecast.

[0715] The server executes a machine learning algorithm using a generative AI model based on the integrated data. The clean data from the previous step is used as input. The output generates predictions of people flow. Specifically, it builds a prediction model using the scikit-learn library and trains the data to predict future fluctuations in people flow.

[0716] Step 4: Proposal for optimizing transportation infrastructure

[0717] The server proposes optimized traffic infrastructure operations based on pedestrian flow prediction results. The input is the pedestrian flow prediction results. The output is a specific operational proposal to the traffic infrastructure management system. The server generates an operational plan based on the prediction and distributes the proposal via an API.

[0718] Step 5: Demand forecasting and energy-saving measures in commercial facilities

[0719] The terminal acquires sensor information installed in commercial facilities and combines it with payment data. Inputs include sensor and electronic transaction information. Outputs include demand forecasting and energy-saving measures for the facility. A Raspberry Pi is used to collect data from sensors and optimize temperature control as needed.

[0720] Step 6: Disaster Risk Assessment

[0721] The server analyzes data from high-altitude observation systems and ground sensors to assess disaster risk. Observation data and sensor data are used as input. The output is the issuance of warnings to local governments and related organizations. Specifically, it assesses risk based on collected data and automatically sends notifications if a certain threshold is exceeded.

[0722] Step 7: Notifying users of the action plan

[0723] Users receive personalized behavioral guidelines through a smartphone app. Inputs include information on pedestrian flow forecasts and energy-saving measures. Outputs include optimal travel routes and energy-saving instructions sent to the user. The app uses push notifications and map display functions to provide suggestions for efficient actions.

[0724] (Application Example 1)

[0725] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0726] In modern cities, there is a need to respond quickly and efficiently to traffic congestion and fluctuations in demand for commercial facilities. Furthermore, environmental considerations necessitate the optimization of energy-saving measures. Meanwhile, in logistics, the use of real-time data is required to enable efficient operations. However, conventional methods have struggled to address these multifaceted challenges in a unified manner. Therefore, the challenge lies in integrating and analyzing data obtained from multiple sources to achieve efficient urban management and optimized logistics.

[0727] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0728] In this invention, the server includes means for acquiring location data from a location information source, means for acquiring search data from an information search source, and means for acquiring transaction data from an electronic transaction source. This makes it possible to provide a system that integrates data, makes predictions based on a learning algorithm, and enables the efficiency of transportation, demand forecasting for commercial facilities, adjustment of energy-saving measures, and optimization of logistics operations.

[0729] A "location information source" is an information source that provides location data and is used to obtain information about geographical location.

[0730] An "information search source" is an information source that provides search data, acquiring data based on users' search queries and actions.

[0731] An "electronic transaction source" is a source of information that provides transaction data, allowing access to transaction details and timing.

[0732] "Data preparation" is the process of correcting acquired data to make it accurate and consistent, and converting it into a state that can be analyzed.

[0733] A "learning algorithm" is a computational method used to make predictions and classifications based on data, and is used in building models in machine learning.

[0734] A "transportation system" is a system of transportation that provides a means of moving goods and people.

[0735] "Energy conservation measures" are policies aimed at efficiently managing and reducing energy consumption.

[0736] A "high-altitude platform" is a foundation for collecting wide-area data from the air and is used for monitoring the environment, disasters, and other phenomena.

[0737] A "smart device" is a computing device that has data communication capabilities and can run various applications.

[0738] "Operational support" refers to the provision of information and assistance to support the efficiency and smooth operation of business processes.

[0739] "Route suggestion" aims to reduce time and costs by proposing the optimal travel route.

[0740] To implement this invention, it is necessary to construct a system that effectively collects data from various data sources and performs analysis based on that data. Specifically, the server acquires corresponding data from location information sources, information search sources, and electronic transaction sources. Location information provides data on geographical location, search data indicates user interests and behavior, and transaction data shows details of commercial activities. This data is integrated and organized on the server, and machine learning algorithms are used to predict human flow and optimize supply and demand.

[0741] The hardware used includes servers for computational processing, utilizing cloud platforms (such as AWS). For software, machine learning libraries like TensorFlow are used for data analysis.

[0742] The terminal uses a smart device to run an application that provides real-time operational support. As part of this operational support, the terminal provides optimal route suggestions for logistics operations.

[0743] For example, in operations at a logistics center in Tokyo, this data can be used to calculate efficient delivery routes for the following day, thus avoiding traffic congestion. This improves logistics efficiency and reduces the burden on drivers. Such proposals are made through generative AI models, and improved prediction accuracy is expected.

[0744] An example of a prompt message is, "Analyze logistics data within Tokyo and propose the optimal delivery route for tomorrow." This prompt message enables concrete operational support.

[0745] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0746] Step 1:

[0747] The server acquires data from location sources, information search sources, and electronic transaction sources. This data acquisition is performed via APIs and sent to the server. The inputs are location data, search data, and transaction data, and the output is a dataset of these data.

[0748] Step 2:

[0749] The server classifies and organizes the acquired data and stores it in a database. This prepares the information necessary for generating prompt statements. Data preparation includes imputing missing data and removing outliers. The input is a raw dataset, and the output is an integrated database.

[0750] Step 3:

[0751] The server utilizes a generative AI model with well-organized data to perform supply and demand forecasting. It applies machine learning algorithms and analyzes past trends to predict future pedestrian and commercial demand. The input is integrated data, and the output is the forecast result.

[0752] Step 4:

[0753] The terminal receives forecast results from the server and calculates a route plan for optimal operation. Specifically, it optimizes vehicle dispatch and delivery routes based on predicted congestion and demand. The input is the supply and demand forecast result, and the output is the optimized route suggestion.

[0754] Step 5:

[0755] Users receive optimized route suggestions via smart devices and perform operations based on them. This enables effective work execution, such as avoiding traffic congestion. The input is optimized route information, and the output is improved efficiency in actual work.

[0756] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0757] This invention expands the functionality of a smart city solution system, originally developed to improve the efficiency of urban functions, by incorporating an emotion engine that recognizes user emotions. This system covers everything from data collection and analysis to prediction and the provision of interactive services to users.

[0758] Data collection and analysis

[0759] The server periodically collects location information, search data, and payment data from each provider via APIs. The emotion engine also collects emotion-related data from the user's digital interactions, and all of this data is integrated into a database on the server.

[0760] Application of the emotion engine

[0761] The server processes the integrated data through an emotion engine to recognize the user's emotional state. This process extracts emotions from sources such as voice tone, text analysis, and payment patterns, and records them in a database.

[0762] pedestrian flow prediction and personalization

[0763] The server leverages integrated data and recognized sentiment data to apply machine learning algorithms and predict pedestrian traffic and urban demand. The compiled dataset is used to predict people's behavioral trends and analyze congestion levels in specific areas.

[0764] Optimization of transportation operations and commercial activities

[0765] The server generates suggestions to optimize transportation operations based on predictions. For example, if there is a high level of negative emotion, it adjusts the transportation plan to alleviate traffic flow in that area. In commercial facilities, it can also change promotions and service offerings based on user emotions.

[0766] User Feedback

[0767] Users receive personalized suggestions based on their emotional state through a smartphone app. For example, if they are experiencing prolonged stress, they may receive notifications via the app suggesting relaxation spots or routes to avoid crowds.

[0768] The system of this invention can improve the quality of urban life by enabling the provision of situation-appropriate services that take into account the user's emotions. This is expected to contribute to improved convenience in urban areas and the realization of a sustainable society.

[0769] The following describes the processing flow.

[0770] Step 1:

[0771] The server periodically collects data from location service providers, information retrieval service providers, and electronic payment service providers via APIs. The collected data includes longitude and latitude, search queries, and transaction amounts.

[0772] Step 2:

[0773] The server uses an emotion engine to analyze emotional data from the user's digital interactions (voice, text, payment patterns, etc.). The emotion engine infers emotions from voice tone and text content and adds emotional states such as positive and negative to the database.

[0774] Step 3:

[0775] The server uses machine learning algorithms to predict pedestrian flow based on all the integrated data. This prediction combines location information and sentiment data to accurately predict congestion levels in specific areas.

[0776] Step 4:

[0777] The server generates suggestions for optimizing transportation operations based on pedestrian flow predictions and sentiment data. For example, if there is a high level of negative sentiment in a particular area, it will suggest adjusting public transportation services to increase those services in that area.

[0778] Step 5:

[0779] The terminal uses sensor and payment data from commercial facilities to forecast demand. It leverages data from an emotion engine to generate suggestions for promotions and service adjustments tailored to the user's emotional state.

[0780] Step 6:

[0781] Users receive personalized recommendations based on emotional data through a smartphone app. If the app determines that a user is experiencing excessive stress, it will notify them of relaxing spots or routes that avoid crowds.

[0782] Step 7:

[0783] The server collects data from a wide area and integrates it with information from ground sensors to assess environmental risks. Especially when disaster risk is high, it proposes crisis management plans to local governments that also take emotional data into consideration, encouraging a swift response.

[0784] (Example 2)

[0785] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0786] Modern cities, facing population growth and rapid urbanization, demand efficient urban management. In particular, predicting traffic congestion and demand for commercial facilities are critical issues that significantly impact residents' quality of life. Furthermore, considering people's emotional states is essential for providing more optimized services. However, conventional systems struggle to integrate various data, perform sentiment analysis, and provide appropriate predictions and feedback.

[0787] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0788] In this invention, the server includes means for acquiring location information from a location information service provider, means for acquiring search data from an information retrieval service provider, and means for acquiring transaction data from an electronic transaction service provider. This enables the efficient operation of urban functions and the optimization of public transportation, taking into account the emotional state of users, as well as improved accuracy in demand forecasting at commercial facilities.

[0789] A "location information service provider" is an entity that is responsible for acquiring and providing location information.

[0790] An "information retrieval service provider" is an entity that operates a system that collects and provides relevant information based on search queries from users.

[0791] An "electronic transaction service provider" is an entity that provides online commercial transactions, including digital payments.

[0792] "Data correction" refers to the process of processing and formatting acquired data to improve its consistency and accuracy.

[0793] A "machine learning algorithm" is a mathematical method used to learn patterns and rules from data and perform predictions and classifications.

[0794] "Population movement forecasting" is the act of analyzing location information and other data to predict people's movement patterns and trends.

[0795] "Sentiment analysis" is a technology that identifies and quantifies a user's emotional state based on their text and actions.

[0796] "Optimizing public transport operations" refers to adjusting operating schedules and routes with the aim of achieving efficient traffic flow and service delivery.

[0797] A "commercial facility" is a facility established for the purpose of providing goods or services.

[0798] "Personalized behavioral guidelines" are specific and appropriate instructions provided based on the circumstances and conditions of a particular user.

[0799] This invention is a system aimed at the efficient operation of urban functions and improving the quality of life for residents. The server collects location information, search data, and transaction data, performs sentiment analysis, and aims to optimize traffic and enhance services at commercial facilities.

[0800] The server first acquires data from location service providers, information retrieval service providers, and electronic transaction service providers via APIs. The server uses Google Maps API, online search engine API, and electronic payment API. The acquired data is integrated into a database. After data correction, this integrated data is used to predict population movement using machine learning algorithms. By using open-source platforms such as TensorFlow and PyTorch, predictions with high accuracy can be achieved.

[0801] Simultaneously, the server processes the user's social media posts and voice interactions through an emotion analysis engine, extracting emotional data using NLP (Neuro-Linguistic Programming) technology. By utilizing Google Cloud Natural Language or similar services, the user's emotional state is quantified and recorded in a database.

[0802] Through the user's device, the server provides personalized suggestions based on the analysis results. The device is the user's smartphone app, which is developed using Flutter or React Native. The optimal action plan generated by the server is sent to the device, and suggestions based on the user's emotional state are delivered via push notifications.

[0803] For example, if a user searches for "tired" several times, the system will provide information about relaxing parks and cafes. An example of a prompt message would be text like this: "Please suggest ways to improve the quality of urban life. The user's current emotional state is 'stressed'. Available resources are parks and cafes."

[0804] It is expected that this system will improve the quality of urban life and create a convenient and sustainable environment for residents.

[0805] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0806] Step 1:

[0807] The server collects data. It receives real-time data as input from location service providers, information retrieval service providers, and e-commerce service providers. The server retrieves this data via APIs and stores it in a database. Specifically, it sends requests to API endpoints every minute, organizes the retrieved data, and stores it in MongoDB or MySQL.

[0808] Step 2:

[0809] The server collects and analyzes emotional data in parallel. Inputs include user social media posts and voice interactions; this data is sent to an emotional analysis engine for processing. The server utilizes NLP and voice analysis technologies to generate emotional scores and record them in a database. Specifically, it uses the Google Cloud Natural Language API to quantify emotional states and adds the results to the dataset.

[0810] Step 3:

[0811] The server integrates and corrects the data. The inputs are location information, search data, transaction data, and sentiment data obtained in steps 1 and 2. The server combines this data and formats it into a consistent format through a data correction process. Specifically, it stores the corrected data in a new table and prepares a data mart dedicated to analysis.

[0812] Step 4:

[0813] The server performs analysis by applying machine learning algorithms. The input is integrated and corrected data, and the server uses TensorFlow to build a population flow prediction model. The output is a forecast of pedestrian flow trends and congestion in a specific area. Specifically, batch processing is performed daily to update the latest population flow model.

[0814] Step 5:

[0815] The server proposes an action plan based on the analysis results. Here, it uses the pedestrian flow prediction data and emotional state data from Step 4 as input. The server adjusts transportation schedules and generates promotional strategies for commercial facilities. Specifically, it sends proposals to business systems via email notifications and APIs, and receives feedback as it progresses.

[0816] Step 6:

[0817] Users receive personalized insights via their devices. Input consists of suggestions received from the server, which users then review using a smartphone app. Output includes information on parks for stress relief and optimal travel routes. In terms of specific actions, users receive push notifications with "recommended actions appropriate to their current state."

[0818] (Application Example 2)

[0819] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0820] In recent years, with the increase in urban populations and the hosting of events, maintaining public safety and providing a comfortable urban environment have become crucial issues. However, currently, it is difficult to adequately address these issues, and there is a particular need for enhanced security in places where large numbers of people gather and smooth management of pedestrian flow. This invention aims to achieve more effective and rapid assurance of public safety and improve the efficiency of urban management by integrating technology that visualizes users' emotional states into urban management systems.

[0821] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0822] In this invention, the server includes means for acquiring spatial data from a location information providing device, means for acquiring search information from an information retrieval providing device, and means for acquiring transaction information from a payment processing providing device. This makes it possible to sense the emotional state of surrounding individuals in real time and provide personalized behavioral guidelines.

[0823] A "location information provider" is a device that collects location-related data and provides it to other devices or systems.

[0824] "Spatial data" refers to data that represents information about geographical location or space.

[0825] An "information retrieval and provision device" is a device that acquires and provides relevant information in response to a user's search request.

[0826] "Search information" refers to data related to keywords and results obtained when a user performs a search.

[0827] A "payment processing and provision device" is a device that collects payment data in transactions and provides it to other devices or systems.

[0828] "Transaction information" refers to data related to purchasing activities conducted through electronic payments.

[0829] "Data cleansing" is a method of processing collected data to improve its quality, such as imputing missing values ​​and correcting outliers.

[0830] A "learning algorithm" is a method that learns patterns based on past data and uses that knowledge to make predictions and classifications about future data.

[0831] A "transportation system" is a system or mechanism that provides means of transporting people and goods.

[0832] "Personalization" refers to providing information and services that are optimized according to each user's characteristics and circumstances.

[0833] A "visual device" is a device that presents external information visually to the wearer and provides it to them.

[0834] The system used to realize this application example takes the form of a server that acquires and analyzes diverse data and provides information to various devices. The server acquires spatial data from a location information provider, search information from an information retrieval provider, and transaction information from a payment processing provider. This data is then integrated and subjected to data cleansing.

[0835] Based on a comprehensive dataset, the server uses learning algorithms to predict pedestrian flow and emotional states. This process utilizes libraries such as Python and TensorFlow to train machine learning models. The server further processes real-time data acquired through sensors mounted on visual devices using an emotion engine. This emotion engine leverages natural language processing technologies like IBM Watson to analyze speech tone and text.

[0836] Users can monitor the emotional state of their surroundings in real time through a visual device. This device consists of smart glasses, such as Google Glass, and the recognized information is presented to the user as a simplified heatmap. This enables rapid responses to public safety and congestion levels. For example, security staff can be effectively deployed to areas where anxiety levels are high during holiday events.

[0837] An example of a prompt to the generating AI model would be, "Identify the area with the most emotionally unstable situation and quickly notify security staff of the details." This would allow security staff to focus their efforts on the area where the situation is most severe.

[0838] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0839] Step 1:

[0840] The server acquires spatial data from a location information provider. The input is real-time latitude and longitude data transmitted from the location information provider, and the output is the accurate storage of that data on the server. The server records this data in a database and prepares it for later analysis.

[0841] Step 2:

[0842] The server retrieves search information from the information retrieval device. The input is the search query entered by the user, and the output is data related to the search results. The server records this information as search trends and uses it in predictive models.

[0843] Step 3:

[0844] The server retrieves transaction information from the payment processing device. The input is detailed data about the user's transactions, and the output is an organized list of that data. The server analyzes transaction patterns and models which time periods have the most purchasing activity.

[0845] Step 4:

[0846] The server performs data cleansing based on the integrated data. The input is all location, search, and transaction data, and the output is a clean dataset with noise removed. The server improves data accuracy by imputing missing values ​​and removing outliers.

[0847] Step 5:

[0848] The server applies a learning algorithm using a clean dataset to predict pedestrian flow and emotional states. The input is a clean dataset, and the output is the prediction result. Python and TensorFlow are used to train the learning model and calculate future pedestrian flow patterns.

[0849] Step 6:

[0850] The server incorporates an emotion engine into the visual device and analyzes data acquired in real time. The input is video and audio data acquired by the visual device, and the output is emotional states obtained through text analysis and speech tone analysis. Natural language processing technologies, such as IBM Watson, are used to visualize these emotions.

[0851] Step 7:

[0852] Users can view the emotional state of their surroundings in real time through a visual device. The input is emotion visualization data transmitted by the server, and the output is a heat map presented to the user. Based on this information, users can quickly respond to the situation in a specific area.

[0853] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0854] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0855] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0856] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0857] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0858] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0859] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0860] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0861] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0862] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0863] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0864] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0865] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0866] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0867] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0868] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0869] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0870] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0871] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0872] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0873] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0874] The following is further disclosed regarding the embodiments described above.

[0875] (Claim 1)

[0876] A means of obtaining location information from a location information service provider,

[0877] A means of obtaining search data from an information retrieval service provider,

[0878] A means of obtaining payment data from an electronic payment service provider,

[0879] A means for integrating acquired location information, search data, and payment data, and performing data cleaning,

[0880] A method for predicting pedestrian flow by applying machine learning algorithms based on integrated data,

[0881] A means of proposing the optimization of transportation operations based on human flow predictions,

[0882] A means of forecasting demand in commercial facilities and adjusting energy conservation measures,

[0883] A means of collecting wide-area data using a high-altitude platform and assessing disaster risk,

[0884] A means of providing personalized behavioral guidelines for users,

[0885] A system that includes this.

[0886] (Claim 2)

[0887] A means of performing pedestrian flow analysis in a specific area using data obtained from a location information service provider,

[0888] A means for detecting abnormal pedestrian flow patterns based on the analysis results,

[0889] The system according to claim 1, further comprising means for issuing a warning to a public authority based on a detected pattern.

[0890] (Claim 3)

[0891] A means of constructing a time-of-day demand fluctuation model for commercial facilities using electronic payment data,

[0892] The system according to claim 1, further comprising means for performing an energy consumption optimization operation based on a constructed model.

[0893] "Example 1"

[0894] (Claim 1)

[0895] A means of obtaining location information from a location data provider,

[0896] A means of obtaining search information from data search providers,

[0897] Means for obtaining transaction information from electronic transaction information providers,

[0898] A means of integrating acquired location information, search information, and transaction information, and performing data scrutiny,

[0899] A method for predicting people's movement by applying machine learning techniques based on integrated data,

[0900] A method for proposing the optimization of transportation infrastructure operations based on predictions of people's movement,

[0901] A means of forecasting demand in commercial facilities and adjusting measures to curb resource consumption,

[0902] A means of collecting wide-area information using a high-altitude observation system and assessing the risk of disaster,

[0903] A means of providing individualized action plans for users,

[0904] A system that includes this.

[0905] (Claim 2)

[0906] A means for performing human movement analysis in a specific area using data obtained from location data providers,

[0907] A means for detecting abnormal human movement patterns based on the analysis results,

[0908] The system according to claim 1, which issues a warning to a public institution based on a detected pattern.

[0909] (Claim 3)

[0910] A means for constructing a time-of-day demand fluctuation model for commercial facilities using electronic transaction information,

[0911] The system according to claim 1, which performs resource consumption optimization operations based on a constructed model.

[0912] "Application Example 1"

[0913] (Claim 1)

[0914] A means of obtaining location data from a location information source,

[0915] Means for obtaining search data from information search sources,

[0916] Means for obtaining transaction data from electronic transaction sources,

[0917] A means for integrating acquired location data, search data, and transaction data, and for preparing the data,

[0918] A method for predicting pedestrian flow by applying a learning algorithm based on integrated data,

[0919] A means of proposing ways to improve the efficiency of transportation operations based on passenger flow predictions,

[0920] A means of forecasting demand in commercial facilities and adjusting energy conservation measures,

[0921] A means of collecting wide-area data using a high-altitude platform and assessing disaster risk,

[0922] A means of providing operational support and suggesting the optimal route using smart devices,

[0923] A system that includes this.

[0924] (Claim 2)

[0925] A means for performing pedestrian flow analysis in a specific area using data obtained from a location information source,

[0926] A means for detecting abnormal pedestrian flow patterns based on the analysis results,

[0927] The system according to claim 1, comprising means for issuing a warning to a public authority based on a detected pattern.

[0928] (Claim 3)

[0929] A means for constructing a time-of-day demand fluctuation model for commercial facilities using electronic transaction data,

[0930] The system according to claim 1, comprising means for performing an energy consumption optimization operation based on a constructed model.

[0931] "Example 2 of combining an emotion engine"

[0932] (Claim 1)

[0933] Means for obtaining location information from location information service providers,

[0934] Means of obtaining search data from information retrieval service providers,

[0935] Means for obtaining transaction data from electronic transaction service providers,

[0936] A means for integrating acquired location information, search data, and transaction data, and for performing data correction,

[0937] A method for predicting population movement by applying machine learning algorithms based on integrated data,

[0938] A means of collecting emotional data from users' digital interactions and performing emotional analysis,

[0939] A means of advising on the optimization of public transport operations while taking into account the emotional state of users,

[0940] A means of adjusting sales promotion and service provision in commercial facilities according to user emotions,

[0941] A means of collecting wide-area data and evaluating crisis response measures,

[0942] A means of providing personalized behavioral guidelines for users,

[0943] A system that includes this.

[0944] (Claim 2)

[0945] The system according to claim 1, which performs population flow analysis in a limited area using data obtained from a location information service provider.

[0946] (Claim 3)

[0947] The system according to claim 1, which develops a time-of-day demand fluctuation model for commercial facilities using electronic transaction data.

[0948] "Application example 2 when combining with an emotional engine"

[0949] (Claim 1)

[0950] A means of acquiring spatial data from a location information provider,

[0951] A means for obtaining search information from an information retrieval and provision device,

[0952] A means for obtaining transaction information from a payment processing device,

[0953] A means for integrating acquired spatial data, search information, and transaction information, and performing data cleansing,

[0954] A method for predicting pedestrian flow by applying a learning algorithm based on integrated information,

[0955] A means of proposing the optimization of transportation operations based on passenger flow predictions,

[0956] A means of forecasting demand in commercial facilities and adjusting energy conservation measures,

[0957] A means of collecting wide-area data using a high-altitude platform and assessing crisis risk,

[0958] A means of providing personalized behavioral guidelines for users,

[0959] A means of sensing the emotional state of surrounding individuals using a visual device,

[0960] A system that includes this.

[0961] (Claim 2)

[0962] A means for performing pedestrian flow analysis in a specific area using data acquired from a spatial information provision device,

[0963] Based on the analysis results, a means to detect abnormal pedestrian flow patterns,

[0964] The system according to claim 1, further comprising means for sending a warning to a public authority based on the discovered pattern.

[0965] (Claim 3)

[0966] A means of constructing a seasonal demand fluctuation model for commercial facilities using transaction information,

[0967] The system according to claim 1, further comprising means for performing an energy consumption optimization operation based on a constructed model. [Explanation of symbols]

[0968] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of obtaining location information from a location information service provider, A means of obtaining search data from an information retrieval service provider, A means of obtaining payment data from an electronic payment service provider, A means for integrating acquired location information, search data, and payment data, and performing data cleaning, A method for predicting pedestrian flow by applying machine learning algorithms based on integrated data, A means of proposing the optimization of transportation operations based on human flow predictions, A means of forecasting demand in commercial facilities and adjusting energy conservation measures, A means of collecting wide-area data using a high-altitude platform and assessing disaster risk, A means of providing personalized behavioral guidelines for users, A system that includes this.

2. A means of performing pedestrian flow analysis in a specific area using data obtained from a location information service provider, A means for detecting abnormal pedestrian flow patterns based on the analysis results, The system according to claim 1, further comprising means for issuing a warning to a public authority based on the detected pattern.

3. A means of constructing a time-of-day demand fluctuation model for commercial facilities using electronic payment data, The system according to claim 1, further comprising means for performing an energy consumption optimization operation based on a constructed model.

Images