system

Abstract

We provide the system. [Solution] A means for collecting information from data acquisition devices and organizing information regarding material inventory and movement, A means of predicting material usage patterns by analyzing past consumption information and external factors, A means of automatically generating material ordering plans and placement proposals that take environmental factors into consideration, and notifying users of them, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] At a construction site, inventory management of materials and optimization of their placement are issues. Especially in large-scale projects, problems such as work interruption due to material shortages or surpluses and increased storage costs occur. Such issues increase the burden on site supervisors and are factors reducing the efficiency of the entire project.

Means for Solving the Problems

[0005] This invention provides a means for collecting data in real time using a device that organizes information regarding the inventory and movement of materials. Furthermore, it provides a means for optimizing material ordering plans and placement suggestions by analyzing past consumption information and external factors to predict material usage patterns. Moreover, by presenting these suggestions to the user and requesting their confirmation, it enables rapid and efficient material management.

[0006] A "data acquisition device" is a device used to collect information in real time from cameras, sensors, and other equipment installed on-site.

[0007] "Inventory" refers to the total quantity of goods and products stored in a specific location, and in this context, it specifically refers to the total amount of materials used at a construction site.

[0008] "Movement flow" refers to the path along which objects and people move, and it is an important factor that affects work efficiency and safety at construction sites.

[0009] "Past consumption information" refers to data that records the amount and patterns of material usage during a specific period.

[0010] "External factors" refer to events or conditions outside the work site that affect material consumption, such as weather or worker shift schedules.

[0011] "Usage pattern" refers to the tendencies and characteristics that indicate how often and under what conditions a material is used.

[0012] "Ordering planning" refers to planning the timing and quantity of materials needed for a specific period.

[0013] A "layout proposal" is the act of presenting recommended plans for the optimal placement of materials on-site.

[0014] "Users" refers to individuals such as site supervisors who use this system to manage materials and approve layout proposals. [Brief explanation of the drawing]

[0015] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0016] An example of an embodiment of the system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] In embodiments of the present invention, multiple components are coordinated to optimize material management. The main elements consist of a data acquisition device, a data processing module, an analysis engine, and a notification system.

[0037] In the data acquisition system, cameras and sensors installed on-site collect information in real time and transmit data on material inventory levels, location information, and movement patterns to a cloud server.

[0038] The server uses a data processing module to organize the collected information, remove noise data, and store it in the database. This makes it possible to always have a grasp of the latest field conditions.

[0039] The server then uses this data to run an analysis engine. The analysis engine uses historical consumption data and external factor data (such as weather information and worker schedules) to predict material consumption patterns.

[0040] Based on the predicted data, the server generates suggestions for optimal ordering plans and material allocation. For example, in a project that uses a lot of materials on rainy days, it will consider the weather forecast in advance and plan to order more than necessary.

[0041] The server generates a plan and proposes it to the user (site supervisor) via an app through a notification system. The user uses the app to review the plan and make adjustments or approvals as needed.

[0042] This system aims to prevent material shortages and excess inventory, maximizing on-site efficiency. This reduces the burden on site supervisors and supports the overall success of the project.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The server receives real-time inventory and location information of materials from on-site data acquisition devices. Each time data from sensors is updated, the server aggregates and centrally manages the data.

[0046] Step 2:

[0047] Before saving received data to the database, the server performs data cleaning. This removes inaccurate data and noise and verifies the integrity of inventory information.

[0048] Step 3:

[0049] The server predicts material consumption by combining historical consumption data with external factors such as weather and worker schedules. This involves using machine learning algorithms to analyze the factors that cause fluctuations in consumption patterns.

[0050] Step 4:

[0051] The server generates an optimal ordering plan for materials based on consumption forecasts. The timing and quantity of orders are determined considering inventory costs and lead times.

[0052] Step 5:

[0053] The server generates optimized material placement suggestions based on on-site layout information, taking into account traffic flow. Materials that are frequently accessed are placed particularly efficiently.

[0054] Step 6:

[0055] The server notifies the user of the generated order plan and placement proposal through the app. The notification includes options to help with decision-making and detailed suggestions.

[0056] Step 7:

[0057] Users check notifications via the app and approve or modify the plan based on the suggestions provided. Once approved, the information is returned to the system and implemented.

[0058] (Example 1)

[0059] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0060] Materials management in the industrial sector constantly faces problems such as inventory shortages and surpluses, and decreased work efficiency due to improper allocation. Furthermore, conventional systems struggle to collect and predict accurate information in real time, resulting in inadequate material procurement and allocation.

[0061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0062] In this invention, the server includes means for receiving data from a sensing device and organizing information regarding the inventory status and movement routes of materials; means for calculating and processing past usage data and external factors to predict material consumption patterns; and means for presenting a material procurement plan and placement proposal to the user and requesting their approval. This makes it possible to streamline material inventory management, improve work efficiency through optimized movement routes, and reduce the risk of insufficient or excessive preparation.

[0063] A "sensing device" refers to hardware used to record on-site conditions in real time and collect information about the inventory and movement of materials.

[0064] A "server" refers to a core information processing system that analyzes received data and generates a resource management plan.

[0065] "Inventory status" refers to data regarding the quantity and types of goods stored at a site or warehouse at a given time.

[0066] "Transportation route" refers to route information that shows how supplies are moved within the site.

[0067] "Computational processing" refers to the process of performing mathematical analysis using collected data, and specifically to the calculations necessary to predict the consumption patterns of materials.

[0068] "Consumption patterns of goods" refer to trends regarding how goods are used and the rate at which they are consumed, based on past usage data and external factors.

[0069] A "procurement plan" refers to a detailed plan for securing necessary supplies in the appropriate time and quantity.

[0070] A "layout proposal" refers to a plan that outlines how materials should be arranged on-site to ensure the most efficient use of them.

[0071] "Information organization" refers to the process of processing received data, removing noise, and prioritizing its importance to make it usable.

[0072] This invention is a system that works in conjunction with multiple components to improve the efficiency of material management.

[0073] The terminal uses sensing devices placed on-site, specifically cameras and sensors, to collect real-time data on the inventory status and movement routes of supplies. This data is immediately transmitted to a server in the cloud.

[0074] The server receives this data and uses a data processing module to organize the information. Image processing technology is used to analyze the video data from the camera, removing noise such as audio data to extract clear information. This organized information is stored in a database, allowing for a constantly updated understanding of the on-site situation.

[0075] Furthermore, the server uses an analysis engine to predict material consumption patterns using machine learning algorithms, based on past material usage data and external factors such as weather and worker schedules. For example, if it rains continuously, the amount of concrete used may increase, so the server plans to order more materials based on that.

[0076] Next, the server notifies the user (site supervisor) of the generated plan via a notification system to their app. The user can review the proposed procurement plan and placement suggestions in the app and make changes as needed. This maximizes operational efficiency and reduces the risk of material shortages or surpluses.

[0077] As a concrete example, a prompt message to the generating AI model, such as "Predict the material consumption pattern of Project X based on weather forecasts and historical data, and propose the optimal ordering plan," will activate the analysis engine and create a useful material management plan.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The terminal (sensing device) collects real-time data on the inventory status and movement routes of supplies from cameras and sensors installed on-site. Inputs include camera video data and location data from sensors. This data is transmitted to a cloud server, providing the necessary information to understand the current state of the supplies. Specifically, cameras capture shelf heights, and sensors track movement paths.

[0081] Step 2:

[0082] The server analyzes the received data using a data processing module and organizes the information. Inputs include image data and numerical data sent from sensing devices. Specific data processing includes material identification through image processing, refinement through noise reduction filtering, and automatic detection of anomalies. Since the data is then stored in a database, the output is organized on-site inventory data.

[0083] Step 3:

[0084] The server uses an analysis engine to analyze organized inventory data, historical usage history, and external factor data (e.g., weather and work schedule) as input. A generative AI model is involved in this analysis, providing "prompt messages" to predict consumption patterns. The output is a predicted demand pattern for materials. For example, it shows increases or decreases in material consumption based on weather forecasts.

[0085] Step 4:

[0086] The server generates procurement plans and placement suggestions based on forecast data. Inputs include predicted demand patterns and current inventory data. An algorithm is used to optimize procurement timing and quantity. The output is the generated material procurement plan and placement suggestions. Specific operations include listing required materials and suggesting optimal placement methods.

[0087] Step 5:

[0088] The server notifies the user (site supervisor) of the generated plan through a notification system. Specifically, the server sends a notification to the user's app, and the user receives and confirms its contents. The input is the generated proposal, and the output is feedback from the user regarding approval or revisions. Here, the user uses a smartphone or tablet to check the details of the plan and take actions such as pressing the approval button.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] In materials management, inventory shortages and excesses can negatively impact project progress. In particular, ordering and distributing materials without considering environmental factors or on-site workflows leads to inefficient management. Currently, optimizing these processes requires significant time and effort, increasing the burden on managers. Therefore, there is a need for a highly versatile system that accurately predicts material usage patterns and automatically proposes efficient ordering and distribution strategies.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes means for collecting information from a data acquisition device and organizing information regarding material inventory and movement; means for analyzing past consumption information and external factors to predict material usage patterns; and means for automatically generating material ordering plans and placement proposals considering environmental factors and notifying the user. This enables more efficient material management.

[0094] A "data acquisition device" is a device installed on-site to collect real-time information on material inventory, location, and movement patterns.

[0095] "Means for collecting information and organizing information regarding material inventory and movement" refers to data organization methods that use collected raw data to understand the inventory situation and efficiency of movement at the site.

[0096] "Methods for predicting material usage patterns by analyzing past consumption information and external factors" refers to analytical methods that analyze past material consumption history and external factors such as weather to estimate future material demand.

[0097] "A means of automatically generating material ordering plans and placement proposals that take environmental factors into consideration and notifying users" refers to a system that creates ordering schedules and placement proposals that respond to environmental changes based on predicted demand, and informs relevant parties of these.

[0098] In an embodiment of this invention, the server provides an integrated system for streamlining material management. Using data acquisition devices installed on-site, specifically cameras and sensors, it collects real-time data on material inventory, location, and movement patterns. This data is transmitted to a cloud server via the internet. The server uses programming languages ​​such as Python to build data processing modules, organize the collected data, remove noise, and store it in a database. This enables up-to-date monitoring of on-site conditions.

[0099] The server then activates an analysis engine to predict material consumption patterns based on data. Specifically, it integrates and analyzes external factors such as past consumption information, weather data, and work schedules to predict future demand. Based on this predicted data, it generates an optimal ordering plan and material allocation proposal. The server communicates the generated proposal to the site supervisor through a notification system and requests confirmation and approval via a smartphone app. This allows managers to efficiently manage materials.

[0100] As a concrete example, in a specific construction project, where work is often carried out on rainy days, it is possible to automatically propose a plan to prepare 20% more materials during weeks when rain is expected. An example of a prompt message based on this would be: "Predict the consumption pattern of material A to be used at the construction site and generate an optimal ordering plan that takes weather forecasts into consideration."

[0101] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0102] Step 1:

[0103] The terminal connects to a data acquisition device and collects data in real time from cameras and sensors installed on-site. This data includes material inventory levels, location information, and movement patterns. This data is transmitted to a cloud server via the internet. The input is raw data acquired via a wireless or wired network, and the output is data transmission to the cloud server.

[0104] Step 2:

[0105] The server uses a data processing module to organize the received data. Specifically, it filters out noisy data and formats it into an appropriate format. The input is the collected raw data, and the output is the organized information. In this process, algorithms are used to cleanse and convert the data format.

[0106] Step 3:

[0107] The server inputs the organized data into the analysis engine to predict material consumption patterns. This analysis utilizes historical consumption data, weather information, and work schedules. The input consists of organized information and data on external factors, and the output is a forecast of future material demand. The analysis engine uses a statistical model to perform demand forecasting.

[0108] Step 4:

[0109] The server generates optimal ordering plans and material allocation suggestions based on predicted data. It applies an algorithm to determine the optimal order quantities and schedules for predicted demand. The input is a forecast of consumption patterns, and the output is an ordering plan and allocation suggestion. The suggestions are automatically formulated based on the calculation results.

[0110] Step 5:

[0111] The server presents the generated proposals to the user via a smartphone app using a notification system. The user reviews the proposals using the app and approves them as needed. The input is the generated proposals, and the output is the notification and feedback to the user. The notification system uses push notification technology to deliver information quickly.

[0112] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0113] In embodiments of the present invention, emotion recognition technology is combined with a materials management system to analyze the emotional state of the user (site supervisor) and, based on that analysis, optimal materials management and recommendations are made.

[0114] First, the server processes inventory information and customer movement data collected from data acquisition devices in real time. In addition, the server uses facial expression data and voice transmitted from the user's terminal to enable an emotion engine to analyze the user's emotional state.

[0115] The emotion engine utilizes machine learning techniques to identify user emotions and evaluate feelings such as stress, satisfaction, and anxiety. This information is then reflected in material ordering plans and placement suggestions. For example, if a user is feeling stressed, the system can reduce the frequency of notifications or simplify the suggestions.

[0116] The server uses predictive algorithms to analyze material consumption patterns and optimize ordering plans. By combining this with evaluations from an emotion engine, the plan adapts to the user's mood.

[0117] The notification system sends planned order and placement proposals to users via the application. Users review the proposals through the app and make any necessary changes. The system learns from user feedback and uses it to improve future proposals.

[0118] This embodiment allows the materials management system to go beyond simply pursuing efficiency and make flexible suggestions that take into account the user's emotional state, aiming to improve operational efficiency on-site.

[0119] The following describes the processing flow.

[0120] Step 1:

[0121] The server collects real-time inventory and movement data of materials supplied from data acquisition devices and records this data in a database. The data is continuously organized to ensure that the latest situation on site is understood.

[0122] Step 2:

[0123] The device uses its built-in camera and microphone to record the user's facial expressions and voice. Data is acquired during everyday use without requiring any special operation from the user.

[0124] Step 3:

[0125] The server analyzes facial expression and voice data transmitted from the terminal using an emotion engine to determine the user's emotional state. It uses machine learning models to identify emotions such as stress and satisfaction.

[0126] Step 4:

[0127] The server adjusts its resource consumption forecast based on the analysis results. For example, if a user is experiencing stress, it may change the timing of notifications and prioritize presenting concise information.

[0128] Step 5:

[0129] The server combines predicted material consumption and sentiment analysis results to generate an optimal material ordering plan and placement proposal. This includes optimizing ordering timing and suggesting efficient placement based on workflow.

[0130] Step 6:

[0131] The server notifies the user of the generated plan and proposal. The notification is sent through the app, and the user can review the proposal.

[0132] Step 7:

[0133] Users review the notified plan within the app and approve or modify it. The feedback is sent back to the server, and the system uses this information to further optimize its features.

[0134] (Example 2)

[0135] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0136] In industrial settings, inventory management systems require not only the pursuit of conventional efficiency but also flexible management that responds to the emotional state of users. However, current systems are unable to optimize inventory management based on users' emotional states, which can lead to decreased management efficiency and user satisfaction.

[0137] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0138] In this invention, the server includes means for accumulating information from data collection elements and organizing information regarding the inventory and movement of items; means for analyzing past consumption information and external factors to predict item usage trends; and means for analyzing the user's facial expressions and voice data and making item management suggestions based on their emotional state. This enables optimal item management and suggestions that take into account the user's emotional state.

[0139] A "data collection element" is hardware or sensor technology that detects inventory information and movement patterns of goods and transmits that data to a server.

[0140] "Inventory of goods" refers to the quantity and condition of goods and materials that need to be managed in industrial sites and warehouses.

[0141] "Movement flow" refers to the routes taken by people and materials within a work site or facility, and is a factor that affects work efficiency and safety.

[0142] "Usage trends" refer to patterns and trends in how goods and materials are consumed over time.

[0143] "Facial expression and audio data" refers to visual and auditory information provided by the user through their device, and is used to analyze their emotional state.

[0144] "Emotional state" refers to the user's mental and emotional condition, including stress, satisfaction, and anxiety.

[0145] "Inventory management proposals" are a process that provides recommendations regarding ordering, inventory management, and placement of goods, enabling users to make efficient and beneficial choices.

[0146] This invention utilizes advanced data processing technology to make suggestions that take into account the emotional state of the user in an item management system. Specifically, the following hardware and software are used.

[0147] The server organizes and manages inventory and movement information obtained from data collection elements in real time. This is done using IoT sensors installed on each shelf and storage area. These sensors determine the quantity of inventory and transmit the data to the server via wireless communication. The server has a built-in database management system that efficiently stores the collected information and processes it as needed.

[0148] The device functions as a device that collects facial and voice data from the user. The device has a built-in camera and microphone, and facial recognition software operates to analyze the video data. Voice recognition software also runs simultaneously, collecting data to determine emotions from the user's voice. This allows for real-time analysis of the user's emotional state.

[0149] The emotion engine is a machine learning algorithm that runs on the server and implements frameworks such as TENSORFLOW® and PyTorch. This allows facial expressions and voice data acquired from the device to be classified into emotions such as stress, satisfaction, or anxiety. This information is then used to inform product ordering plans and placement suggestions, and customized to the user's mental state.

[0150] As a concrete example, consider a situation at a construction site where a site supervisor uses a tablet to monitor daily work. If the system determines that the supervisor is experiencing stress, it can automatically adjust to reduce the frequency of notifications and display only concise suggestions.

[0151] An example of an input prompt for a generative AI model is, "Suggest an algorithm that adjusts notification frequency using user sentiment data." This prompt prompts the system to generate an adaptive strategy to optimize management tasks while taking sentiment data into consideration.

[0152] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0153] Step 1:

[0154] The server receives inventory information and movement data from data collection elements. This input data includes the quantity and location information of items detected by IoT sensors. The server organizes this data and stores it in a database. This process provides the basic data necessary for understanding the current state of items and optimizing their movement.

[0155] Step 2:

[0156] The device collects the user's facial expressions and voice data through its camera and microphone. Specifically, facial recognition and voice analysis software operates using the video and audio obtained from the device as input. Based on the data, the analysis software generates output that classifies the user's emotional state into emotions such as stress, satisfaction, and anxiety. This makes it possible to understand the user's mental state in real time.

[0157] Step 3:

[0158] The server uses an emotion engine to analyze emotional data transmitted from the terminal. The input is the user's emotional state, which the emotion engine analyzes using a machine learning model and outputs a specific emotional evaluation. This evaluation result will be an important element in future material management proposals. The server records the results of the emotional analysis and uses them to optimize material management.

[0159] Step 4:

[0160] The server uses historical consumption data and current inventory data as input to execute a predictive algorithm. This calculation estimates future demand for goods, taking into account consumption trends and external factors, and produces an output that creates an ordering plan. Furthermore, it develops a plan that incorporates emotional evaluation, creating flexible suggestions tailored to the user's emotional state.

[0161] Step 5:

[0162] Users receive order plans and deployment proposals from the server via the application. These proposals are pre-adjusted to suit the user's situation, and users review the proposals and make any necessary modifications. User feedback is provided as input, and the results are fed back to the server as output. This feedback is used to improve the accuracy of future proposals.

[0163] Through this series of processes, the system can not only improve the efficiency of inventory management but also provide management suggestions that respond to the emotional state of the users.

[0164] (Application Example 2)

[0165] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0166] Traditional materials management systems contribute to increased efficiency on-site, but they fail to consider the emotional state of users. As a result, work efficiency and user satisfaction may decrease, and in situations where stress and anxiety are high, this can negatively impact materials management decision-making. Therefore, there is a need for a materials management system that utilizes emotion recognition technology to adapt to the emotional state of users.

[0167] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0168] In this invention, the server includes means for organizing information on the inventory and movement of materials collected from a data acquisition device; means for analyzing past consumption information and external factors to predict material usage patterns; means for utilizing emotion recognition technology to analyze the emotional state of users; means for optimizing material ordering plans and placement suggestions based on the emotion analysis results; and means for simplifying or adjusting notification content according to the emotional state. This enables flexible material management in accordance with the user's emotional state, improving work efficiency and satisfaction with on-site management.

[0169] A "data acquisition device" is a device used to collect information on the inventory status and movement patterns of materials.

[0170] "Means of organizing information" refer to methods and techniques for efficiently classifying collected data and presenting it in an easily understandable format.

[0171] "Consumption information" refers to information that shows the usage history and consumption trends of materials.

[0172] "External factors" refer to external environmental factors such as weather conditions and market trends that affect material consumption.

[0173] "Methods for predicting usage patterns" refer to methods or algorithms that predict future trends in material usage based on analyzed past data.

[0174] "Emotion recognition technology" is a technology used to analyze a user's emotions and psychological state, making judgments based on data such as voice and facial expressions.

[0175] "Means for optimizing ordering plans and allocation proposals" refers to methods and systems for proposing appropriate ordering and allocation of materials according to the user's emotional state.

[0176] "Means of simplifying or adjusting notification content" refers to methods of flexibly changing the amount of information and the method of delivery of notifications, taking into consideration the user's feelings.

[0177] The system in this invention mainly consists of three elements: a server, a terminal, and a user. The server organizes information on material inventory and movement patterns sent from the data acquisition device in real time. It uses Amazon Web Services (AWS®) data management tools to efficiently organize the information. The server also analyzes past consumption information and external factors, using machine learning algorithms to predict material usage patterns. This utilizes predictive models from Microsoft® Azure®. Furthermore, the server is equipped with an emotion recognition engine to analyze the user's emotional state. User facial expression data and voice data are collected by smart glasses and used for analysis.

[0178] On the user's device, emotion recognition technology is used to analyze the user's emotional state in real time, and based on this, the ordering plan and placement suggestions for materials are optimized. In this process, the emotion engine determines the user's stress and satisfaction levels and simplifies or adjusts the content of notifications. This information is visually communicated through the user's smart glasses.

[0179] For example, if a user is stressed due to a shortage of materials, the server can sense this emotion, simplify the suggestions, and display only important notifications to the user's view, thereby supporting quick decision-making on-site. An example of a prompt using a generative AI model is, "How can I simplify notifications when a site supervisor is stressed?"

[0180] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0181] Step 1:

[0182] The server receives inventory and movement data for materials from data acquisition devices. Input is real-time data from sensors and RFID readers, and output is an integrated database of inventory and movement information. The data is organized and efficiently stored in the database using AWS data management tools.

[0183] Step 2:

[0184] The server acquires historical consumption information and external factors (e.g., weather data and market trends) and analyzes them. Here, the input is historical transaction records and data acquired from external sources, and the output is a predictive model of material usage patterns. Machine learning algorithms from Microsoft Azure are used to perform predictions through time series analysis and regression analysis.

[0185] Step 3:

[0186] The user's device transmits audio and facial expression data collected through smart glasses to a server. The input consists of video and audio data, while the output is an input dataset for the emotion recognition engine. The data is first compressed using a codec on the local device before being securely transmitted.

[0187] Step 4:

[0188] The server analyzes the received user data using an emotion recognition engine to evaluate the user's emotional state. The input here is a dataset of voice and facial expressions, and the output is a determination of the emotional state (stress, satisfaction, anxiety, etc.). Emotional analysis technology using a generative AI model is employed to determine emotions with high accuracy.

[0189] Step 5:

[0190] Based on the user's emotional state, the server adjusts the material ordering plan and placement suggestions. The input is emotional state evaluation data, and the output is an optimized ordering plan and placement suggestion. The suggestion content and frequency of notifications are simplified as needed.

[0191] Step 6:

[0192] Optimized suggestions and tailored notifications are visually transmitted to the user's smart glasses. The input is material management suggestions, and the output is notifications displayed in the user's field of vision. At this point, the user can review the suggestions and make any necessary changes immediately.

[0193] Step 7:

[0194] Users make decisions based on the suggestions and send feedback to the system. The input here is user feedback information, and the output is adjustment data that helps in future suggestions. This allows the system to learn and further optimize its processes.

[0195] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0196] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0197] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0198] [Second Embodiment]

[0199] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0200] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0201] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0202] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0203] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0204] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0205] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0206] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0207] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0208] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0209] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0210] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0211] In embodiments of the present invention, multiple components are coordinated to optimize material management. The main elements consist of a data acquisition device, a data processing module, an analysis engine, and a notification system.

[0212] In the data acquisition system, cameras and sensors installed on-site collect information in real time and transmit data on material inventory levels, location information, and movement patterns to a cloud server.

[0213] The server uses a data processing module to organize the collected information, remove noise data, and store it in the database. This makes it possible to always have a grasp of the latest field conditions.

[0214] The server then uses this data to run an analysis engine. The analysis engine uses historical consumption data and external factor data (such as weather information and worker schedules) to predict material consumption patterns.

[0215] Based on the predicted data, the server generates suggestions for optimal ordering plans and material allocation. For example, in a project that uses a lot of materials on rainy days, it will consider the weather forecast in advance and plan to order more than necessary.

[0216] The server generates a plan and proposes it to the user (site supervisor) via an app through a notification system. The user uses the app to review the plan and make adjustments or approvals as needed.

[0217] This system aims to prevent material shortages and excess inventory, maximizing on-site efficiency. This reduces the burden on site supervisors and supports the overall success of the project.

[0218] The following describes the processing flow.

[0219] Step 1:

[0220] The server receives real-time inventory and location information of materials from on-site data acquisition devices. Each time data from sensors is updated, the server aggregates and centrally manages the data.

[0221] Step 2:

[0222] Before saving received data to the database, the server performs data cleaning. This removes inaccurate data and noise and verifies the integrity of inventory information.

[0223] Step 3:

[0224] The server predicts material consumption by combining historical consumption data with external factors such as weather and worker schedules. This involves using machine learning algorithms to analyze the factors that cause fluctuations in consumption patterns.

[0225] Step 4:

[0226] The server generates an optimal ordering plan for materials based on consumption forecasts. The timing and quantity of orders are determined considering inventory costs and lead times.

[0227] Step 5:

[0228] The server generates optimized material placement suggestions based on on-site layout information, taking into account traffic flow. Materials that are frequently accessed are placed particularly efficiently.

[0229] Step 6:

[0230] The server notifies the user of the generated order plan and placement proposal through the app. The notification includes options to help with decision-making and detailed suggestions.

[0231] Step 7:

[0232] Users check notifications via the app and approve or modify the plan based on the suggestions provided. Once approved, the information is returned to the system and implemented.

[0233] (Example 1)

[0234] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0235] Materials management in the industrial sector constantly faces problems such as inventory shortages and surpluses, and decreased work efficiency due to improper allocation. Furthermore, conventional systems struggle to collect and predict accurate information in real time, resulting in inadequate material procurement and allocation.

[0236] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0237] In this invention, the server includes means for receiving data from a sensing device and organizing information regarding the inventory status and movement routes of materials; means for calculating and processing past usage data and external factors to predict material consumption patterns; and means for presenting a material procurement plan and placement proposal to the user and requesting their approval. This makes it possible to streamline material inventory management, improve work efficiency through optimized movement routes, and reduce the risk of insufficient or excessive preparation.

[0238] A "sensing device" refers to hardware used to record on-site conditions in real time and collect information about the inventory and movement of materials.

[0239] A "server" refers to a core information processing system that analyzes received data and generates a resource management plan.

[0240] "Inventory status" refers to data regarding the quantity and types of goods stored at a site or warehouse at a given time.

[0241] "Transportation route" refers to route information that shows how supplies are moved within the site.

[0242] "Computational processing" refers to the process of performing mathematical analysis using collected data, and specifically to the calculations necessary to predict the consumption patterns of materials.

[0243] "Consumption patterns of goods" refer to trends regarding how goods are used and the rate at which they are consumed, based on past usage data and external factors.

[0244] A "procurement plan" refers to a detailed plan for securing necessary supplies in the appropriate time and quantity.

[0245] A "layout proposal" refers to a plan that outlines how materials should be arranged on-site to ensure the most efficient use of them.

[0246] "Information organization" refers to the process of processing received data, removing noise, and prioritizing its importance to make it usable.

[0247] This invention is a system that works in conjunction with multiple components to improve the efficiency of material management.

[0248] The terminal uses sensing devices placed on-site, specifically cameras and sensors, to collect real-time data on the inventory status and movement routes of supplies. This data is immediately transmitted to a server in the cloud.

[0249] The server receives this data and uses a data processing module to organize the information. Image processing technology is used to analyze the video data from the camera, removing noise such as audio data to extract clear information. This organized information is stored in a database, allowing for a constantly updated understanding of the on-site situation.

[0250] Furthermore, the server uses an analysis engine to predict material consumption patterns using machine learning algorithms, based on past material usage data and external factors such as weather and worker schedules. For example, if it rains continuously, the amount of concrete used may increase, so the server plans to order more materials based on that.

[0251] Next, the server notifies the user (site supervisor) of the generated plan via a notification system to their app. The user can review the proposed procurement plan and placement suggestions in the app and make changes as needed. This maximizes operational efficiency and reduces the risk of material shortages or surpluses.

[0252] As a concrete example, a prompt message to the generating AI model, such as "Predict the material consumption pattern of Project X based on weather forecasts and historical data, and propose the optimal ordering plan," will activate the analysis engine and create a useful material management plan.

[0253] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0254] Step 1:

[0255] The terminal (sensing device) collects real-time data on the inventory status and movement routes of supplies from cameras and sensors installed on-site. Inputs include camera video data and location data from sensors. This data is transmitted to a cloud server, providing the necessary information to understand the current state of the supplies. Specifically, cameras capture shelf heights, and sensors track movement paths.

[0256] Step 2:

[0257] The server analyzes the received data using a data processing module and organizes the information. Inputs include image data and numerical data sent from sensing devices. Specific data processing includes material identification through image processing, refinement through noise reduction filtering, and automatic detection of anomalies. Since the data is then stored in a database, the output is organized on-site inventory data.

[0258] Step 3:

[0259] The server uses an analysis engine to analyze organized inventory data, historical usage history, and external factor data (e.g., weather and work schedule) as input. A generative AI model is involved in this analysis, providing "prompt messages" to predict consumption patterns. The output is a predicted demand pattern for materials. For example, it shows increases or decreases in material consumption based on weather forecasts.

[0260] Step 4:

[0261] The server generates procurement plans and placement suggestions based on forecast data. Inputs include predicted demand patterns and current inventory data. An algorithm is used to optimize procurement timing and quantity. The output is the generated material procurement plan and placement suggestions. Specific operations include listing required materials and suggesting optimal placement methods.

[0262] Step 5:

[0263] The server notifies the user (site supervisor) of the generated plan through a notification system. Specifically, the server sends a notification to the user's app, and the user receives and confirms its contents. The input is the generated proposal, and the output is feedback from the user regarding approval or revisions. Here, the user uses a smartphone or tablet to check the details of the plan and take actions such as pressing the approval button.

[0264] (Application Example 1)

[0265] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0266] In materials management, inventory shortages and excesses can negatively impact project progress. In particular, ordering and distributing materials without considering environmental factors or on-site workflows leads to inefficient management. Currently, optimizing these processes requires significant time and effort, increasing the burden on managers. Therefore, there is a need for a highly versatile system that accurately predicts material usage patterns and automatically proposes efficient ordering and distribution strategies.

[0267] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0268] In this invention, the server includes means for collecting information from a data acquisition device and organizing information regarding material inventory and movement; means for analyzing past consumption information and external factors to predict material usage patterns; and means for automatically generating material ordering plans and placement proposals considering environmental factors and notifying the user. This enables more efficient material management.

[0269] A "data acquisition device" is a device installed on-site to collect real-time information on material inventory, location, and movement patterns.

[0270] "Means for collecting information and organizing information regarding material inventory and movement" refers to data organization methods that use collected raw data to understand the inventory situation and efficiency of movement at the site.

[0271] "Methods for predicting material usage patterns by analyzing past consumption information and external factors" refers to analytical methods that analyze past material consumption history and external factors such as weather to estimate future material demand.

[0272] "A means of automatically generating material ordering plans and placement proposals that take environmental factors into consideration and notifying users" refers to a system that creates ordering schedules and placement proposals that respond to environmental changes based on predicted demand, and informs relevant parties of these.

[0273] In an embodiment of this invention, the server provides an integrated system for streamlining material management. Using data acquisition devices installed on-site, specifically cameras and sensors, it collects real-time data on material inventory, location, and movement patterns. This data is transmitted to a cloud server via the internet. The server uses programming languages ​​such as Python to build data processing modules, organize the collected data, remove noise, and store it in a database. This enables up-to-date monitoring of on-site conditions.

[0274] The server then activates an analysis engine to predict material consumption patterns based on data. Specifically, it integrates and analyzes external factors such as past consumption information, weather data, and work schedules to predict future demand. Based on this predicted data, it generates an optimal ordering plan and material allocation proposal. The server communicates the generated proposal to the site supervisor through a notification system and requests confirmation and approval via a smartphone app. This allows managers to efficiently manage materials.

[0275] As a concrete example, in a specific construction project, where work is often carried out on rainy days, it is possible to automatically propose a plan to prepare 20% more materials during weeks when rain is expected. An example of a prompt message based on this would be: "Predict the consumption pattern of material A to be used at the construction site and generate an optimal ordering plan that takes weather forecasts into consideration."

[0276] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0277] Step 1:

[0278] The terminal connects to a data acquisition device and collects data in real time from cameras and sensors installed on-site. This data includes material inventory levels, location information, and movement patterns. This data is transmitted to a cloud server via the internet. The input is raw data acquired via a wireless or wired network, and the output is data transmission to the cloud server.

[0279] Step 2:

[0280] The server uses a data processing module to organize the received data. Specifically, it filters out noisy data and formats it into an appropriate format. The input is the collected raw data, and the output is the organized information. In this process, algorithms are used to cleanse and convert the data format.

[0281] Step 3:

[0282] The server inputs the sorted data into the analysis engine to predict the consumption pattern of materials. Past consumption data, weather information, and work schedules are utilized for this analysis. The input is the sorted information and data on external factors, and the output is the predicted value of future material demand. The analysis engine uses a statistical model to perform demand prediction.

[0283] Step 4:

[0284] The server generates a proposal for the optimal ordering plan and material allocation based on the predicted data. Here, an algorithm for determining the optimal order quantity and schedule for the predicted demand is applied. The input is the prediction of the consumption pattern, and the output is the ordering plan and allocation plan. Based on the calculation results, a proposal is automatically formulated.

[0285] Step 5:

[0286] The server presents the generated proposal to the user through the smartphone app using the notification system. The user checks the proposal using the app and approves it if necessary. The input is the generated proposal, and the output is the notification and feedback to the user. The notification system uses push notification technology to deliver information quickly.

[0287] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion recognition model 59 and perform specific processing using the user's emotion.

[0288] In an embodiment for implementing the present invention, by combining emotion recognition technology with the material management system, the emotional state of the user (on-site supervisor) is analyzed, and optimal material management and proposals are made based on it.

[0289] First, the server processes inventory information and customer movement data collected from data acquisition devices in real time. In addition, the server uses facial expression data and voice transmitted from the user's terminal to enable an emotion engine to analyze the user's emotional state.

[0290] The emotion engine utilizes machine learning techniques to identify user emotions and evaluate feelings such as stress, satisfaction, and anxiety. This information is then reflected in material ordering plans and placement suggestions. For example, if a user is feeling stressed, the system can reduce the frequency of notifications or simplify the suggestions.

[0291] The server uses predictive algorithms to analyze material consumption patterns and optimize ordering plans. By combining this with evaluations from an emotion engine, the plan adapts to the user's mood.

[0292] The notification system sends planned order and placement proposals to users via the application. Users review the proposals through the app and make any necessary changes. The system learns from user feedback and uses it to improve future proposals.

[0293] This embodiment allows the materials management system to go beyond simply pursuing efficiency and make flexible suggestions that take into account the user's emotional state, aiming to improve operational efficiency on-site.

[0294] The following describes the processing flow.

[0295] Step 1:

[0296] The server collects real-time inventory and movement data of materials supplied from data acquisition devices and records this data in a database. The data is continuously organized to ensure that the latest situation on site is understood.

[0297] Step 2:

[0298] The terminal uses the built-in camera and microphone to record the user's expressions and voice. Without any special operations by the user, data is acquired during daily use.

[0299] Step 3:

[0300] The server analyzes the facial expression data and voice data transmitted from the terminal using an emotion engine to determine the user's emotional state. Machine learning models are used to identify emotions such as stress and satisfaction.

[0301] Step 4:

[0302] Based on the analysis results, the server adjusts the consumption prediction of materials. For example, if the user is feeling stressed, measures such as changing the notification timing and prioritizing the presentation of concise information are taken.

[0303] Step 5:

[0304] The server combines the predicted consumption volume of materials and the results of the emotion analysis to generate an optimal material ordering plan and layout proposal. This includes optimizing the ordering timing and proposing an efficient layout based on the flow line.

[0305] Step 6:

[0306] The server notifies the user of the generated plan and proposal. The notification is made through the app, and the user can view the content of the proposal.

[0307] Step 7:

[0308] The user views the notified plan on the app and approves or modifies it. The feedback is sent back to the server, and the system further optimizes based on this information.

[0309] (Example 2)

[0310] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0311] In industrial settings, inventory management systems require not only the pursuit of conventional efficiency but also flexible management that responds to the emotional state of users. However, current systems are unable to optimize inventory management based on users' emotional states, which can lead to decreased management efficiency and user satisfaction.

[0312] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0313] In this invention, the server includes means for accumulating information from data collection elements and organizing information regarding the inventory and movement of items; means for analyzing past consumption information and external factors to predict item usage trends; and means for analyzing the user's facial expressions and voice data and making item management suggestions based on their emotional state. This enables optimal item management and suggestions that take into account the user's emotional state.

[0314] A "data collection element" is hardware or sensor technology that detects inventory information and movement patterns of goods and transmits that data to a server.

[0315] "Inventory of goods" refers to the quantity and condition of goods and materials that need to be managed in industrial sites and warehouses.

[0316] "Movement flow" refers to the routes taken by people and materials within a work site or facility, and is a factor that affects work efficiency and safety.

[0317] "Usage trends" refer to patterns and trends in how goods and materials are consumed over time.

[0318] "Facial expression and audio data" refers to visual and auditory information provided by the user through their device, and is used to analyze their emotional state.

[0319] "Emotional state" refers to the user's mental and emotional condition, including stress, satisfaction, and anxiety.

[0320] "Inventory management proposals" are a process that provides recommendations regarding ordering, inventory management, and placement of goods, enabling users to make efficient and beneficial choices.

[0321] This invention utilizes advanced data processing technology to make suggestions that take into account the emotional state of the user in an item management system. Specifically, the following hardware and software are used.

[0322] The server organizes and manages inventory and movement information obtained from data collection elements in real time. This is done using IoT sensors installed on each shelf and storage area. These sensors determine the quantity of inventory and transmit the data to the server via wireless communication. The server has a built-in database management system that efficiently stores the collected information and processes it as needed.

[0323] The device functions as a device that collects facial and voice data from the user. The device has a built-in camera and microphone, and facial recognition software operates to analyze the video data. Voice recognition software also runs simultaneously, collecting data to determine emotions from the user's voice. This allows for real-time analysis of the user's emotional state.

[0324] The emotion engine is a machine learning algorithm that runs on the server and is implemented using frameworks such as TensorFlow and PyTorch. This allows facial expressions and voice data acquired from the device to be classified into emotions such as stress, satisfaction, or anxiety. This information is then used to inform ordering plans and placement suggestions for goods, resulting in customized experiences tailored to the user's mental state.

[0325] As a concrete example, consider a situation at a construction site where a site supervisor uses a tablet to monitor daily work. If the system determines that the supervisor is experiencing stress, it can automatically adjust to reduce the frequency of notifications and display only concise suggestions.

[0326] An example of an input prompt for a generative AI model is, "Suggest an algorithm that adjusts notification frequency using user sentiment data." This prompt prompts the system to generate an adaptive strategy to optimize management tasks while taking sentiment data into consideration.

[0327] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0328] Step 1:

[0329] The server receives inventory information and movement data from data collection elements. This input data includes the quantity and location information of items detected by IoT sensors. The server organizes this data and stores it in a database. This process provides the basic data necessary for understanding the current state of items and optimizing their movement.

[0330] Step 2:

[0331] The device collects the user's facial expressions and voice data through its camera and microphone. Specifically, facial recognition and voice analysis software operates using the video and audio obtained from the device as input. Based on the data, the analysis software generates output that classifies the user's emotional state into emotions such as stress, satisfaction, and anxiety. This makes it possible to understand the user's mental state in real time.

[0332] Step 3:

[0333] The server uses an emotion engine to analyze emotional data transmitted from the terminal. The input is the user's emotional state, which the emotion engine analyzes using a machine learning model and outputs a specific emotional evaluation. This evaluation result will be an important element in future material management proposals. The server records the results of the emotional analysis and uses them to optimize material management.

[0334] Step 4:

[0335] The server uses historical consumption data and current inventory data as input to execute a predictive algorithm. This calculation estimates future demand for goods, taking into account consumption trends and external factors, and produces an output that creates an ordering plan. Furthermore, it develops a plan that incorporates emotional evaluation, creating flexible suggestions tailored to the user's emotional state.

[0336] Step 5:

[0337] Users receive order plans and deployment proposals from the server via the application. These proposals are pre-adjusted to suit the user's situation, and users review the proposals and make any necessary modifications. User feedback is provided as input, and the results are fed back to the server as output. This feedback is used to improve the accuracy of future proposals.

[0338] Through this series of processes, the system can not only improve the efficiency of inventory management but also provide management suggestions that respond to the emotional state of the users.

[0339] (Application Example 2)

[0340] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0341] Traditional materials management systems contribute to increased efficiency on-site, but they fail to consider the emotional state of users. As a result, work efficiency and user satisfaction may decrease, and in situations where stress and anxiety are high, this can negatively impact materials management decision-making. Therefore, there is a need for a materials management system that utilizes emotion recognition technology to adapt to the emotional state of users.

[0342] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0343] In this invention, the server includes means for organizing information on the inventory and movement of materials collected from a data acquisition device; means for analyzing past consumption information and external factors to predict material usage patterns; means for utilizing emotion recognition technology to analyze the emotional state of users; means for optimizing material ordering plans and placement suggestions based on the emotion analysis results; and means for simplifying or adjusting notification content according to the emotional state. This enables flexible material management in accordance with the user's emotional state, improving work efficiency and satisfaction with on-site management.

[0344] A "data acquisition device" is a device used to collect information on the inventory status and movement patterns of materials.

[0345] "Means of organizing information" refer to methods and techniques for efficiently classifying collected data and presenting it in an easily understandable format.

[0346] "Consumption information" refers to information that shows the usage history and consumption trends of materials.

[0347] "External factors" refer to external environmental factors such as weather conditions and market trends that affect material consumption.

[0348] "Methods for predicting usage patterns" refer to methods or algorithms that predict future trends in material usage based on analyzed past data.

[0349] "Emotion recognition technology" is a technology used to analyze a user's emotions and psychological state, making judgments based on data such as voice and facial expressions.

[0350] "Means for optimizing ordering plans and allocation proposals" refers to methods and systems for proposing appropriate ordering and allocation of materials according to the user's emotional state.

[0351] "Means of simplifying or adjusting notification content" refers to methods of flexibly changing the amount of information and the method of delivery of notifications, taking into consideration the user's feelings.

[0352] The system in this invention mainly consists of three elements: a server, a terminal, and a user. The server organizes information on material inventory and movement patterns sent from the data acquisition device in real time. It uses Amazon Web Services (AWS) data management tools to efficiently organize the information. The server also analyzes past consumption information and external factors, employing machine learning algorithms to predict material usage patterns. Microsoft Azure's predictive models are utilized for this purpose. Furthermore, the server is equipped with an emotion recognition engine to analyze the user's emotional state. User facial expression data and voice data are collected by smart glasses and used for analysis.

[0353] On the user's device, emotion recognition technology is used to analyze the user's emotional state in real time, and based on this, the ordering plan and placement suggestions for materials are optimized. In this process, the emotion engine determines the user's stress and satisfaction levels and simplifies or adjusts the content of notifications. This information is visually communicated through the user's smart glasses.

[0354] For example, if a user is stressed due to a shortage of materials, the server can sense this emotion, simplify the suggestions, and display only important notifications to the user's view, thereby supporting quick decision-making on-site. An example of a prompt using a generative AI model is, "How can I simplify notifications when a site supervisor is stressed?"

[0355] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0356] Step 1:

[0357] The server receives inventory and movement data for materials from data acquisition devices. Input is real-time data from sensors and RFID readers, and output is an integrated database of inventory and movement information. The data is organized and efficiently stored in the database using AWS data management tools.

[0358] Step 2:

[0359] The server acquires historical consumption information and external factors (e.g., weather data and market trends) and analyzes them. Here, the input is historical transaction records and data acquired from external sources, and the output is a predictive model of material usage patterns. Machine learning algorithms from Microsoft Azure are used to perform predictions through time series analysis and regression analysis.

[0360] Step 3:

[0361] The user's device transmits audio and facial expression data collected through smart glasses to a server. The input consists of video and audio data, while the output is an input dataset for the emotion recognition engine. The data is first compressed using a codec on the local device before being securely transmitted.

[0362] Step 4:

[0363] The server analyzes the received user data using an emotion recognition engine to evaluate the user's emotional state. The input here is a dataset of voice and facial expressions, and the output is a determination of the emotional state (stress, satisfaction, anxiety, etc.). Emotional analysis technology using a generative AI model is employed to determine emotions with high accuracy.

[0364] Step 5:

[0365] Based on the user's emotional state, the server adjusts the material ordering plan and placement suggestions. The input is emotional state evaluation data, and the output is an optimized ordering plan and placement suggestion. The suggestion content and frequency of notifications are simplified as needed.

[0366] Step 6:

[0367] Optimized suggestions and tailored notifications are visually transmitted to the user's smart glasses. The input is material management suggestions, and the output is notifications displayed in the user's field of vision. At this point, the user can review the suggestions and make any necessary changes immediately.

[0368] Step 7:

[0369] Users make decisions based on the suggestions and send feedback to the system. The input here is user feedback information, and the output is adjustment data that helps in future suggestions. This allows the system to learn and further optimize its processes.

[0370] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0371] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0372] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0373] [Third Embodiment]

[0374] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0375] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0376] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0377] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0378] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0379] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0380] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0381] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0382] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0383] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0384] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0385] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0386] In embodiments of the present invention, multiple components are coordinated to optimize material management. The main elements consist of a data acquisition device, a data processing module, an analysis engine, and a notification system.

[0387] In the data acquisition system, cameras and sensors installed on-site collect information in real time and transmit data on material inventory levels, location information, and movement patterns to a cloud server.

[0388] The server uses a data processing module to organize the collected information, remove noise data, and store it in the database. This makes it possible to always have a grasp of the latest field conditions.

[0389] The server then uses this data to run an analysis engine. The analysis engine uses historical consumption data and external factor data (such as weather information and worker schedules) to predict material consumption patterns.

[0390] Based on the predicted data, the server generates suggestions for optimal ordering plans and material allocation. For example, in a project that uses a lot of materials on rainy days, it will consider the weather forecast in advance and plan to order more than necessary.

[0391] The server generates a plan and proposes it to the user (site supervisor) via an app through a notification system. The user uses the app to review the plan and make adjustments or approvals as needed.

[0392] This system aims to prevent material shortages and excess inventory, maximizing on-site efficiency. This reduces the burden on site supervisors and supports the overall success of the project.

[0393] The following describes the processing flow.

[0394] Step 1:

[0395] The server receives real-time inventory and location information of materials from on-site data acquisition devices. Each time data from sensors is updated, the server aggregates and centrally manages the data.

[0396] Step 2:

[0397] Before saving received data to the database, the server performs data cleaning. This removes inaccurate data and noise and verifies the integrity of inventory information.

[0398] Step 3:

[0399] The server predicts material consumption by combining historical consumption data with external factors such as weather and worker schedules. This involves using machine learning algorithms to analyze the factors that cause fluctuations in consumption patterns.

[0400] Step 4:

[0401] The server generates an optimal ordering plan for materials based on consumption forecasts. The timing and quantity of orders are determined considering inventory costs and lead times.

[0402] Step 5:

[0403] The server generates optimized material placement suggestions based on on-site layout information, taking into account traffic flow. Materials that are frequently accessed are placed particularly efficiently.

[0404] Step 6:

[0405] The server notifies the user of the generated order plan and placement proposal through the app. The notification includes options to help with decision-making and detailed suggestions.

[0406] Step 7:

[0407] Users check notifications via the app and approve or modify the plan based on the suggestions provided. Once approved, the information is returned to the system and implemented.

[0408] (Example 1)

[0409] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0410] Materials management in the industrial sector constantly faces problems such as inventory shortages and surpluses, and decreased work efficiency due to improper allocation. Furthermore, conventional systems struggle to collect and predict accurate information in real time, resulting in inadequate material procurement and allocation.

[0411] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0412] In this invention, the server includes means for receiving data from a sensing device and organizing information regarding the inventory status and movement routes of materials; means for calculating and processing past usage data and external factors to predict material consumption patterns; and means for presenting a material procurement plan and placement proposal to the user and requesting their approval. This makes it possible to streamline material inventory management, improve work efficiency through optimized movement routes, and reduce the risk of insufficient or excessive preparation.

[0413] A "sensing device" refers to hardware used to record on-site conditions in real time and collect information about the inventory and movement of materials.

[0414] A "server" refers to a core information processing system that analyzes received data and generates a resource management plan.

[0415] "Inventory status" refers to data regarding the quantity and types of goods stored at a site or warehouse at a given time.

[0416] "Transportation route" refers to route information that shows how supplies are moved within the site.

[0417] "Computational processing" refers to the process of performing mathematical analysis using collected data, and specifically to the calculations necessary to predict the consumption patterns of materials.

[0418] "Consumption patterns of goods" refer to trends regarding how goods are used and the rate at which they are consumed, based on past usage data and external factors.

[0419] A "procurement plan" refers to a detailed plan for securing necessary supplies in the appropriate time and quantity.

[0420] A "layout proposal" refers to a plan that outlines how materials should be arranged on-site to ensure the most efficient use of them.

[0421] "Information organization" refers to the process of processing received data, removing noise, and prioritizing its importance to make it usable.

[0422] This invention is a system that works in conjunction with multiple components to improve the efficiency of material management.

[0423] The terminal uses sensing devices placed on-site, specifically cameras and sensors, to collect real-time data on the inventory status and movement routes of supplies. This data is immediately transmitted to a server in the cloud.

[0424] The server receives this data and uses a data processing module to organize the information. Image processing technology is used to analyze the video data from the camera, removing noise such as audio data to extract clear information. This organized information is stored in a database, allowing for a constantly updated understanding of the on-site situation.

[0425] Furthermore, the server uses an analysis engine to predict material consumption patterns using machine learning algorithms, based on past material usage data and external factors such as weather and worker schedules. For example, if it rains continuously, the amount of concrete used may increase, so the server plans to order more materials based on that.

[0426] Next, the server notifies the user (site supervisor) of the generated plan via a notification system to their app. The user can review the proposed procurement plan and placement suggestions in the app and make changes as needed. This maximizes operational efficiency and reduces the risk of material shortages or surpluses.

[0427] As a concrete example, a prompt message to the generating AI model, such as "Predict the material consumption pattern of Project X based on weather forecasts and historical data, and propose the optimal ordering plan," will activate the analysis engine and create a useful material management plan.

[0428] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0429] Step 1:

[0430] The terminal (sensing device) collects real-time data on the inventory status and movement routes of supplies from cameras and sensors installed on-site. Inputs include camera video data and location data from sensors. This data is transmitted to a cloud server, providing the necessary information to understand the current state of the supplies. Specifically, cameras capture shelf heights, and sensors track movement paths.

[0431] Step 2:

[0432] The server analyzes the received data using a data processing module and organizes the information. Inputs include image data and numerical data sent from sensing devices. Specific data processing includes material identification through image processing, refinement through noise reduction filtering, and automatic detection of anomalies. Since the data is then stored in a database, the output is organized on-site inventory data.

[0433] Step 3:

[0434] The server uses an analysis engine to analyze organized inventory data, historical usage history, and external factor data (e.g., weather and work schedule) as input. A generative AI model is involved in this analysis, providing "prompt messages" to predict consumption patterns. The output is a predicted demand pattern for materials. For example, it shows increases or decreases in material consumption based on weather forecasts.

[0435] Step 4:

[0436] The server generates procurement plans and placement suggestions based on forecast data. Inputs include predicted demand patterns and current inventory data. An algorithm is used to optimize procurement timing and quantity. The output is the generated material procurement plan and placement suggestions. Specific operations include listing required materials and suggesting optimal placement methods.

[0437] Step 5:

[0438] The server notifies the user (site supervisor) of the generated plan through a notification system. Specifically, the server sends a notification to the user's app, and the user receives and confirms its contents. The input is the generated proposal, and the output is feedback from the user regarding approval or revisions. Here, the user uses a smartphone or tablet to check the details of the plan and take actions such as pressing the approval button.

[0439] (Application Example 1)

[0440] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0441] In materials management, inventory shortages and excesses can negatively impact project progress. In particular, ordering and distributing materials without considering environmental factors or on-site workflows leads to inefficient management. Currently, optimizing these processes requires significant time and effort, increasing the burden on managers. Therefore, there is a need for a highly versatile system that accurately predicts material usage patterns and automatically proposes efficient ordering and distribution strategies.

[0442] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0443] In this invention, the server includes means for collecting information from a data acquisition device and organizing information regarding material inventory and movement; means for analyzing past consumption information and external factors to predict material usage patterns; and means for automatically generating material ordering plans and placement proposals considering environmental factors and notifying the user. This enables more efficient material management.

[0444] A "data acquisition device" is a device installed on-site to collect real-time information on material inventory, location, and movement patterns.

[0445] "Means for collecting information and organizing information regarding material inventory and movement" refers to data organization methods that use collected raw data to understand the inventory situation and efficiency of movement at the site.

[0446] "Methods for predicting material usage patterns by analyzing past consumption information and external factors" refers to analytical methods that analyze past material consumption history and external factors such as weather to estimate future material demand.

[0447] "A means of automatically generating material ordering plans and placement proposals that take environmental factors into consideration and notifying users" refers to a system that creates ordering schedules and placement proposals that respond to environmental changes based on predicted demand, and informs relevant parties of these.

[0448] In an embodiment of this invention, the server provides an integrated system for streamlining material management. Using data acquisition devices installed on-site, specifically cameras and sensors, it collects real-time data on material inventory, location, and movement patterns. This data is transmitted to a cloud server via the internet. The server uses programming languages ​​such as Python to build data processing modules, organize the collected data, remove noise, and store it in a database. This enables up-to-date monitoring of on-site conditions.

[0449] The server then activates an analysis engine to predict material consumption patterns based on data. Specifically, it integrates and analyzes external factors such as past consumption information, weather data, and work schedules to predict future demand. Based on this predicted data, it generates an optimal ordering plan and material allocation proposal. The server communicates the generated proposal to the site supervisor through a notification system and requests confirmation and approval via a smartphone app. This allows managers to efficiently manage materials.

[0450] As a concrete example, in a specific construction project, where work is often carried out on rainy days, it is possible to automatically propose a plan to prepare 20% more materials during weeks when rain is expected. An example of a prompt message based on this would be: "Predict the consumption pattern of material A to be used at the construction site and generate an optimal ordering plan that takes weather forecasts into consideration."

[0451] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0452] Step 1:

[0453] The terminal connects to a data acquisition device and collects data in real time from cameras and sensors installed on-site. This data includes material inventory levels, location information, and movement patterns. This data is transmitted to a cloud server via the internet. The input is raw data acquired via a wireless or wired network, and the output is data transmission to the cloud server.

[0454] Step 2:

[0455] The server uses a data processing module to organize the received data. Specifically, it filters out noisy data and formats it into an appropriate format. The input is the collected raw data, and the output is the organized information. In this process, algorithms are used to cleanse and convert the data format.

[0456] Step 3:

[0457] The server inputs the organized data into the analysis engine to predict material consumption patterns. This analysis utilizes historical consumption data, weather information, and work schedules. The input consists of organized information and data on external factors, and the output is a forecast of future material demand. The analysis engine uses a statistical model to perform demand forecasting.

[0458] Step 4:

[0459] The server generates optimal ordering plans and material allocation suggestions based on predicted data. It applies an algorithm to determine the optimal order quantities and schedules for predicted demand. The input is a forecast of consumption patterns, and the output is an ordering plan and allocation suggestion. The suggestions are automatically formulated based on the calculation results.

[0460] Step 5:

[0461] The server presents the generated proposals to the user via a smartphone app using a notification system. The user reviews the proposals using the app and approves them as needed. The input is the generated proposals, and the output is the notification and feedback to the user. The notification system uses push notification technology to deliver information quickly.

[0462] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0463] In embodiments of the present invention, emotion recognition technology is combined with a materials management system to analyze the emotional state of the user (site supervisor) and, based on that analysis, optimal materials management and recommendations are made.

[0464] First, the server processes inventory information and customer movement data collected from data acquisition devices in real time. In addition, the server uses facial expression data and voice transmitted from the user's terminal to enable an emotion engine to analyze the user's emotional state.

[0465] The emotion engine utilizes machine learning techniques to identify user emotions and evaluate feelings such as stress, satisfaction, and anxiety. This information is then reflected in material ordering plans and placement suggestions. For example, if a user is feeling stressed, the system can reduce the frequency of notifications or simplify the suggestions.

[0466] The server uses predictive algorithms to analyze material consumption patterns and optimize ordering plans. By combining this with evaluations from an emotion engine, the plan adapts to the user's mood.

[0467] The notification system sends planned order and placement proposals to users via the application. Users review the proposals through the app and make any necessary changes. The system learns from user feedback and uses it to improve future proposals.

[0468] This embodiment allows the materials management system to go beyond simply pursuing efficiency and make flexible suggestions that take into account the user's emotional state, aiming to improve operational efficiency on-site.

[0469] The following describes the processing flow.

[0470] Step 1:

[0471] The server collects real-time inventory and movement data of materials supplied from data acquisition devices and records this data in a database. The data is continuously organized to ensure that the latest situation on site is understood.

[0472] Step 2:

[0473] The device uses its built-in camera and microphone to record the user's facial expressions and voice. Data is acquired during everyday use without requiring any special operation from the user.

[0474] Step 3:

[0475] The server analyzes facial expression and voice data transmitted from the terminal using an emotion engine to determine the user's emotional state. It uses machine learning models to identify emotions such as stress and satisfaction.

[0476] Step 4:

[0477] The server adjusts its resource consumption forecast based on the analysis results. For example, if a user is experiencing stress, it may change the timing of notifications and prioritize presenting concise information.

[0478] Step 5:

[0479] The server combines predicted material consumption and sentiment analysis results to generate an optimal material ordering plan and placement proposal. This includes optimizing ordering timing and suggesting efficient placement based on workflow.

[0480] Step 6:

[0481] The server notifies the user of the generated plan and proposal. The notification is sent through the app, and the user can review the proposal.

[0482] Step 7:

[0483] Users review the notified plan within the app and approve or modify it. The feedback is sent back to the server, and the system uses this information to further optimize its features.

[0484] (Example 2)

[0485] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0486] In industrial settings, inventory management systems require not only the pursuit of conventional efficiency but also flexible management that responds to the emotional state of users. However, current systems are unable to optimize inventory management based on users' emotional states, which can lead to decreased management efficiency and user satisfaction.

[0487] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0488] In this invention, the server includes means for accumulating information from data collection elements and organizing information regarding the inventory and movement of items; means for analyzing past consumption information and external factors to predict item usage trends; and means for analyzing the user's facial expressions and voice data and making item management suggestions based on their emotional state. This enables optimal item management and suggestions that take into account the user's emotional state.

[0489] A "data collection element" is hardware or sensor technology that detects inventory information and movement patterns of goods and transmits that data to a server.

[0490] "Inventory of goods" refers to the quantity and condition of goods and materials that need to be managed in industrial sites and warehouses.

[0491] "Movement flow" refers to the routes taken by people and materials within a work site or facility, and is a factor that affects work efficiency and safety.

[0492] "Usage trends" refer to patterns and trends in how goods and materials are consumed over time.

[0493] "Facial expression and audio data" refers to visual and auditory information provided by the user through their device, and is used to analyze their emotional state.

[0494] "Emotional state" refers to the user's mental and emotional condition, including stress, satisfaction, and anxiety.

[0495] "Inventory management proposals" are a process that provides recommendations regarding ordering, inventory management, and placement of goods, enabling users to make efficient and beneficial choices.

[0496] This invention utilizes advanced data processing technology to make suggestions that take into account the emotional state of the user in an item management system. Specifically, the following hardware and software are used.

[0497] The server organizes and manages inventory and movement information obtained from data collection elements in real time. This is done using IoT sensors installed on each shelf and storage area. These sensors determine the quantity of inventory and transmit the data to the server via wireless communication. The server has a built-in database management system that efficiently stores the collected information and processes it as needed.

[0498] The device functions as a device that collects facial and voice data from the user. The device has a built-in camera and microphone, and facial recognition software operates to analyze the video data. Voice recognition software also runs simultaneously, collecting data to determine emotions from the user's voice. This allows for real-time analysis of the user's emotional state.

[0499] The emotion engine is a machine learning algorithm that runs on the server and is implemented using frameworks such as TensorFlow and PyTorch. This allows facial expressions and voice data acquired from the device to be classified into emotions such as stress, satisfaction, or anxiety. This information is then used to inform ordering plans and placement suggestions for goods, resulting in customized experiences tailored to the user's mental state.

[0500] As a concrete example, consider a situation at a construction site where a site supervisor uses a tablet to monitor daily work. If the system determines that the supervisor is experiencing stress, it can automatically adjust to reduce the frequency of notifications and display only concise suggestions.

[0501] An example of an input prompt for a generative AI model is, "Suggest an algorithm that adjusts notification frequency using user sentiment data." This prompt prompts the system to generate an adaptive strategy to optimize management tasks while taking sentiment data into consideration.

[0502] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0503] Step 1:

[0504] The server receives inventory information and movement data from data collection elements. This input data includes the quantity and location information of items detected by IoT sensors. The server organizes this data and stores it in a database. This process provides the basic data necessary for understanding the current state of items and optimizing their movement.

[0505] Step 2:

[0506] The device collects the user's facial expressions and voice data through its camera and microphone. Specifically, facial recognition and voice analysis software operates using the video and audio obtained from the device as input. Based on the data, the analysis software generates output that classifies the user's emotional state into emotions such as stress, satisfaction, and anxiety. This makes it possible to understand the user's mental state in real time.

[0507] Step 3:

[0508] The server uses an emotion engine to analyze emotional data transmitted from the terminal. The input is the user's emotional state, which the emotion engine analyzes using a machine learning model and outputs a specific emotional evaluation. This evaluation result will be an important element in future material management proposals. The server records the results of the emotional analysis and uses them to optimize material management.

[0509] Step 4:

[0510] The server uses historical consumption data and current inventory data as input to execute a predictive algorithm. This calculation estimates future demand for goods, taking into account consumption trends and external factors, and produces an output that creates an ordering plan. Furthermore, it develops a plan that incorporates emotional evaluation, creating flexible suggestions tailored to the user's emotional state.

[0511] Step 5:

[0512] Users receive order plans and deployment proposals from the server via the application. These proposals are pre-adjusted to suit the user's situation, and users review the proposals and make any necessary modifications. User feedback is provided as input, and the results are fed back to the server as output. This feedback is used to improve the accuracy of future proposals.

[0513] Through this series of processes, the system can not only improve the efficiency of inventory management but also provide management suggestions that respond to the emotional state of the users.

[0514] (Application Example 2)

[0515] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0516] Traditional materials management systems contribute to increased efficiency on-site, but they fail to consider the emotional state of users. As a result, work efficiency and user satisfaction may decrease, and in situations where stress and anxiety are high, this can negatively impact materials management decision-making. Therefore, there is a need for a materials management system that utilizes emotion recognition technology to adapt to the emotional state of users.

[0517] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0518] In this invention, the server includes means for organizing information on the inventory and movement of materials collected from a data acquisition device; means for analyzing past consumption information and external factors to predict material usage patterns; means for utilizing emotion recognition technology to analyze the emotional state of users; means for optimizing material ordering plans and placement suggestions based on the emotion analysis results; and means for simplifying or adjusting notification content according to the emotional state. This enables flexible material management in accordance with the user's emotional state, improving work efficiency and satisfaction with on-site management.

[0519] A "data acquisition device" is a device used to collect information on the inventory status and movement patterns of materials.

[0520] "Means of organizing information" refer to methods and techniques for efficiently classifying collected data and presenting it in an easily understandable format.

[0521] "Consumption information" refers to information that shows the usage history and consumption trends of materials.

[0522] "External factors" refer to external environmental factors such as weather conditions and market trends that affect material consumption.

[0523] "Methods for predicting usage patterns" refer to methods or algorithms that predict future trends in material usage based on analyzed past data.

[0524] "Emotion recognition technology" is a technology used to analyze a user's emotions and psychological state, making judgments based on data such as voice and facial expressions.

[0525] "Means for optimizing ordering plans and allocation proposals" refers to methods and systems for proposing appropriate ordering and allocation of materials according to the user's emotional state.

[0526] "Means of simplifying or adjusting notification content" refers to methods of flexibly changing the amount of information and the method of delivery of notifications, taking into consideration the user's feelings.

[0527] The system in this invention mainly consists of three elements: a server, a terminal, and a user. The server organizes information on material inventory and movement patterns sent from the data acquisition device in real time. It uses Amazon Web Services (AWS) data management tools to efficiently organize the information. The server also analyzes past consumption information and external factors, employing machine learning algorithms to predict material usage patterns. Microsoft Azure's predictive models are utilized for this purpose. Furthermore, the server is equipped with an emotion recognition engine to analyze the user's emotional state. User facial expression data and voice data are collected by smart glasses and used for analysis.

[0528] On the user's device, emotion recognition technology is used to analyze the user's emotional state in real time, and based on this, the ordering plan and placement suggestions for materials are optimized. In this process, the emotion engine determines the user's stress and satisfaction levels and simplifies or adjusts the content of notifications. This information is visually communicated through the user's smart glasses.

[0529] For example, if a user is stressed due to a shortage of materials, the server can sense this emotion, simplify the suggestions, and display only important notifications to the user's view, thereby supporting quick decision-making on-site. An example of a prompt using a generative AI model is, "How can I simplify notifications when a site supervisor is stressed?"

[0530] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0531] Step 1:

[0532] The server receives inventory and movement data for materials from data acquisition devices. Input is real-time data from sensors and RFID readers, and output is an integrated database of inventory and movement information. The data is organized and efficiently stored in the database using AWS data management tools.

[0533] Step 2:

[0534] The server acquires historical consumption information and external factors (e.g., weather data and market trends) and analyzes them. Here, the input is historical transaction records and data acquired from external sources, and the output is a predictive model of material usage patterns. Machine learning algorithms from Microsoft Azure are used to perform predictions through time series analysis and regression analysis.

[0535] Step 3:

[0536] The user's device transmits audio and facial expression data collected through smart glasses to a server. The input consists of video and audio data, while the output is an input dataset for the emotion recognition engine. The data is first compressed using a codec on the local device before being securely transmitted.

[0537] Step 4:

[0538] The server analyzes the received user data using an emotion recognition engine to evaluate the user's emotional state. The input here is a dataset of voice and facial expressions, and the output is a determination of the emotional state (stress, satisfaction, anxiety, etc.). Emotional analysis technology using a generative AI model is employed to determine emotions with high accuracy.

[0539] Step 5:

[0540] Based on the user's emotional state, the server adjusts the material ordering plan and placement suggestions. The input is emotional state evaluation data, and the output is an optimized ordering plan and placement suggestion. The suggestion content and frequency of notifications are simplified as needed.

[0541] Step 6:

[0542] Optimized suggestions and tailored notifications are visually transmitted to the user's smart glasses. The input is material management suggestions, and the output is notifications displayed in the user's field of vision. At this point, the user can review the suggestions and make any necessary changes immediately.

[0543] Step 7:

[0544] Users make decisions based on the suggestions and send feedback to the system. The input here is user feedback information, and the output is adjustment data that helps in future suggestions. This allows the system to learn and further optimize its processes.

[0545] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0546] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0547] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0548] [Fourth Embodiment]

[0549] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0550] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0551] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0552] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0553] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0554] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0555] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0556] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0557] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0558] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0559] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0560] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0561] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0562] In embodiments of the present invention, multiple components are coordinated to optimize material management. The main elements consist of a data acquisition device, a data processing module, an analysis engine, and a notification system.

[0563] In the data acquisition system, cameras and sensors installed on-site collect information in real time and transmit data on material inventory levels, location information, and movement patterns to a cloud server.

[0564] The server uses a data processing module to organize the collected information, remove noise data, and store it in the database. This makes it possible to always have a grasp of the latest field conditions.

[0565] The server then uses this data to run an analysis engine. The analysis engine uses historical consumption data and external factor data (such as weather information and worker schedules) to predict material consumption patterns.

[0566] Based on the predicted data, the server generates suggestions for optimal ordering plans and material allocation. For example, in a project that uses a lot of materials on rainy days, it will consider the weather forecast in advance and plan to order more than necessary.

[0567] The server generates a plan and proposes it to the user (site supervisor) via an app through a notification system. The user uses the app to review the plan and make adjustments or approvals as needed.

[0568] This system aims to prevent material shortages and excess inventory, maximizing on-site efficiency. This reduces the burden on site supervisors and supports the overall success of the project.

[0569] The following describes the processing flow.

[0570] Step 1:

[0571] The server receives real-time inventory and location information of materials from on-site data acquisition devices. Each time data from sensors is updated, the server aggregates and centrally manages the data.

[0572] Step 2:

[0573] Before saving received data to the database, the server performs data cleaning. This removes inaccurate data and noise and verifies the integrity of inventory information.

[0574] Step 3:

[0575] The server predicts material consumption by combining historical consumption data with external factors such as weather and worker schedules. This involves using machine learning algorithms to analyze the factors that cause fluctuations in consumption patterns.

[0576] Step 4:

[0577] The server generates an optimal ordering plan for materials based on consumption forecasts. The timing and quantity of orders are determined considering inventory costs and lead times.

[0578] Step 5:

[0579] The server generates optimized material placement suggestions based on on-site layout information, taking into account traffic flow. Materials that are frequently accessed are placed particularly efficiently.

[0580] Step 6:

[0581] The server notifies the user of the generated order plan and placement proposal through the app. The notification includes options to help with decision-making and detailed suggestions.

[0582] Step 7:

[0583] Users check notifications via the app and approve or modify the plan based on the suggestions provided. Once approved, the information is returned to the system and implemented.

[0584] (Example 1)

[0585] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0586] Materials management in the industrial sector constantly faces problems such as inventory shortages and surpluses, and decreased work efficiency due to improper allocation. Furthermore, conventional systems struggle to collect and predict accurate information in real time, resulting in inadequate material procurement and allocation.

[0587] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0588] In this invention, the server includes means for receiving data from a sensing device and organizing information regarding the inventory status and movement routes of materials; means for calculating and processing past usage data and external factors to predict material consumption patterns; and means for presenting a material procurement plan and placement proposal to the user and requesting their approval. This makes it possible to streamline material inventory management, improve work efficiency through optimized movement routes, and reduce the risk of insufficient or excessive preparation.

[0589] A "sensing device" refers to hardware used to record on-site conditions in real time and collect information about the inventory and movement of materials.

[0590] A "server" refers to a core information processing system that analyzes received data and generates a resource management plan.

[0591] "Inventory status" refers to data regarding the quantity and types of goods stored at a site or warehouse at a given time.

[0592] "Transportation route" refers to route information that shows how supplies are moved within the site.

[0593] "Computational processing" refers to the process of performing mathematical analysis using collected data, and specifically to the calculations necessary to predict the consumption patterns of materials.

[0594] "Consumption patterns of goods" refer to trends regarding how goods are used and the rate at which they are consumed, based on past usage data and external factors.

[0595] A "procurement plan" refers to a detailed plan for securing necessary supplies in the appropriate time and quantity.

[0596] A "layout proposal" refers to a plan that outlines how materials should be arranged on-site to ensure the most efficient use of them.

[0597] "Information organization" refers to the process of processing received data, removing noise, and prioritizing its importance to make it usable.

[0598] This invention is a system that works in conjunction with multiple components to improve the efficiency of material management.

[0599] The terminal uses sensing devices placed on-site, specifically cameras and sensors, to collect real-time data on the inventory status and movement routes of supplies. This data is immediately transmitted to a server in the cloud.

[0600] The server receives this data and uses a data processing module to organize the information. Image processing technology is used to analyze the video data from the camera, removing noise such as audio data to extract clear information. This organized information is stored in a database, allowing for a constantly updated understanding of the on-site situation.

[0601] Furthermore, the server uses an analysis engine to predict material consumption patterns using machine learning algorithms, based on past material usage data and external factors such as weather and worker schedules. For example, if it rains continuously, the amount of concrete used may increase, so the server plans to order more materials based on that.

[0602] Next, the server notifies the user (site supervisor) of the generated plan via a notification system to their app. The user can review the proposed procurement plan and placement suggestions in the app and make changes as needed. This maximizes operational efficiency and reduces the risk of material shortages or surpluses.

[0603] As a concrete example, a prompt message to the generating AI model, such as "Predict the material consumption pattern of Project X based on weather forecasts and historical data, and propose the optimal ordering plan," will activate the analysis engine and create a useful material management plan.

[0604] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0605] Step 1:

[0606] The terminal (sensing device) collects real-time data on the inventory status and movement routes of supplies from cameras and sensors installed on-site. Inputs include camera video data and location data from sensors. This data is transmitted to a cloud server, providing the necessary information to understand the current state of the supplies. Specifically, cameras capture shelf heights, and sensors track movement paths.

[0607] Step 2:

[0608] The server analyzes the received data using a data processing module and organizes the information. Inputs include image data and numerical data sent from sensing devices. Specific data processing includes material identification through image processing, refinement through noise reduction filtering, and automatic detection of anomalies. Since the data is then stored in a database, the output is organized on-site inventory data.

[0609] Step 3:

[0610] The server uses an analysis engine to analyze organized inventory data, historical usage history, and external factor data (e.g., weather and work schedule) as input. A generative AI model is involved in this analysis, providing "prompt messages" to predict consumption patterns. The output is a predicted demand pattern for materials. For example, it shows increases or decreases in material consumption based on weather forecasts.

[0611] Step 4:

[0612] The server generates procurement plans and placement suggestions based on forecast data. Inputs include predicted demand patterns and current inventory data. An algorithm is used to optimize procurement timing and quantity. The output is the generated material procurement plan and placement suggestions. Specific operations include listing required materials and suggesting optimal placement methods.

[0613] Step 5:

[0614] The server notifies the user (site supervisor) of the generated plan through a notification system. Specifically, the server sends a notification to the user's app, and the user receives and confirms its contents. The input is the generated proposal, and the output is feedback from the user regarding approval or revisions. Here, the user uses a smartphone or tablet to check the details of the plan and take actions such as pressing the approval button.

[0615] (Application Example 1)

[0616] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0617] In materials management, inventory shortages and excesses can negatively impact project progress. In particular, ordering and distributing materials without considering environmental factors or on-site workflows leads to inefficient management. Currently, optimizing these processes requires significant time and effort, increasing the burden on managers. Therefore, there is a need for a highly versatile system that accurately predicts material usage patterns and automatically proposes efficient ordering and distribution strategies.

[0618] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0619] In this invention, the server includes means for collecting information from a data acquisition device and organizing information regarding material inventory and movement; means for analyzing past consumption information and external factors to predict material usage patterns; and means for automatically generating material ordering plans and placement proposals considering environmental factors and notifying the user. This enables more efficient material management.

[0620] A "data acquisition device" is a device installed on-site to collect real-time information on material inventory, location, and movement patterns.

[0621] "Means for collecting information and organizing information regarding material inventory and movement" refers to data organization methods that use collected raw data to understand the inventory situation and efficiency of movement at the site.

[0622] "Methods for predicting material usage patterns by analyzing past consumption information and external factors" refers to analytical methods that analyze past material consumption history and external factors such as weather to estimate future material demand.

[0623] "A means of automatically generating material ordering plans and placement proposals that take environmental factors into consideration and notifying users" refers to a system that creates ordering schedules and placement proposals that respond to environmental changes based on predicted demand, and informs relevant parties of these.

[0624] In an embodiment of this invention, the server provides an integrated system for streamlining material management. Using data acquisition devices installed on-site, specifically cameras and sensors, it collects real-time data on material inventory, location, and movement patterns. This data is transmitted to a cloud server via the internet. The server uses programming languages ​​such as Python to build data processing modules, organize the collected data, remove noise, and store it in a database. This enables up-to-date monitoring of on-site conditions.

[0625] The server then activates an analysis engine to predict material consumption patterns based on data. Specifically, it integrates and analyzes external factors such as past consumption information, weather data, and work schedules to predict future demand. Based on this predicted data, it generates an optimal ordering plan and material allocation proposal. The server communicates the generated proposal to the site supervisor through a notification system and requests confirmation and approval via a smartphone app. This allows managers to efficiently manage materials.

[0626] As a concrete example, in a specific construction project, where work is often carried out on rainy days, it is possible to automatically propose a plan to prepare 20% more materials during weeks when rain is expected. An example of a prompt message based on this would be: "Predict the consumption pattern of material A to be used at the construction site and generate an optimal ordering plan that takes weather forecasts into consideration."

[0627] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0628] Step 1:

[0629] The terminal connects to a data acquisition device and collects data in real time from cameras and sensors installed on-site. This data includes material inventory levels, location information, and movement patterns. This data is transmitted to a cloud server via the internet. The input is raw data acquired via a wireless or wired network, and the output is data transmission to the cloud server.

[0630] Step 2:

[0631] The server uses a data processing module to organize the received data. Specifically, it filters out noisy data and formats it into an appropriate format. The input is the collected raw data, and the output is the organized information. In this process, algorithms are used to cleanse and convert the data format.

[0632] Step 3:

[0633] The server inputs the organized data into the analysis engine to predict material consumption patterns. This analysis utilizes historical consumption data, weather information, and work schedules. The input consists of organized information and data on external factors, and the output is a forecast of future material demand. The analysis engine uses a statistical model to perform demand forecasting.

[0634] Step 4:

[0635] The server generates optimal ordering plans and material allocation suggestions based on predicted data. It applies an algorithm to determine the optimal order quantities and schedules for predicted demand. The input is a forecast of consumption patterns, and the output is an ordering plan and allocation suggestion. The suggestions are automatically formulated based on the calculation results.

[0636] Step 5:

[0637] The server presents the generated proposals to the user via a smartphone app using a notification system. The user reviews the proposals using the app and approves them as needed. The input is the generated proposals, and the output is the notification and feedback to the user. The notification system uses push notification technology to deliver information quickly.

[0638] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0639] In embodiments of the present invention, emotion recognition technology is combined with a materials management system to analyze the emotional state of the user (site supervisor) and, based on that analysis, optimal materials management and recommendations are made.

[0640] First, the server processes inventory information and customer movement data collected from data acquisition devices in real time. In addition, the server uses facial expression data and voice transmitted from the user's terminal to enable an emotion engine to analyze the user's emotional state.

[0641] The emotion engine utilizes machine learning techniques to identify user emotions and evaluate feelings such as stress, satisfaction, and anxiety. This information is then reflected in material ordering plans and placement suggestions. For example, if a user is feeling stressed, the system can reduce the frequency of notifications or simplify the suggestions.

[0642] The server uses predictive algorithms to analyze material consumption patterns and optimize ordering plans. By combining this with evaluations from an emotion engine, the plan adapts to the user's mood.

[0643] The notification system sends planned order and placement proposals to users via the application. Users review the proposals through the app and make any necessary changes. The system learns from user feedback and uses it to improve future proposals.

[0644] This embodiment allows the materials management system to go beyond simply pursuing efficiency and make flexible suggestions that take into account the user's emotional state, aiming to improve operational efficiency on-site.

[0645] The following describes the processing flow.

[0646] Step 1:

[0647] The server collects real-time inventory and movement data of materials supplied from data acquisition devices and records this data in a database. The data is continuously organized to ensure that the latest situation on site is understood.

[0648] Step 2:

[0649] The device uses its built-in camera and microphone to record the user's facial expressions and voice. Data is acquired during everyday use without requiring any special operation from the user.

[0650] Step 3:

[0651] The server analyzes facial expression and voice data transmitted from the terminal using an emotion engine to determine the user's emotional state. It uses machine learning models to identify emotions such as stress and satisfaction.

[0652] Step 4:

[0653] The server adjusts its resource consumption forecast based on the analysis results. For example, if a user is experiencing stress, it may change the timing of notifications and prioritize presenting concise information.

[0654] Step 5:

[0655] The server combines predicted material consumption and sentiment analysis results to generate an optimal material ordering plan and placement proposal. This includes optimizing ordering timing and suggesting efficient placement based on workflow.

[0656] Step 6:

[0657] The server notifies the user of the generated plan and proposal. The notification is sent through the app, and the user can review the proposal.

[0658] Step 7:

[0659] Users review the notified plan within the app and approve or modify it. The feedback is sent back to the server, and the system uses this information to further optimize its features.

[0660] (Example 2)

[0661] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0662] In industrial settings, inventory management systems require not only the pursuit of conventional efficiency but also flexible management that responds to the emotional state of users. However, current systems are unable to optimize inventory management based on users' emotional states, which can lead to decreased management efficiency and user satisfaction.

[0663] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0664] In this invention, the server includes means for accumulating information from data collection elements and organizing information regarding the inventory and movement of items; means for analyzing past consumption information and external factors to predict item usage trends; and means for analyzing the user's facial expressions and voice data and making item management suggestions based on their emotional state. This enables optimal item management and suggestions that take into account the user's emotional state.

[0665] A "data collection element" is hardware or sensor technology that detects inventory information and movement patterns of goods and transmits that data to a server.

[0666] "Inventory of goods" refers to the quantity and condition of goods and materials that need to be managed in industrial sites and warehouses.

[0667] "Movement flow" refers to the routes taken by people and materials within a work site or facility, and is a factor that affects work efficiency and safety.

[0668] "Usage trends" refer to patterns and trends in how goods and materials are consumed over time.

[0669] "Facial expression and audio data" refers to visual and auditory information provided by the user through their device, and is used to analyze their emotional state.

[0670] "Emotional state" refers to the user's mental and emotional condition, including stress, satisfaction, and anxiety.

[0671] "Inventory management proposals" are a process that provides recommendations regarding ordering, inventory management, and placement of goods, enabling users to make efficient and beneficial choices.

[0672] This invention utilizes advanced data processing technology to make suggestions that take into account the emotional state of the user in an item management system. Specifically, the following hardware and software are used.

[0673] The server organizes and manages inventory and movement information obtained from data collection elements in real time. This is done using IoT sensors installed on each shelf and storage area. These sensors determine the quantity of inventory and transmit the data to the server via wireless communication. The server has a built-in database management system that efficiently stores the collected information and processes it as needed.

[0674] The device functions as a device that collects facial and voice data from the user. The device has a built-in camera and microphone, and facial recognition software operates to analyze the video data. Voice recognition software also runs simultaneously, collecting data to determine emotions from the user's voice. This allows for real-time analysis of the user's emotional state.

[0675] The emotion engine is a machine learning algorithm that runs on the server and is implemented using frameworks such as TensorFlow and PyTorch. This allows facial expressions and voice data acquired from the device to be classified into emotions such as stress, satisfaction, or anxiety. This information is then used to inform ordering plans and placement suggestions for goods, resulting in customized experiences tailored to the user's mental state.

[0676] As a concrete example, consider a situation at a construction site where a site supervisor uses a tablet to monitor daily work. If the system determines that the supervisor is experiencing stress, it can automatically adjust to reduce the frequency of notifications and display only concise suggestions.

[0677] An example of an input prompt for a generative AI model is, "Suggest an algorithm that adjusts notification frequency using user sentiment data." This prompt prompts the system to generate an adaptive strategy to optimize management tasks while taking sentiment data into consideration.

[0678] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0679] Step 1:

[0680] The server receives inventory information and movement data from data collection elements. This input data includes the quantity and location information of items detected by IoT sensors. The server organizes this data and stores it in a database. This process provides the basic data necessary for understanding the current state of items and optimizing their movement.

[0681] Step 2:

[0682] The device collects the user's facial expressions and voice data through its camera and microphone. Specifically, facial recognition and voice analysis software operates using the video and audio obtained from the device as input. Based on the data, the analysis software generates output that classifies the user's emotional state into emotions such as stress, satisfaction, and anxiety. This makes it possible to understand the user's mental state in real time.

[0683] Step 3:

[0684] The server uses an emotion engine to analyze emotional data transmitted from the terminal. The input is the user's emotional state, which the emotion engine analyzes using a machine learning model and outputs a specific emotional evaluation. This evaluation result will be an important element in future material management proposals. The server records the results of the emotional analysis and uses them to optimize material management.

[0685] Step 4:

[0686] The server uses historical consumption data and current inventory data as input to execute a predictive algorithm. This calculation estimates future demand for goods, taking into account consumption trends and external factors, and produces an output that creates an ordering plan. Furthermore, it develops a plan that incorporates emotional evaluation, creating flexible suggestions tailored to the user's emotional state.

[0687] Step 5:

[0688] Users receive order plans and deployment proposals from the server via the application. These proposals are pre-adjusted to suit the user's situation, and users review the proposals and make any necessary modifications. User feedback is provided as input, and the results are fed back to the server as output. This feedback is used to improve the accuracy of future proposals.

[0689] Through this series of processes, the system can not only improve the efficiency of inventory management but also provide management suggestions that respond to the emotional state of the users.

[0690] (Application Example 2)

[0691] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0692] Traditional materials management systems contribute to increased efficiency on-site, but they fail to consider the emotional state of users. As a result, work efficiency and user satisfaction may decrease, and in situations where stress and anxiety are high, this can negatively impact materials management decision-making. Therefore, there is a need for a materials management system that utilizes emotion recognition technology to adapt to the emotional state of users.

[0693] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0694] In this invention, the server includes means for organizing information on the inventory and movement of materials collected from a data acquisition device; means for analyzing past consumption information and external factors to predict material usage patterns; means for utilizing emotion recognition technology to analyze the emotional state of users; means for optimizing material ordering plans and placement suggestions based on the emotion analysis results; and means for simplifying or adjusting notification content according to the emotional state. This enables flexible material management in accordance with the user's emotional state, improving work efficiency and satisfaction with on-site management.

[0695] A "data acquisition device" is a device used to collect information on the inventory status and movement patterns of materials.

[0696] "Means of organizing information" refer to methods and techniques for efficiently classifying collected data and presenting it in an easily understandable format.

[0697] "Consumption information" refers to information that shows the usage history and consumption trends of materials.

[0698] "External factors" refer to external environmental factors such as weather conditions and market trends that affect material consumption.

[0699] "Methods for predicting usage patterns" refer to methods or algorithms that predict future trends in material usage based on analyzed past data.

[0700] "Emotion recognition technology" is a technology used to analyze a user's emotions and psychological state, making judgments based on data such as voice and facial expressions.

[0701] "Means for optimizing ordering plans and allocation proposals" refers to methods and systems for proposing appropriate ordering and allocation of materials according to the user's emotional state.

[0702] "Means of simplifying or adjusting notification content" refers to methods of flexibly changing the amount of information and the method of delivery of notifications, taking into consideration the user's feelings.

[0703] The system in this invention mainly consists of three elements: a server, a terminal, and a user. The server organizes information on material inventory and movement patterns sent from the data acquisition device in real time. It uses Amazon Web Services (AWS) data management tools to efficiently organize the information. The server also analyzes past consumption information and external factors, employing machine learning algorithms to predict material usage patterns. Microsoft Azure's predictive models are utilized for this purpose. Furthermore, the server is equipped with an emotion recognition engine to analyze the user's emotional state. User facial expression data and voice data are collected by smart glasses and used for analysis.

[0704] On the user's device, emotion recognition technology is used to analyze the user's emotional state in real time, and based on this, the ordering plan and placement suggestions for materials are optimized. In this process, the emotion engine determines the user's stress and satisfaction levels and simplifies or adjusts the content of notifications. This information is visually communicated through the user's smart glasses.

[0705] For example, if a user is stressed due to a shortage of materials, the server can sense this emotion, simplify the suggestions, and display only important notifications to the user's view, thereby supporting quick decision-making on-site. An example of a prompt using a generative AI model is, "How can I simplify notifications when a site supervisor is stressed?"

[0706] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0707] Step 1:

[0708] The server receives inventory and movement data for materials from data acquisition devices. Input is real-time data from sensors and RFID readers, and output is an integrated database of inventory and movement information. The data is organized and efficiently stored in the database using AWS data management tools.

[0709] Step 2:

[0710] The server acquires historical consumption information and external factors (e.g., weather data and market trends) and analyzes them. Here, the input is historical transaction records and data acquired from external sources, and the output is a predictive model of material usage patterns. Machine learning algorithms from Microsoft Azure are used to perform predictions through time series analysis and regression analysis.

[0711] Step 3:

[0712] The user's device transmits audio and facial expression data collected through smart glasses to a server. The input consists of video and audio data, while the output is an input dataset for the emotion recognition engine. The data is first compressed using a codec on the local device before being securely transmitted.

[0713] Step 4:

[0714] The server analyzes the received user data using an emotion recognition engine to evaluate the user's emotional state. The input here is a dataset of voice and facial expressions, and the output is a determination of the emotional state (stress, satisfaction, anxiety, etc.). Emotional analysis technology using a generative AI model is employed to determine emotions with high accuracy.

[0715] Step 5:

[0716] Based on the user's emotional state, the server adjusts the material ordering plan and placement suggestions. The input is emotional state evaluation data, and the output is an optimized ordering plan and placement suggestion. The suggestion content and frequency of notifications are simplified as needed.

[0717] Step 6:

[0718] Optimized suggestions and tailored notifications are visually transmitted to the user's smart glasses. The input is material management suggestions, and the output is notifications displayed in the user's field of vision. At this point, the user can review the suggestions and make any necessary changes immediately.

[0719] Step 7:

[0720] Users make decisions based on the suggestions and send feedback to the system. The input here is user feedback information, and the output is adjustment data that helps in future suggestions. This allows the system to learn and further optimize its processes.

[0721] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0722] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0723] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0724] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0725] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0726] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0727] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0728] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0729] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0730] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0731] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0732] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0733] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0734] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0735] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0736] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0737] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0738] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0739] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0740] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0741] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0742] The following is further disclosed regarding the embodiments described above.

[0743] (Claim 1)

[0744] A means for collecting information from data acquisition devices and organizing information regarding material inventory and movement,

[0745] A means of predicting material usage patterns by analyzing past consumption information and external factors,

[0746] A means of presenting the user with a material ordering plan and placement proposal and requesting their confirmation,

[0747] A system that includes this.

[0748] (Claim 2)

[0749] The system according to claim 1, which proposes a material ordering plan using an algorithm that optimizes the timing and quantity of orders.

[0750] (Claim 3)

[0751] The system according to claim 1, in which the placement of materials is proposed taking into account the flow of movement at the site and in order to improve work efficiency.

[0752] "Example 1"

[0753] (Claim 1)

[0754] A means for receiving data from a sensing device and organizing information regarding the inventory status and movement routes of materials,

[0755] A means for predicting the consumption patterns of materials by processing past usage data and external factors,

[0756] A means of presenting a procurement plan and placement proposal for supplies to users and seeking their approval,

[0757] A method for refining received data by performing noise reduction using acoustic and image processing,

[0758] A method for analyzing the prediction of material use using machine learning,

[0759] A system that includes this.

[0760] (Claim 2)

[0761] The system according to claim 1, which proposes a procurement plan for materials and utilizes a calculation process to optimize the timing and quantity of procurement.

[0762] (Claim 3)

[0763] The system according to claim 1, which proposes the placement of materials while considering the flow of movement at the work site in order to improve work efficiency.

[0764] "Application Example 1"

[0765] (Claim 1)

[0766] A means for collecting information from data acquisition devices and organizing information regarding material inventory and movement,

[0767] A means of predicting material usage patterns by analyzing past consumption information and external factors,

[0768] A means of automatically generating material ordering plans and placement proposals that take environmental factors into consideration, and notifying users of them,

[0769] A system that includes this.

[0770] (Claim 2)

[0771] The system according to claim 1, which proposes a material ordering plan using an algorithm that optimizes the timing and quantity of orders, and makes proposals in response to changes in the external environment.

[0772] (Claim 3)

[0773] The system according to claim 1, which proposes the placement of materials considering the flow of movement at the site and in order to improve work efficiency, and dynamically adjusts based on environmental conditions.

[0774] "Example 2 of combining an emotion engine"

[0775] (Claim 1)

[0776] A means for collecting information from data collection elements and organizing information regarding the inventory and movement of goods,

[0777] A means for analyzing past consumption information and external factors to predict trends in the use of goods,

[0778] A means of analyzing the user's facial expressions and voice data and making item management suggestions based on their emotional state,

[0779] A means of presenting the user with an order plan and placement proposal for goods and requesting their confirmation,

[0780] A system that includes this.

[0781] (Claim 2)

[0782] The system according to claim 1, which proposes an order plan for goods, using a calculation means to optimize the timing and quantity of orders.

[0783] (Claim 3)

[0784] The system according to claim 1, wherein the arrangement of items is proposed taking into account the flow of movement at the site in order to improve work efficiency.

[0785] "Application example 2 when combining with an emotional engine"

[0786] (Claim 1)

[0787] A means for organizing information on the inventory and movement of materials collected from a data acquisition device,

[0788] A means of predicting material usage patterns by analyzing past consumption information and external factors,

[0789] A means of using emotion recognition technology to analyze the emotional state of users,

[0790] A means for optimizing material ordering plans and placement proposals based on emotion analysis results,

[0791] A means of presenting the user with a material ordering plan and placement proposal and requesting their confirmation,

[0792] Means for simplifying or adjusting notification content according to emotional state,

[0793] A system that includes this.

[0794] (Claim 2)

[0795] The system according to claim 1, which proposes a material ordering plan using an algorithm that optimizes the timing and quantity of orders.

[0796] (Claim 3)

[0797] The system according to claim 1, in which the placement of materials is proposed taking into account the flow of movement at the site and in order to improve work efficiency. [Explanation of Symbols]

[0798] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting information from data acquisition devices and organizing information regarding material inventory and movement, A means of predicting material usage patterns by analyzing past consumption information and external factors, A means of automatically generating material ordering plans and placement proposals that take environmental factors into consideration, and notifying users of them, A system that includes this.

2. The system according to claim 1, which proposes material ordering plans using an algorithm that optimizes the timing and quantity of orders, and makes proposals in response to changes in the external environment.

3. The system according to claim 1, which proposes the placement of materials considering the flow of movement at the site and in order to improve work efficiency, and dynamically adjusts based on environmental conditions.

Images