system

Abstract

We provide the system. [Solution] A means for collecting and pre-processing data from a monitoring device, A means for analyzing collected data to predict the amount of materials used and the timing of necessary replenishment, A means for automatically generating a material ordering plan based on the analyzed results, A means for generating work instructions for on-site workers and notifying them via a display device, A means of assessing future risks and notifying the site supervisor of proposed countermeasures, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] At a construction site, there are frequent problems of material shortages and excessive inventory, which hinder the progress of work and lead to an increase in storage costs. In addition, on-site managers and supervisors have to manage materials while considering progress, weather, and the working status of workers, which is a heavy burden. Furthermore, it is difficult to predict risks in advance and formulate countermeasures, and there is a problem that unplanned troubles are likely to occur.

Means for Solving the Problems

[0005] This invention provides means for collecting data from monitoring devices and performing real-time preprocessing to accurately grasp the inventory status of materials. Furthermore, by providing means for analyzing the collected data and predicting material usage and replenishment timing, it automatically generates an optimal ordering plan. Based on this ordering plan, it is possible to prevent excess inventory and shortages of materials. In addition, by providing means for generating work instructions for on-site workers and notifying them via a display device, work efficiency is improved. Moreover, by providing means for evaluating future risks and notifying the site supervisor of proposed countermeasures, it is possible to prevent problems before they occur. As a result, the burden on the site supervisor is reduced and the accuracy and efficiency of material management are improved.

[0006] A "monitoring device" is a device installed at a construction site that has the function of collecting data in real time on the inventory status of materials and the progress of work.

[0007] "Preprocessing" is the process of removing noise from collected data and preparing it for analysis.

[0008] "Analysis" is the process of evaluating the current situation using scientific and statistical methods based on collected data, and predicting future material usage and replenishment timing.

[0009] A "procurement plan" is a plan that serves as a guideline for procuring materials, determining the optimal timing and quantity based on the results of the analysis.

[0010] "Work instructions" refer to specific instructions given to on-site workers regarding the placement of materials and work procedures.

[0011] A "display device" is a device used by workers and supervisors to visually confirm instructions and information from a system.

[0012] "Risk assessment" is the process of identifying potential problems that may arise in the future and conducting analyses to minimize their impact.

[0013] A "plan of countermeasures" is a list of specific countermeasures formulated to prevent or mitigate risks that are predicted to occur.

[0014] "Notification" is the act of transmitting necessary information or instructions to a specific person, usually using electronic means. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Embodiment for Carrying out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention is a system for streamlining material management at construction sites. It optimizes material ordering plans by collecting and analyzing data in real time using sensors and cameras. This system mainly consists of server, terminal, and user components.

[0037] Server Role

[0038] The server plays a central role, collecting data from multiple sensors and cameras installed on-site. The server preprocesses the data and then analyzes it using a generative AI model. Based on the analysis, it predicts material usage and replenishment timing, and automatically creates an ordering plan. This process also considers past consumption data, on-site work schedules, and even weather data. The server also assesses future risks and notifies the site supervisor of countermeasures as needed.

[0039] Terminal role

[0040] The terminal is for field workers to receive information. It receives work instructions generated from the server and provides specific instructions to the workers via a display device. This includes optimal placement of materials and work procedures. The terminal also assists the system's learning process by receiving feedback from workers and sending it to the server.

[0041] User roles

[0042] Users, namely site supervisors and workers, utilize information from the system to manage materials and plan work. Site supervisors make quick decisions based on risk notifications and ordering plans from the server, while workers efficiently carry out their tasks by following instructions displayed on their terminals. Feedback from users is stored on the server and used to improve the system's accuracy.

[0043] Specific example

[0044] For example, at a construction site, a server acquires real-time data on the remaining amount of materials through cameras and sensors. The server analyzes this data, comparing the current consumption rate with the future work plan, and predicts that a particular material will be in short supply in a week. Based on this, the server automatically calculates the optimal order quantity and places an order with the material supplier so that it can be replenished by the next day. Furthermore, if bad weather is predicted, the server notifies the site supervisor with suggested changes to the work schedule, thus preventing risks before they occur. Workers receive instructions on terminals and can quickly place new materials, improving the overall work efficiency of the site.

[0045] As a result, the present invention can dramatically improve the accuracy and efficiency of material management at construction sites.

[0046] The following describes the processing flow.

[0047] Step 1:

[0048] The server collects real-time data on material inventory, work progress, and weather from cameras and sensors installed at the construction site. This includes periodically receiving data from each sensor and storing it in a central database.

[0049] Step 2:

[0050] The server preprocesses the collected data. Specifically, it removes noise from the sensor data and fills in missing data. Furthermore, it converts the data into a consistent format and prepares it for analysis.

[0051] Step 3:

[0052] The server uses pre-processed data to perform analysis using an AI model. This analysis utilizes time series analysis and regression analysis based on historical and current data to predict future material usage and replenishment timing.

[0053] Step 4:

[0054] The server automatically generates a materials ordering plan based on the analysis results. This plan includes specific information such as the type and quantity of materials needed and the optimal ordering timing. The generated ordering plan is automatically notified to the relevant suppliers.

[0055] Step 5:

[0056] The server generates instructions for workers to improve the efficiency of material placement and work procedures. These instructions are sent to terminals, allowing workers to review placement procedures and work changes. This process includes displaying the instructions on a screen.

[0057] Step 6:

[0058] The server predicts future risks based on weather and work progress, and notifies the site supervisor with suggestions tailored to those risks. This risk assessment includes suggestions for changes to the work schedule due to sudden weather changes and warnings about the risk of delays in material procurement.

[0059] Step 7:

[0060] Users, such as workers and site supervisors, efficiently carry out their tasks and manage materials based on instructions and notifications received from terminals and servers. By providing feedback and inputting work status in digital format, they contribute to further system optimization by the server.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] Improving the efficiency and accuracy of material management at construction sites is a critical challenge. Traditional material management methods made it difficult to grasp site conditions in real time and create accurate ordering plans. Furthermore, they failed to adequately consider the impact of weather fluctuations and work schedule adjustments on material consumption. This resulted in unnecessary inventory and over-ordering, reducing site efficiency.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for collecting data from monitoring devices, means for preprocessing the data and converting it into a standard format, and means for inputting the collected data into a generating AI model for analysis. This enables accurate real-time data analysis and automatic generation of material ordering plans. Furthermore, the accuracy of the system can be improved by incorporating user feedback, enabling flexible material management in accordance with weather conditions and work schedules.

[0066] A "monitoring device" is a piece of equipment used at a construction site to acquire real-time information on the status of materials, and it consists of sensors, cameras, and other components.

[0067] "Preprocessing" refers to a series of operations that remove noise from acquired data and appropriately impute missing values, thereby transforming it into a state suitable for analysis.

[0068] A "generative AI model" is an analytical model that utilizes artificial intelligence technology to predict material consumption patterns and shortages using collected data.

[0069] A "procurement plan" is a plan created to secure the necessary materials based on the predicted consumption and supply timing of those materials.

[0070] "Work instructions" are instructions provided to on-site workers to encourage specific actions, such as how to arrange materials and the procedures for performing tasks.

[0071] "Feedback" refers to performance information and opinions provided by field workers and supervisors, which are used to improve the system and enhance its accuracy.

[0072] "Risk assessment" is a process for analyzing the impact of future weather conditions and changes in work schedules on material management and predicting potential problems.

[0073] This invention provides a system using sensors and cameras with the aim of improving the efficiency and accuracy of material management at construction sites. This system consists of a server, a terminal, and a user.

[0074] The server collects data in real time from sensors and cameras installed at the construction site. Specific hardware used includes weight sensors, RFID tag readers, and cameras. The data acquired from these devices is first pre-processed on the server. Pre-processing includes data filtering and imputation of missing values, and data processing software such as Python or R may be used.

[0075] The pre-processed data is input into a generative AI model. This generative AI model predicts future material consumption based on past consumption data, work schedules, and weather data. Examples of software used include machine learning libraries such as TENSORFLOW® and PyTorch. As a concrete example of this analysis process, the prompt "Predict future material consumption" is input into the model, and the expected material consumption and replenishment timing are calculated.

[0076] Based on the analysis results, the server automatically generates a materials ordering plan. This plan includes the required quantities of materials and the optimal delivery schedule. Furthermore, a risk assessment is performed, taking into account on-site weather conditions and work plans, and the site supervisor (the user) is notified as needed.

[0077] The terminals are responsible for providing work instructions from the server to field workers. Workers receive instructions from the terminals and efficiently arrange materials and manage the progress of their work. The terminals also send feedback from workers to the server, contributing to the improvement of the system's accuracy.

[0078] Users utilize information from the system to manage materials and plan work. Specifically, site supervisors can receive risk notifications from the server and take swift countermeasures. Furthermore, workers can improve overall site efficiency by quickly completing material placement based on instructions from their terminals.

[0079] Implementing this system will enable high accuracy and efficiency in on-site material management.

[0080] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0081] Step 1:

[0082] The server collects data in real time from sensors and cameras installed at the construction site. Inputs include material quantities and location information. The server receives this data, analyzes the signals sent from each device, and obtains specific information about the materials. The output generates a list of each material type and its quantity.

[0083] Step 2:

[0084] The server preprocesses the collected data. At this stage, noise and outliers are removed, and missing data is imputed with historical data or estimates. Specifically, data filtering is performed using Python. The input is the material information obtained in the previous step, and the output is a formatted material information dataset.

[0085] Step 3:

[0086] The server inputs pre-processed data into a generating AI model for analysis. This input includes formatted material information data, weather data, and work schedules. The server then prompts the generating AI model with the message, "Predict future material consumption," and performs the prediction. The output is a list of predicted material usage and replenishment timings.

[0087] Step 4:

[0088] The server automatically generates a material ordering plan based on the analysis results. The inputs are the analysis results and past ordering history. The server incorporates predictions from a machine learning model to determine the required material quantities and delivery schedule. The output is an order form to be sent to material suppliers.

[0089] Step 5:

[0090] The server generates work instructions for field workers and sends them to terminals. Input includes the ordering plan and site conditions. The server determines the specific placement of materials and work procedures, and creates instruction sheets. The output is the work instructions displayed on the terminals.

[0091] Step 6:

[0092] The terminal displays work instructions received from the server to the worker. The worker receives the instructions and performs the material placement work. The input is a work instruction sheet, and the output is a report of the completed work results. The terminal sends this report to the server, where it is stored as feedback.

[0093] Step 7:

[0094] Users manage materials using information obtained from the system. Site supervisors make accurate decisions based on risk notifications from the server, ensuring safe and efficient work. User feedback is incorporated into the system and used to improve the accuracy of future analyses.

[0095] (Application Example 1)

[0096] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0097] Efficient resource management is crucial at construction sites. However, conventional methods make it difficult to predict real-time resource usage and replenishment timing, and resource shortages can impact the progress of construction. Furthermore, adjusting work plans to accommodate fluctuations in environmental conditions is challenging. These challenges need to be addressed to achieve more flexible and efficient resource management.

[0098] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0099] In this invention, the server includes means for collecting information from data measurement devices and performing initial processing, means for analyzing the collected information to predict resource usage and the required replenishment timing, and means for providing the status of resource management to the operator in real time via communication equipment. This enables efficient resource management and flexible adjustment of work plans in response to changes in environmental conditions.

[0100] A "data measurement device" is a device used to measure the quantity, location, and condition of resources at a site and acquire them as digital data.

[0101] A "server" is a central processing unit that receives data collected from measuring devices, performs analysis and processing, and provides the results to various terminals.

[0102] "Initial processing" refers to the process of preparing data acquired from data measurement devices for analysis by performing noise reduction and format conversion.

[0103] "Real-time" is a concept that refers to a state in which information is processed or transmitted immediately, with virtually no time delay.

[0104] A "resource procurement plan" is a plan for procuring the necessary resources in the right quantity at the right time, and is based on predictions of resource usage and replenishment timing.

[0105] "Risk assessment" is the process of predicting potential risks and problems that may arise in the future and considering measures to mitigate their impact.

[0106] "Environmental conditions" refer to external natural factors that affect work and resource management at a construction site, such as weather and temperature.

[0107] The system implementing this invention is designed to streamline resource management at construction sites. This system consists of data measurement devices, servers, communication equipment, work terminals, and the like.

[0108] The server collects resource data transmitted from data measurement devices and performs initial processing. This initial processing includes noise removal and conversion to the required data format. Subsequently, the server uses a generative AI model to predict resource usage and the necessary replenishment timing. This automatically generates a resource procurement plan, enabling efficient resource management. The server also assesses future risks and notifies field managers of appropriate countermeasures.

[0109] Communication equipment plays a role in providing workers with real-time information on resource management status. This enables rapid responses tailored to the situation on site. For example, if a shortage of a certain resource is predicted, the server automatically generates a procurement plan and notifies the worker via their terminal, allowing for rapid resource replenishment.

[0110] This system is effective, for example, when "adjusting resource usage and work plans in response to fluctuations in environmental conditions." This enables efficient work progress while flexibly responding to environmental changes.

[0111] The following are examples of prompt statements that can be used.

[0112] "Based on real-time material management, please propose the next necessary actions. This should take into account material usage forecasts and proactive risk management."

[0113] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0114] Step 1:

[0115] The server receives digital data from data measurement devices, including the quantity and location information of resources. The server performs initial processing on this data and removes noise. The input is raw data, and the output is data in a clean format.

[0116] Step 2:

[0117] The server inputs the initially processed data into a generating AI model to predict resource usage and replenishment timing. Data processing considers past consumption data and environmental conditions, and the predictive model is used. The output is the predicted resource consumption and optimal replenishment timing.

[0118] Step 3:

[0119] The server automatically generates a resource procurement plan based on the forecast results. The generated plan enables the procurement of resources in the optimal quantity and timing. The input is forecast data, and the output is the finalized procurement plan.

[0120] Step 4:

[0121] The server transmits the procurement plan to field work terminals, displaying it to field personnel in real time. The input is the generated plan information, and the output is a notification to the personnel. The terminal displays this and instructs them on the necessary actions.

[0122] Step 5:

[0123] The server also assesses future risks and notifies field supervisors of relevant information. Inputs here are normal operational and environmental data, while outputs are risk notifications and proposed countermeasures.

[0124] Step 6:

[0125] The field staff, acting as users, perform tasks based on information received through their terminals and submit completion reports as feedback to the server. This feedback helps improve the accuracy of the next predictive model. The input is the feedback information, and the output is its storage in the training database.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] This invention combines a materials management system for construction sites with an emotion engine that recognizes user emotions. The aim is to improve the efficiency and safety of on-site work and reduce the burden on workers and managers. This system consists of a server, terminals, and user components.

[0128] Server Role

[0129] The server collects data from various sensors and cameras at the construction site and uses an emotion engine to analyze workers' emotions from their facial expressions and tone of voice. This allows the server to understand in real time whether workers are stressed or highly motivated. The server preprocesses the collected data, uses an AI model to forecast material demand and assess risks, and generates ordering plans. It also generates appropriate work instructions and feedback based on the emotion analysis results and provides them to the user.

[0130] Terminal role

[0131] The terminal is a device that allows workers to receive information and instructions from the server. Through a display device, it provides real-time work instructions and feedback, as well as displaying motivational messages generated by an emotion engine. This allows workers to receive appropriate support tailored to their emotional state. The terminal also functions as a means of transmitting worker feedback and emotional information to the server via voice input and camera.

[0132] User roles

[0133] This system enables site supervisors and workers, as users, to efficiently manage materials and plan work. Site supervisors receive risk notifications, work instructions, and management support based on their emotional state from the server, improving overall operational efficiency on site. Workers receive feedback tailored to their individual emotional state displayed on their terminals, reducing stress and allowing them to work more safely and comfortably.

[0134] Specific example

[0135] For example, if a worker is performing heavy labor for several consecutive days on a hot day, and the emotion engine detects that their stress levels are rising based on their facial expression and tone of voice, the server analyzes this information and displays a message on the terminal prompting them to take a break. It also provides positive feedback in real time to improve motivation. This helps ensure worker safety and maintain overall performance at the worksite.

[0136] Based on the above, the present invention provides a system that contributes to material management that takes emotions into consideration and to improving work efficiency.

[0137] The following describes the processing flow.

[0138] Step 1:

[0139] The server collects real-time data from sensors such as cameras and microphones installed at the construction site, including worker movements, audio data, and material inventory status. This data is initially processed to remove noise and prepare it for easy analysis.

[0140] Step 2:

[0141] The server activates an emotion engine to analyze the worker's facial expressions and tone of voice from the collected data. Based on this analysis, it determines the worker's current emotional state, such as "stress" or "fatigue," in real time.

[0142] Step 3:

[0143] As a notification to the site supervisor (the user), the server compiles the emotional state of each worker, as determined by the emotion engine, and displays it on a management dashboard. This allows the site supervisor to quickly understand which workers require special attention.

[0144] Step 4:

[0145] If the emotional state exceeds a certain threshold, the server sends a message generated by the emotion engine to the terminal. This message may include specific suggestions for taking a break or advice on mental health care.

[0146] Step 5:

[0147] The terminal displays emotion-based messages received from the server, and the worker reviews the content. For example, they might receive a notification such as, "Your stress level is high, please take a 10-minute break," and be advised to temporarily stop working and take a break.

[0148] Step 6:

[0149] The server continuously records emotional and work efficiency data for long-term analysis. This allows for feedback to be obtained for improving the work environment and optimizing material management. The feedback is provided to the administrator at the end of the project.

[0150] Step 7:

[0151] Users, i.e., workers, can adjust their daily tasks based on feedback from their devices and review their workload and efficiency. Furthermore, they can proactively seek areas for improvement to enhance their performance in the field, based on the continuously provided feedback.

[0152] (Example 2)

[0153] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0154] Conventional construction site management systems struggle with effective material management and timely ordering planning, and lack work instructions that adequately consider the feelings and safety of workers. Therefore, it is crucial to reduce worker stress and improve motivation, while simultaneously demanding increased efficiency in material management and improved accuracy in risk prediction.

[0155] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0156] In this invention, the server includes means for collecting and pre-processing data from a monitoring device, means for analyzing the collected data to predict the amount of materials used and the timing of necessary replenishment, and means for analyzing the emotional state of workers. This enables appropriate work instructions and feedback based on the emotional state of workers.

[0157] A "monitoring device" is a piece of equipment installed to understand the working environment and to acquire data.

[0158] "Preprocessing" refers to the initial stages of processing to prepare collected data into a format suitable for analysis.

[0159] "Data analysis" is the process of analyzing collected information to identify specific patterns or trends.

[0160] "Material usage" is an indicator that refers to the amount of materials consumed at a construction site.

[0161] "Replenishment timing" refers to the period when additional materials need to be procured, and is a point in time identified for efficient management.

[0162] An "automatic ordering plan" is an ordering plan automatically generated by a computer system based on the amount of materials used and the timing of replenishment.

[0163] "Emotional analysis" is a process that determines the emotional state of a worker based on information such as their facial expressions and tone of voice.

[0164] "Risk assessment" is a procedure for identifying potential problems or obstacles that may arise in the future, and for analyzing and evaluating their impact.

[0165] "Work instructions" refer to specific guidance provided to on-site workers to ensure they perform their tasks effectively.

[0166] "Feedback" refers to reactions and advice given to workers based on the results of their work and their emotional state.

[0167] This invention is a system aimed at improving material management and work efficiency at construction sites, and consists of three elements: a server, a terminal, and a user. The roles of each element and specific embodiments are described in detail below.

[0168] The server collects data from multiple sensors and cameras placed at the construction site. These devices are used to monitor the work environment and the status of workers in real time. The server preprocesses the collected data, removing noise and standardizing it. After the data is preprocessed, the server uses an emotion analysis engine to evaluate the emotional state of the workers. This includes facial recognition and voice analysis of the workers, applying AI technology to understand their emotional state.

[0169] The results of sentiment analysis and other data are sent to a generative AI model. The AI ​​model used here is trained with prompts for forecasting material demand and assessing risks. For example, a specific prompt such as, "What materials are needed when worker A's stress level is high?" is used. The resulting forecast information is then used to create efficient material usage and timely ordering plans.

[0170] The terminal is a device directly operated by the worker, displaying instructions and feedback from the server in real time. Support messages and work instructions tailored to the worker's emotional state are displayed on the terminal, allowing the worker to take appropriate action based on this information. The terminal can also send feedback back to the server, which can be done via voice input or touch operation.

[0171] Users include site supervisors and workers who utilize the system to improve work efficiency and safety. Site supervisors use the provided data and risk assessments to optimize overall site management. Workers can perform their tasks more safely and stress-free by receiving instructions and feedback based on their emotional state.

[0172] This system improves the quality of material management and work instruction by incorporating sentiment analysis into construction sites, ultimately increasing on-site productivity and safety.

[0173] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0174] Step 1:

[0175] The server collects data from sensors and cameras placed at the construction site. Inputs include work environment data acquired from various sensors, as well as data on workers' movements, voices, and facial expressions. Its specific operation here is real-time data collection from each device. The output is the collected raw data.

[0176] Step 2:

[0177] The server preprocesses the collected data. At this stage, it uses the raw data obtained in the previous step as input and performs data processing such as noise reduction and data standardization. Specifically, this involves clearing audio data and adjusting the resolution of image data. The output is analyzable, formatted data.

[0178] Step 3:

[0179] The server inputs the formatted data into the emotion analysis engine to analyze the workers' emotional state. The input in this step is pre-processed data. The emotion analysis engine performs specific actions to determine whether the workers are experiencing stress by analyzing their facial expressions and tone of voice. The output provides insights into their emotional state (e.g., stress level and motivation).

[0180] Step 4:

[0181] The server uses a generated AI model based on the results of sentiment analysis and other data to forecast material demand and assess risks. The input for this step is information about the workers' emotional states and field data. The server creates prompts and performs specific actions to input them into the AI ​​model. The output includes an efficient material ordering plan and a risk assessment report.

[0182] Step 5:

[0183] The server generates work instructions and feedback based on the generated information. The input is the output data of the AI ​​model. Specifically, it creates work instructions and feedback messages tailored to the emotional state of a particular worker. The output consists of work instructions and feedback.

[0184] Step 6:

[0185] The terminal notifies workers of generated work instructions and feedback. The input here is information received from the server. Specifically, it displays and broadcasts messages via the terminal's display or voice call. The output is the worker's feedback response.

[0186] Step 7:

[0187] The user, a worker, sends feedback back to the server via a terminal. Inputs include the worker's voice instructions and emotional feedback, and specific actions such as providing feedback using voice recognition or text input. The server processes this feedback and uses it to improve instructions and predictions in the next cycle. The output is the system's response, improved by the feedback.

[0188] (Application Example 2)

[0189] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0190] Problems at construction sites include decreased work efficiency, insufficient safety, and mental fatigue among workers. In particular, issuing uniform work instructions without considering the emotional state of workers can lead to unstable work performance. This not only affects the overall productivity of the site but can also threaten worker safety. Furthermore, if proper material management is not carried out, there is a risk of waste or shortage of materials.

[0191] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0192] In this invention, the server includes means for collecting and pre-processing information from a monitoring device, means for analyzing the collected information to predict the amount of materials used and the timing of necessary replenishment, means for recognizing the emotional state of field workers, generating work instructions based on the analysis results, and notifying them through a display device, and means for providing feedback based on the emotional state to improve work efficiency and safety. This enables the provision of individualized and appropriate work instructions that reflect the emotional state of workers, as well as efficient management of materials.

[0193] A "monitoring device" is a device used to collect necessary data at a construction site, and may include sensors and cameras.

[0194] "Preprocessing" refers to the initial stages of processing performed to effectively utilize collected data, and includes tasks such as formatting and filtering the data.

[0195] "Analysis" is the process of analyzing collected data, extracting information, and finding meaning in it.

[0196] "Material usage and required replenishment timing" is an evaluation index used to assess the current inventory and future demand for materials at a construction site.

[0197] A "procurement plan" is a plan for ordering materials based on the required timing and quantity, with the aim of efficient material procurement.

[0198] "The emotional state of field workers" refers to the type and intensity of emotions that workers are currently experiencing, and may include stress levels and motivation levels.

[0199] "Emotion-based feedback" refers to advice and information provided to improve work efficiency and safety by taking into account the emotions of the workers.

[0200] "Risk assessment" is the process of identifying potential risks that may arise in the future and evaluating their impact and likelihood of occurrence.

[0201] A "plan of countermeasures" outlines specific actions and strategies to be taken in response to the assessed risks.

[0202] The system implementing this invention involves the cooperative operation of three elements: a server, a terminal, and a user, in order to improve work efficiency and safety at construction sites.

[0203] First, the server collects information from various sensors and cameras installed at the construction site as a monitoring device and performs preprocessing. Preprocessing includes shaping and filtering the data acquired from the sensors, which allows for efficient data analysis. Based on this information, the server uses a generative AI model to predict the amount of materials used and when they will need to be replenished. Based on this prediction, the server automatically generates a material ordering plan and notifies the administrator.

[0204] Meanwhile, the emotion engine analyzes the worker's facial expressions and tone of voice in real time through smart devices (such as smartphones and smart glasses) to recognize their emotional state. The server incorporates the results of this emotion analysis and uses them to create individual work instructions and feedback. This feedback may include positive messages that motivate the worker, contributing to improved work efficiency and safety.

[0205] Furthermore, the server assesses potential risks that may arise in the future and notifies the on-site manager of proposed countermeasures. This risk assessment also takes into account weather conditions and work progress. For example, in the event of a sudden deterioration in weather, it will propose adjusting the work schedule in advance.

[0206] Specifically, when the server detects worker fatigue, it notifies the worker via their terminal with a message such as, "We recommend a 5-minute break," and also displays a motivational message like, "You're doing a great job!" This process balances worker safety and efficiency.

[0207] An example of a prompt message for a generative AI model is as follows: "Generate work instruction messages based on the work environment data and worker sentiment data for this construction site." This allows the entire system to work together to provide a more efficient and safer work environment.

[0208] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0209] Step 1:

[0210] The server collects data in real time from sensors and cameras installed at the construction site. This data includes temperature, humidity, images of workers' facial expressions, and audio data. The collected data is first pre-processed to remove noise and format the data. This pre-processing enables efficient data analysis in the next analysis step.

[0211] Step 2:

[0212] The server analyzes pre-processed data and uses a generating AI model to predict material usage and required replenishment timing. Current material inventory data and construction progress information are used as input. The analysis results in the output of specific replenishment timings and quantities, and a material ordering plan is automatically generated. This reduces the risk of shortages or surpluses.

[0213] Step 3:

[0214] The server uses an emotion engine to analyze the workers' emotional state based on the collected data. This step uses the workers' facial images and audio data as input. The emotion analysis algorithm then assesses the workers' current stress or motivation levels. The output is a classification of their emotional state.

[0215] Step 4:

[0216] The server generates individual work instructions and feedback based on the results of sentiment analysis. It uses information about the worker's emotional state as input and outputs appropriate work instructions and motivational messages. These messages are communicated to the worker via the terminal's display device. For example, a message such as "We recommend you take a short break" might be generated.

[0217] Step 5:

[0218] The terminal displays work instructions and feedback messages received from the server to the worker in real time. The input is messages from the server, and the output is the information displayed on the terminal's screen. This allows the worker to take necessary actions immediately.

[0219] Step 6:

[0220] The server assesses future risks based on on-site work environment data and emotional state data, and notifies the site manager. Weather data and work progress are used as input for the risk assessment. A risk level assessment and proposed countermeasures are generated as output and notified to the site manager. This enables a rapid response.

[0221] The generated AI model and prompt messages are crucial elements for appropriately creating work instruction messages and risk assessments at each of these steps.

[0222] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0223] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0224] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0225] [Second Embodiment]

[0226] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0227] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0228] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0229] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0230] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0231] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0232] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0233] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0234] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0235] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0236] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0237] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0238] This invention is a system for streamlining material management at construction sites. It optimizes material ordering plans by collecting and analyzing data in real time using sensors and cameras. This system mainly consists of server, terminal, and user components.

[0239] Server Role

[0240] The server plays a central role, collecting data from multiple sensors and cameras installed on-site. The server preprocesses the data and then analyzes it using a generative AI model. Based on the analysis, it predicts material usage and replenishment timing, and automatically creates an ordering plan. This process also considers past consumption data, on-site work schedules, and even weather data. The server also assesses future risks and notifies the site supervisor of countermeasures as needed.

[0241] Terminal role

[0242] The terminal is for field workers to receive information. It receives work instructions generated from the server and provides specific instructions to the workers via a display device. This includes optimal placement of materials and work procedures. The terminal also assists the system's learning process by receiving feedback from workers and sending it to the server.

[0243] User roles

[0244] Users, namely site supervisors and workers, utilize information from the system to manage materials and plan work. Site supervisors make quick decisions based on risk notifications and ordering plans from the server, while workers efficiently carry out their tasks by following instructions displayed on their terminals. Feedback from users is stored on the server and used to improve the system's accuracy.

[0245] Specific example

[0246] For example, at a construction site, a server acquires real-time data on the remaining amount of materials through cameras and sensors. The server analyzes this data, comparing the current consumption rate with the future work plan, and predicts that a particular material will be in short supply in a week. Based on this, the server automatically calculates the optimal order quantity and places an order with the material supplier so that it can be replenished by the next day. Furthermore, if bad weather is predicted, the server notifies the site supervisor with suggested changes to the work schedule, thus preventing risks before they occur. Workers receive instructions on terminals and can quickly place new materials, improving the overall work efficiency of the site.

[0247] As a result, the present invention can dramatically improve the accuracy and efficiency of material management at construction sites.

[0248] The following describes the processing flow.

[0249] Step 1:

[0250] The server collects real-time data on material inventory, work progress, and weather from cameras and sensors installed at the construction site. This includes periodically receiving data from each sensor and storing it in a central database.

[0251] Step 2:

[0252] The server preprocesses the collected data. Specifically, it removes noise from the sensor data and fills in missing data. Furthermore, it converts the data into a consistent format and prepares it for analysis.

[0253] Step 3:

[0254] The server uses pre-processed data to perform analysis using an AI model. This analysis utilizes time series analysis and regression analysis based on historical and current data to predict future material usage and replenishment timing.

[0255] Step 4:

[0256] The server automatically generates a materials ordering plan based on the analysis results. This plan includes specific information such as the type and quantity of materials needed and the optimal ordering timing. The generated ordering plan is automatically notified to the relevant suppliers.

[0257] Step 5:

[0258] The server generates instructions for workers to improve the efficiency of material placement and work procedures. These instructions are sent to terminals, allowing workers to review placement procedures and work changes. This process includes displaying the instructions on a screen.

[0259] Step 6:

[0260] The server predicts future risks based on weather and work progress, and notifies the site supervisor with suggestions tailored to those risks. This risk assessment includes suggestions for changes to the work schedule due to sudden weather changes and warnings about the risk of delays in material procurement.

[0261] Step 7:

[0262] Users, such as workers and site supervisors, efficiently carry out their tasks and manage materials based on instructions and notifications received from terminals and servers. By providing feedback and inputting work status in digital format, they contribute to further system optimization by the server.

[0263] (Example 1)

[0264] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0265] Improving the efficiency and accuracy of material management at construction sites is a critical challenge. Traditional material management methods made it difficult to grasp site conditions in real time and create accurate ordering plans. Furthermore, they failed to adequately consider the impact of weather fluctuations and work schedule adjustments on material consumption. This resulted in unnecessary inventory and over-ordering, reducing site efficiency.

[0266] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0267] In this invention, the server includes means for collecting data from monitoring devices, means for preprocessing the data and converting it into a standard format, and means for inputting the collected data into a generating AI model for analysis. This enables accurate real-time data analysis and automatic generation of material ordering plans. Furthermore, the accuracy of the system can be improved by incorporating user feedback, enabling flexible material management in accordance with weather conditions and work schedules.

[0268] A "monitoring device" is a piece of equipment used at a construction site to acquire real-time information on the status of materials, and it consists of sensors, cameras, and other components.

[0269] "Preprocessing" refers to a series of operations that remove noise from acquired data and appropriately impute missing values, thereby transforming it into a state suitable for analysis.

[0270] A "generative AI model" is an analytical model that utilizes artificial intelligence technology to predict material consumption patterns and shortages using collected data.

[0271] A "procurement plan" is a plan created to secure the necessary materials based on the predicted consumption and supply timing of those materials.

[0272] "Work instructions" are instructions provided to on-site workers to encourage specific actions, such as how to arrange materials and the procedures for performing tasks.

[0273] "Feedback" refers to performance information and opinions provided by field workers and supervisors, which are used to improve the system and enhance its accuracy.

[0274] "Risk assessment" is a process for analyzing the impact of future weather conditions and changes in work schedules on material management and predicting potential problems.

[0275] This invention provides a system using sensors and cameras with the aim of improving the efficiency and accuracy of material management at construction sites. This system consists of a server, a terminal, and a user.

[0276] The server collects data in real time from sensors and cameras installed at the construction site. Specific hardware used includes weight sensors, RFID tag readers, and cameras. The data acquired from these devices is first pre-processed on the server. Pre-processing includes data filtering and imputation of missing values, and data processing software such as Python or R may be used.

[0277] The pre-processed data is input into a generative AI model. This generative AI model predicts future material consumption based on past consumption data, work schedules, and weather data. Examples of software used include machine learning libraries such as TensorFlow and PyTorch. As a concrete example of this analysis process, the prompt "Predict future material consumption" is input into the model, and the expected material consumption and replenishment timing are calculated.

[0278] Based on the analysis results, the server automatically generates a procurement plan for materials. This plan includes the quantity of required materials and the optimal delivery schedule. Furthermore, a risk assessment considering the on-site weather conditions and work plan is conducted and, if necessary, notified to the on-site supervisor, who is the user.

[0279] The terminal is responsible for providing work instructions from the server to on-site workers. The workers receive instructions from the terminal and efficiently arrange materials and progress with the work. The terminal also sends feedback from the workers to the server, contributing to improving the accuracy of the system.

[0280] The user utilizes the information from the system to execute materials management and work plans. As a specific usage method, the on-site supervisor can receive risk notifications from the server and promptly take countermeasures. Also, the workers can complete material placement promptly based on the instructions from the terminal, thereby improving the work efficiency of the entire site.

[0281] By implementing this system, high accuracy and efficiency can be achieved in on-site materials management.

[0282] The flow of the specific process in Example 1 will be described using FIG. 11.

[0283] Step 1:

[0284] The server collects data in real-time from sensors and cameras installed at the construction site. The input includes the quantity and location information of materials. The server receives these data, analyzes the signals sent from each device, and obtains specific information about the materials. As output, a list of the type and quantity of each material is generated.

[0285] Step 2:

[0286] The server preprocesses the collected data. At this stage, noise and outliers are removed, and missing data is supplemented with past data or estimated values. Specifically, data filtering is performed using Python. The input is the material information obtained in the previous step, and the output is a formatted dataset of material information.

[0287] Step 3:

[0288] The server inputs the preprocessed data into the generative AI model for analysis. The inputs include the formatted material information data, weather data, and work schedules. The server sends a prompt sentence "Please predict future material consumption" to the generative AI model to predict material consumption. The output is a list of predicted material usage amounts and their replenishment times.

[0289] Step 4:

[0290] Based on the analysis results, the server automatically generates a procurement plan for materials. The inputs are the analysis results and past procurement histories. The server reflects the predictions of the machine learning model and determines the required material quantities and delivery schedules. The output is a purchase order for transmission to the material suppliers.

[0291] Step 5:

[0292] The server generates work instructions for on-site workers and transmits them to the terminals. The inputs include the procurement plan and the on-site situation. The server determines the specific locations for material placement and work procedures and creates an instruction sheet. As output, work instructions to be displayed on the terminals are generated.

[0293] Step 6:

[0294] The terminal displays the work instructions received from the server to the workers. The workers receive the instructions and carry out the material placement work. The input is the work instruction sheet, and the output is a report on the completed work results. The terminal transmits this report to the server and accumulates it as feedback.

[0295] Step 7:

[0296] Users manage materials using information obtained from the system. Site supervisors make accurate decisions based on risk notifications from the server, ensuring safe and efficient work. User feedback is incorporated into the system and used to improve the accuracy of future analyses.

[0297] (Application Example 1)

[0298] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0299] Efficient resource management is crucial at construction sites. However, conventional methods make it difficult to predict real-time resource usage and replenishment timing, and resource shortages can impact the progress of construction. Furthermore, adjusting work plans to accommodate fluctuations in environmental conditions is challenging. These challenges need to be addressed to achieve more flexible and efficient resource management.

[0300] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0301] In this invention, the server includes means for collecting information from data measurement devices and performing initial processing, means for analyzing the collected information to predict resource usage and the required replenishment timing, and means for providing the status of resource management to the operator in real time via communication equipment. This enables efficient resource management and flexible adjustment of work plans in response to changes in environmental conditions.

[0302] A "data measurement device" is a device used to measure the quantity, location, and condition of resources at a site and acquire them as digital data.

[0303] A "server" is a central processing unit that receives data collected from measuring devices, performs analysis and processing, and provides the results to various terminals.

[0304] "Initial processing" refers to the operation of preparing the data acquired from the data measurement device into a state suitable for analysis by performing noise removal and format conversion on the data.

[0305] "Real-time" refers to the concept that certain information is processed or transmitted immediately with almost no time delay.

[0306] "Resource procurement plan" is a plan for procuring the necessary resources in appropriate amounts at appropriate times, based on the prediction of the usage amount and replenishment time of resources.

[0307] "Risk assessment" is a process of predicting possible future risks and problems and considering countermeasures to mitigate their impacts.

[0308] "Environmental conditions" refer to external natural factors such as weather and temperature at the construction site that affect work and resource management.

[0309] The system for implementing the present invention is designed to improve the efficiency of resource management at the construction site. This system is composed of a data measurement device, a server, communication equipment, a work terminal, etc.

[0310] The server collects the data related to resources transmitted from the data measurement device and performs initial processing there. In this initial processing, noise removal and conversion to the required data format are performed. Then, the server uses the generated AI model to predict the usage amount of resources and the required replenishment time. As a result, a resource procurement plan is automatically generated, enabling efficient resource management. In addition, the server evaluates future risks and notifies appropriate countermeasure plans to the on-site supervisor.

[0311] Communication equipment plays a role in providing workers with real-time information on resource management status. This enables rapid responses tailored to the situation on site. For example, if a shortage of a certain resource is predicted, the server automatically generates a procurement plan and notifies the worker via their terminal, allowing for rapid resource replenishment.

[0312] This system is effective, for example, when "adjusting resource usage and work plans in response to fluctuations in environmental conditions." This enables efficient work progress while flexibly responding to environmental changes.

[0313] The following are examples of prompt statements that can be used.

[0314] "Based on real-time material management, please propose the next necessary actions. This should take into account material usage forecasts and proactive risk management."

[0315] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0316] Step 1:

[0317] The server receives digital data from data measurement devices, including the quantity and location information of resources. The server performs initial processing on this data and removes noise. The input is raw data, and the output is data in a clean format.

[0318] Step 2:

[0319] The server inputs the initially processed data into a generating AI model to predict resource usage and replenishment timing. Data processing considers past consumption data and environmental conditions, and the predictive model is used. The output is the predicted resource consumption and optimal replenishment timing.

[0320] Step 3:

[0321] The server automatically generates a resource procurement plan based on the forecast results. The generated plan enables the procurement of resources in the optimal quantity and timing. The input is forecast data, and the output is the finalized procurement plan.

[0322] Step 4:

[0323] The server transmits the procurement plan to field work terminals, displaying it to field personnel in real time. The input is the generated plan information, and the output is a notification to the personnel. The terminal displays this and instructs them on the necessary actions.

[0324] Step 5:

[0325] The server also assesses future risks and notifies field supervisors of relevant information. Inputs here are normal operational and environmental data, while outputs are risk notifications and proposed countermeasures.

[0326] Step 6:

[0327] The field staff, acting as users, perform tasks based on information received through their terminals and submit completion reports as feedback to the server. This feedback helps improve the accuracy of the next predictive model. The input is the feedback information, and the output is its storage in the training database.

[0328] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0329] This invention combines a materials management system for construction sites with an emotion engine that recognizes user emotions. The aim is to improve the efficiency and safety of on-site work and reduce the burden on workers and managers. This system consists of a server, terminals, and user components.

[0330] Server Role

[0331] The server collects data from various sensors and cameras at the construction site and uses an emotion engine to analyze workers' emotions from their facial expressions and tone of voice. This allows the server to understand in real time whether workers are stressed or highly motivated. The server preprocesses the collected data, uses an AI model to forecast material demand and assess risks, and generates ordering plans. It also generates appropriate work instructions and feedback based on the emotion analysis results and provides them to the user.

[0332] Terminal role

[0333] The terminal is a device that allows workers to receive information and instructions from the server. Through a display device, it provides real-time work instructions and feedback, as well as displaying motivational messages generated by an emotion engine. This allows workers to receive appropriate support tailored to their emotional state. The terminal also functions as a means of transmitting worker feedback and emotional information to the server via voice input and camera.

[0334] User roles

[0335] This system enables site supervisors and workers, as users, to efficiently manage materials and plan work. Site supervisors receive risk notifications, work instructions, and management support based on their emotional state from the server, improving overall operational efficiency on site. Workers receive feedback tailored to their individual emotional state displayed on their terminals, reducing stress and allowing them to work more safely and comfortably.

[0336] Specific example

[0337] For example, if a worker is performing heavy labor for several consecutive days on a hot day, and the emotion engine detects that their stress levels are rising based on their facial expression and tone of voice, the server analyzes this information and displays a message on the terminal prompting them to take a break. It also provides positive feedback in real time to improve motivation. This helps ensure worker safety and maintain overall performance at the worksite.

[0338] Based on the above, the present invention provides a system that contributes to material management that takes emotions into consideration and to improving work efficiency.

[0339] The following describes the processing flow.

[0340] Step 1:

[0341] The server collects real-time data from sensors such as cameras and microphones installed at the construction site, including worker movements, audio data, and material inventory status. This data is initially processed to remove noise and prepare it for easy analysis.

[0342] Step 2:

[0343] The server activates an emotion engine to analyze the worker's facial expressions and tone of voice from the collected data. Based on this analysis, it determines the worker's current emotional state, such as "stress" or "fatigue," in real time.

[0344] Step 3:

[0345] As a notification to the site supervisor (the user), the server compiles the emotional state of each worker, as determined by the emotion engine, and displays it on a management dashboard. This allows the site supervisor to quickly understand which workers require special attention.

[0346] Step 4:

[0347] If the emotional state exceeds a certain threshold, the server sends a message generated by the emotion engine to the terminal. This message may include specific suggestions for taking a break or advice on mental health care.

[0348] Step 5:

[0349] The terminal displays emotion-based messages received from the server, and the worker reviews the content. For example, they might receive a notification such as, "Your stress level is high, please take a 10-minute break," and be advised to temporarily stop working and take a break.

[0350] Step 6:

[0351] The server continuously records emotional and work efficiency data for long-term analysis. This allows for feedback to be obtained for improving the work environment and optimizing material management. The feedback is provided to the administrator at the end of the project.

[0352] Step 7:

[0353] Users, i.e., workers, can adjust their daily tasks based on feedback from their devices and review their workload and efficiency. Furthermore, they can proactively seek areas for improvement to enhance their performance in the field, based on the continuously provided feedback.

[0354] (Example 2)

[0355] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0356] Conventional construction site management systems struggle with effective material management and timely ordering planning, and lack work instructions that adequately consider the feelings and safety of workers. Therefore, it is crucial to reduce worker stress and improve motivation, while simultaneously demanding increased efficiency in material management and improved accuracy in risk prediction.

[0357] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0358] In this invention, the server includes means for collecting and pre-processing data from a monitoring device, means for analyzing the collected data to predict the amount of materials used and the timing of necessary replenishment, and means for analyzing the emotional state of workers. This enables appropriate work instructions and feedback based on the emotional state of workers.

[0359] A "monitoring device" is a piece of equipment installed to understand the working environment and to acquire data.

[0360] "Preprocessing" refers to the initial stages of processing to prepare collected data into a format suitable for analysis.

[0361] "Data analysis" is the process of analyzing collected information to identify specific patterns or trends.

[0362] "Material usage" is an indicator that refers to the amount of materials consumed at a construction site.

[0363] "Replenishment timing" refers to the period when additional materials need to be procured, and is a point in time identified for efficient management.

[0364] An "automatic ordering plan" is an ordering plan automatically generated by a computer system based on the amount of materials used and the timing of replenishment.

[0365] "Emotional analysis" is a process that determines the emotional state of a worker based on information such as their facial expressions and tone of voice.

[0366] "Risk assessment" is a procedure for identifying potential problems or obstacles that may arise in the future, and for analyzing and evaluating their impact.

[0367] "Work instructions" refer to specific guidance provided to on-site workers to ensure they perform their tasks effectively.

[0368] "Feedback" refers to reactions and advice given to workers based on the results of their work and their emotional state.

[0369] This invention is a system aimed at improving material management and work efficiency at construction sites, and consists of three elements: a server, a terminal, and a user. The roles of each element and specific embodiments are described in detail below.

[0370] The server collects data from multiple sensors and cameras placed at the construction site. These devices are used to monitor the work environment and the status of workers in real time. The server preprocesses the collected data, removing noise and standardizing it. After the data is preprocessed, the server uses an emotion analysis engine to evaluate the emotional state of the workers. This includes facial recognition and voice analysis of the workers, applying AI technology to understand their emotional state.

[0371] The results of sentiment analysis and other data are sent to a generative AI model. The AI ​​model used here is trained with prompts for forecasting material demand and assessing risks. For example, a specific prompt such as, "What materials are needed when worker A's stress level is high?" is used. The resulting forecast information is then used to create efficient material usage and timely ordering plans.

[0372] The terminal is a device directly operated by the worker, displaying instructions and feedback from the server in real time. Support messages and work instructions tailored to the worker's emotional state are displayed on the terminal, allowing the worker to take appropriate action based on this information. The terminal can also send feedback back to the server, which can be done via voice input or touch operation.

[0373] Users include site supervisors and workers who utilize the system to improve work efficiency and safety. Site supervisors use the provided data and risk assessments to optimize overall site management. Workers can perform their tasks more safely and stress-free by receiving instructions and feedback based on their emotional state.

[0374] This system improves the quality of material management and work instruction by incorporating sentiment analysis into construction sites, ultimately increasing on-site productivity and safety.

[0375] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0376] Step 1:

[0377] The server collects data from sensors and cameras placed at the construction site. Inputs include work environment data acquired from various sensors, as well as data on workers' movements, voices, and facial expressions. Its specific operation here is real-time data collection from each device. The output is the collected raw data.

[0378] Step 2:

[0379] The server preprocesses the collected data. At this stage, it uses the raw data obtained in the previous step as input and performs data processing such as noise reduction and data standardization. Specifically, this involves clearing audio data and adjusting the resolution of image data. The output is analyzable, formatted data.

[0380] Step 3:

[0381] The server inputs the formatted data into the emotion analysis engine to analyze the workers' emotional state. The input in this step is pre-processed data. The emotion analysis engine performs specific actions to determine whether the workers are experiencing stress by analyzing their facial expressions and tone of voice. The output provides insights into their emotional state (e.g., stress level and motivation).

[0382] Step 4:

[0383] The server uses a generated AI model based on the results of sentiment analysis and other data to forecast material demand and assess risks. The input for this step is information about the workers' emotional states and field data. The server creates prompts and performs specific actions to input them into the AI ​​model. The output includes an efficient material ordering plan and a risk assessment report.

[0384] Step 5:

[0385] The server generates work instructions and feedback based on the generated information. The input is the output data of the AI ​​model. Specifically, it creates work instructions and feedback messages tailored to the emotional state of a particular worker. The output consists of work instructions and feedback.

[0386] Step 6:

[0387] The terminal notifies workers of generated work instructions and feedback. The input here is information received from the server. Specifically, it displays and broadcasts messages via the terminal's display or voice call. The output is the worker's feedback response.

[0388] Step 7:

[0389] The user, a worker, sends feedback back to the server via a terminal. Inputs include the worker's voice instructions and emotional feedback, and specific actions such as providing feedback using voice recognition or text input. The server processes this feedback and uses it to improve instructions and predictions in the next cycle. The output is the system's response, improved by the feedback.

[0390] (Application Example 2)

[0391] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0392] Problems at construction sites include decreased work efficiency, insufficient safety, and mental fatigue among workers. In particular, issuing uniform work instructions without considering the emotional state of workers can lead to unstable work performance. This not only affects the overall productivity of the site but can also threaten worker safety. Furthermore, if proper material management is not carried out, there is a risk of waste or shortage of materials.

[0393] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0394] In this invention, the server includes means for collecting and pre-processing information from a monitoring device, means for analyzing the collected information to predict the amount of materials used and the timing of necessary replenishment, means for recognizing the emotional state of field workers, generating work instructions based on the analysis results, and notifying them through a display device, and means for providing feedback based on the emotional state to improve work efficiency and safety. This enables the provision of individualized and appropriate work instructions that reflect the emotional state of workers, as well as efficient management of materials.

[0395] A "monitoring device" is a device used to collect necessary data at a construction site, and may include sensors and cameras.

[0396] "Preprocessing" refers to the initial stages of processing performed to effectively utilize collected data, and includes tasks such as formatting and filtering the data.

[0397] "Analysis" is the process of analyzing collected data, extracting information, and finding meaning in it.

[0398] "Material usage and required replenishment timing" is an evaluation index used to assess the current inventory and future demand for materials at a construction site.

[0399] A "procurement plan" is a plan for ordering materials based on the required timing and quantity, with the aim of efficient material procurement.

[0400] "The emotional state of field workers" refers to the type and intensity of emotions that workers are currently experiencing, and may include stress levels and motivation levels.

[0401] "Emotion-based feedback" refers to advice and information provided to improve work efficiency and safety by taking into account the emotions of the workers.

[0402] "Risk assessment" is the process of identifying potential risks that may arise in the future and evaluating their impact and likelihood of occurrence.

[0403] A "plan of countermeasures" outlines specific actions and strategies to be taken in response to the assessed risks.

[0404] The system implementing this invention involves the cooperative operation of three elements: a server, a terminal, and a user, in order to improve work efficiency and safety at construction sites.

[0405] First, the server collects information from various sensors and cameras installed at the construction site as a monitoring device and performs preprocessing. Preprocessing includes shaping and filtering the data acquired from the sensors, which allows for efficient data analysis. Based on this information, the server uses a generative AI model to predict the amount of materials used and when they will need to be replenished. Based on this prediction, the server automatically generates a material ordering plan and notifies the administrator.

[0406] Meanwhile, the emotion engine analyzes the worker's facial expressions and tone of voice in real time through smart devices (such as smartphones and smart glasses) to recognize their emotional state. The server incorporates the results of this emotion analysis and uses them to create individual work instructions and feedback. This feedback may include positive messages that motivate the worker, contributing to improved work efficiency and safety.

[0407] Furthermore, the server assesses potential risks that may arise in the future and notifies the on-site manager of proposed countermeasures. This risk assessment also takes into account weather conditions and work progress. For example, in the event of a sudden deterioration in weather, it will propose adjusting the work schedule in advance.

[0408] Specifically, when the server detects worker fatigue, it notifies the worker via their terminal with a message such as, "We recommend a 5-minute break," and also displays a motivational message like, "You're doing a great job!" This process balances worker safety and efficiency.

[0409] An example of a prompt message for a generative AI model is as follows: "Generate work instruction messages based on the work environment data and worker sentiment data for this construction site." This allows the entire system to work together to provide a more efficient and safer work environment.

[0410] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0411] Step 1:

[0412] The server collects data in real time from sensors and cameras installed at the construction site. This data includes temperature, humidity, images of workers' facial expressions, and audio data. The collected data is first pre-processed to remove noise and format the data. This pre-processing enables efficient data analysis in the next analysis step.

[0413] Step 2:

[0414] The server analyzes pre-processed data and uses a generating AI model to predict material usage and required replenishment timing. Current material inventory data and construction progress information are used as input. The analysis results in the output of specific replenishment timings and quantities, and a material ordering plan is automatically generated. This reduces the risk of shortages or surpluses.

[0415] Step 3:

[0416] The server uses an emotion engine to analyze the workers' emotional state based on the collected data. This step uses the workers' facial images and audio data as input. The emotion analysis algorithm then assesses the workers' current stress or motivation levels. The output is a classification of their emotional state.

[0417] Step 4:

[0418] The server generates individual work instructions and feedback based on the results of sentiment analysis. It uses information about the worker's emotional state as input and outputs appropriate work instructions and motivational messages. These messages are communicated to the worker via the terminal's display device. For example, a message such as "We recommend you take a short break" might be generated.

[0419] Step 5:

[0420] The terminal displays work instructions and feedback messages received from the server to the worker in real time. The input is messages from the server, and the output is the information displayed on the terminal's screen. This allows the worker to take necessary actions immediately.

[0421] Step 6:

[0422] The server assesses future risks based on on-site work environment data and emotional state data, and notifies the site manager. Weather data and work progress are used as input for the risk assessment. A risk level assessment and proposed countermeasures are generated as output and notified to the site manager. This enables a rapid response.

[0423] The generated AI model and prompt messages are crucial elements for appropriately creating work instruction messages and risk assessments at each of these steps.

[0424] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0425] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0426] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0427] [Third Embodiment]

[0428] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0429] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0430] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0431] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0432] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0433] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0434] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0435] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0436] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0437] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0438] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0439] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0440] This invention is a system for streamlining material management at construction sites. It optimizes material ordering plans by collecting and analyzing data in real time using sensors and cameras. This system mainly consists of server, terminal, and user components.

[0441] Server Role

[0442] The server plays a central role, collecting data from multiple sensors and cameras installed on-site. The server preprocesses the data and then analyzes it using a generative AI model. Based on the analysis, it predicts material usage and replenishment timing, and automatically creates an ordering plan. This process also considers past consumption data, on-site work schedules, and even weather data. The server also assesses future risks and notifies the site supervisor of countermeasures as needed.

[0443] Terminal role

[0444] The terminal is for field workers to receive information. It receives work instructions generated from the server and provides specific instructions to the workers via a display device. This includes optimal placement of materials and work procedures. The terminal also assists the system's learning process by receiving feedback from workers and sending it to the server.

[0445] User roles

[0446] Users, namely site supervisors and workers, utilize information from the system to manage materials and plan work. Site supervisors make quick decisions based on risk notifications and ordering plans from the server, while workers efficiently carry out their tasks by following instructions displayed on their terminals. Feedback from users is stored on the server and used to improve the system's accuracy.

[0447] Specific example

[0448] For example, at a construction site, a server acquires real-time data on the remaining amount of materials through cameras and sensors. The server analyzes this data, comparing the current consumption rate with the future work plan, and predicts that a particular material will be in short supply in a week. Based on this, the server automatically calculates the optimal order quantity and places an order with the material supplier so that it can be replenished by the next day. Furthermore, if bad weather is predicted, the server notifies the site supervisor with suggested changes to the work schedule, thus preventing risks before they occur. Workers receive instructions on terminals and can quickly place new materials, improving the overall work efficiency of the site.

[0449] As a result, the present invention can dramatically improve the accuracy and efficiency of material management at construction sites.

[0450] The following describes the processing flow.

[0451] Step 1:

[0452] The server collects real-time data on material inventory, work progress, and weather from cameras and sensors installed at the construction site. This includes periodically receiving data from each sensor and storing it in a central database.

[0453] Step 2:

[0454] The server preprocesses the collected data. Specifically, it removes noise from the sensor data and fills in missing data. Furthermore, it converts the data into a consistent format and prepares it for analysis.

[0455] Step 3:

[0456] The server uses pre-processed data to perform analysis using an AI model. This analysis utilizes time series analysis and regression analysis based on historical and current data to predict future material usage and replenishment timing.

[0457] Step 4:

[0458] The server automatically generates a materials ordering plan based on the analysis results. This plan includes specific information such as the type and quantity of materials needed and the optimal ordering timing. The generated ordering plan is automatically notified to the relevant suppliers.

[0459] Step 5:

[0460] The server generates instructions for workers to improve the efficiency of material placement and work procedures. These instructions are sent to terminals, allowing workers to review placement procedures and work changes. This process includes displaying the instructions on a screen.

[0461] Step 6:

[0462] The server predicts future risks based on weather and work progress, and notifies the site supervisor with suggestions tailored to those risks. This risk assessment includes suggestions for changes to the work schedule due to sudden weather changes and warnings about the risk of delays in material procurement.

[0463] Step 7:

[0464] Users, such as workers and site supervisors, efficiently carry out their tasks and manage materials based on instructions and notifications received from terminals and servers. By providing feedback and inputting work status in digital format, they contribute to further system optimization by the server.

[0465] (Example 1)

[0466] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0467] Improving the efficiency and accuracy of material management at construction sites is a critical challenge. Traditional material management methods made it difficult to grasp site conditions in real time and create accurate ordering plans. Furthermore, they failed to adequately consider the impact of weather fluctuations and work schedule adjustments on material consumption. This resulted in unnecessary inventory and over-ordering, reducing site efficiency.

[0468] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0469] In this invention, the server includes means for collecting data from monitoring devices, means for preprocessing the data and converting it into a standard format, and means for inputting the collected data into a generating AI model for analysis. This enables accurate real-time data analysis and automatic generation of material ordering plans. Furthermore, the accuracy of the system can be improved by incorporating user feedback, enabling flexible material management in accordance with weather conditions and work schedules.

[0470] A "monitoring device" is a piece of equipment used at a construction site to acquire real-time information on the status of materials, and it consists of sensors, cameras, and other components.

[0471] "Preprocessing" refers to a series of operations that remove noise from acquired data and appropriately impute missing values, thereby transforming it into a state suitable for analysis.

[0472] A "generative AI model" is an analytical model that utilizes artificial intelligence technology to predict material consumption patterns and shortages using collected data.

[0473] A "procurement plan" is a plan created to secure the necessary materials based on the predicted consumption and supply timing of those materials.

[0474] "Work instructions" are instructions provided to on-site workers to encourage specific actions, such as how to arrange materials and the procedures for performing tasks.

[0475] "Feedback" refers to performance information and opinions provided by field workers and supervisors, which are used to improve the system and enhance its accuracy.

[0476] "Risk assessment" is a process for analyzing the impact of future weather conditions and changes in work schedules on material management and predicting potential problems.

[0477] This invention provides a system using sensors and cameras with the aim of improving the efficiency and accuracy of material management at construction sites. This system consists of a server, a terminal, and a user.

[0478] The server collects data in real time from sensors and cameras installed at the construction site. Specific hardware used includes weight sensors, RFID tag readers, and cameras. The data acquired from these devices is first pre-processed on the server. Pre-processing includes data filtering and imputation of missing values, and data processing software such as Python or R may be used.

[0479] The pre-processed data is input into a generative AI model. This generative AI model predicts future material consumption based on past consumption data, work schedules, and weather data. Examples of software used include machine learning libraries such as TensorFlow and PyTorch. As a concrete example of this analysis process, the prompt "Predict future material consumption" is input into the model, and the expected material consumption and replenishment timing are calculated.

[0480] Based on the analysis results, the server automatically generates a materials ordering plan. This plan includes the required quantities of materials and the optimal delivery schedule. Furthermore, a risk assessment is performed, taking into account on-site weather conditions and work plans, and the site supervisor (the user) is notified as needed.

[0481] The terminals are responsible for providing work instructions from the server to field workers. Workers receive instructions from the terminals and efficiently arrange materials and manage the progress of their work. The terminals also send feedback from workers to the server, contributing to the improvement of the system's accuracy.

[0482] Users utilize information from the system to manage materials and plan work. Specifically, site supervisors can receive risk notifications from the server and take swift countermeasures. Furthermore, workers can improve overall site efficiency by quickly completing material placement based on instructions from their terminals.

[0483] Implementing this system will enable high accuracy and efficiency in on-site material management.

[0484] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0485] Step 1:

[0486] The server collects data in real time from sensors and cameras installed at the construction site. Inputs include material quantities and location information. The server receives this data, analyzes the signals sent from each device, and obtains specific information about the materials. The output generates a list of each material type and its quantity.

[0487] Step 2:

[0488] The server preprocesses the collected data. At this stage, noise and outliers are removed, and missing data is imputed with historical data or estimates. Specifically, data filtering is performed using Python. The input is the material information obtained in the previous step, and the output is a formatted material information dataset.

[0489] Step 3:

[0490] The server inputs pre-processed data into a generating AI model for analysis. This input includes formatted material information data, weather data, and work schedules. The server then prompts the generating AI model with the message, "Predict future material consumption," and performs the prediction. The output is a list of predicted material usage and replenishment timings.

[0491] Step 4:

[0492] The server automatically generates a material ordering plan based on the analysis results. The inputs are the analysis results and past ordering history. The server incorporates predictions from a machine learning model to determine the required material quantities and delivery schedule. The output is an order form to be sent to material suppliers.

[0493] Step 5:

[0494] The server generates work instructions for field workers and sends them to terminals. Input includes the ordering plan and site conditions. The server determines the specific placement of materials and work procedures, and creates instruction sheets. The output is the work instructions displayed on the terminals.

[0495] Step 6:

[0496] The terminal displays work instructions received from the server to the worker. The worker receives the instructions and performs the material placement work. The input is a work instruction sheet, and the output is a report of the completed work results. The terminal sends this report to the server, where it is stored as feedback.

[0497] Step 7:

[0498] Users manage materials using information obtained from the system. Site supervisors make accurate decisions based on risk notifications from the server, ensuring safe and efficient work. User feedback is incorporated into the system and used to improve the accuracy of future analyses.

[0499] (Application Example 1)

[0500] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0501] Efficient resource management is crucial at construction sites. However, conventional methods make it difficult to predict real-time resource usage and replenishment timing, and resource shortages can impact the progress of construction. Furthermore, adjusting work plans to accommodate fluctuations in environmental conditions is challenging. These challenges need to be addressed to achieve more flexible and efficient resource management.

[0502] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0503] In this invention, the server includes means for collecting information from data measurement devices and performing initial processing, means for analyzing the collected information to predict resource usage and the required replenishment timing, and means for providing the status of resource management to the operator in real time via communication equipment. This enables efficient resource management and flexible adjustment of work plans in response to changes in environmental conditions.

[0504] A "data measurement device" is a device used to measure the quantity, location, and condition of resources at a site and acquire them as digital data.

[0505] A "server" is a central processing unit that receives data collected from measuring devices, performs analysis and processing, and provides the results to various terminals.

[0506] "Initial processing" refers to the process of preparing data acquired from data measurement devices for analysis by performing noise reduction and format conversion.

[0507] "Real-time" is a concept that refers to a state in which information is processed or transmitted immediately, with virtually no time delay.

[0508] A "resource procurement plan" is a plan for procuring the necessary resources in the right quantity at the right time, and is based on predictions of resource usage and replenishment timing.

[0509] "Risk assessment" is the process of predicting potential risks and problems that may arise in the future and considering measures to mitigate their impact.

[0510] "Environmental conditions" refer to external natural factors that affect work and resource management at a construction site, such as weather and temperature.

[0511] The system implementing this invention is designed to streamline resource management at construction sites. This system consists of data measurement devices, servers, communication equipment, work terminals, and the like.

[0512] The server collects resource data transmitted from data measurement devices and performs initial processing. This initial processing includes noise removal and conversion to the required data format. Subsequently, the server uses a generative AI model to predict resource usage and the necessary replenishment timing. This automatically generates a resource procurement plan, enabling efficient resource management. The server also assesses future risks and notifies field managers of appropriate countermeasures.

[0513] Communication equipment plays a role in providing workers with real-time information on resource management status. This enables rapid responses tailored to the situation on site. For example, if a shortage of a certain resource is predicted, the server automatically generates a procurement plan and notifies the worker via their terminal, allowing for rapid resource replenishment.

[0514] This system is effective, for example, when "adjusting resource usage and work plans in response to fluctuations in environmental conditions." This enables efficient work progress while flexibly responding to environmental changes.

[0515] The following are examples of prompt statements that can be used.

[0516] "Based on real-time material management, please propose the next necessary actions. This should take into account material usage forecasts and proactive risk management."

[0517] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0518] Step 1:

[0519] The server receives digital data from data measurement devices, including the quantity and location information of resources. The server performs initial processing on this data and removes noise. The input is raw data, and the output is data in a clean format.

[0520] Step 2:

[0521] The server inputs the initially processed data into a generating AI model to predict resource usage and replenishment timing. Data processing considers past consumption data and environmental conditions, and the predictive model is used. The output is the predicted resource consumption and optimal replenishment timing.

[0522] Step 3:

[0523] The server automatically generates a resource procurement plan based on the forecast results. The generated plan enables the procurement of resources in the optimal quantity and timing. The input is forecast data, and the output is the finalized procurement plan.

[0524] Step 4:

[0525] The server transmits the procurement plan to field work terminals, displaying it to field personnel in real time. The input is the generated plan information, and the output is a notification to the personnel. The terminal displays this and instructs them on the necessary actions.

[0526] Step 5:

[0527] The server also assesses future risks and notifies field supervisors of relevant information. Inputs here are normal operational and environmental data, while outputs are risk notifications and proposed countermeasures.

[0528] Step 6:

[0529] The field staff, acting as users, perform tasks based on information received through their terminals and submit completion reports as feedback to the server. This feedback helps improve the accuracy of the next predictive model. The input is the feedback information, and the output is its storage in the training database.

[0530] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0531] This invention combines a materials management system for construction sites with an emotion engine that recognizes user emotions. The aim is to improve the efficiency and safety of on-site work and reduce the burden on workers and managers. This system consists of a server, terminals, and user components.

[0532] Server Role

[0533] The server collects data from various sensors and cameras at the construction site and uses an emotion engine to analyze workers' emotions from their facial expressions and tone of voice. This allows the server to understand in real time whether workers are stressed or highly motivated. The server preprocesses the collected data, uses an AI model to forecast material demand and assess risks, and generates ordering plans. It also generates appropriate work instructions and feedback based on the emotion analysis results and provides them to the user.

[0534] Terminal role

[0535] The terminal is a device that allows workers to receive information and instructions from the server. Through a display device, it provides real-time work instructions and feedback, as well as displaying motivational messages generated by an emotion engine. This allows workers to receive appropriate support tailored to their emotional state. The terminal also functions as a means of transmitting worker feedback and emotional information to the server via voice input and camera.

[0536] User roles

[0537] This system enables site supervisors and workers, as users, to efficiently manage materials and plan work. Site supervisors receive risk notifications, work instructions, and management support based on their emotional state from the server, improving overall operational efficiency on site. Workers receive feedback tailored to their individual emotional state displayed on their terminals, reducing stress and allowing them to work more safely and comfortably.

[0538] Specific example

[0539] For example, if a worker is performing heavy labor for several consecutive days on a hot day, and the emotion engine detects that their stress levels are rising based on their facial expression and tone of voice, the server analyzes this information and displays a message on the terminal prompting them to take a break. It also provides positive feedback in real time to improve motivation. This helps ensure worker safety and maintain overall performance at the worksite.

[0540] Based on the above, the present invention provides a system that contributes to material management that takes emotions into consideration and to improving work efficiency.

[0541] The following describes the processing flow.

[0542] Step 1:

[0543] The server collects real-time data from sensors such as cameras and microphones installed at the construction site, including worker movements, audio data, and material inventory status. This data is initially processed to remove noise and prepare it for easy analysis.

[0544] Step 2:

[0545] The server activates an emotion engine to analyze the worker's facial expressions and tone of voice from the collected data. Based on this analysis, it determines the worker's current emotional state, such as "stress" or "fatigue," in real time.

[0546] Step 3:

[0547] As a notification to the site supervisor (the user), the server compiles the emotional state of each worker, as determined by the emotion engine, and displays it on a management dashboard. This allows the site supervisor to quickly understand which workers require special attention.

[0548] Step 4:

[0549] If the emotional state exceeds a certain threshold, the server sends a message generated by the emotion engine to the terminal. This message may include specific suggestions for taking a break or advice on mental health care.

[0550] Step 5:

[0551] The terminal displays emotion-based messages received from the server, and the worker reviews the content. For example, they might receive a notification such as, "Your stress level is high, please take a 10-minute break," and be advised to temporarily stop working and take a break.

[0552] Step 6:

[0553] The server continuously records emotional and work efficiency data for long-term analysis. This allows for feedback to be obtained for improving the work environment and optimizing material management. The feedback is provided to the administrator at the end of the project.

[0554] Step 7:

[0555] Users, i.e., workers, can adjust their daily tasks based on feedback from their devices and review their workload and efficiency. Furthermore, they can proactively seek areas for improvement to enhance their performance in the field, based on the continuously provided feedback.

[0556] (Example 2)

[0557] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0558] Conventional construction site management systems struggle with effective material management and timely ordering planning, and lack work instructions that adequately consider the feelings and safety of workers. Therefore, it is crucial to reduce worker stress and improve motivation, while simultaneously demanding increased efficiency in material management and improved accuracy in risk prediction.

[0559] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0560] In this invention, the server includes means for collecting and pre-processing data from a monitoring device, means for analyzing the collected data to predict the amount of materials used and the timing of necessary replenishment, and means for analyzing the emotional state of workers. This enables appropriate work instructions and feedback based on the emotional state of workers.

[0561] A "monitoring device" is a piece of equipment installed to understand the working environment and to acquire data.

[0562] "Preprocessing" refers to the initial stages of processing to prepare collected data into a format suitable for analysis.

[0563] "Data analysis" is the process of analyzing collected information to identify specific patterns or trends.

[0564] "Material usage" is an indicator that refers to the amount of materials consumed at a construction site.

[0565] "Replenishment timing" refers to the period when additional materials need to be procured, and is a point in time identified for efficient management.

[0566] An "automatic ordering plan" is an ordering plan automatically generated by a computer system based on the amount of materials used and the timing of replenishment.

[0567] "Emotional analysis" is a process that determines the emotional state of a worker based on information such as their facial expressions and tone of voice.

[0568] "Risk assessment" is a procedure for identifying potential problems or obstacles that may arise in the future, and for analyzing and evaluating their impact.

[0569] "Work instructions" refer to specific guidance provided to on-site workers to ensure they perform their tasks effectively.

[0570] "Feedback" refers to reactions and advice given to workers based on the results of their work and their emotional state.

[0571] This invention is a system aimed at improving material management and work efficiency at construction sites, and consists of three elements: a server, a terminal, and a user. The roles of each element and specific embodiments are described in detail below.

[0572] The server collects data from multiple sensors and cameras placed at the construction site. These devices are used to monitor the work environment and the status of workers in real time. The server preprocesses the collected data, removing noise and standardizing it. After the data is preprocessed, the server uses an emotion analysis engine to evaluate the emotional state of the workers. This includes facial recognition and voice analysis of the workers, applying AI technology to understand their emotional state.

[0573] The results of sentiment analysis and other data are sent to a generative AI model. The AI ​​model used here is trained with prompts for forecasting material demand and assessing risks. For example, a specific prompt such as, "What materials are needed when worker A's stress level is high?" is used. The resulting forecast information is then used to create efficient material usage and timely ordering plans.

[0574] The terminal is a device directly operated by the worker, displaying instructions and feedback from the server in real time. Support messages and work instructions tailored to the worker's emotional state are displayed on the terminal, allowing the worker to take appropriate action based on this information. The terminal can also send feedback back to the server, which can be done via voice input or touch operation.

[0575] Users include site supervisors and workers who utilize the system to improve work efficiency and safety. Site supervisors use the provided data and risk assessments to optimize overall site management. Workers can perform their tasks more safely and stress-free by receiving instructions and feedback based on their emotional state.

[0576] This system improves the quality of material management and work instruction by incorporating sentiment analysis into construction sites, ultimately increasing on-site productivity and safety.

[0577] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0578] Step 1:

[0579] The server collects data from sensors and cameras placed at the construction site. Inputs include work environment data acquired from various sensors, as well as data on workers' movements, voices, and facial expressions. Its specific operation here is real-time data collection from each device. The output is the collected raw data.

[0580] Step 2:

[0581] The server preprocesses the collected data. At this stage, it uses the raw data obtained in the previous step as input and performs data processing such as noise reduction and data standardization. Specifically, this involves clearing audio data and adjusting the resolution of image data. The output is analyzable, formatted data.

[0582] Step 3:

[0583] The server inputs the formatted data into the emotion analysis engine to analyze the workers' emotional state. The input in this step is pre-processed data. The emotion analysis engine performs specific actions to determine whether the workers are experiencing stress by analyzing their facial expressions and tone of voice. The output provides insights into their emotional state (e.g., stress level and motivation).

[0584] Step 4:

[0585] The server uses a generated AI model based on the results of sentiment analysis and other data to forecast material demand and assess risks. The input for this step is information about the workers' emotional states and field data. The server creates prompts and performs specific actions to input them into the AI ​​model. The output includes an efficient material ordering plan and a risk assessment report.

[0586] Step 5:

[0587] The server generates work instructions and feedback based on the generated information. The input is the output data of the AI ​​model. Specifically, it creates work instructions and feedback messages tailored to the emotional state of a particular worker. The output consists of work instructions and feedback.

[0588] Step 6:

[0589] The terminal notifies workers of generated work instructions and feedback. The input here is information received from the server. Specifically, it displays and broadcasts messages via the terminal's display or voice call. The output is the worker's feedback response.

[0590] Step 7:

[0591] The user, a worker, sends feedback back to the server via a terminal. Inputs include the worker's voice instructions and emotional feedback, and specific actions such as providing feedback using voice recognition or text input. The server processes this feedback and uses it to improve instructions and predictions in the next cycle. The output is the system's response, improved by the feedback.

[0592] (Application Example 2)

[0593] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0594] Problems at construction sites include decreased work efficiency, insufficient safety, and mental fatigue among workers. In particular, issuing uniform work instructions without considering the emotional state of workers can lead to unstable work performance. This not only affects the overall productivity of the site but can also threaten worker safety. Furthermore, if proper material management is not carried out, there is a risk of waste or shortage of materials.

[0595] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0596] In this invention, the server includes means for collecting and pre-processing information from a monitoring device, means for analyzing the collected information to predict the amount of materials used and the timing of necessary replenishment, means for recognizing the emotional state of field workers, generating work instructions based on the analysis results, and notifying them through a display device, and means for providing feedback based on the emotional state to improve work efficiency and safety. This enables the provision of individualized and appropriate work instructions that reflect the emotional state of workers, as well as efficient management of materials.

[0597] A "monitoring device" is a device used to collect necessary data at a construction site, and may include sensors and cameras.

[0598] "Preprocessing" refers to the initial stages of processing performed to effectively utilize collected data, and includes tasks such as formatting and filtering the data.

[0599] "Analysis" is the process of analyzing collected data, extracting information, and finding meaning in it.

[0600] "Material usage and required replenishment timing" is an evaluation index used to assess the current inventory and future demand for materials at a construction site.

[0601] A "procurement plan" is a plan for ordering materials based on the required timing and quantity, with the aim of efficient material procurement.

[0602] "The emotional state of field workers" refers to the type and intensity of emotions that workers are currently experiencing, and may include stress levels and motivation levels.

[0603] "Emotion-based feedback" refers to advice and information provided to improve work efficiency and safety by taking into account the emotions of the workers.

[0604] "Risk assessment" is the process of identifying potential risks that may arise in the future and evaluating their impact and likelihood of occurrence.

[0605] A "plan of countermeasures" outlines specific actions and strategies to be taken in response to the assessed risks.

[0606] The system implementing this invention involves the cooperative operation of three elements: a server, a terminal, and a user, in order to improve work efficiency and safety at construction sites.

[0607] First, the server collects information from various sensors and cameras installed at the construction site as a monitoring device and performs preprocessing. Preprocessing includes shaping and filtering the data acquired from the sensors, which allows for efficient data analysis. Based on this information, the server uses a generative AI model to predict the amount of materials used and when they will need to be replenished. Based on this prediction, the server automatically generates a material ordering plan and notifies the administrator.

[0608] Meanwhile, the emotion engine analyzes the worker's facial expressions and tone of voice in real time through smart devices (such as smartphones and smart glasses) to recognize their emotional state. The server incorporates the results of this emotion analysis and uses them to create individual work instructions and feedback. This feedback may include positive messages that motivate the worker, contributing to improved work efficiency and safety.

[0609] Furthermore, the server assesses potential risks that may arise in the future and notifies the on-site manager of proposed countermeasures. This risk assessment also takes into account weather conditions and work progress. For example, in the event of a sudden deterioration in weather, it will propose adjusting the work schedule in advance.

[0610] Specifically, when the server detects worker fatigue, it notifies the worker via their terminal with a message such as, "We recommend a 5-minute break," and also displays a motivational message like, "You're doing a great job!" This process balances worker safety and efficiency.

[0611] An example of a prompt message for a generative AI model is as follows: "Generate work instruction messages based on the work environment data and worker sentiment data for this construction site." This allows the entire system to work together to provide a more efficient and safer work environment.

[0612] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0613] Step 1:

[0614] The server collects data in real time from sensors and cameras installed at the construction site. This data includes temperature, humidity, images of workers' facial expressions, and audio data. The collected data is first pre-processed to remove noise and format the data. This pre-processing enables efficient data analysis in the next analysis step.

[0615] Step 2:

[0616] The server analyzes pre-processed data and uses a generating AI model to predict material usage and required replenishment timing. Current material inventory data and construction progress information are used as input. The analysis results in the output of specific replenishment timings and quantities, and a material ordering plan is automatically generated. This reduces the risk of shortages or surpluses.

[0617] Step 3:

[0618] The server uses an emotion engine to analyze the workers' emotional state based on the collected data. This step uses the workers' facial images and audio data as input. The emotion analysis algorithm then assesses the workers' current stress or motivation levels. The output is a classification of their emotional state.

[0619] Step 4:

[0620] The server generates individual work instructions and feedback based on the results of sentiment analysis. It uses information about the worker's emotional state as input and outputs appropriate work instructions and motivational messages. These messages are communicated to the worker via the terminal's display device. For example, a message such as "We recommend you take a short break" might be generated.

[0621] Step 5:

[0622] The terminal displays work instructions and feedback messages received from the server to the worker in real time. The input is messages from the server, and the output is the information displayed on the terminal's screen. This allows the worker to take necessary actions immediately.

[0623] Step 6:

[0624] The server assesses future risks based on on-site work environment data and emotional state data, and notifies the site manager. Weather data and work progress are used as input for the risk assessment. A risk level assessment and proposed countermeasures are generated as output and notified to the site manager. This enables a rapid response.

[0625] The generated AI model and prompt messages are crucial elements for appropriately creating work instruction messages and risk assessments at each of these steps.

[0626] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0627] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0628] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0629] [Fourth Embodiment]

[0630] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0631] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0632] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0633] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0634] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0635] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0636] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0637] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0638] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0639] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0640] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0641] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0642] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0643] This invention is a system for streamlining material management at construction sites. It optimizes material ordering plans by collecting and analyzing data in real time using sensors and cameras. This system mainly consists of server, terminal, and user components.

[0644] Server Role

[0645] The server plays a central role, collecting data from multiple sensors and cameras installed on-site. The server preprocesses the data and then analyzes it using a generative AI model. Based on the analysis, it predicts material usage and replenishment timing, and automatically creates an ordering plan. This process also considers past consumption data, on-site work schedules, and even weather data. The server also assesses future risks and notifies the site supervisor of countermeasures as needed.

[0646] Terminal role

[0647] The terminal is for field workers to receive information. It receives work instructions generated from the server and provides specific instructions to the workers via a display device. This includes optimal placement of materials and work procedures. The terminal also assists the system's learning process by receiving feedback from workers and sending it to the server.

[0648] User roles

[0649] Users, namely site supervisors and workers, utilize information from the system to manage materials and plan work. Site supervisors make quick decisions based on risk notifications and ordering plans from the server, while workers efficiently carry out their tasks by following instructions displayed on their terminals. Feedback from users is stored on the server and used to improve the system's accuracy.

[0650] Specific example

[0651] For example, at a construction site, a server acquires real-time data on the remaining amount of materials through cameras and sensors. The server analyzes this data, comparing the current consumption rate with the future work plan, and predicts that a particular material will be in short supply in a week. Based on this, the server automatically calculates the optimal order quantity and places an order with the material supplier so that it can be replenished by the next day. Furthermore, if bad weather is predicted, the server notifies the site supervisor with suggested changes to the work schedule, thus preventing risks before they occur. Workers receive instructions on terminals and can quickly place new materials, improving the overall work efficiency of the site.

[0652] As a result, the present invention can dramatically improve the accuracy and efficiency of material management at construction sites.

[0653] The following describes the processing flow.

[0654] Step 1:

[0655] The server collects real-time data on material inventory, work progress, and weather from cameras and sensors installed at the construction site. This includes periodically receiving data from each sensor and storing it in a central database.

[0656] Step 2:

[0657] The server preprocesses the collected data. Specifically, it removes noise from the sensor data and fills in missing data. Furthermore, it converts the data into a consistent format and prepares it for analysis.

[0658] Step 3:

[0659] The server uses pre-processed data to perform analysis using an AI model. This analysis utilizes time series analysis and regression analysis based on historical and current data to predict future material usage and replenishment timing.

[0660] Step 4:

[0661] The server automatically generates a materials ordering plan based on the analysis results. This plan includes specific information such as the type and quantity of materials needed and the optimal ordering timing. The generated ordering plan is automatically notified to the relevant suppliers.

[0662] Step 5:

[0663] The server generates instructions for workers to improve the efficiency of material placement and work procedures. These instructions are sent to terminals, allowing workers to review placement procedures and work changes. This process includes displaying the instructions on a screen.

[0664] Step 6:

[0665] The server predicts future risks based on weather and work progress, and notifies the site supervisor with suggestions tailored to those risks. This risk assessment includes suggestions for changes to the work schedule due to sudden weather changes and warnings about the risk of delays in material procurement.

[0666] Step 7:

[0667] Users, such as workers and site supervisors, efficiently carry out their tasks and manage materials based on instructions and notifications received from terminals and servers. By providing feedback and inputting work status in digital format, they contribute to further system optimization by the server.

[0668] (Example 1)

[0669] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0670] Improving the efficiency and accuracy of material management at construction sites is a critical challenge. Traditional material management methods made it difficult to grasp site conditions in real time and create accurate ordering plans. Furthermore, they failed to adequately consider the impact of weather fluctuations and work schedule adjustments on material consumption. This resulted in unnecessary inventory and over-ordering, reducing site efficiency.

[0671] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0672] In this invention, the server includes means for collecting data from monitoring devices, means for preprocessing the data and converting it into a standard format, and means for inputting the collected data into a generating AI model for analysis. This enables accurate real-time data analysis and automatic generation of material ordering plans. Furthermore, the accuracy of the system can be improved by incorporating user feedback, enabling flexible material management in accordance with weather conditions and work schedules.

[0673] A "monitoring device" is a piece of equipment used at a construction site to acquire real-time information on the status of materials, and it consists of sensors, cameras, and other components.

[0674] "Preprocessing" refers to a series of operations that remove noise from acquired data and appropriately impute missing values, thereby transforming it into a state suitable for analysis.

[0675] A "generative AI model" is an analytical model that utilizes artificial intelligence technology to predict material consumption patterns and shortages using collected data.

[0676] A "procurement plan" is a plan created to secure the necessary materials based on the predicted consumption and supply timing of those materials.

[0677] "Work instructions" are instructions provided to on-site workers to encourage specific actions, such as how to arrange materials and the procedures for performing tasks.

[0678] "Feedback" refers to performance information and opinions provided by field workers and supervisors, which are used to improve the system and enhance its accuracy.

[0679] "Risk assessment" is a process for analyzing the impact of future weather conditions and changes in work schedules on material management and predicting potential problems.

[0680] This invention provides a system using sensors and cameras with the aim of improving the efficiency and accuracy of material management at construction sites. This system consists of a server, a terminal, and a user.

[0681] The server collects data in real time from sensors and cameras installed at the construction site. Specific hardware used includes weight sensors, RFID tag readers, and cameras. The data acquired from these devices is first pre-processed on the server. Pre-processing includes data filtering and imputation of missing values, and data processing software such as Python or R may be used.

[0682] The pre-processed data is input into a generative AI model. This generative AI model predicts future material consumption based on past consumption data, work schedules, and weather data. Examples of software used include machine learning libraries such as TensorFlow and PyTorch. As a concrete example of this analysis process, the prompt "Predict future material consumption" is input into the model, and the expected material consumption and replenishment timing are calculated.

[0683] Based on the analysis results, the server automatically generates a materials ordering plan. This plan includes the required quantities of materials and the optimal delivery schedule. Furthermore, a risk assessment is performed, taking into account on-site weather conditions and work plans, and the site supervisor (the user) is notified as needed.

[0684] The terminals are responsible for providing work instructions from the server to field workers. Workers receive instructions from the terminals and efficiently arrange materials and manage the progress of their work. The terminals also send feedback from workers to the server, contributing to the improvement of the system's accuracy.

[0685] Users utilize information from the system to manage materials and plan work. Specifically, site supervisors can receive risk notifications from the server and take swift countermeasures. Furthermore, workers can improve overall site efficiency by quickly completing material placement based on instructions from their terminals.

[0686] Implementing this system will enable high accuracy and efficiency in on-site material management.

[0687] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0688] Step 1:

[0689] The server collects data in real time from sensors and cameras installed at the construction site. Inputs include material quantities and location information. The server receives this data, analyzes the signals sent from each device, and obtains specific information about the materials. The output generates a list of each material type and its quantity.

[0690] Step 2:

[0691] The server preprocesses the collected data. At this stage, noise and outliers are removed, and missing data is imputed with historical data or estimates. Specifically, data filtering is performed using Python. The input is the material information obtained in the previous step, and the output is a formatted material information dataset.

[0692] Step 3:

[0693] The server inputs pre-processed data into a generating AI model for analysis. This input includes formatted material information data, weather data, and work schedules. The server then prompts the generating AI model with the message, "Predict future material consumption," and performs the prediction. The output is a list of predicted material usage and replenishment timings.

[0694] Step 4:

[0695] The server automatically generates a material ordering plan based on the analysis results. The inputs are the analysis results and past ordering history. The server incorporates predictions from a machine learning model to determine the required material quantities and delivery schedule. The output is an order form to be sent to material suppliers.

[0696] Step 5:

[0697] The server generates work instructions for field workers and sends them to terminals. Input includes the ordering plan and site conditions. The server determines the specific placement of materials and work procedures, and creates instruction sheets. The output is the work instructions displayed on the terminals.

[0698] Step 6:

[0699] The terminal displays work instructions received from the server to the worker. The worker receives the instructions and performs the material placement work. The input is a work instruction sheet, and the output is a report of the completed work results. The terminal sends this report to the server, where it is stored as feedback.

[0700] Step 7:

[0701] Users manage materials using information obtained from the system. Site supervisors make accurate decisions based on risk notifications from the server, ensuring safe and efficient work. User feedback is incorporated into the system and used to improve the accuracy of future analyses.

[0702] (Application Example 1)

[0703] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0704] Efficient resource management is crucial at construction sites. However, conventional methods make it difficult to predict real-time resource usage and replenishment timing, and resource shortages can impact the progress of construction. Furthermore, adjusting work plans to accommodate fluctuations in environmental conditions is challenging. These challenges need to be addressed to achieve more flexible and efficient resource management.

[0705] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0706] In this invention, the server includes means for collecting information from data measurement devices and performing initial processing, means for analyzing the collected information to predict resource usage and the required replenishment timing, and means for providing the status of resource management to the operator in real time via communication equipment. This enables efficient resource management and flexible adjustment of work plans in response to changes in environmental conditions.

[0707] A "data measurement device" is a device used to measure the quantity, location, and condition of resources at a site and acquire them as digital data.

[0708] A "server" is a central processing unit that receives data collected from measuring devices, performs analysis and processing, and provides the results to various terminals.

[0709] "Initial processing" refers to the process of preparing data acquired from data measurement devices for analysis by performing noise reduction and format conversion.

[0710] "Real-time" is a concept that refers to a state in which information is processed or transmitted immediately, with virtually no time delay.

[0711] A "resource procurement plan" is a plan for procuring the necessary resources in the right quantity at the right time, and is based on predictions of resource usage and replenishment timing.

[0712] "Risk assessment" is the process of predicting potential risks and problems that may arise in the future and considering measures to mitigate their impact.

[0713] "Environmental conditions" refer to external natural factors that affect work and resource management at a construction site, such as weather and temperature.

[0714] The system implementing this invention is designed to streamline resource management at construction sites. This system consists of data measurement devices, servers, communication equipment, work terminals, and the like.

[0715] The server collects resource data transmitted from data measurement devices and performs initial processing. This initial processing includes noise removal and conversion to the required data format. Subsequently, the server uses a generative AI model to predict resource usage and the necessary replenishment timing. This automatically generates a resource procurement plan, enabling efficient resource management. The server also assesses future risks and notifies field managers of appropriate countermeasures.

[0716] Communication equipment plays a role in providing workers with real-time information on resource management status. This enables rapid responses tailored to the situation on site. For example, if a shortage of a certain resource is predicted, the server automatically generates a procurement plan and notifies the worker via their terminal, allowing for rapid resource replenishment.

[0717] This system is effective, for example, when "adjusting resource usage and work plans in response to fluctuations in environmental conditions." This enables efficient work progress while flexibly responding to environmental changes.

[0718] The following are examples of prompt statements that can be used.

[0719] "Based on real-time material management, please propose the next necessary actions. This should take into account material usage forecasts and proactive risk management."

[0720] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0721] Step 1:

[0722] The server receives digital data from data measurement devices, including the quantity and location information of resources. The server performs initial processing on this data and removes noise. The input is raw data, and the output is data in a clean format.

[0723] Step 2:

[0724] The server inputs the initially processed data into a generating AI model to predict resource usage and replenishment timing. Data processing considers past consumption data and environmental conditions, and the predictive model is used. The output is the predicted resource consumption and optimal replenishment timing.

[0725] Step 3:

[0726] The server automatically generates a resource procurement plan based on the forecast results. The generated plan enables the procurement of resources in the optimal quantity and timing. The input is forecast data, and the output is the finalized procurement plan.

[0727] Step 4:

[0728] The server transmits the procurement plan to field work terminals, displaying it to field personnel in real time. The input is the generated plan information, and the output is a notification to the personnel. The terminal displays this and instructs them on the necessary actions.

[0729] Step 5:

[0730] The server also assesses future risks and notifies field supervisors of relevant information. Inputs here are normal operational and environmental data, while outputs are risk notifications and proposed countermeasures.

[0731] Step 6:

[0732] The field staff, acting as users, perform tasks based on information received through their terminals and submit completion reports as feedback to the server. This feedback helps improve the accuracy of the next predictive model. The input is the feedback information, and the output is its storage in the training database.

[0733] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0734] This invention combines a materials management system for construction sites with an emotion engine that recognizes user emotions. The aim is to improve the efficiency and safety of on-site work and reduce the burden on workers and managers. This system consists of a server, terminals, and user components.

[0735] Server Role

[0736] The server collects data from various sensors and cameras at the construction site and uses an emotion engine to analyze workers' emotions from their facial expressions and tone of voice. This allows the server to understand in real time whether workers are stressed or highly motivated. The server preprocesses the collected data, uses an AI model to forecast material demand and assess risks, and generates ordering plans. It also generates appropriate work instructions and feedback based on the emotion analysis results and provides them to the user.

[0737] Terminal role

[0738] The terminal is a device that allows workers to receive information and instructions from the server. Through a display device, it provides real-time work instructions and feedback, as well as displaying motivational messages generated by an emotion engine. This allows workers to receive appropriate support tailored to their emotional state. The terminal also functions as a means of transmitting worker feedback and emotional information to the server via voice input and camera.

[0739] User roles

[0740] This system enables site supervisors and workers, as users, to efficiently manage materials and plan work. Site supervisors receive risk notifications, work instructions, and management support based on their emotional state from the server, improving overall operational efficiency on site. Workers receive feedback tailored to their individual emotional state displayed on their terminals, reducing stress and allowing them to work more safely and comfortably.

[0741] Specific example

[0742] For example, if a worker is performing heavy labor for several consecutive days on a hot day, and the emotion engine detects that their stress levels are rising based on their facial expression and tone of voice, the server analyzes this information and displays a message on the terminal prompting them to take a break. It also provides positive feedback in real time to improve motivation. This helps ensure worker safety and maintain overall performance at the worksite.

[0743] Based on the above, the present invention provides a system that contributes to material management that takes emotions into consideration and to improving work efficiency.

[0744] The following describes the processing flow.

[0745] Step 1:

[0746] The server collects real-time data from sensors such as cameras and microphones installed at the construction site, including worker movements, audio data, and material inventory status. This data is initially processed to remove noise and prepare it for easy analysis.

[0747] Step 2:

[0748] The server activates an emotion engine to analyze the worker's facial expressions and tone of voice from the collected data. Based on this analysis, it determines the worker's current emotional state, such as "stress" or "fatigue," in real time.

[0749] Step 3:

[0750] As a notification to the site supervisor (the user), the server compiles the emotional state of each worker, as determined by the emotion engine, and displays it on a management dashboard. This allows the site supervisor to quickly understand which workers require special attention.

[0751] Step 4:

[0752] If the emotional state exceeds a certain threshold, the server sends a message generated by the emotion engine to the terminal. This message may include specific suggestions for taking a break or advice on mental health care.

[0753] Step 5:

[0754] The terminal displays emotion-based messages received from the server, and the worker reviews the content. For example, they might receive a notification such as, "Your stress level is high, please take a 10-minute break," and be advised to temporarily stop working and take a break.

[0755] Step 6:

[0756] The server continuously records emotional and work efficiency data for long-term analysis. This allows for feedback to be obtained for improving the work environment and optimizing material management. The feedback is provided to the administrator at the end of the project.

[0757] Step 7:

[0758] Users, i.e., workers, can adjust their daily tasks based on feedback from their devices and review their workload and efficiency. Furthermore, they can proactively seek areas for improvement to enhance their performance in the field, based on the continuously provided feedback.

[0759] (Example 2)

[0760] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0761] Conventional construction site management systems struggle with effective material management and timely ordering planning, and lack work instructions that adequately consider the feelings and safety of workers. Therefore, it is crucial to reduce worker stress and improve motivation, while simultaneously demanding increased efficiency in material management and improved accuracy in risk prediction.

[0762] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0763] In this invention, the server includes means for collecting and pre-processing data from a monitoring device, means for analyzing the collected data to predict the amount of materials used and the timing of necessary replenishment, and means for analyzing the emotional state of workers. This enables appropriate work instructions and feedback based on the emotional state of workers.

[0764] A "monitoring device" is a piece of equipment installed to understand the working environment and to acquire data.

[0765] "Preprocessing" refers to the initial stages of processing to prepare collected data into a format suitable for analysis.

[0766] "Data analysis" is the process of analyzing collected information to identify specific patterns or trends.

[0767] "Material usage" is an indicator that refers to the amount of materials consumed at a construction site.

[0768] "Replenishment timing" refers to the period when additional materials need to be procured, and is a point in time identified for efficient management.

[0769] An "automatic ordering plan" is an ordering plan automatically generated by a computer system based on the amount of materials used and the timing of replenishment.

[0770] "Emotional analysis" is a process that determines the emotional state of a worker based on information such as their facial expressions and tone of voice.

[0771] "Risk assessment" is a procedure for identifying potential problems or obstacles that may arise in the future, and for analyzing and evaluating their impact.

[0772] "Work instructions" refer to specific guidance provided to on-site workers to ensure they perform their tasks effectively.

[0773] "Feedback" refers to reactions and advice given to workers based on the results of their work and their emotional state.

[0774] This invention is a system aimed at improving material management and work efficiency at construction sites, and consists of three elements: a server, a terminal, and a user. The roles of each element and specific embodiments are described in detail below.

[0775] The server collects data from multiple sensors and cameras placed at the construction site. These devices are used to monitor the work environment and the status of workers in real time. The server preprocesses the collected data, removing noise and standardizing it. After the data is preprocessed, the server uses an emotion analysis engine to evaluate the emotional state of the workers. This includes facial recognition and voice analysis of the workers, applying AI technology to understand their emotional state.

[0776] The results of sentiment analysis and other data are sent to a generative AI model. The AI ​​model used here is trained with prompts for forecasting material demand and assessing risks. For example, a specific prompt such as, "What materials are needed when worker A's stress level is high?" is used. The resulting forecast information is then used to create efficient material usage and timely ordering plans.

[0777] The terminal is a device directly operated by the worker, displaying instructions and feedback from the server in real time. Support messages and work instructions tailored to the worker's emotional state are displayed on the terminal, allowing the worker to take appropriate action based on this information. The terminal can also send feedback back to the server, which can be done via voice input or touch operation.

[0778] Users include site supervisors and workers who utilize the system to improve work efficiency and safety. Site supervisors use the provided data and risk assessments to optimize overall site management. Workers can perform their tasks more safely and stress-free by receiving instructions and feedback based on their emotional state.

[0779] This system improves the quality of material management and work instruction by incorporating sentiment analysis into construction sites, ultimately increasing on-site productivity and safety.

[0780] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0781] Step 1:

[0782] The server collects data from sensors and cameras placed at the construction site. Inputs include work environment data acquired from various sensors, as well as data on workers' movements, voices, and facial expressions. Its specific operation here is real-time data collection from each device. The output is the collected raw data.

[0783] Step 2:

[0784] The server preprocesses the collected data. At this stage, it uses the raw data obtained in the previous step as input and performs data processing such as noise reduction and data standardization. Specifically, this involves clearing audio data and adjusting the resolution of image data. The output is analyzable, formatted data.

[0785] Step 3:

[0786] The server inputs the formatted data into the emotion analysis engine to analyze the workers' emotional state. The input in this step is pre-processed data. The emotion analysis engine performs specific actions to determine whether the workers are experiencing stress by analyzing their facial expressions and tone of voice. The output provides insights into their emotional state (e.g., stress level and motivation).

[0787] Step 4:

[0788] The server uses a generated AI model based on the results of sentiment analysis and other data to forecast material demand and assess risks. The input for this step is information about the workers' emotional states and field data. The server creates prompts and performs specific actions to input them into the AI ​​model. The output includes an efficient material ordering plan and a risk assessment report.

[0789] Step 5:

[0790] The server generates work instructions and feedback based on the generated information. The input is the output data of the AI ​​model. Specifically, it creates work instructions and feedback messages tailored to the emotional state of a particular worker. The output consists of work instructions and feedback.

[0791] Step 6:

[0792] The terminal notifies workers of generated work instructions and feedback. The input here is information received from the server. Specifically, it displays and broadcasts messages via the terminal's display or voice call. The output is the worker's feedback response.

[0793] Step 7:

[0794] The user, a worker, sends feedback back to the server via a terminal. Inputs include the worker's voice instructions and emotional feedback, and specific actions such as providing feedback using voice recognition or text input. The server processes this feedback and uses it to improve instructions and predictions in the next cycle. The output is the system's response, improved by the feedback.

[0795] (Application Example 2)

[0796] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0797] Problems at construction sites include decreased work efficiency, insufficient safety, and mental fatigue among workers. In particular, issuing uniform work instructions without considering the emotional state of workers can lead to unstable work performance. This not only affects the overall productivity of the site but can also threaten worker safety. Furthermore, if proper material management is not carried out, there is a risk of waste or shortage of materials.

[0798] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0799] In this invention, the server includes means for collecting and pre-processing information from a monitoring device, means for analyzing the collected information to predict the amount of materials used and the timing of necessary replenishment, means for recognizing the emotional state of field workers, generating work instructions based on the analysis results, and notifying them through a display device, and means for providing feedback based on the emotional state to improve work efficiency and safety. This enables the provision of individualized and appropriate work instructions that reflect the emotional state of workers, as well as efficient management of materials.

[0800] A "monitoring device" is a device used to collect necessary data at a construction site, and may include sensors and cameras.

[0801] "Preprocessing" refers to the initial stages of processing performed to effectively utilize collected data, and includes tasks such as formatting and filtering the data.

[0802] "Analysis" is the process of analyzing collected data, extracting information, and finding meaning in it.

[0803] "Material usage and required replenishment timing" is an evaluation index used to assess the current inventory and future demand for materials at a construction site.

[0804] A "procurement plan" is a plan for ordering materials based on the required timing and quantity, with the aim of efficient material procurement.

[0805] "The emotional state of field workers" refers to the type and intensity of emotions that workers are currently experiencing, and may include stress levels and motivation levels.

[0806] "Emotion-based feedback" refers to advice and information provided to improve work efficiency and safety by taking into account the emotions of the workers.

[0807] "Risk assessment" is the process of identifying potential risks that may arise in the future and evaluating their impact and likelihood of occurrence.

[0808] A "plan of countermeasures" outlines specific actions and strategies to be taken in response to the assessed risks.

[0809] The system implementing this invention involves the cooperative operation of three elements: a server, a terminal, and a user, in order to improve work efficiency and safety at construction sites.

[0810] First, the server collects information from various sensors and cameras installed at the construction site as a monitoring device and performs preprocessing. Preprocessing includes shaping and filtering the data acquired from the sensors, which allows for efficient data analysis. Based on this information, the server uses a generative AI model to predict the amount of materials used and when they will need to be replenished. Based on this prediction, the server automatically generates a material ordering plan and notifies the administrator.

[0811] Meanwhile, the emotion engine analyzes the worker's facial expressions and tone of voice in real time through smart devices (such as smartphones and smart glasses) to recognize their emotional state. The server incorporates the results of this emotion analysis and uses them to create individual work instructions and feedback. This feedback may include positive messages that motivate the worker, contributing to improved work efficiency and safety.

[0812] Furthermore, the server assesses potential risks that may arise in the future and notifies the on-site manager of proposed countermeasures. This risk assessment also takes into account weather conditions and work progress. For example, in the event of a sudden deterioration in weather, it will propose adjusting the work schedule in advance.

[0813] Specifically, when the server detects worker fatigue, it notifies the worker via their terminal with a message such as, "We recommend a 5-minute break," and also displays a motivational message like, "You're doing a great job!" This process balances worker safety and efficiency.

[0814] An example of a prompt message for a generative AI model is as follows: "Generate work instruction messages based on the work environment data and worker sentiment data for this construction site." This allows the entire system to work together to provide a more efficient and safer work environment.

[0815] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0816] Step 1:

[0817] The server collects data in real time from sensors and cameras installed at the construction site. This data includes temperature, humidity, images of workers' facial expressions, and audio data. The collected data is first pre-processed to remove noise and format the data. This pre-processing enables efficient data analysis in the next analysis step.

[0818] Step 2:

[0819] The server analyzes pre-processed data and uses a generating AI model to predict material usage and required replenishment timing. Current material inventory data and construction progress information are used as input. The analysis results in the output of specific replenishment timings and quantities, and a material ordering plan is automatically generated. This reduces the risk of shortages or surpluses.

[0820] Step 3:

[0821] The server uses an emotion engine to analyze the workers' emotional state based on the collected data. This step uses the workers' facial images and audio data as input. The emotion analysis algorithm then assesses the workers' current stress or motivation levels. The output is a classification of their emotional state.

[0822] Step 4:

[0823] The server generates individual work instructions and feedback based on the results of sentiment analysis. It uses information about the worker's emotional state as input and outputs appropriate work instructions and motivational messages. These messages are communicated to the worker via the terminal's display device. For example, a message such as "We recommend you take a short break" might be generated.

[0824] Step 5:

[0825] The terminal displays work instructions and feedback messages received from the server to the worker in real time. The input is messages from the server, and the output is the information displayed on the terminal's screen. This allows the worker to take necessary actions immediately.

[0826] Step 6:

[0827] The server assesses future risks based on on-site work environment data and emotional state data, and notifies the site manager. Weather data and work progress are used as input for the risk assessment. A risk level assessment and proposed countermeasures are generated as output and notified to the site manager. This enables a rapid response.

[0828] The generated AI model and prompt messages are crucial elements for appropriately creating work instruction messages and risk assessments at each of these steps.

[0829] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0830] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0831] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0832] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0833] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0834] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0835] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0836] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0837] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0838] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0839] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0840] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0841] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0842] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0843] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0844] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0845] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0846] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0847] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0848] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0849] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0850] The following is further disclosed regarding the embodiments described above.

[0851] (Claim 1)

[0852] A means for collecting and pre-processing data from a monitoring device,

[0853] A means for analyzing collected data to predict the amount of materials used and the timing of necessary replenishment,

[0854] A means for automatically generating a material ordering plan based on the analyzed results,

[0855] A means for generating work instructions for on-site workers and notifying them via a display device,

[0856] A means of assessing future risks and notifying the site supervisor of proposed countermeasures,

[0857] A system that includes this.

[0858] (Claim 2)

[0859] The system according to claim 1 for optimizing the flow of material placement.

[0860] (Claim 3)

[0861] The system according to claim 1, which adjusts material usage and work plans in consideration of changes in weather conditions.

[0862] "Example 1"

[0863] (Claim 1)

[0864] Means for collecting data from monitoring devices,

[0865] A means of performing preprocessing and converting data to a standard format,

[0866] A means of inputting the collected data into a generating AI model and analyzing it,

[0867] A means for predicting the amount of materials used and the timing of necessary replenishment,

[0868] A means for automatically generating a material ordering plan based on the analyzed results,

[0869] A means for generating work instructions for on-site workers and notifying them via a display device,

[0870] A means of collecting user feedback and improving the accuracy of the system,

[0871] A means of assessing future risks and notifying the site supervisor of proposed countermeasures,

[0872] A system that includes this.

[0873] (Claim 2)

[0874] The system according to claim 1 for optimizing the flow of materials management.

[0875] (Claim 3)

[0876] The system according to claim 1, which adjusts material usage and work plans in consideration of changes in weather conditions.

[0877] "Application Example 1"

[0878] (Claim 1)

[0879] A means for collecting information from data measurement devices and performing initial processing,

[0880] A means for analyzing collected information to predict resource usage and the necessary replenishment timing,

[0881] A means for automatically generating a resource procurement plan based on the analyzed results,

[0882] A means for generating work instructions for workers and notifying them via a display device,

[0883] A means of assessing future risks and notifying on-site supervisors of proposed countermeasures,

[0884] A means of providing the status of resource management to the person in charge in real time via communication equipment,

[0885] A system that includes this.

[0886] (Claim 2)

[0887] The system according to claim 1 for optimizing the flow of resource installation.

[0888] (Claim 3)

[0889] The system according to claim 1, which adjusts resource use and work plans in consideration of fluctuations in environmental conditions.

[0890] "Example 2 of combining an emotion engine"

[0891] (Claim 1)

[0892] A means for collecting and pre-processing data from a monitoring device,

[0893] A means for analyzing collected data to predict the amount of materials used and the timing of necessary replenishment,

[0894] A means for automatically generating a material ordering plan based on the analyzed results,

[0895] A means of analyzing the emotional state of field workers,

[0896] A means of preparing appropriate work instructions and feedback based on the emotional state of the worker and notifying them through a display device,

[0897] A means of assessing future risks and notifying the site supervisor of proposed countermeasures,

[0898] A system that includes this.

[0899] (Claim 2)

[0900] The system according to claim 1 for optimizing the flow of material placement.

[0901] (Claim 3)

[0902] The system according to claim 1, which adjusts material usage and work plans in consideration of changes in weather conditions.

[0903] "Application example 2 of combining emotional engines"

[0904] (Claim 1)

[0905] A means for collecting information from a monitoring device and performing preprocessing,

[0906] A means for analyzing collected information to predict the amount of materials used and the timing of necessary replenishment,

[0907] A means for automatically generating a material ordering plan based on the analyzed results,

[0908] A means for recognizing the emotional state of on-site workers, generating work instructions based on the analysis results, and notifying them via a display device,

[0909] A means of improving work efficiency and safety by providing feedback based on emotional state,

[0910] A means of assessing future risks and notifying on-site managers of proposed countermeasures,

[0911] A system that includes this.

[0912] (Claim 2)

[0913] The system according to claim 1, which provides feedback to increase worker motivation based on the results of emotion analysis.

[0914] (Claim 3)

[0915] The system according to claim 1, which uses a smart device to monitor the emotional state of workers in real time. [Explanation of Symbols]

[0916] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting and pre-processing data from a monitoring device, A means for analyzing collected data to predict the amount of materials used and the timing of necessary replenishment, A means for automatically generating a material ordering plan based on the analyzed results, A means for generating work instructions for on-site workers and notifying them via a display device, A means of assessing future risks and notifying the site supervisor of proposed countermeasures, A system that includes this.

2. The system according to claim 1 for optimizing the flow of movement of materials.

3. The system according to claim 1, which adjusts material usage and work plans in consideration of changes in weather conditions.

Images