system

Abstract

We provide a system that uses data collected from sensors installed in buildings to predict the deterioration of buildings over time and carry out repairs and maintenance work at the optimal time. [Solution] A system including: a data collection means for collecting data from multiple sensors installed in a building; a data preprocessing means for preprocessing the collected data and detecting and removing outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of deterioration over time; a deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; a warning generation means for identifying problem areas based on the prediction results and generating warnings; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; and a user interface means for displaying the proposed repair plan on a user terminal and allowing the user to select and execute the optimal plan.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional building maintenance management, countermeasures are often taken when deterioration over time is discovered, but repair work after the deterioration has progressed is costly and time-consuming, resulting in problems such as a worsening of the living environment and a decrease in safety. In addition, there is no way to accurately predict the progression of deterioration or the optimal timing for repairs, and unnecessary repairs are often carried out. In light of this situation, there is a need for the development of a system that can predict the deterioration of buildings in advance and carry out repairs and maintenance work at the optimal time. [Means for solving the problem]

[0005] The present invention provides a system for predicting the deterioration of a building over time in advance by using data collected from sensors installed in the building, and for carrying out repair and maintenance work at the optimal timing. The system includes the following means.

[0006] The first method is to establish a data collection means that collects data from multiple sensors installed in the building.

[0007] The second means is to provide a data preprocessing means for preprocessing the collected data and detecting and removing outliers and missing values.

[0008] The third means is to provide a data analysis means for analyzing the preprocessed data to detect signs of deterioration over time.

[0009] As a fourth means, a degradation prediction model generation means is provided that models the progression pattern of degradation and predicts future degradation.

[0010] The fifth means is to provide a warning generation means that identifies problem areas based on the prediction results and generates warnings.

[0011] The sixth measure is to provide a proposal generation means that optimizes the timing of repair and maintenance work and generates optimal construction plans that take into account budgets and construction periods.

[0012] As a seventh means, a user interface means is provided for displaying the proposed repair plans on the user's terminal and for the user to select and execute the optimal plan.

[0013] This allows us to predict the deterioration of buildings over time in advance and carry out repairs and maintenance work at the optimal time, thereby maintaining the living environment over the long term and improving safety.

[0014] A "data collection means" is a device or method for collecting data from multiple sensors installed in a building.

[0015] "Data preprocessing means" refers to a device or method for preprocessing collected data and detecting and removing outliers and missing values.

[0016] "Data analysis means" refers to a device or method for analyzing pre-processed data and detecting signs of aging degradation.

[0017] The "deterioration prediction model generating means" is a device or method for modeling the deterioration progression pattern and predicting future deterioration.

[0018] The "warning generation means" refers to a device or method for identifying problem areas based on the results of the deterioration prediction model and generating a warning.

[0019] The "proposal generation means" refers to a device or method for optimizing the timing of repair and maintenance work and generating an optimal construction plan that takes into account the budget and construction period.

[0020] The "user interface means" refers to a device or method that displays proposed repair plans on a user terminal and allows the user to select and execute the most suitable plan.

[0021] A "sensor" is a device that measures a specific physical phenomenon (such as temperature, humidity, or vibration) and collects that information as data. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] The present invention relates to a system that predicts the deterioration of buildings over time and performs repair and maintenance work at the optimal timing. To implement the present invention, three main components are required: a server, a sensor, and a user terminal. A specific embodiment of the system is shown below.

[0044] This system collects data from multiple sensors installed in buildings and predicts deterioration over time based on the collected data. The server analyzes the data and predicts deterioration, and based on the results, sends warnings and repair proposals to the user. The user can then review the proposals and select and implement the optimal repair plan.

[0045] Data collection

[0046] server

[0047] Data is collected periodically from various sensors in the building (temperature sensors, humidity sensors, vibration sensors, etc.) The server receives the data sent from these sensors and stores it in a database.

[0048] Data Preprocessing

[0049] server

[0050] Preprocessing is performed on the collected data, specifically detecting outliers and missing values ​​and removing or correcting them as necessary. This process improves data quality.

[0051] Data analysis

[0052] server

[0053] The pre-processed data is analyzed to detect signs of aging deterioration. Various algorithms (e.g., time series analysis, clustering) are used for data analysis.

[0054] Deterioration prediction model generation

[0055] server

[0056] Based on the analysis results, a machine learning algorithm is implemented to model the deterioration progression pattern. The deterioration prediction model is used to predict future deterioration.

[0057] Warning generation

[0058] server

[0059] Based on the results of the degradation prediction model, areas where problems are likely to occur are identified and warnings are generated, which are notified to the user in real time.

[0060] Proposal generation

[0061] server

[0062] The system uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. The proposed repair plan takes into account budget, construction time, materials used, and other factors.

[0063] Check and execute on the user interface

[0064] User

[0065] Users access the system using a terminal, check the warnings sent from the server and the proposed repair plans, select the repair plan they think is most appropriate, and carry out the specific work.

[0066] Specific examples

[0067] Example 1: Predicting and repairing cracks in exterior walls

[0068] 1. Data Collection

[0069] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[0070] 2. Data Preprocessing

[0071] An anomaly detection algorithm removes false sensor detection data.

[0072] 3. Data Analysis

[0073] Time series analysis is performed to detect crack progression patterns.

[0074] 4. Deterioration prediction model generation

[0075] Machine learning algorithms are used to generate predictive models of crack progression.

[0076] 5. Warning generation

[0077] Before the cracks become serious, the server generates a warning to notify the user.

[0078] 6. Proposal generation

[0079] The server will suggest the best time and method for repairing the crack, for example, indicating that repairs are needed within the next two months.

[0080] 7. User Interface

[0081] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[0082] In this way, this system predicts the deterioration of buildings over time in advance and takes preventative measures at the appropriate time, making it possible to maintain the living environment over the long term.

[0083] The processing flow will be explained below.

[0084] Step 1:

[0085] server

[0086] Data is collected in real time from multiple sensors installed in buildings (temperature sensors, humidity sensors, vibration sensors, etc.) The data is sent via the Internet, and the server receives it and stores it in a database.

[0087] Step 2:

[0088] server

[0089] Preprocessing the collected data involves running algorithms to detect outliers and missing values ​​and remove or correct them as needed, creating a dataset suitable for analysis.

[0090] Step 3:

[0091] server

[0092] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering are performed to find patterns of fluctuations and abnormalities in the data.

[0093] Step 4:

[0094] server

[0095] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[0096] Step 5:

[0097] server

[0098] Analyzes the results of the deterioration prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records them in a database.

[0099] Step 6:

[0100] server

[0101] Optimize the timing of repair and maintenance work using a multivariate optimization algorithm. Create the optimal repair plan taking into account conditions such as budget, construction period, and materials used.

[0102] Step 7:

[0103] server

[0104] The generated repair plan is sent to the user interface, which includes the details of the repair plan for easy user review.

[0105] Step 8:

[0106] User

[0107] Users access the system from their devices (smartphones or PCs) and check the warnings and repair proposals sent from the server. Users can then view the proposed plans and select the most appropriate one.

[0108] Step 9:

[0109] User

[0110] Based on the repair plan selected by the user, the system schedules specific work, contacts repair contractors as necessary, and carries out the actual repair and maintenance work.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] In recent years, problems caused by the deterioration of buildings over time have been increasing. This often results in unexpected repair costs and the need for long-term construction work. However, with current systems, it is difficult to accurately predict the progression of deterioration in advance and carry out repair and maintenance work at the optimal time. Therefore, there is a need for a system that can accurately predict the deterioration of buildings over time and propose repair and maintenance work at the optimal time, thereby preventing problems before they occur and optimizing costs.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in a building; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; a notification means for notifying a user terminal of the prediction results in real time based on the constructed deterioration prediction model; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; a user interface means for displaying the proposed repair plans on the user terminal and allowing the user to select and execute the optimal plan; and a deterioration progress prediction algorithm means for predicting the progression of deterioration of a building using a machine learning algorithm from the collected data. This makes it possible to accurately predict building deterioration and perform repair and maintenance work at the optimal time.

[0116] A "data collection means" is a means for collecting data from multiple sensors installed in a building.

[0117] The "data preprocessing means" is a means for preprocessing collected data to detect and remove outliers and missing values.

[0118] The "data analysis means" is a means for analyzing the pre-processed data to detect signs of aging deterioration.

[0119] The "deterioration prediction model generating means" is a means for modeling the deterioration progression pattern and predicting future deterioration.

[0120] The "notification means" is a means for notifying the user terminal of the prediction results in real time based on the constructed deterioration prediction model.

[0121] The "proposal generation means" is a means for optimizing the timing of repair and maintenance work and generating an optimal construction plan that takes into account the budget and construction period.

[0122] The "user interface means" is a means for displaying proposed repair plans on a user terminal and allowing the user to select and execute the most suitable plan.

[0123] The "deterioration progress prediction algorithm means" is a means for predicting the deterioration progress of a building using a machine learning algorithm from collected data.

[0124] The present invention relates to a system for predicting deterioration of a building over time and performing repair and maintenance work at an optimal timing. Specific embodiments for carrying out the present invention are described below.

[0125] The system consists of three main components: a server, sensors, and user terminals. The entire system runs through the following stages: data collection, data preprocessing, data analysis, degradation prediction model generation, warning generation, suggestion generation, and user interface.

[0126] Data collection

[0127] The server periodically collects data from multiple sensors installed in the building, such as temperature sensors, humidity sensors, and vibration sensors. This collected data is then stored in a database by the server. The database used is MySQL (registered trademark).

[0128] Data Preprocessing

[0129] The server preprocesses the collected data to detect and remove outliers and missing values. It uses the PyOD library to detect outliers and imputes missing values ​​with the most recent valid data, thereby improving data quality.

[0130] Data analysis

[0131] Based on the preprocessed data, the server analyzes signs of aging deterioration. For the analysis, it uses "scikit-learn" and "statsmodels" to perform time series analysis and clustering. For example, it applies the "ARIMA model" to vibration data from building exterior walls to detect fluctuation patterns.

[0132] Deterioration prediction model generation

[0133] The server generates a deterioration prediction model based on the results of the data analysis. This model is generated using a deep learning algorithm, specifically LSTM (Long Short Term Memory), that uses "TENSORFLOW (registered trademark)." The generated model is used to predict future deterioration and can accurately capture the progression pattern of deterioration.

[0134] Warning generation

[0135] The server identifies problem areas and generates warnings based on the results of the deterioration prediction model. For example, if a crack in an exterior wall exceeds 5 mm in depth, it will generate a warning as a serious error. This warning is sent to the user in real time via email or SMS using a messaging API such as "Twilio."

[0136] Proposal generation

[0137] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. Proposed repair plans are created taking into account budget, construction time, materials used, and more. The SciPy library is used for optimization, generating specific suggestions, such as "repairs are required within the next two months."

[0138] User Interface

[0139] Users access the system using a terminal and check the warnings and proposed repair plans sent from the server. The user interface is built using React and Angular, and information can be viewed in real time. For example, users can click the "Accept repair proposal" button, which will send a notification to the repair company.

[0140] Specific examples

[0141] Example 1: Predicting and repairing cracks in exterior walls

[0142] 1. Data Collection

[0143] The server collects crack data every hour from vibration sensors installed on the exterior walls.

[0144] Example: "Sensor ID: 123, Time: 10:00, Vibration Data: 0.05"

[0145] 2. Data Preprocessing

[0146] An outlier (0.10) is detected, removed, and supplemented with the latest valid data (0.05).

[0147] 3. Data Analysis

[0148] The server applies the ARIMA model and generates a forecast using the most recent three months of data.

[0149] 4. Deterioration prediction model generation

[0150] Generate a degradation prediction model using the SaaS deep learning algorithm LSTM. Example: "LSTM model generation complete, accuracy: 95%."

[0151] 5. Warning generation

[0152] Send a warning to the user if the crack depth exceeds 5mm (10:05)

[0153] 6. Proposal generation

[0154] The proposed implementation plan is to repair within the next two months, with materials A and construction time of 3 days.

[0155] 7. User Interface

[0156] The server sends a notification to the contractor when the user confirms the proposal and clicks the "Request Repair" button (10:10)

[0157] Example of input prompt for generative AI model

[0158] "Please predict cracks in the exterior walls of buildings and suggest appropriate repair timing."

[0159] This enables the system to accurately predict deterioration of buildings over time and carry out repairs and maintenance work at the optimal time, preventing problems before they occur and optimizing costs.

[0160] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0161] Step 1: Data collection

[0162] server

[0163] The server collects data from temperature sensors, humidity sensors, vibration sensors, etc. installed in the building. This data collection is done periodically, for example, every hour, and data from the sensors is sent to the server. The collected data includes information such as the sensor ID, time, and measurement value. The server receives this data and stores it in a database. The database used is "MySQL."

[0164] Input: Measurement data from sensors (temperature, humidity, vibration)

[0165] Output: Sensor data stored in a database

[0166] Step 2: Data Preprocessing

[0167] server

[0168] The server preprocesses the collected data, including detecting and removing outliers and missing values. For example, it uses the PyOD library to detect outliers and removes outliers with data points exceeding three sigma. Additionally, if missing values ​​are found, they are imputed with the most recent valid data.

[0169] Input: Sensor data stored in a database

[0170] Output: Pre-processed high-quality sensor data

[0171] Step 3: Data analysis

[0172] server

[0173] The server analyzes the preprocessed data. For this analysis, it uses "scikit-learn" and "statsmodels" to perform time series analysis and clustering. For example, for vibration data on building exterior walls, it uses the "ARIMA model" to analyze data fluctuation patterns.

[0174] Input: Preprocessed sensor data

[0175] Output: Analysis results (deterioration progression pattern, etc.)

[0176] Step 4: Deterioration prediction model generation

[0177] server

[0178] The server generates a degradation prediction model based on the results of the data analysis. This model is generated using a deep learning algorithm using TensorFlow, specifically LSTM (Long Short Term Memory). LSTM is suitable for building prediction models based on time series data.

[0179] Input: Analysis results

[0180] Output: Deterioration prediction model (model for predicting future deterioration)

[0181] Step 5: Generate warnings

[0182] server

[0183] The server identifies problem areas and generates warnings based on the results of the deterioration prediction model. For example, if a crack in an exterior wall exceeds 5 mm in depth, it will generate a warning as a serious error. This warning is sent to the user in real time via email or SMS using a messaging API such as "Twilio."

[0184] Input: Results of the deterioration prediction model

[0185] Output: A warning notice to the user

[0186] Step 6: Proposal Generation

[0187] server

[0188] The server uses multivariate optimization algorithms to optimize the timing of repairs and maintenance work. Proposals are created taking into account budget, construction time, materials used, etc. For example, it uses the SciPy optimization algorithm to notify you that repairs are needed within the next two months.

[0189] Input: Warning Notification

[0190] Output: Optimized repair plan

[0191] Step 7: Check and execute in the user interface

[0192] User

[0193] Users access the system using a terminal and check the warnings and proposed repair plans sent from the server. The user interface is built using React and Angular, allowing users to check information in real time. By clicking the "Accept Repair Proposal" button, an automatic notification is sent to the repair company.

[0194] Input: Warning and suggestion notifications from the server

[0195] Output: Confirmation of repair execution (execution of the repair plan selected by the user)

[0196] (Application example 1)

[0197] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0198] Conventional building and equipment maintenance management systems had low accuracy in predicting deterioration, making it difficult to carry out repairs at the appropriate time. Furthermore, there was a lack of means for on-site staff to receive signs of deterioration and repair proposals in real time. This increased the risk of sudden breakdowns and malfunctions in buildings and equipment, leading to increased maintenance costs and reduced work efficiency. To solve these issues, a system that can more accurately predict deterioration and make more efficient repair proposals is needed.

[0199] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0200] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in a building or facility; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; a warning generation means for identifying problem areas based on the prediction results and generating warnings; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; a user interface means for displaying the proposed repair plan on a user terminal or wearable device and allowing the user to select and execute the optimal plan; and a notification means for sending notifications to on-site staff in real time based on the data collected from the sensors and the generated deterioration prediction model. This enables highly accurate deterioration predictions and efficient repair proposals, preventing unexpected breakdowns of buildings and facilities, reducing maintenance costs, and improving work efficiency.

[0201] "Data collection means" refers to devices or software that function to collect data from multiple sensors installed in buildings or facilities.

[0202] "Data preprocessing means" refers to devices or software that have the function of preprocessing collected data and detecting and removing outliers and missing values.

[0203] "Data analysis means" refers to devices or software that have the function of analyzing preprocessed data and detecting signs of deterioration over time.

[0204] The "deterioration prediction model generation means" refers to a device or software that uses techniques such as machine learning algorithms to model the progression pattern of deterioration and predict future deterioration.

[0205] The "warning generation means" is a device or software that has the function of identifying problem areas based on the results of deterioration prediction and generating a warning.

[0206] The "proposal generation means" refers to a device or software that has the function of optimizing the timing of repair and maintenance work and generating an optimal construction plan that takes into account the budget and construction period.

[0207] A "user interface means" is a device or software that has the function of displaying proposed repair plans on a user terminal or wearable device and allowing the user to select and execute the optimal plan.

[0208] "Notification means" refers to devices or software that have the ability to send notifications to on-site staff in real time based on data collected from sensors and the generated deterioration prediction model.

[0209] The present invention relates to a system that predicts the deterioration of buildings and facilities over time and performs repairs and maintenance work at the optimal time. Specifically, the system uses a server, sensors, user terminals, wearable devices, etc. Specific embodiments of the system are described below.

[0210] System Configuration

[0211] The system includes the following major components:

[0212] 1. Server

[0213] The server includes a data collection means, a data preprocessing means, a data analysis means, a degradation prediction model generation means, a warning generation means, a proposal generation means, and a notification means. The server plays a central role in collecting and analyzing data from sensors, generating models, etc.

[0214] 2. Sensors

[0215] Data is collected from multiple sensors (e.g., temperature, humidity, and vibration sensors) installed in buildings and facilities. These sensors are used to monitor changes in the state of the objects in real time.

[0216] 3. User Device

[0217] This device displays repair plans and warnings to the user, and has a user interface function that allows the user to select and implement the proposed repair plans.

[0218] 4. Wearable devices

[0219] A device worn by field staff (e.g., smart glasses) used to receive real-time notifications.

[0220] Data collection

[0221] The server collects data from sensors installed in buildings and facilities, including information on temperature, humidity, vibration, and other factors.

[0222] Data Preprocessing

[0223] The server performs preprocessing on the collected data, detecting and removing outliers and missing values ​​to improve data quality.

[0224] Data analysis

[0225] The pre-processed data is analyzed to detect signs of aging. The server analyzes the data using algorithms such as time series analysis and clustering.

[0226] Deterioration prediction model generation

[0227] Based on the analysis results, a machine learning algorithm is run to model the deterioration progression pattern. The generated predictive model is used to predict future deterioration.

[0228] Warning generation

[0229] Based on the results of the degradation prediction model, areas where problems are likely to occur are identified and warnings are generated, which are sent in real time via user terminals or wearable devices.

[0230] Proposal generation

[0231] Using a multivariable optimization algorithm, the timing of repairs and maintenance work is optimized. The server generates an optimal construction plan taking into account budget, construction period, materials used, etc.

[0232] User Interface

[0233] Users access the system using a terminal or wearable device, check the warnings and repair plans sent from the server, select the repair plan they think is most appropriate, and carry out the specific work.

[0234] Specific examples

[0235] Example 1: Vibration detection and repair of factory equipment

[0236] 1. Data Collection

[0237] The server periodically collects data from vibration sensors installed within the factory.

[0238] 2. Data Preprocessing

[0239] An anomaly detection algorithm removes false sensor detection data.

[0240] 3. Data Analysis

[0241] The server performs time series analysis and analyzes the vibration patterns of the equipment.

[0242] 4. Deterioration prediction model generation

[0243] A machine learning algorithm is used to generate a predictive model of vibration progression.

[0244] 5. Warning generation

[0245] The server generates a warning before vibrations become abnormally high and notifies on-site staff.

[0246] 6. Proposal generation

[0247] The server will suggest the optimal timing and method for repairs to reduce vibrations, for example, informing users that maintenance is required by the end of the next month.

[0248] 7. User Interface and Notifications

[0249] On-site staff can view warnings and suggestions through smart glasses and instruct on specific repair work.

[0250] Example prompt sentence:

[0251] text

[0252] The following data has been collected from temperature, humidity, and vibration sensors in the factory. Please analyze this data to predict equipment deterioration and propose the optimal timing for repairs.

[0253] data:

[0254] Temperature: 20 ± 2 degrees

[0255] Humidity: 30 ± 5%

[0256] Vibration: 0 ± 0.1g

[0257] Data were recorded every hour.

[0258] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0259] Step 1:

[0260] Data collection

[0261] The server periodically collects data from sensors (such as temperature sensors, humidity sensors, and vibration sensors) installed in buildings and facilities. The input data from the sensors includes temperature, humidity, and vibration values. The server receives the input data from the sensors and stores it in a database. The output of this step is the raw data stored in the database.

[0262] Step 2:

[0263] Data Preprocessing

[0264] The server performs preprocessing on the raw data stored in the database. The input data may contain missing or outliers, and an outlier detection algorithm is used to detect and remove them. Specifically, outliers are detected using statistical methods and machine learning models, and missing values ​​are imputed using statistical imputation methods. The preprocessed data is output in a form that can be used for data analysis.

[0265] Step 3:

[0266] Data analysis

[0267] The server analyzes the preprocessed data. It uses the preprocessed data as input data. It applies algorithms such as time series analysis and clustering to detect abnormal equipment behavior and signs of deterioration. The server obtains the analysis results and uses them in the next step. The output of the analysis results includes data that identifies abnormal areas and indicates signs of progressing deterioration.

[0268] Step 4:

[0269] Deterioration prediction model generation

[0270] The server generates a deterioration prediction model based on the results of the data analysis. The input data includes the analyzed signs and patterns. A machine learning algorithm (e.g., random forest, LSTM, etc.) is used to train the prediction model and create a model for predicting future deterioration. The output is a prediction model of deterioration progression.

[0271] Step 5:

[0272] Warning generation

[0273] The server identifies problem areas based on the results of the degradation prediction model and generates warnings. It uses the results of the prediction model as input data. The server generates warnings when problem areas exceed a certain degradation risk and sends notifications to user terminals or wearable devices. The output is a real-time warning sent to the user.

[0274] Step 6:

[0275] Proposal generation

[0276] The server optimizes the timing of repair and maintenance work and generates an optimal construction plan that takes into account budget and construction time. Input data includes the results of the deterioration prediction model and warning data. Using a multivariable optimization algorithm, the optimal repair plan is constructed and provided to the user via a user interface. The output is a detailed repair plan.

[0277] Step 7:

[0278] User Interface and Notifications

[0279] Users access the system using a terminal or wearable device to check the warnings and repair plans sent from the server. Input data includes warning data and repair plans. The user checks the repair plan that seems most appropriate and carries out the specific repair work. Field staff can also receive real-time notifications via the wearable device. The output is a repair plan that the user and field staff can check and carry out.

[0280] The above is the specific processing flow of this system.

[0281] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0282] This invention combines a system that predicts the deterioration of buildings over time and carries out repair and maintenance work at the optimal time with an emotion engine that recognizes the user's emotions. This invention makes it possible to provide efficient repair plans while reducing the user's stress and anxiety.

[0283] To implement the present invention, four main components are required: a server, a sensor, a user terminal, and an emotion engine. The specific embodiment is shown below.

[0284] Data collection

[0285] server

[0286] It collects data in real time from various sensors in the building (temperature sensors, humidity sensors, vibration sensors, etc.) The server receives the data sent from these sensors and stores it in a database.

[0287] Data Preprocessing

[0288] server

[0289] Preprocessing is performed on the collected data, detecting outliers and missing values ​​and removing or correcting them as necessary, to create a dataset suitable for analysis.

[0290] Data analysis

[0291] server

[0292] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering are performed to find patterns of fluctuations and abnormalities in the data.

[0293] Deterioration prediction model generation

[0294] server

[0295] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[0296] Warning generation

[0297] server

[0298] Analyzes the results of the deterioration prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records them in a database.

[0299] Proposal generation

[0300] server

[0301] Optimize the timing of repair and maintenance work using a multivariate optimization algorithm. Create the optimal repair plan taking into account conditions such as budget, construction period, and materials used.

[0302] Emotion recognition by emotion engine

[0303] server

[0304] Using the camera and microphone installed on the user's device, the system recognizes emotions from the user's facial expressions and tone of voice, and uses an emotion engine to determine whether the user is feeling stressed.

[0305] Emotion-based suggestion adjustment

[0306] server

[0307] The emotion engine recognizes the user's emotional state and adjusts the content and presentation of repair suggestions based on that. For example, if the user is feeling stressed, the suggestions will be presented with more detailed explanations.

[0308] Check and execute on the user interface

[0309] User

[0310] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. Users then view the proposed plans and select the most appropriate one.

[0311] Specific examples

[0312] Example 1: Predicting and repairing cracks in exterior walls

[0313] 1. Data Collection

[0314] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[0315] 2. Data Preprocessing

[0316] An anomaly detection algorithm removes false sensor detection data.

[0317] 3. Data Analysis

[0318] Time series analysis is performed to detect crack progression patterns.

[0319] 4. Deterioration prediction model generation

[0320] Machine learning algorithms are used to generate predictive models of crack progression.

[0321] 5. Warning generation

[0322] Before the cracks become serious, the server generates a warning to notify the user.

[0323] 6. Proposal generation

[0324] Based on a multivariate optimization algorithm, it will suggest repairs that are needed within the next two months.

[0325] 7. Emotion Engine

[0326] When a user operates the device, the emotion engine recognizes the user's emotions from their facial expressions and voice.

[0327] 8. Adjusting suggestions based on emotions

[0328] If the user is stressed, the server will adjust the way the repair plan is presented, providing more detailed instructions.

[0329] 9. User Interface

[0330] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[0331] In this way, this system can predict deterioration of buildings over time and carry out repairs at the appropriate time, not only enabling the long-term maintenance of living environments but also reducing stress and anxiety for users.

[0332] The processing flow will be explained below.

[0333] Step 1:

[0334] server

[0335] It collects data in real time from multiple sensors (temperature sensors, humidity sensors, vibration sensors, etc.) installed in buildings. The sensors send the data via the Internet, and the server receives the data and stores it in a database.

[0336] Step 2:

[0337] server

[0338] Preprocessing the collected data involves running algorithms to detect outliers and missing values ​​and remove or correct them as needed, preparing a clean dataset suitable for analysis.

[0339] Step 3:

[0340] server

[0341] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering algorithms are used to identify patterns of fluctuation and abnormalities in the data.

[0342] Step 4:

[0343] server

[0344] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[0345] Step 5:

[0346] server

[0347] Analyzes the results of the degradation prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records the information in a database.

[0348] Step 6:

[0349] server

[0350] The timing of repair and maintenance work is optimized using a multivariate optimization algorithm, taking into account conditions such as budget, construction period, and materials used to generate the optimal repair plan.

[0351] Step 7:

[0352] server

[0353] The generated repair plan is sent to the user interface in cooperation with the emotion engine. The emotion engine recognizes the user's emotional state, and if the user is feeling stressed, the emotion engine provides the user with a more detailed explanation of the repair plan.

[0354] Step 8:

[0355] Terminal

[0356] The user's device (smartphone or PC) receives and displays the warnings and repair plans sent from the server. The emotion engine analyzes the user's facial expressions and tone of voice to recognize the user's emotions.

[0357] Step 9:

[0358] User

[0359] Users can check the repair plans on their device and select the most suitable one. Based on the advice and information provided by the emotion engine, users can make a repair selection with peace of mind.

[0360] Step 10:

[0361] User

[0362] Schedule specific repair work based on the selected repair plan, and contact repair contractors as necessary to request repair work.

[0363] In this way, the system can predict the deterioration of buildings over time and carry out repairs at the appropriate time and in the appropriate manner, while also recognizing the user's emotions and providing appropriate information, thereby reducing the user's stress and anxiety.

[0364] Example 2

[0365] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0366] Conventional building deterioration prediction systems are limited to predicting deterioration and proposing repair timing, and do not include work plans or proposals that take the user's emotional state into account. This can cause users to feel stressed or anxious about repair proposals, which can result in delays in appropriate repairs. Therefore, there is a need for a system that provides efficient repair plans that take the user's emotional state into account and minimizes their emotional burden.

[0367] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0368] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in the building; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the progression pattern of deterioration and predicting future deterioration; a warning generation means for identifying problem areas based on the prediction results and generating warnings; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; an emotion recognition means for recognizing the user's emotion; a proposal adjustment means for adjusting the proposal content based on the recognized emotion; and a user interface means for displaying the proposed repair plan on a user terminal and allowing the user to select and implement the optimal plan. This makes it possible to predict aging deterioration and provide an optimal repair plan while taking into account the user's emotional state.

[0369] "Data collection means" is a function for collecting data from multiple sensors installed in a building.

[0370] The "data preprocessing means" is a function that preprocesses collected data and detects and removes outliers and missing values.

[0371] The "data analysis means" is a function that analyzes the preprocessed data to detect signs of deterioration over time.

[0372] The "deterioration prediction model generation means" is a function that models the deterioration progression pattern and executes a machine learning algorithm to predict future deterioration.

[0373] The "warning generation means" is a function that identifies problem areas based on the prediction results and generates a warning.

[0374] The "proposal generation means" is a function that optimizes the timing of repair and maintenance work and generates the optimal construction plan taking into account the budget and construction period.

[0375] The "emotion recognition means" is a function for recognizing the user's emotions, and uses a camera and microphone to analyze the user's facial expressions and tone of voice.

[0376] The "proposal adjustment means" is a function that adjusts the content and presentation method of repair proposals based on the recognized emotions.

[0377] The "user interface means" is a function that displays proposed repair plans on a user terminal and allows the user to select and execute the most suitable plan.

[0378] MODE FOR CARRYING OUT THE INVENTION

[0379] The present invention combines a system that predicts the deterioration of buildings over time and performs repair and maintenance work at the optimal timing with a function that recognizes user emotions. This system makes it possible to provide efficient repair plans while reducing user stress and anxiety. Specific embodiments of the system are described below.

[0380] Hardware and software used

[0381] server

[0382] Data collection, preprocessing, analysis, model generation, warning generation, and suggestion generation

[0383] Specific software used: Python, TensorFlow, Scikit-learn, OpenCV

[0384] sensor

[0385] Temperature sensor, humidity sensor, vibration sensor

[0386] User terminal

[0387] Smartphones, PCs

[0388] Emotion Recognition Engine

[0389] Specific software used: Python, OpenCV, NVIDIA GPU (CUDA)

[0390] System configuration and functions

[0391] The server has the following functions:

[0392] 1. Data Collection Methods

[0393] The server collects data in real time from temperature, humidity, and vibration sensors installed in the building. The sensors measure environmental data from each part and send it to the server. The data is then stored in the server's database.

[0394] 2. Data preprocessing methods

[0395] The server preprocesses the collected data, detecting outliers and missing values ​​and removing or imputing them as necessary. Preprocessing involves the use of outlier detection algorithms (e.g., Z-score analysis) and data imputation algorithms (e.g., linear interpolation).

[0396] 3. Data Analysis Methods

[0397] The server analyzes the preprocessed data to detect signs of aging degradation, and uses time series analysis (e.g., ARIMA model) and clustering (e.g., K-means method) to identify patterns of fluctuation and abnormalities in the data.

[0398] 4. Deterioration prediction model generation method

[0399] Based on the analysis results, the server runs a machine learning algorithm to model the progression pattern of deterioration, using recurrent neural networks (RNNs) and random forests to generate a deterioration prediction model.

[0400] 5. Warning generation means

[0401] The server identifies problem areas based on the prediction results and generates warnings as necessary, which are stored in a database and sent to the user's device.

[0402] 6. Proposal generation means

[0403] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work, for example, generating a repair plan to be carried out within the next two months, taking into account budget and construction time.

[0404] 7. Emotion recognition means

[0405] The server uses the camera and microphone installed on the user's device to recognize the user's emotions from their facial expressions and tone of voice, and the emotion engine determines in real time whether the user is feeling stressed.

[0406] 8. Proposed adjustment measures

[0407] The server adjusts the content and presentation of repair suggestions based on the user's emotional state, acquired through emotion recognition. If the user is feeling stressed, the server displays suggestions with more detailed explanations.

[0408] User accesses the device:

[0409] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. They then view the proposed plans, select the most appropriate one, and implement it. If necessary, they can request repairs from a repair company.

[0410] Specific examples

[0411] Example 1: Predicting and repairing cracks in exterior walls

[0412] 1. Data Collection

[0413] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[0414] 2. Data Preprocessing

[0415] An anomaly detection algorithm removes false sensor detection data.

[0416] 3. Data Analysis

[0417] Time series analysis is performed to detect crack progression patterns.

[0418] 4. Deterioration prediction model generation

[0419] Machine learning algorithms are used to generate predictive models of crack progression.

[0420] 5. Warning generation

[0421] Before the cracks become serious, the server generates a warning to notify the user.

[0422] 6. Proposal generation

[0423] Based on a multivariate optimization algorithm, it will suggest repairs that are needed within the next two months.

[0424] 7. Emotion recognition means

[0425] When a user operates the device, the emotion engine recognizes the user's emotions from their facial expressions and voice.

[0426] 8. Proposed adjustment measures

[0427] If the user is stressed, the server will adjust the way the repair plan is presented, providing more detailed instructions.

[0428] 9. User Interface

[0429] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[0430] Example prompts to input to the generative AI model

[0431] "Your goal is to generate a machine learning algorithm that analyzes data from vibration sensors installed on the building's exterior walls and predicts the progression pattern of cracks. You also integrate an emotion engine to adjust the way repair plans are presented to users based on their emotional state."

[0432] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0433] Step 1:

[0434] The server collects data in real time from multiple sensors (temperature, humidity, vibration, etc.) installed in the building. The input is environmental data sent from the sensors, specifically temperature, humidity, and vibration values. The server stores this data in a database. The output is a raw dataset for preprocessing.

[0435] Step 2:

[0436] The server preprocesses the collected dataset. The input is the raw data collected in step 1. The server detects outliers and missing values, removes them using an outlier detection algorithm (e.g., Z-score analysis), and imputes missing values ​​using a missing value imputation algorithm (e.g., linear interpolation). The output is a clean dataset.

[0437] Step 3:

[0438] The server analyzes the clean dataset. The input is preprocessed data. It performs time series analysis (e.g., ARIMA model) and clustering (e.g., K-means) to detect signs of aging. Specifically, it extracts patterns of fluctuation and abnormalities in the data. The output is the analysis results.

[0439] Step 4:

[0440] The server then runs a machine learning algorithm based on the analysis results to model the deterioration progression pattern. The input is the data analysis results. A deterioration prediction model is generated using a recurrent neural network (RNN) or random forest. The output is the deterioration prediction model.

[0441] Step 5:

[0442] The server generates warnings using the results of the degradation prediction model. The input is the prediction results of the degradation prediction model. The server identifies high-risk problem areas, records warnings in a database, and sends notifications to user terminals in real time. The output is a warning message.

[0443] Step 6:

[0444] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. The inputs are warning messages and user constraints (budget, construction period, materials used, etc.). The server generates an optimal construction plan and proposes the timing of repairs and maintenance work. The output is a repair proposal.

[0445] Step 7:

[0446] The server uses an emotion recognition engine to recognize emotions from the user's facial expressions and tone of voice using the camera and microphone installed on the user's device. The input is camera video and audio data. The server determines and records the user's emotional state (e.g., stress, anxiety). The output is the user's emotional data.

[0447] Step 8:

[0448] The server adjusts the content and presentation method of repair suggestions based on the user's emotional state, obtained through emotion recognition. The input is the user's emotional data and repair suggestions. If the user is feeling stressed, the server presents repair suggestions with detailed explanations. The output is an adjusted repair suggestion.

[0449] Step 9:

[0450] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. The input is the adjusted repair proposal. The user reviews the proposal, selects the optimal plan, and executes it. The output is the implementation of the selected repair plan.

[0451] By following the above process, it is possible to predict the deterioration of a building over time in advance and provide an optimal repair plan that takes into account the user's emotional state.

[0452] (Application example 2)

[0453] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0454] In recent years, there has been an increasing demand for efficient management of building deterioration over time. However, conventional systems are limited to predicting physical deterioration and do not take into account the psychological burden on users. As a result, unexpected repair costs and prolonged construction periods have led to increased stress and anxiety among users. Furthermore, data analysis and predictive modeling to determine the appropriate timing for repairs are not fully functional. Therefore, a new system that can simultaneously effectively manage deterioration over time and reduce the psychological burden on users is needed.

[0455] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: data collection means for collecting data from multiple sensors installed in the building; data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; warning generation means for identifying problem areas based on the prediction results and generating warnings; proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; emotion engine means for recognizing emotions from the user's facial expressions and tone of voice using a camera and microphone installed in the user terminal; emotion-based proposal adjustment means for adjusting the content and presentation method of repair proposals based on the recognized user's emotional state; and user interface means for displaying proposed repair plans on the user terminal and allowing the user to select and implement the optimal plan. This makes it possible to predict building deterioration and perform effective repairs while reducing user stress and anxiety.

[0456] A "data collection means" is a device or method for collecting data from multiple sensors installed in a building.

[0457] "Data preprocessing means" refers to a device or method that preprocesses collected data and detects and removes outliers and missing values.

[0458] "Data analysis means" refers to a device or method that analyzes the pre-processed data to detect signs of aging.

[0459] The "deterioration prediction model generating means" is a device or method that models the deterioration progression pattern and predicts future deterioration.

[0460] The "warning generation means" is a device or method that identifies problem areas based on the prediction results and generates a warning.

[0461] The "proposal generation means" is a device or method that optimizes the timing of repair and maintenance work and generates an optimal construction plan that takes into account the budget and construction period.

[0462] The "emotion engine means" is a device or method that uses a camera or microphone mounted on the user terminal to recognize emotions from the user's facial expressions and tone of voice.

[0463] An "emotion-based suggestion adjustment means" is a device or method that adjusts the content and presentation of repair suggestions based on the recognized emotional state of the user.

[0464] The "user interface means" is a device or method for displaying proposed repair plans on a user terminal and allowing the user to select and execute the most suitable plan.

[0465] This invention is a system that predicts building deterioration and provides repair plans based on the user's emotional state. The system includes four main components: a server, sensors, a user terminal, and an emotion engine.

[0466] Data collection

[0467] The server collects data in real time from multiple sensors installed in the building, such as temperature sensors, humidity sensors, and vibration sensors, allowing the server to understand the current state of the building and secure the necessary data.

[0468] Data Preprocessing

[0469] The server performs preprocessing on the collected data, detecting outliers and missing values ​​and correcting or removing data as needed. This process results in a clean dataset suitable for data analysis.

[0470] Data analysis

[0471] The server analyzes the preprocessed data to detect signs of building deterioration over time, using time series analysis and clustering algorithms to identify patterns of fluctuations and anomalies in the data.

[0472] Deterioration prediction model generation

[0473] Based on the analysis results, the server runs a machine learning algorithm to model the progression of deterioration, which can then be used to predict future deterioration locations.

[0474] Warning generation

[0475] The server analyzes the results of the degradation prediction model and identifies areas where problems are likely to occur. It generates warnings for problem areas and records them in a database. Users are notified in real time.

[0476] Proposal generation

[0477] The server uses a multivariate optimization algorithm to optimize the timing of repair and maintenance work, taking into account factors such as budget, construction time, and materials used to create the optimal repair plan.

[0478] Emotion recognition by emotion engine

[0479] The system uses a camera and microphone installed on the user's device to recognize emotions from the user's facial expressions and tone of voice. The server's emotion engine analyzes this data and determines whether the user is feeling stressed.

[0480] Emotion-based suggestion adjustment

[0481] The server adjusts the repair suggestions and how they are presented based on the results of the emotion engine, for example, providing more detailed and thoughtful explanations if the user indicates high stress levels.

[0482] User Interface

[0483] The user's device displays the warnings and repair proposals sent from the server. The user can view the proposals and select and implement the repair plan that best suits them. For example, if cracks are predicted in the exterior wall, a notification will appear on the user's device saying, "Cracks have been detected in the exterior wall. Repairs are required," and the user can check the repair plan in detail.

[0484] Prompt Sentence Examples

[0485] "Describe a program that uses smart glasses to display real-time emotion recognition and maintenance information within a building. The program uses a pre-trained emotion recognition model and uses OpenCV for face detection. The program should also include the ability to display repair suggestions based on the user's emotional state."

[0486] This system makes it possible to predict building deterioration and carry out appropriate repairs, while also reducing the psychological burden on users.

[0487] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0488] Step 1:

[0489] The server collects data in real time from various sensors installed in the building (e.g., temperature, humidity, and vibration sensors). The collected data is input into the server and stored in a database. This allows the current status of the building to be monitored at any time.

[0490] Step 2:

[0491] The server performs pre-processing on the collected data. During this pre-processing stage, outliers and missing values ​​are detected and corrected or removed as necessary. For example, if abnormally high temperature data is detected, it is removed to generate a clean data set. This clean data is then sent to the next analysis step.

[0492] Step 3:

[0493] The server analyzes the preprocessed data to detect signs of aging deterioration. Specifically, it uses time series analysis and clustering algorithms. For example, if vibration sensor data from a specific part of a building shows a certain pattern, that pattern can be analyzed to find signs of deterioration. The results of this analysis become the input data.

[0494] Step 4:

[0495] The server then runs a machine learning algorithm based on the analysis results to model the deterioration progression pattern. This model is then used to predict the deterioration progression of the building. For example, it predicts future deterioration progression patterns based on past data. This predicted data is then input into the next step.

[0496] Step 5:

[0497] The server analyzes the output of the deterioration prediction model and identifies areas where problems are likely to occur. At this stage, the server generates a warning, records it in a database, and sends notifications to users in real time. For example, if cracks are progressing in the exterior wall, the server will generate a warning such as "Cracks are progressing in the exterior wall."

[0498] Step 6:

[0499] The server uses a multivariate optimization algorithm to optimize the timing of repair and maintenance work. It creates an optimal repair plan taking into account conditions such as budget, construction period, and materials used. For example, it proposes a repair plan that can be carried out within the shortest construction period and budget. This proposed plan is then sent to the user's device.

[0500] Step 7:

[0501] The server's emotion engine uses the camera and microphone installed on the user's device to recognize emotions from the user's facial expressions and tone of voice. For example, when a user checks a repair plan, the camera reads their facial expressions and the microphone analyzes their tone of voice to determine whether the user is feeling stressed. The recognized emotion data is input into the next step.

[0502] Step 8:

[0503] The server adjusts the repair suggestion content and presentation method based on the recognition results of the emotion engine. For example, if the user is in a high stress state, the server will provide detailed and thorough repair instructions to reduce anxiety. The adjustment results are then sent to the user's device.

[0504] Step 9:

[0505] The user's device displays the warnings and repair proposals sent from the server. The user can view the proposed plans and select and implement the repair plan that best suits them. For example, a notification will be displayed saying, "Cracks have been detected in the exterior wall. Repairs are required." The user can check the detailed information and select a repair plan.

[0506] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0507] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0508] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0509] [Second embodiment]

[0510] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0511] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0512] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0513] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0514] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0515] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0516] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0517] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0518] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0519] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0520] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0521] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0522] The present invention relates to a system that predicts the deterioration of buildings over time and performs repair and maintenance work at the optimal timing. To implement the present invention, three main components are required: a server, a sensor, and a user terminal. A specific embodiment of the system is shown below.

[0523] This system collects data from multiple sensors installed in buildings and predicts deterioration over time based on the collected data. The server analyzes the data and predicts deterioration, and based on the results, sends warnings and repair proposals to the user. The user can then review the proposals and select and implement the optimal repair plan.

[0524] Data collection

[0525] server

[0526] Data is collected periodically from various sensors in the building (temperature sensors, humidity sensors, vibration sensors, etc.) The server receives the data sent from these sensors and stores it in a database.

[0527] Data Preprocessing

[0528] server

[0529] Preprocessing is performed on the collected data, specifically detecting outliers and missing values ​​and removing or correcting them as necessary. This process improves data quality.

[0530] Data analysis

[0531] server

[0532] The pre-processed data is analyzed to detect signs of aging deterioration. Various algorithms (e.g., time series analysis, clustering) are used for data analysis.

[0533] Deterioration prediction model generation

[0534] server

[0535] Based on the analysis results, a machine learning algorithm is implemented to model the deterioration progression pattern. The deterioration prediction model is used to predict future deterioration.

[0536] Warning generation

[0537] server

[0538] Based on the results of the degradation prediction model, areas where problems are likely to occur are identified and warnings are generated, which are notified to the user in real time.

[0539] Proposal generation

[0540] server

[0541] The system uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. The proposed repair plan takes into account budget, construction time, materials used, and other factors.

[0542] Check and execute on the user interface

[0543] User

[0544] Users access the system using a terminal, check the warnings sent from the server and the proposed repair plans, select the repair plan they think is most appropriate, and carry out the specific work.

[0545] Specific examples

[0546] Example 1: Predicting and repairing cracks in exterior walls

[0547] 1. Data Collection

[0548] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[0549] 2. Data Preprocessing

[0550] An anomaly detection algorithm removes false sensor detection data.

[0551] 3. Data Analysis

[0552] Time series analysis is performed to detect crack progression patterns.

[0553] 4. Deterioration prediction model generation

[0554] Machine learning algorithms are used to generate predictive models of crack progression.

[0555] 5. Warning generation

[0556] Before the cracks become serious, the server generates a warning to notify the user.

[0557] 6. Proposal generation

[0558] The server will suggest the best time and method for repairing the crack, for example, indicating that repairs are needed within the next two months.

[0559] 7. User Interface

[0560] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[0561] In this way, this system predicts the deterioration of buildings over time in advance and takes preventative measures at the appropriate time, making it possible to maintain the living environment over the long term.

[0562] The processing flow will be explained below.

[0563] Step 1:

[0564] server

[0565] Data is collected in real time from multiple sensors installed in buildings (temperature sensors, humidity sensors, vibration sensors, etc.) The data is sent via the Internet, and the server receives it and stores it in a database.

[0566] Step 2:

[0567] server

[0568] Preprocessing the collected data involves running algorithms to detect outliers and missing values ​​and remove or correct them as needed, creating a dataset suitable for analysis.

[0569] Step 3:

[0570] server

[0571] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering are performed to find patterns of fluctuations and abnormalities in the data.

[0572] Step 4:

[0573] server

[0574] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[0575] Step 5:

[0576] server

[0577] Analyzes the results of the deterioration prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records them in a database.

[0578] Step 6:

[0579] server

[0580] Optimize the timing of repair and maintenance work using a multivariate optimization algorithm. Create the optimal repair plan taking into account conditions such as budget, construction period, and materials used.

[0581] Step 7:

[0582] server

[0583] The generated repair plan is sent to the user interface, which includes the details of the repair plan for easy user review.

[0584] Step 8:

[0585] User

[0586] Users access the system from their devices (smartphones or PCs) and check the warnings and repair proposals sent from the server. Users can then view the proposed plans and select the most appropriate one.

[0587] Step 9:

[0588] User

[0589] Based on the repair plan selected by the user, the system schedules specific work, contacts repair contractors as necessary, and carries out the actual repair and maintenance work.

[0590] Example 1

[0591] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0592] In recent years, problems caused by the deterioration of buildings over time have been increasing. This often results in unexpected repair costs and the need for long-term construction work. However, with current systems, it is difficult to accurately predict the progression of deterioration in advance and carry out repair and maintenance work at the optimal time. Therefore, there is a need for a system that can accurately predict the deterioration of buildings over time and propose repair and maintenance work at the optimal time, thereby preventing problems before they occur and optimizing costs.

[0593] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0594] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in a building; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; a notification means for notifying a user terminal of the prediction results in real time based on the constructed deterioration prediction model; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; a user interface means for displaying the proposed repair plans on the user terminal and allowing the user to select and execute the optimal plan; and a deterioration progress prediction algorithm means for predicting the progression of deterioration of a building using a machine learning algorithm from the collected data. This makes it possible to accurately predict building deterioration and perform repair and maintenance work at the optimal time.

[0595] A "data collection means" is a means for collecting data from multiple sensors installed in a building.

[0596] The "data preprocessing means" is a means for preprocessing collected data to detect and remove outliers and missing values.

[0597] The "data analysis means" is a means for analyzing the pre-processed data to detect signs of aging deterioration.

[0598] The "deterioration prediction model generating means" is a means for modeling the deterioration progression pattern and predicting future deterioration.

[0599] The "notification means" is a means for notifying the user terminal of the prediction results in real time based on the constructed deterioration prediction model.

[0600] The "proposal generation means" is a means for optimizing the timing of repair and maintenance work and generating an optimal construction plan that takes into account the budget and construction period.

[0601] The "user interface means" is a means for displaying proposed repair plans on a user terminal and allowing the user to select and execute the most suitable plan.

[0602] The "deterioration progress prediction algorithm means" is a means for predicting the deterioration progress of a building using a machine learning algorithm from collected data.

[0603] The present invention relates to a system for predicting deterioration of a building over time and performing repair and maintenance work at an optimal timing. Specific embodiments for carrying out the present invention are described below.

[0604] The system consists of three main components: a server, sensors, and user terminals. The entire system runs through the following stages: data collection, data preprocessing, data analysis, degradation prediction model generation, warning generation, suggestion generation, and user interface.

[0605] Data collection

[0606] The server periodically collects data from multiple sensors installed in the building, such as temperature sensors, humidity sensors, and vibration sensors. This collected data is then stored in a database by the server. The database used is MySQL.

[0607] Data Preprocessing

[0608] The server preprocesses the collected data to detect and remove outliers and missing values. It uses the PyOD library to detect outliers and imputes missing values ​​with the most recent valid data, thereby improving data quality.

[0609] Data analysis

[0610] Based on the preprocessed data, the server analyzes signs of aging deterioration. For the analysis, it uses "scikit-learn" and "statsmodels" to perform time series analysis and clustering. For example, it applies the "ARIMA model" to vibration data from building exterior walls to detect fluctuation patterns.

[0611] Deterioration prediction model generation

[0612] The server generates a deterioration prediction model based on the results of the data analysis. This model is generated using a deep learning algorithm using TensorFlow, specifically LSTM (Long Short Term Memory). The generated model is used to predict future deterioration and can accurately capture the progression pattern of deterioration.

[0613] Warning generation

[0614] The server identifies problem areas and generates warnings based on the results of the deterioration prediction model. For example, if a crack in an exterior wall exceeds 5 mm in depth, it will generate a warning as a serious error. This warning is sent to the user in real time via email or SMS using a messaging API such as "Twilio."

[0615] Proposal generation

[0616] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. Proposed repair plans are created taking into account budget, construction time, materials used, and more. The SciPy library is used for optimization, generating specific suggestions, such as "repairs are required within the next two months."

[0617] User Interface

[0618] Users access the system using a terminal and check the warnings and proposed repair plans sent from the server. The user interface is built using React and Angular, and information can be viewed in real time. For example, users can click the "Accept repair proposal" button, which will send a notification to the repair company.

[0619] Specific examples

[0620] Example 1: Predicting and repairing cracks in exterior walls

[0621] 1. Data Collection

[0622] The server collects crack data every hour from vibration sensors installed on the exterior walls.

[0623] Example: "Sensor ID: 123, Time: 10:00, Vibration Data: 0.05"

[0624] 2. Data Preprocessing

[0625] An outlier (0.10) is detected, removed, and supplemented with the latest valid data (0.05).

[0626] 3. Data Analysis

[0627] The server applies the ARIMA model and generates a forecast using the most recent three months of data.

[0628] 4. Deterioration prediction model generation

[0629] Generate a degradation prediction model using the SaaS deep learning algorithm LSTM. Example: "LSTM model generation complete, accuracy: 95%."

[0630] 5. Warning generation

[0631] Send a warning to the user if the crack depth exceeds 5mm (10:05)

[0632] 6. Proposal generation

[0633] The proposed implementation plan is to repair within the next two months, with materials A and construction time of 3 days.

[0634] 7. User Interface

[0635] The server sends a notification to the contractor when the user confirms the proposal and clicks the "Request Repair" button (10:10)

[0636] Example of input prompt for generative AI model

[0637] "Please predict cracks in the exterior walls of buildings and suggest appropriate repair timing."

[0638] This enables the system to accurately predict deterioration of buildings over time and carry out repairs and maintenance work at the optimal time, preventing problems before they occur and optimizing costs.

[0639] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0640] Step 1: Data collection

[0641] server

[0642] The server collects data from temperature sensors, humidity sensors, vibration sensors, etc. installed in the building. This data collection is done periodically, for example, every hour, and data from the sensors is sent to the server. The collected data includes information such as the sensor ID, time, and measurement value. The server receives this data and stores it in a database. The database used is "MySQL."

[0643] Input: Measurement data from sensors (temperature, humidity, vibration)

[0644] Output: Sensor data stored in a database

[0645] Step 2: Data Preprocessing

[0646] server

[0647] The server preprocesses the collected data, including detecting and removing outliers and missing values. For example, it uses the PyOD library to detect outliers and removes outliers with data points exceeding three sigma. Additionally, if missing values ​​are found, they are imputed with the most recent valid data.

[0648] Input: Sensor data stored in a database

[0649] Output: Pre-processed high-quality sensor data

[0650] Step 3: Data analysis

[0651] server

[0652] The server analyzes the preprocessed data. For this analysis, it uses "scikit-learn" and "statsmodels" to perform time series analysis and clustering. For example, for vibration data on building exterior walls, it uses the "ARIMA model" to analyze data fluctuation patterns.

[0653] Input: Preprocessed sensor data

[0654] Output: Analysis results (deterioration progression pattern, etc.)

[0655] Step 4: Deterioration prediction model generation

[0656] server

[0657] The server generates a degradation prediction model based on the results of the data analysis. This model is generated using a deep learning algorithm using TensorFlow, specifically LSTM (Long Short Term Memory). LSTM is suitable for building prediction models based on time series data.

[0658] Input: Analysis results

[0659] Output: Deterioration prediction model (model for predicting future deterioration)

[0660] Step 5: Generate warnings

[0661] server

[0662] The server identifies problem areas and generates warnings based on the results of the deterioration prediction model. For example, if a crack in an exterior wall exceeds 5 mm in depth, it will generate a warning as a serious error. This warning is sent to the user in real time via email or SMS using a messaging API such as "Twilio."

[0663] Input: Results of the deterioration prediction model

[0664] Output: A warning notice to the user

[0665] Step 6: Proposal Generation

[0666] server

[0667] The server uses multivariate optimization algorithms to optimize the timing of repairs and maintenance work. Proposals are created taking into account budget, construction time, materials used, etc. For example, it uses the SciPy optimization algorithm to notify you that repairs are needed within the next two months.

[0668] Input: Warning Notification

[0669] Output: Optimized repair plan

[0670] Step 7: Check and execute in the user interface

[0671] User

[0672] Users access the system using a terminal and check the warnings and proposed repair plans sent from the server. The user interface is built using React and Angular, allowing users to check information in real time. By clicking the "Accept Repair Proposal" button, an automatic notification is sent to the repair company.

[0673] Input: Warning and suggestion notifications from the server

[0674] Output: Confirmation of repair execution (execution of the repair plan selected by the user)

[0675] (Application example 1)

[0676] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0677] Conventional building and equipment maintenance management systems had low accuracy in predicting deterioration, making it difficult to carry out repairs at the appropriate time. Furthermore, there was a lack of means for on-site staff to receive signs of deterioration and repair proposals in real time. This increased the risk of sudden breakdowns and malfunctions in buildings and equipment, leading to increased maintenance costs and reduced work efficiency. To solve these issues, a system that can more accurately predict deterioration and make more efficient repair proposals is needed.

[0678] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0679] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in a building or facility; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; a warning generation means for identifying problem areas based on the prediction results and generating warnings; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; a user interface means for displaying the proposed repair plan on a user terminal or wearable device and allowing the user to select and execute the optimal plan; and a notification means for sending notifications to on-site staff in real time based on the data collected from the sensors and the generated deterioration prediction model. This enables highly accurate deterioration predictions and efficient repair proposals, preventing unexpected breakdowns of buildings and facilities, reducing maintenance costs, and improving work efficiency.

[0680] "Data collection means" refers to devices or software that function to collect data from multiple sensors installed in buildings or facilities.

[0681] "Data preprocessing means" refers to devices or software that have the function of preprocessing collected data and detecting and removing outliers and missing values.

[0682] "Data analysis means" refers to devices or software that have the function of analyzing preprocessed data and detecting signs of deterioration over time.

[0683] The "deterioration prediction model generation means" refers to a device or software that uses techniques such as machine learning algorithms to model the progression pattern of deterioration and predict future deterioration.

[0684] The "warning generation means" is a device or software that has the function of identifying problem areas based on the results of deterioration prediction and generating a warning.

[0685] The "proposal generation means" refers to a device or software that has the function of optimizing the timing of repair and maintenance work and generating an optimal construction plan that takes into account the budget and construction period.

[0686] A "user interface means" is a device or software that has the function of displaying proposed repair plans on a user terminal or wearable device and allowing the user to select and execute the optimal plan.

[0687] "Notification means" refers to devices or software that have the ability to send notifications to on-site staff in real time based on data collected from sensors and the generated deterioration prediction model.

[0688] The present invention relates to a system that predicts the deterioration of buildings and facilities over time and performs repairs and maintenance work at the optimal time. Specifically, the system uses a server, sensors, user terminals, wearable devices, etc. Specific embodiments of the system are described below.

[0689] System Configuration

[0690] The system includes the following major components:

[0691] 1. Server

[0692] The server includes a data collection means, a data preprocessing means, a data analysis means, a degradation prediction model generation means, a warning generation means, a proposal generation means, and a notification means. The server plays a central role in collecting and analyzing data from sensors, generating models, etc.

[0693] 2. Sensors

[0694] Data is collected from multiple sensors (e.g., temperature, humidity, and vibration sensors) installed in buildings and facilities. These sensors are used to monitor changes in the state of the objects in real time.

[0695] 3. User Device

[0696] This device displays repair plans and warnings to the user, and has a user interface function that allows the user to select and implement the proposed repair plans.

[0697] 4. Wearable devices

[0698] A device worn by field staff (e.g., smart glasses) used to receive real-time notifications.

[0699] Data collection

[0700] The server collects data from sensors installed in buildings and facilities, including information on temperature, humidity, vibration, and other factors.

[0701] Data Preprocessing

[0702] The server performs preprocessing on the collected data, detecting and removing outliers and missing values ​​to improve data quality.

[0703] Data analysis

[0704] The pre-processed data is analyzed to detect signs of aging. The server analyzes the data using algorithms such as time series analysis and clustering.

[0705] Deterioration prediction model generation

[0706] Based on the analysis results, a machine learning algorithm is run to model the deterioration progression pattern. The generated predictive model is used to predict future deterioration.

[0707] Warning generation

[0708] Based on the results of the degradation prediction model, areas where problems are likely to occur are identified and warnings are generated, which are sent in real time via user terminals or wearable devices.

[0709] Proposal generation

[0710] Using a multivariable optimization algorithm, the timing of repairs and maintenance work is optimized. The server generates an optimal construction plan taking into account budget, construction period, materials used, etc.

[0711] User Interface

[0712] Users access the system using a terminal or wearable device, check the warnings and repair plans sent from the server, select the repair plan they think is most appropriate, and carry out the specific work.

[0713] Specific examples

[0714] Example 1: Vibration detection and repair of factory equipment

[0715] 1. Data Collection

[0716] The server periodically collects data from vibration sensors installed within the factory.

[0717] 2. Data Preprocessing

[0718] An anomaly detection algorithm removes false sensor detection data.

[0719] 3. Data Analysis

[0720] The server performs time series analysis and analyzes the vibration patterns of the equipment.

[0721] 4. Deterioration prediction model generation

[0722] A machine learning algorithm is used to generate a predictive model of vibration progression.

[0723] 5. Warning generation

[0724] The server generates a warning before vibrations become abnormally high and notifies on-site staff.

[0725] 6. Proposal generation

[0726] The server will suggest the optimal timing and method for repairs to reduce vibrations, for example, informing users that maintenance is required by the end of the next month.

[0727] 7. User Interface and Notifications

[0728] On-site staff can view warnings and suggestions through smart glasses and instruct on specific repair work.

[0729] Example prompt sentence:

[0730] text

[0731] The following data has been collected from temperature, humidity, and vibration sensors in the factory. Please analyze this data to predict equipment deterioration and propose the optimal timing for repairs.

[0732] data:

[0733] Temperature: 20 ± 2 degrees

[0734] Humidity: 30 ± 5%

[0735] Vibration: 0 ± 0.1g

[0736] Data were recorded every hour.

[0737] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0738] Step 1:

[0739] Data collection

[0740] The server periodically collects data from sensors (such as temperature sensors, humidity sensors, and vibration sensors) installed in buildings and facilities. The input data from the sensors includes temperature, humidity, and vibration values. The server receives the input data from the sensors and stores it in a database. The output of this step is the raw data stored in the database.

[0741] Step 2:

[0742] Data Preprocessing

[0743] The server performs preprocessing on the raw data stored in the database. The input data may contain missing or outliers, and an outlier detection algorithm is used to detect and remove them. Specifically, outliers are detected using statistical methods and machine learning models, and missing values ​​are imputed using statistical imputation methods. The preprocessed data is output in a form that can be used for data analysis.

[0744] Step 3:

[0745] Data analysis

[0746] The server analyzes the preprocessed data. It uses the preprocessed data as input data. It applies algorithms such as time series analysis and clustering to detect abnormal equipment behavior and signs of deterioration. The server obtains the analysis results and uses them in the next step. The output of the analysis results includes data that identifies abnormal areas and indicates signs of progressing deterioration.

[0747] Step 4:

[0748] Deterioration prediction model generation

[0749] The server generates a deterioration prediction model based on the results of the data analysis. The input data includes the analyzed signs and patterns. A machine learning algorithm (e.g., random forest, LSTM, etc.) is used to train the prediction model and create a model for predicting future deterioration. The output is a prediction model of deterioration progression.

[0750] Step 5:

[0751] Warning generation

[0752] The server identifies problem areas based on the results of the degradation prediction model and generates warnings. It uses the results of the prediction model as input data. The server generates warnings when problem areas exceed a certain degradation risk and sends notifications to user terminals or wearable devices. The output is a real-time warning sent to the user.

[0753] Step 6:

[0754] Proposal generation

[0755] The server optimizes the timing of repair and maintenance work and generates an optimal construction plan that takes into account budget and construction time. Input data includes the results of the deterioration prediction model and warning data. Using a multivariable optimization algorithm, the optimal repair plan is constructed and provided to the user via a user interface. The output is a detailed repair plan.

[0756] Step 7:

[0757] User Interface and Notifications

[0758] Users access the system using a terminal or wearable device to check the warnings and repair plans sent from the server. Input data includes warning data and repair plans. The user checks the repair plan that seems most appropriate and carries out the specific repair work. Field staff can also receive real-time notifications via the wearable device. The output is a repair plan that the user and field staff can check and carry out.

[0759] The above is the specific processing flow of this system.

[0760] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0761] This invention combines a system that predicts the deterioration of buildings over time and carries out repair and maintenance work at the optimal time with an emotion engine that recognizes the user's emotions. This invention makes it possible to provide efficient repair plans while reducing the user's stress and anxiety.

[0762] To implement the present invention, four main components are required: a server, a sensor, a user terminal, and an emotion engine. The specific embodiment is shown below.

[0763] Data collection

[0764] server

[0765] It collects data in real time from various sensors in the building (temperature sensors, humidity sensors, vibration sensors, etc.) The server receives the data sent from these sensors and stores it in a database.

[0766] Data Preprocessing

[0767] server

[0768] Preprocessing is performed on the collected data, detecting outliers and missing values ​​and removing or correcting them as necessary, to create a dataset suitable for analysis.

[0769] Data analysis

[0770] server

[0771] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering are performed to find patterns of fluctuations and abnormalities in the data.

[0772] Deterioration prediction model generation

[0773] server

[0774] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[0775] Warning generation

[0776] server

[0777] Analyzes the results of the deterioration prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records them in a database.

[0778] Proposal generation

[0779] server

[0780] Optimize the timing of repair and maintenance work using a multivariate optimization algorithm. Create the optimal repair plan taking into account conditions such as budget, construction period, and materials used.

[0781] Emotion recognition by emotion engine

[0782] server

[0783] Using the camera and microphone installed on the user's device, the system recognizes emotions from the user's facial expressions and tone of voice, and uses an emotion engine to determine whether the user is feeling stressed.

[0784] Emotion-based suggestion adjustment

[0785] server

[0786] The emotion engine recognizes the user's emotional state and adjusts the content and presentation of repair suggestions based on that. For example, if the user is feeling stressed, the suggestions will be presented with more detailed explanations.

[0787] Check and execute on the user interface

[0788] User

[0789] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. Users then view the proposed plans and select the most appropriate one.

[0790] Specific examples

[0791] Example 1: Predicting and repairing cracks in exterior walls

[0792] 1. Data Collection

[0793] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[0794] 2. Data Preprocessing

[0795] An anomaly detection algorithm removes false sensor detection data.

[0796] 3. Data Analysis

[0797] Time series analysis is performed to detect crack progression patterns.

[0798] 4. Deterioration prediction model generation

[0799] Machine learning algorithms are used to generate predictive models of crack progression.

[0800] 5. Warning generation

[0801] Before the cracks become serious, the server generates a warning to notify the user.

[0802] 6. Proposal generation

[0803] Based on a multivariate optimization algorithm, it will suggest repairs that are needed within the next two months.

[0804] 7. Emotion Engine

[0805] When a user operates the device, the emotion engine recognizes the user's emotions from their facial expressions and voice.

[0806] 8. Adjusting suggestions based on emotions

[0807] If the user is stressed, the server will adjust the way the repair plan is presented, providing more detailed instructions.

[0808] 9. User Interface

[0809] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[0810] In this way, this system can predict deterioration of buildings over time and carry out repairs at the appropriate time, not only enabling the long-term maintenance of living environments but also reducing stress and anxiety for users.

[0811] The processing flow will be explained below.

[0812] Step 1:

[0813] server

[0814] It collects data in real time from multiple sensors (temperature sensors, humidity sensors, vibration sensors, etc.) installed in buildings. The sensors send the data via the Internet, and the server receives the data and stores it in a database.

[0815] Step 2:

[0816] server

[0817] Preprocessing the collected data involves running algorithms to detect outliers and missing values ​​and remove or correct them as needed, preparing a clean dataset suitable for analysis.

[0818] Step 3:

[0819] server

[0820] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering algorithms are used to identify patterns of fluctuation and abnormalities in the data.

[0821] Step 4:

[0822] server

[0823] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[0824] Step 5:

[0825] server

[0826] Analyzes the results of the degradation prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records the information in a database.

[0827] Step 6:

[0828] server

[0829] The timing of repair and maintenance work is optimized using a multivariate optimization algorithm, taking into account conditions such as budget, construction period, and materials used to generate the optimal repair plan.

[0830] Step 7:

[0831] server

[0832] The generated repair plan is sent to the user interface in cooperation with the emotion engine. The emotion engine recognizes the user's emotional state, and if the user is feeling stressed, the emotion engine provides the user with a more detailed explanation of the repair plan.

[0833] Step 8:

[0834] Terminal

[0835] The user's device (smartphone or PC) receives and displays the warnings and repair plans sent from the server. The emotion engine analyzes the user's facial expressions and tone of voice to recognize the user's emotions.

[0836] Step 9:

[0837] User

[0838] Users can check the repair plans on their device and select the most suitable one. Based on the advice and information provided by the emotion engine, users can make a repair selection with peace of mind.

[0839] Step 10:

[0840] User

[0841] Schedule specific repair work based on the selected repair plan, and contact repair contractors as necessary to request repair work.

[0842] In this way, the system can predict the deterioration of buildings over time and carry out repairs at the appropriate time and in the appropriate manner, while also recognizing the user's emotions and providing appropriate information, thereby reducing the user's stress and anxiety.

[0843] Example 2

[0844] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0845] Conventional building deterioration prediction systems are limited to predicting deterioration and proposing repair timing, and do not include work plans or proposals that take the user's emotional state into account. This can cause users to feel stressed or anxious about repair proposals, which can result in delays in appropriate repairs. Therefore, there is a need for a system that provides efficient repair plans that take the user's emotional state into account and minimizes their emotional burden.

[0846] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0847] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in the building; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the progression pattern of deterioration and predicting future deterioration; a warning generation means for identifying problem areas based on the prediction results and generating warnings; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; an emotion recognition means for recognizing the user's emotion; a proposal adjustment means for adjusting the proposal content based on the recognized emotion; and a user interface means for displaying the proposed repair plan on a user terminal and allowing the user to select and implement the optimal plan. This makes it possible to predict aging deterioration and provide an optimal repair plan while taking into account the user's emotional state.

[0848] "Data collection means" is a function for collecting data from multiple sensors installed in a building.

[0849] The "data preprocessing means" is a function that preprocesses collected data and detects and removes outliers and missing values.

[0850] The "data analysis means" is a function that analyzes the preprocessed data to detect signs of deterioration over time.

[0851] The "deterioration prediction model generation means" is a function that models the deterioration progression pattern and executes a machine learning algorithm to predict future deterioration.

[0852] The "warning generation means" is a function that identifies problem areas based on the prediction results and generates a warning.

[0853] The "proposal generation means" is a function that optimizes the timing of repair and maintenance work and generates the optimal construction plan taking into account the budget and construction period.

[0854] The "emotion recognition means" is a function for recognizing the user's emotions, and uses a camera and microphone to analyze the user's facial expressions and tone of voice.

[0855] The "proposal adjustment means" is a function that adjusts the content and presentation method of repair proposals based on the recognized emotions.

[0856] The "user interface means" is a function that displays proposed repair plans on a user terminal and allows the user to select and execute the most suitable plan.

[0857] MODE FOR CARRYING OUT THE INVENTION

[0858] The present invention combines a system that predicts the deterioration of buildings over time and performs repair and maintenance work at the optimal timing with a function that recognizes user emotions. This system makes it possible to provide efficient repair plans while reducing user stress and anxiety. Specific embodiments of the system are described below.

[0859] Hardware and software used

[0860] server

[0861] Data collection, preprocessing, analysis, model generation, warning generation, and suggestion generation

[0862] Specific software used: Python, TensorFlow, Scikit-learn, OpenCV

[0863] sensor

[0864] Temperature sensor, humidity sensor, vibration sensor

[0865] User terminal

[0866] Smartphones, PCs

[0867] Emotion Recognition Engine

[0868] Specific software used: Python, OpenCV, NVIDIA GPU (CUDA)

[0869] System configuration and functions

[0870] The server has the following functions:

[0871] 1. Data Collection Methods

[0872] The server collects data in real time from temperature, humidity, and vibration sensors installed in the building. The sensors measure environmental data from each part and send it to the server. The data is then stored in the server's database.

[0873] 2. Data preprocessing methods

[0874] The server preprocesses the collected data, detecting outliers and missing values ​​and removing or imputing them as necessary. Preprocessing involves the use of outlier detection algorithms (e.g., Z-score analysis) and data imputation algorithms (e.g., linear interpolation).

[0875] 3. Data Analysis Methods

[0876] The server analyzes the preprocessed data to detect signs of aging degradation, and uses time series analysis (e.g., ARIMA model) and clustering (e.g., K-means method) to identify patterns of fluctuation and abnormalities in the data.

[0877] 4. Deterioration prediction model generation method

[0878] Based on the analysis results, the server runs a machine learning algorithm to model the progression pattern of deterioration, using recurrent neural networks (RNNs) and random forests to generate a deterioration prediction model.

[0879] 5. Warning generation means

[0880] The server identifies problem areas based on the prediction results and generates warnings as necessary, which are stored in a database and sent to the user's device.

[0881] 6. Proposal generation means

[0882] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work, for example, generating a repair plan to be carried out within the next two months, taking into account budget and construction time.

[0883] 7. Emotion recognition means

[0884] The server uses the camera and microphone installed on the user's device to recognize the user's emotions from their facial expressions and tone of voice, and the emotion engine determines in real time whether the user is feeling stressed.

[0885] 8. Proposed adjustment measures

[0886] The server adjusts the content and presentation of repair suggestions based on the user's emotional state, acquired through emotion recognition. If the user is feeling stressed, the server displays suggestions with more detailed explanations.

[0887] User accesses the device:

[0888] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. They then view the proposed plans, select the most appropriate one, and implement it. If necessary, they can request repairs from a repair company.

[0889] Specific examples

[0890] Example 1: Predicting and repairing cracks in exterior walls

[0891] 1. Data Collection

[0892] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[0893] 2. Data Preprocessing

[0894] An anomaly detection algorithm removes false sensor detection data.

[0895] 3. Data Analysis

[0896] Time series analysis is performed to detect crack progression patterns.

[0897] 4. Deterioration prediction model generation

[0898] Machine learning algorithms are used to generate predictive models of crack progression.

[0899] 5. Warning generation

[0900] Before the cracks become serious, the server generates a warning to notify the user.

[0901] 6. Proposal generation

[0902] Based on a multivariate optimization algorithm, it will suggest repairs that are needed within the next two months.

[0903] 7. Emotion recognition means

[0904] When a user operates the device, the emotion engine recognizes the user's emotions from their facial expressions and voice.

[0905] 8. Proposed adjustment measures

[0906] If the user is stressed, the server will adjust the way the repair plan is presented, providing more detailed instructions.

[0907] 9. User Interface

[0908] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[0909] Example prompts to input to the generative AI model

[0910] "Your goal is to generate a machine learning algorithm that analyzes data from vibration sensors installed on the building's exterior walls and predicts the progression pattern of cracks. You also integrate an emotion engine to adjust the way repair plans are presented to users based on their emotional state."

[0911] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0912] Step 1:

[0913] The server collects data in real time from multiple sensors (temperature, humidity, vibration, etc.) installed in the building. The input is environmental data sent from the sensors, specifically temperature, humidity, and vibration values. The server stores this data in a database. The output is a raw dataset for preprocessing.

[0914] Step 2:

[0915] The server preprocesses the collected dataset. The input is the raw data collected in step 1. The server detects outliers and missing values, removes them using an outlier detection algorithm (e.g., Z-score analysis), and imputes missing values ​​using a missing value imputation algorithm (e.g., linear interpolation). The output is a clean dataset.

[0916] Step 3:

[0917] The server analyzes the clean dataset. The input is preprocessed data. It performs time series analysis (e.g., ARIMA model) and clustering (e.g., K-means) to detect signs of aging. Specifically, it extracts patterns of fluctuation and abnormalities in the data. The output is the analysis results.

[0918] Step 4:

[0919] The server then runs a machine learning algorithm based on the analysis results to model the deterioration progression pattern. The input is the data analysis results. A deterioration prediction model is generated using a recurrent neural network (RNN) or random forest. The output is the deterioration prediction model.

[0920] Step 5:

[0921] The server generates warnings using the results of the degradation prediction model. The input is the prediction results of the degradation prediction model. The server identifies high-risk problem areas, records warnings in a database, and sends notifications to user terminals in real time. The output is a warning message.

[0922] Step 6:

[0923] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. The inputs are warning messages and user constraints (budget, construction period, materials used, etc.). The server generates an optimal construction plan and proposes the timing of repairs and maintenance work. The output is a repair proposal.

[0924] Step 7:

[0925] The server uses an emotion recognition engine to recognize emotions from the user's facial expressions and tone of voice using the camera and microphone installed on the user's device. The input is camera video and audio data. The server determines and records the user's emotional state (e.g., stress, anxiety). The output is the user's emotional data.

[0926] Step 8:

[0927] The server adjusts the content and presentation method of repair suggestions based on the user's emotional state, obtained through emotion recognition. The input is the user's emotional data and repair suggestions. If the user is feeling stressed, the server presents repair suggestions with detailed explanations. The output is an adjusted repair suggestion.

[0928] Step 9:

[0929] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. The input is the adjusted repair proposal. The user reviews the proposal, selects the optimal plan, and executes it. The output is the implementation of the selected repair plan.

[0930] By following the above process, it is possible to predict the deterioration of a building over time in advance and provide an optimal repair plan that takes into account the user's emotional state.

[0931] (Application example 2)

[0932] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0933] In recent years, there has been an increasing demand for efficient management of building deterioration over time. However, conventional systems are limited to predicting physical deterioration and do not take into account the psychological burden on users. As a result, unexpected repair costs and prolonged construction periods have led to increased stress and anxiety among users. Furthermore, data analysis and predictive modeling to determine the appropriate timing for repairs are not fully functional. Therefore, a new system that can simultaneously effectively manage deterioration over time and reduce the psychological burden on users is needed.

[0934] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: data collection means for collecting data from multiple sensors installed in the building; data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; warning generation means for identifying problem areas based on the prediction results and generating warnings; proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; emotion engine means for recognizing emotions from the user's facial expressions and tone of voice using a camera and microphone installed in the user terminal; emotion-based proposal adjustment means for adjusting the content and presentation method of repair proposals based on the recognized user's emotional state; and user interface means for displaying proposed repair plans on the user terminal and allowing the user to select and implement the optimal plan. This makes it possible to predict building deterioration and perform effective repairs while reducing user stress and anxiety.

[0935] A "data collection means" is a device or method for collecting data from multiple sensors installed in a building.

[0936] "Data preprocessing means" refers to a device or method that preprocesses collected data and detects and removes outliers and missing values.

[0937] "Data analysis means" refers to a device or method that analyzes the pre-processed data to detect signs of aging.

[0938] The "deterioration prediction model generating means" is a device or method that models the deterioration progression pattern and predicts future deterioration.

[0939] The "warning generation means" is a device or method that identifies problem areas based on the prediction results and generates a warning.

[0940] The "proposal generation means" is a device or method that optimizes the timing of repair and maintenance work and generates an optimal construction plan that takes into account the budget and construction period.

[0941] The "emotion engine means" is a device or method that uses a camera or microphone mounted on the user terminal to recognize emotions from the user's facial expressions and tone of voice.

[0942] An "emotion-based suggestion adjustment means" is a device or method that adjusts the content and presentation of repair suggestions based on the recognized emotional state of the user.

[0943] The "user interface means" is a device or method for displaying proposed repair plans on a user terminal and allowing the user to select and execute the most suitable plan.

[0944] This invention is a system that predicts building deterioration and provides repair plans based on the user's emotional state. The system includes four main components: a server, sensors, a user terminal, and an emotion engine.

[0945] Data collection

[0946] The server collects data in real time from multiple sensors installed in the building, such as temperature sensors, humidity sensors, and vibration sensors, allowing the server to understand the current state of the building and secure the necessary data.

[0947] Data Preprocessing

[0948] The server performs preprocessing on the collected data, detecting outliers and missing values ​​and correcting or removing data as needed. This process results in a clean dataset suitable for data analysis.

[0949] Data analysis

[0950] The server analyzes the preprocessed data to detect signs of building deterioration over time, using time series analysis and clustering algorithms to identify patterns of fluctuations and anomalies in the data.

[0951] Deterioration prediction model generation

[0952] Based on the analysis results, the server runs a machine learning algorithm to model the progression of deterioration, which can then be used to predict future deterioration locations.

[0953] Warning generation

[0954] The server analyzes the results of the degradation prediction model and identifies areas where problems are likely to occur. It generates warnings for problem areas and records them in a database. Users are notified in real time.

[0955] Proposal generation

[0956] The server uses a multivariate optimization algorithm to optimize the timing of repair and maintenance work, taking into account factors such as budget, construction time, and materials used to create the optimal repair plan.

[0957] Emotion recognition by emotion engine

[0958] The system uses a camera and microphone installed on the user's device to recognize emotions from the user's facial expressions and tone of voice. The server's emotion engine analyzes this data and determines whether the user is feeling stressed.

[0959] Emotion-based suggestion adjustment

[0960] The server adjusts the repair suggestions and how they are presented based on the results of the emotion engine, for example, providing more detailed and thoughtful explanations if the user indicates high stress levels.

[0961] User Interface

[0962] The user's device displays the warnings and repair proposals sent from the server. The user can view the proposals and select and implement the repair plan that best suits them. For example, if cracks are predicted in the exterior wall, a notification will appear on the user's device saying, "Cracks have been detected in the exterior wall. Repairs are required," and the user can check the repair plan in detail.

[0963] Prompt Sentence Examples

[0964] "Describe a program that uses smart glasses to display real-time emotion recognition and maintenance information within a building. The program uses a pre-trained emotion recognition model and uses OpenCV for face detection. The program should also include the ability to display repair suggestions based on the user's emotional state."

[0965] This system makes it possible to predict building deterioration and carry out appropriate repairs, while also reducing the psychological burden on users.

[0966] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0967] Step 1:

[0968] The server collects data in real time from various sensors installed in the building (e.g., temperature, humidity, and vibration sensors). The collected data is input into the server and stored in a database. This allows the current status of the building to be monitored at any time.

[0969] Step 2:

[0970] The server performs pre-processing on the collected data. During this pre-processing stage, outliers and missing values ​​are detected and corrected or removed as necessary. For example, if abnormally high temperature data is detected, it is removed to generate a clean data set. This clean data is then sent to the next analysis step.

[0971] Step 3:

[0972] The server analyzes the preprocessed data to detect signs of aging deterioration. Specifically, it uses time series analysis and clustering algorithms. For example, if vibration sensor data from a specific part of a building shows a certain pattern, that pattern can be analyzed to find signs of deterioration. The results of this analysis become the input data.

[0973] Step 4:

[0974] The server then runs a machine learning algorithm based on the analysis results to model the deterioration progression pattern. This model is then used to predict the deterioration progression of the building. For example, it predicts future deterioration progression patterns based on past data. This predicted data is then input into the next step.

[0975] Step 5:

[0976] The server analyzes the output of the deterioration prediction model and identifies areas where problems are likely to occur. At this stage, the server generates a warning, records it in a database, and sends notifications to users in real time. For example, if cracks are progressing in the exterior wall, the server will generate a warning such as "Cracks are progressing in the exterior wall."

[0977] Step 6:

[0978] The server uses a multivariate optimization algorithm to optimize the timing of repair and maintenance work. It creates an optimal repair plan taking into account conditions such as budget, construction period, and materials used. For example, it proposes a repair plan that can be carried out within the shortest construction period and budget. This proposed plan is then sent to the user's device.

[0979] Step 7:

[0980] The server's emotion engine uses the camera and microphone installed on the user's device to recognize emotions from the user's facial expressions and tone of voice. For example, when a user checks a repair plan, the camera reads their facial expressions and the microphone analyzes their tone of voice to determine whether the user is feeling stressed. The recognized emotion data is input into the next step.

[0981] Step 8:

[0982] The server adjusts the repair suggestion content and presentation method based on the recognition results of the emotion engine. For example, if the user is in a high stress state, the server will provide detailed and thorough repair instructions to reduce anxiety. The adjustment results are then sent to the user's device.

[0983] Step 9:

[0984] The user's device displays the warnings and repair proposals sent from the server. The user can view the proposed plans and select and implement the repair plan that best suits them. For example, a notification will be displayed saying, "Cracks have been detected in the exterior wall. Repairs are required." The user can check the detailed information and select a repair plan.

[0985] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0986] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0987] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0988] [Third embodiment]

[0989] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0990] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0991] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0992] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0993] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0994] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0995] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0996] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0997] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0998] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0999] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1000] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1001] The present invention relates to a system that predicts the deterioration of buildings over time and performs repair and maintenance work at the optimal timing. To implement the present invention, three main components are required: a server, a sensor, and a user terminal. A specific embodiment of the system is shown below.

[1002] This system collects data from multiple sensors installed in buildings and predicts deterioration over time based on the collected data. The server analyzes the data and predicts deterioration, and based on the results, sends warnings and repair proposals to the user. The user can then review the proposals and select and implement the optimal repair plan.

[1003] Data collection

[1004] server

[1005] Data is collected periodically from various sensors in the building (temperature sensors, humidity sensors, vibration sensors, etc.) The server receives the data sent from these sensors and stores it in a database.

[1006] Data Preprocessing

[1007] server

[1008] Preprocessing is performed on the collected data, specifically detecting outliers and missing values ​​and removing or correcting them as necessary. This process improves data quality.

[1009] Data analysis

[1010] server

[1011] The pre-processed data is analyzed to detect signs of aging deterioration. Various algorithms (e.g., time series analysis, clustering) are used for data analysis.

[1012] Deterioration prediction model generation

[1013] server

[1014] Based on the analysis results, a machine learning algorithm is implemented to model the deterioration progression pattern. The deterioration prediction model is used to predict future deterioration.

[1015] Warning generation

[1016] server

[1017] Based on the results of the degradation prediction model, areas where problems are likely to occur are identified and warnings are generated, which are notified to the user in real time.

[1018] Proposal generation

[1019] server

[1020] The system uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. The proposed repair plan takes into account budget, construction time, materials used, and other factors.

[1021] Check and execute on the user interface

[1022] User

[1023] Users access the system using a terminal, check the warnings sent from the server and the proposed repair plans, select the repair plan they think is most appropriate, and carry out the specific work.

[1024] Specific examples

[1025] Example 1: Predicting and repairing cracks in exterior walls

[1026] 1. Data Collection

[1027] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[1028] 2. Data Preprocessing

[1029] An anomaly detection algorithm removes false sensor detection data.

[1030] 3. Data Analysis

[1031] Time series analysis is performed to detect crack progression patterns.

[1032] 4. Deterioration prediction model generation

[1033] Machine learning algorithms are used to generate predictive models of crack progression.

[1034] 5. Warning generation

[1035] Before the cracks become serious, the server generates a warning to notify the user.

[1036] 6. Proposal generation

[1037] The server will suggest the best time and method for repairing the crack, for example, indicating that repairs are needed within the next two months.

[1038] 7. User Interface

[1039] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[1040] In this way, this system predicts the deterioration of buildings over time in advance and takes preventative measures at the appropriate time, making it possible to maintain the living environment over the long term.

[1041] The processing flow will be explained below.

[1042] Step 1:

[1043] server

[1044] Data is collected in real time from multiple sensors installed in buildings (temperature sensors, humidity sensors, vibration sensors, etc.) The data is sent via the Internet, and the server receives it and stores it in a database.

[1045] Step 2:

[1046] server

[1047] Preprocessing the collected data involves running algorithms to detect outliers and missing values ​​and remove or correct them as needed, creating a dataset suitable for analysis.

[1048] Step 3:

[1049] server

[1050] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering are performed to find patterns of fluctuations and abnormalities in the data.

[1051] Step 4:

[1052] server

[1053] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[1054] Step 5:

[1055] server

[1056] Analyzes the results of the deterioration prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records them in a database.

[1057] Step 6:

[1058] server

[1059] Optimize the timing of repair and maintenance work using a multivariate optimization algorithm. Create the optimal repair plan taking into account conditions such as budget, construction period, and materials used.

[1060] Step 7:

[1061] server

[1062] The generated repair plan is sent to the user interface, which includes the details of the repair plan for easy user review.

[1063] Step 8:

[1064] User

[1065] Users access the system from their devices (smartphones or PCs) and check the warnings and repair proposals sent from the server. Users can then view the proposed plans and select the most appropriate one.

[1066] Step 9:

[1067] User

[1068] Based on the repair plan selected by the user, the system schedules specific work, contacts repair contractors as necessary, and carries out the actual repair and maintenance work.

[1069] Example 1

[1070] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1071] In recent years, problems caused by the deterioration of buildings over time have been increasing. This often results in unexpected repair costs and the need for long-term construction work. However, with current systems, it is difficult to accurately predict the progression of deterioration in advance and carry out repair and maintenance work at the optimal time. Therefore, there is a need for a system that can accurately predict the deterioration of buildings over time and propose repair and maintenance work at the optimal time, thereby preventing problems before they occur and optimizing costs.

[1072] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1073] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in a building; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; a notification means for notifying a user terminal of the prediction results in real time based on the constructed deterioration prediction model; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; a user interface means for displaying the proposed repair plans on the user terminal and allowing the user to select and execute the optimal plan; and a deterioration progress prediction algorithm means for predicting the progression of deterioration of a building using a machine learning algorithm from the collected data. This makes it possible to accurately predict building deterioration and perform repair and maintenance work at the optimal time.

[1074] A "data collection means" is a means for collecting data from multiple sensors installed in a building.

[1075] The "data preprocessing means" is a means for preprocessing collected data to detect and remove outliers and missing values.

[1076] The "data analysis means" is a means for analyzing the pre-processed data to detect signs of aging deterioration.

[1077] The "deterioration prediction model generating means" is a means for modeling the deterioration progression pattern and predicting future deterioration.

[1078] The "notification means" is a means for notifying the user terminal of the prediction results in real time based on the constructed deterioration prediction model.

[1079] The "proposal generation means" is a means for optimizing the timing of repair and maintenance work and generating an optimal construction plan that takes into account the budget and construction period.

[1080] The "user interface means" is a means for displaying proposed repair plans on a user terminal and allowing the user to select and execute the most suitable plan.

[1081] The "deterioration progress prediction algorithm means" is a means for predicting the deterioration progress of a building using a machine learning algorithm from collected data.

[1082] The present invention relates to a system for predicting deterioration of a building over time and performing repair and maintenance work at an optimal timing. Specific embodiments for carrying out the present invention are described below.

[1083] The system consists of three main components: a server, sensors, and user terminals. The entire system runs through the following stages: data collection, data preprocessing, data analysis, degradation prediction model generation, warning generation, suggestion generation, and user interface.

[1084] Data collection

[1085] The server periodically collects data from multiple sensors installed in the building, such as temperature sensors, humidity sensors, and vibration sensors. This collected data is then stored in a database by the server. The database used is MySQL.

[1086] Data Preprocessing

[1087] The server preprocesses the collected data to detect and remove outliers and missing values. It uses the PyOD library to detect outliers and imputes missing values ​​with the most recent valid data, thereby improving data quality.

[1088] Data analysis

[1089] Based on the preprocessed data, the server analyzes signs of aging deterioration. For the analysis, it uses "scikit-learn" and "statsmodels" to perform time series analysis and clustering. For example, it applies the "ARIMA model" to vibration data from building exterior walls to detect fluctuation patterns.

[1090] Deterioration prediction model generation

[1091] The server generates a deterioration prediction model based on the results of the data analysis. This model is generated using a deep learning algorithm using TensorFlow, specifically LSTM (Long Short Term Memory). The generated model is used to predict future deterioration and can accurately capture the progression pattern of deterioration.

[1092] Warning generation

[1093] The server identifies problem areas and generates warnings based on the results of the deterioration prediction model. For example, if a crack in an exterior wall exceeds 5 mm in depth, it will generate a warning as a serious error. This warning is sent to the user in real time via email or SMS using a messaging API such as "Twilio."

[1094] Proposal generation

[1095] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. Proposed repair plans are created taking into account budget, construction time, materials used, and more. The SciPy library is used for optimization, generating specific suggestions, such as "repairs are required within the next two months."

[1096] User Interface

[1097] Users access the system using a terminal and check the warnings and proposed repair plans sent from the server. The user interface is built using React and Angular, and information can be viewed in real time. For example, users can click the "Accept repair proposal" button, which will send a notification to the repair company.

[1098] Specific examples

[1099] Example 1: Predicting and repairing cracks in exterior walls

[1100] 1. Data Collection

[1101] The server collects crack data every hour from vibration sensors installed on the exterior walls.

[1102] Example: "Sensor ID: 123, Time: 10:00, Vibration Data: 0.05"

[1103] 2. Data Preprocessing

[1104] An outlier (0.10) is detected, removed, and supplemented with the latest valid data (0.05).

[1105] 3. Data Analysis

[1106] The server applies the ARIMA model and generates a forecast using the most recent three months of data.

[1107] 4. Deterioration prediction model generation

[1108] Generate a degradation prediction model using the SaaS deep learning algorithm LSTM. Example: "LSTM model generation complete, accuracy: 95%."

[1109] 5. Warning generation

[1110] Send a warning to the user if the crack depth exceeds 5mm (10:05)

[1111] 6. Proposal generation

[1112] The proposed implementation plan is to repair within the next two months, with materials A and construction time of 3 days.

[1113] 7. User Interface

[1114] The server sends a notification to the contractor when the user confirms the proposal and clicks the "Request Repair" button (10:10)

[1115] Example of input prompt for generative AI model

[1116] "Please predict cracks in the exterior walls of buildings and suggest appropriate repair timing."

[1117] This enables the system to accurately predict deterioration of buildings over time and carry out repairs and maintenance work at the optimal time, preventing problems before they occur and optimizing costs.

[1118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1119] Step 1: Data collection

[1120] server

[1121] The server collects data from temperature sensors, humidity sensors, vibration sensors, etc. installed in the building. This data collection is done periodically, for example, every hour, and data from the sensors is sent to the server. The collected data includes information such as the sensor ID, time, and measurement value. The server receives this data and stores it in a database. The database used is "MySQL."

[1122] Input: Measurement data from sensors (temperature, humidity, vibration)

[1123] Output: Sensor data stored in a database

[1124] Step 2: Data Preprocessing

[1125] server

[1126] The server preprocesses the collected data, including detecting and removing outliers and missing values. For example, it uses the PyOD library to detect outliers and removes outliers with data points exceeding three sigma. Additionally, if missing values ​​are found, they are imputed with the most recent valid data.

[1127] Input: Sensor data stored in a database

[1128] Output: Pre-processed high-quality sensor data

[1129] Step 3: Data analysis

[1130] server

[1131] The server analyzes the preprocessed data. For this analysis, it uses "scikit-learn" and "statsmodels" to perform time series analysis and clustering. For example, for vibration data on building exterior walls, it uses the "ARIMA model" to analyze data fluctuation patterns.

[1132] Input: Preprocessed sensor data

[1133] Output: Analysis results (deterioration progression pattern, etc.)

[1134] Step 4: Deterioration prediction model generation

[1135] server

[1136] The server generates a degradation prediction model based on the results of the data analysis. This model is generated using a deep learning algorithm using TensorFlow, specifically LSTM (Long Short Term Memory). LSTM is suitable for building prediction models based on time series data.

[1137] Input: Analysis results

[1138] Output: Deterioration prediction model (model for predicting future deterioration)

[1139] Step 5: Generate warnings

[1140] server

[1141] The server identifies problem areas and generates warnings based on the results of the deterioration prediction model. For example, if a crack in an exterior wall exceeds 5 mm in depth, it will generate a warning as a serious error. This warning is sent to the user in real time via email or SMS using a messaging API such as "Twilio."

[1142] Input: Results of the deterioration prediction model

[1143] Output: A warning notice to the user

[1144] Step 6: Proposal Generation

[1145] server

[1146] The server uses multivariate optimization algorithms to optimize the timing of repairs and maintenance work. Proposals are created taking into account budget, construction time, materials used, etc. For example, it uses the SciPy optimization algorithm to notify you that repairs are needed within the next two months.

[1147] Input: Warning Notification

[1148] Output: Optimized repair plan

[1149] Step 7: Check and execute in the user interface

[1150] User

[1151] Users access the system using a terminal and check the warnings and proposed repair plans sent from the server. The user interface is built using React and Angular, allowing users to check information in real time. By clicking the "Accept Repair Proposal" button, an automatic notification is sent to the repair company.

[1152] Input: Warning and suggestion notifications from the server

[1153] Output: Confirmation of repair execution (execution of the repair plan selected by the user)

[1154] (Application example 1)

[1155] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1156] Conventional building and equipment maintenance management systems had low accuracy in predicting deterioration, making it difficult to carry out repairs at the appropriate time. Furthermore, there was a lack of means for on-site staff to receive signs of deterioration and repair proposals in real time. This increased the risk of sudden breakdowns and malfunctions in buildings and equipment, leading to increased maintenance costs and reduced work efficiency. To solve these issues, a system that can more accurately predict deterioration and make more efficient repair proposals is needed.

[1157] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1158] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in a building or facility; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; a warning generation means for identifying problem areas based on the prediction results and generating warnings; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; a user interface means for displaying the proposed repair plan on a user terminal or wearable device and allowing the user to select and execute the optimal plan; and a notification means for sending notifications to on-site staff in real time based on the data collected from the sensors and the generated deterioration prediction model. This enables highly accurate deterioration predictions and efficient repair proposals, preventing unexpected breakdowns of buildings and facilities, reducing maintenance costs, and improving work efficiency.

[1159] "Data collection means" refers to devices or software that function to collect data from multiple sensors installed in buildings or facilities.

[1160] "Data preprocessing means" refers to devices or software that have the function of preprocessing collected data and detecting and removing outliers and missing values.

[1161] "Data analysis means" refers to devices or software that have the function of analyzing preprocessed data and detecting signs of deterioration over time.

[1162] The "deterioration prediction model generation means" refers to a device or software that uses techniques such as machine learning algorithms to model the progression pattern of deterioration and predict future deterioration.

[1163] The "warning generation means" is a device or software that has the function of identifying problem areas based on the results of deterioration prediction and generating a warning.

[1164] The "proposal generation means" refers to a device or software that has the function of optimizing the timing of repair and maintenance work and generating an optimal construction plan that takes into account the budget and construction period.

[1165] A "user interface means" is a device or software that has the function of displaying proposed repair plans on a user terminal or wearable device and allowing the user to select and execute the optimal plan.

[1166] "Notification means" refers to devices or software that have the ability to send notifications to on-site staff in real time based on data collected from sensors and the generated deterioration prediction model.

[1167] The present invention relates to a system that predicts the deterioration of buildings and facilities over time and performs repairs and maintenance work at the optimal time. Specifically, the system uses a server, sensors, user terminals, wearable devices, etc. Specific embodiments of the system are described below.

[1168] System Configuration

[1169] The system includes the following major components:

[1170] 1. Server

[1171] The server includes a data collection means, a data preprocessing means, a data analysis means, a degradation prediction model generation means, a warning generation means, a proposal generation means, and a notification means. The server plays a central role in collecting and analyzing data from sensors, generating models, etc.

[1172] 2. Sensors

[1173] Data is collected from multiple sensors (e.g., temperature, humidity, and vibration sensors) installed in buildings and facilities. These sensors are used to monitor changes in the state of the objects in real time.

[1174] 3. User Device

[1175] This device displays repair plans and warnings to the user, and has a user interface function that allows the user to select and implement the proposed repair plans.

[1176] 4. Wearable devices

[1177] A device worn by field staff (e.g., smart glasses) used to receive real-time notifications.

[1178] Data collection

[1179] The server collects data from sensors installed in buildings and facilities, including information on temperature, humidity, vibration, and other factors.

[1180] Data Preprocessing

[1181] The server performs preprocessing on the collected data, detecting and removing outliers and missing values ​​to improve data quality.

[1182] Data analysis

[1183] The pre-processed data is analyzed to detect signs of aging. The server analyzes the data using algorithms such as time series analysis and clustering.

[1184] Deterioration prediction model generation

[1185] Based on the analysis results, a machine learning algorithm is run to model the deterioration progression pattern. The generated predictive model is used to predict future deterioration.

[1186] Warning generation

[1187] Based on the results of the degradation prediction model, areas where problems are likely to occur are identified and warnings are generated, which are sent in real time via user terminals or wearable devices.

[1188] Proposal generation

[1189] Using a multivariable optimization algorithm, the timing of repairs and maintenance work is optimized. The server generates an optimal construction plan taking into account budget, construction period, materials used, etc.

[1190] User Interface

[1191] Users access the system using a terminal or wearable device, check the warnings and repair plans sent from the server, select the repair plan they think is most appropriate, and carry out the specific work.

[1192] Specific examples

[1193] Example 1: Vibration detection and repair of factory equipment

[1194] 1. Data Collection

[1195] The server periodically collects data from vibration sensors installed within the factory.

[1196] 2. Data Preprocessing

[1197] An anomaly detection algorithm removes false sensor detection data.

[1198] 3. Data Analysis

[1199] The server performs time series analysis and analyzes the vibration patterns of the equipment.

[1200] 4. Deterioration prediction model generation

[1201] A machine learning algorithm is used to generate a predictive model of vibration progression.

[1202] 5. Warning generation

[1203] The server generates a warning before vibrations become abnormally high and notifies on-site staff.

[1204] 6. Proposal generation

[1205] The server will suggest the optimal timing and method for repairs to reduce vibrations, for example, informing users that maintenance is required by the end of the next month.

[1206] 7. User Interface and Notifications

[1207] On-site staff can view warnings and suggestions through smart glasses and instruct on specific repair work.

[1208] Example prompt sentence:

[1209] text

[1210] The following data has been collected from temperature, humidity, and vibration sensors in the factory. Please analyze this data to predict equipment deterioration and propose the optimal timing for repairs.

[1211] data:

[1212] Temperature: 20 ± 2 degrees

[1213] Humidity: 30 ± 5%

[1214] Vibration: 0 ± 0.1g

[1215] Data were recorded every hour.

[1216] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1217] Step 1:

[1218] Data collection

[1219] The server periodically collects data from sensors (such as temperature sensors, humidity sensors, and vibration sensors) installed in buildings and facilities. The input data from the sensors includes temperature, humidity, and vibration values. The server receives the input data from the sensors and stores it in a database. The output of this step is the raw data stored in the database.

[1220] Step 2:

[1221] Data Preprocessing

[1222] The server performs preprocessing on the raw data stored in the database. The input data may contain missing or outliers, and an outlier detection algorithm is used to detect and remove them. Specifically, outliers are detected using statistical methods and machine learning models, and missing values ​​are imputed using statistical imputation methods. The preprocessed data is output in a form that can be used for data analysis.

[1223] Step 3:

[1224] Data analysis

[1225] The server analyzes the preprocessed data. It uses the preprocessed data as input data. It applies algorithms such as time series analysis and clustering to detect abnormal equipment behavior and signs of deterioration. The server obtains the analysis results and uses them in the next step. The output of the analysis results includes data that identifies abnormal areas and indicates signs of progressing deterioration.

[1226] Step 4:

[1227] Deterioration prediction model generation

[1228] The server generates a deterioration prediction model based on the results of the data analysis. The input data includes the analyzed signs and patterns. A machine learning algorithm (e.g., random forest, LSTM, etc.) is used to train the prediction model and create a model for predicting future deterioration. The output is a prediction model of deterioration progression.

[1229] Step 5:

[1230] Warning generation

[1231] The server identifies problem areas based on the results of the degradation prediction model and generates warnings. It uses the results of the prediction model as input data. The server generates warnings when problem areas exceed a certain degradation risk and sends notifications to user terminals or wearable devices. The output is a real-time warning sent to the user.

[1232] Step 6:

[1233] Proposal generation

[1234] The server optimizes the timing of repair and maintenance work and generates an optimal construction plan that takes into account budget and construction time. Input data includes the results of the deterioration prediction model and warning data. Using a multivariable optimization algorithm, the optimal repair plan is constructed and provided to the user via a user interface. The output is a detailed repair plan.

[1235] Step 7:

[1236] User Interface and Notifications

[1237] Users access the system using a terminal or wearable device to check the warnings and repair plans sent from the server. Input data includes warning data and repair plans. The user checks the repair plan that seems most appropriate and carries out the specific repair work. Field staff can also receive real-time notifications via the wearable device. The output is a repair plan that the user and field staff can check and carry out.

[1238] The above is the specific processing flow of this system.

[1239] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1240] This invention combines a system that predicts the deterioration of buildings over time and carries out repair and maintenance work at the optimal time with an emotion engine that recognizes the user's emotions. This invention makes it possible to provide efficient repair plans while reducing the user's stress and anxiety.

[1241] To implement the present invention, four main components are required: a server, a sensor, a user terminal, and an emotion engine. The specific embodiment is shown below.

[1242] Data collection

[1243] server

[1244] It collects data in real time from various sensors in the building (temperature sensors, humidity sensors, vibration sensors, etc.) The server receives the data sent from these sensors and stores it in a database.

[1245] Data Preprocessing

[1246] server

[1247] Preprocessing is performed on the collected data, detecting outliers and missing values ​​and removing or correcting them as necessary, to create a dataset suitable for analysis.

[1248] Data analysis

[1249] server

[1250] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering are performed to find patterns of fluctuations and abnormalities in the data.

[1251] Deterioration prediction model generation

[1252] server

[1253] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[1254] Warning generation

[1255] server

[1256] Analyzes the results of the deterioration prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records them in a database.

[1257] Proposal generation

[1258] server

[1259] Optimize the timing of repair and maintenance work using a multivariate optimization algorithm. Create the optimal repair plan taking into account conditions such as budget, construction period, and materials used.

[1260] Emotion recognition by emotion engine

[1261] server

[1262] Using the camera and microphone installed on the user's device, the system recognizes emotions from the user's facial expressions and tone of voice, and uses an emotion engine to determine whether the user is feeling stressed.

[1263] Emotion-based suggestion adjustment

[1264] server

[1265] The emotion engine recognizes the user's emotional state and adjusts the content and presentation of repair suggestions based on that. For example, if the user is feeling stressed, the suggestions will be presented with more detailed explanations.

[1266] Check and execute on the user interface

[1267] User

[1268] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. Users then view the proposed plans and select the most appropriate one.

[1269] Specific examples

[1270] Example 1: Predicting and repairing cracks in exterior walls

[1271] 1. Data Collection

[1272] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[1273] 2. Data Preprocessing

[1274] An anomaly detection algorithm removes false sensor detection data.

[1275] 3. Data Analysis

[1276] Time series analysis is performed to detect crack progression patterns.

[1277] 4. Deterioration prediction model generation

[1278] Machine learning algorithms are used to generate predictive models of crack progression.

[1279] 5. Warning generation

[1280] Before the cracks become serious, the server generates a warning to notify the user.

[1281] 6. Proposal generation

[1282] Based on a multivariate optimization algorithm, it will suggest repairs that are needed within the next two months.

[1283] 7. Emotion Engine

[1284] When a user operates the device, the emotion engine recognizes the user's emotions from their facial expressions and voice.

[1285] 8. Adjusting suggestions based on emotions

[1286] If the user is stressed, the server will adjust the way the repair plan is presented, providing more detailed instructions.

[1287] 9. User Interface

[1288] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[1289] In this way, this system can predict deterioration of buildings over time and carry out repairs at the appropriate time, not only enabling the long-term maintenance of living environments but also reducing stress and anxiety for users.

[1290] The processing flow will be explained below.

[1291] Step 1:

[1292] server

[1293] It collects data in real time from multiple sensors (temperature sensors, humidity sensors, vibration sensors, etc.) installed in buildings. The sensors send the data via the Internet, and the server receives the data and stores it in a database.

[1294] Step 2:

[1295] server

[1296] Preprocessing the collected data involves running algorithms to detect outliers and missing values ​​and remove or correct them as needed, preparing a clean dataset suitable for analysis.

[1297] Step 3:

[1298] server

[1299] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering algorithms are used to identify patterns of fluctuation and abnormalities in the data.

[1300] Step 4:

[1301] server

[1302] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[1303] Step 5:

[1304] server

[1305] Analyzes the results of the degradation prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records the information in a database.

[1306] Step 6:

[1307] server

[1308] The timing of repair and maintenance work is optimized using a multivariate optimization algorithm, taking into account conditions such as budget, construction period, and materials used to generate the optimal repair plan.

[1309] Step 7:

[1310] server

[1311] The generated repair plan is sent to the user interface in cooperation with the emotion engine. The emotion engine recognizes the user's emotional state, and if the user is feeling stressed, the emotion engine provides the user with a more detailed explanation of the repair plan.

[1312] Step 8:

[1313] Terminal

[1314] The user's device (smartphone or PC) receives and displays the warnings and repair plans sent from the server. The emotion engine analyzes the user's facial expressions and tone of voice to recognize the user's emotions.

[1315] Step 9:

[1316] User

[1317] Users can check the repair plans on their device and select the most suitable one. Based on the advice and information provided by the emotion engine, users can make a repair selection with peace of mind.

[1318] Step 10:

[1319] User

[1320] Schedule specific repair work based on the selected repair plan, and contact repair contractors as necessary to request repair work.

[1321] In this way, the system can predict the deterioration of buildings over time and carry out repairs at the appropriate time and in the appropriate manner, while also recognizing the user's emotions and providing appropriate information, thereby reducing the user's stress and anxiety.

[1322] Example 2

[1323] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1324] Conventional building deterioration prediction systems are limited to predicting deterioration and proposing repair timing, and do not include work plans or proposals that take the user's emotional state into account. This can cause users to feel stressed or anxious about repair proposals, which can result in delays in appropriate repairs. Therefore, there is a need for a system that provides efficient repair plans that take the user's emotional state into account and minimizes their emotional burden.

[1325] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1326] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in the building; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the progression pattern of deterioration and predicting future deterioration; a warning generation means for identifying problem areas based on the prediction results and generating warnings; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; an emotion recognition means for recognizing the user's emotion; a proposal adjustment means for adjusting the proposal content based on the recognized emotion; and a user interface means for displaying the proposed repair plan on a user terminal and allowing the user to select and implement the optimal plan. This makes it possible to predict aging deterioration and provide an optimal repair plan while taking into account the user's emotional state.

[1327] "Data collection means" is a function for collecting data from multiple sensors installed in a building.

[1328] The "data preprocessing means" is a function that preprocesses collected data and detects and removes outliers and missing values.

[1329] The "data analysis means" is a function that analyzes the preprocessed data to detect signs of deterioration over time.

[1330] The "deterioration prediction model generation means" is a function that models the deterioration progression pattern and executes a machine learning algorithm to predict future deterioration.

[1331] The "warning generation means" is a function that identifies problem areas based on the prediction results and generates a warning.

[1332] The "proposal generation means" is a function that optimizes the timing of repair and maintenance work and generates the optimal construction plan taking into account the budget and construction period.

[1333] The "emotion recognition means" is a function for recognizing the user's emotions, and uses a camera and microphone to analyze the user's facial expressions and tone of voice.

[1334] The "proposal adjustment means" is a function that adjusts the content and presentation method of repair proposals based on the recognized emotions.

[1335] The "user interface means" is a function that displays proposed repair plans on a user terminal and allows the user to select and execute the most suitable plan.

[1336] MODE FOR CARRYING OUT THE INVENTION

[1337] The present invention combines a system that predicts the deterioration of buildings over time and performs repair and maintenance work at the optimal timing with a function that recognizes user emotions. This system makes it possible to provide efficient repair plans while reducing user stress and anxiety. Specific embodiments of the system are described below.

[1338] Hardware and software used

[1339] server

[1340] Data collection, preprocessing, analysis, model generation, warning generation, and suggestion generation

[1341] Specific software used: Python, TensorFlow, Scikit-learn, OpenCV

[1342] sensor

[1343] Temperature sensor, humidity sensor, vibration sensor

[1344] User terminal

[1345] Smartphones, PCs

[1346] Emotion Recognition Engine

[1347] Specific software used: Python, OpenCV, NVIDIA GPU (CUDA)

[1348] System configuration and functions

[1349] The server has the following functions:

[1350] 1. Data Collection Methods

[1351] The server collects data in real time from temperature, humidity, and vibration sensors installed in the building. The sensors measure environmental data from each part and send it to the server. The data is then stored in the server's database.

[1352] 2. Data preprocessing methods

[1353] The server preprocesses the collected data, detecting outliers and missing values ​​and removing or imputing them as necessary. Preprocessing involves the use of outlier detection algorithms (e.g., Z-score analysis) and data imputation algorithms (e.g., linear interpolation).

[1354] 3. Data Analysis Methods

[1355] The server analyzes the preprocessed data to detect signs of aging degradation, and uses time series analysis (e.g., ARIMA model) and clustering (e.g., K-means method) to identify patterns of fluctuation and abnormalities in the data.

[1356] 4. Deterioration prediction model generation method

[1357] Based on the analysis results, the server runs a machine learning algorithm to model the progression pattern of deterioration, using recurrent neural networks (RNNs) and random forests to generate a deterioration prediction model.

[1358] 5. Warning generation means

[1359] The server identifies problem areas based on the prediction results and generates warnings as necessary, which are stored in a database and sent to the user's device.

[1360] 6. Proposal generation means

[1361] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work, for example, generating a repair plan to be carried out within the next two months, taking into account budget and construction time.

[1362] 7. Emotion recognition means

[1363] The server uses the camera and microphone installed on the user's device to recognize the user's emotions from their facial expressions and tone of voice, and the emotion engine determines in real time whether the user is feeling stressed.

[1364] 8. Proposed adjustment measures

[1365] The server adjusts the content and presentation of repair suggestions based on the user's emotional state, acquired through emotion recognition. If the user is feeling stressed, the server displays suggestions with more detailed explanations.

[1366] User accesses the device:

[1367] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. They then view the proposed plans, select the most appropriate one, and implement it. If necessary, they can request repairs from a repair company.

[1368] Specific examples

[1369] Example 1: Predicting and repairing cracks in exterior walls

[1370] 1. Data Collection

[1371] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[1372] 2. Data Preprocessing

[1373] An anomaly detection algorithm removes false sensor detection data.

[1374] 3. Data Analysis

[1375] Time series analysis is performed to detect crack progression patterns.

[1376] 4. Deterioration prediction model generation

[1377] Machine learning algorithms are used to generate predictive models of crack progression.

[1378] 5. Warning generation

[1379] Before the cracks become serious, the server generates a warning to notify the user.

[1380] 6. Proposal generation

[1381] Based on a multivariate optimization algorithm, it will suggest repairs that are needed within the next two months.

[1382] 7. Emotion recognition means

[1383] When a user operates the device, the emotion engine recognizes the user's emotions from their facial expressions and voice.

[1384] 8. Proposed adjustment measures

[1385] If the user is stressed, the server will adjust the way the repair plan is presented, providing more detailed instructions.

[1386] 9. User Interface

[1387] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[1388] Example prompts to input to the generative AI model

[1389] "Your goal is to generate a machine learning algorithm that analyzes data from vibration sensors installed on the building's exterior walls and predicts the progression pattern of cracks. You also integrate an emotion engine to adjust the way repair plans are presented to users based on their emotional state."

[1390] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1391] Step 1:

[1392] The server collects data in real time from multiple sensors (temperature, humidity, vibration, etc.) installed in the building. The input is environmental data sent from the sensors, specifically temperature, humidity, and vibration values. The server stores this data in a database. The output is a raw dataset for preprocessing.

[1393] Step 2:

[1394] The server preprocesses the collected dataset. The input is the raw data collected in step 1. The server detects outliers and missing values, removes them using an outlier detection algorithm (e.g., Z-score analysis), and imputes missing values ​​using a missing value imputation algorithm (e.g., linear interpolation). The output is a clean dataset.

[1395] Step 3:

[1396] The server analyzes the clean dataset. The input is preprocessed data. It performs time series analysis (e.g., ARIMA model) and clustering (e.g., K-means) to detect signs of aging. Specifically, it extracts patterns of fluctuation and abnormalities in the data. The output is the analysis results.

[1397] Step 4:

[1398] The server then runs a machine learning algorithm based on the analysis results to model the deterioration progression pattern. The input is the data analysis results. A deterioration prediction model is generated using a recurrent neural network (RNN) or random forest. The output is the deterioration prediction model.

[1399] Step 5:

[1400] The server generates warnings using the results of the degradation prediction model. The input is the prediction results of the degradation prediction model. The server identifies high-risk problem areas, records warnings in a database, and sends notifications to user terminals in real time. The output is a warning message.

[1401] Step 6:

[1402] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. The inputs are warning messages and user constraints (budget, construction period, materials used, etc.). The server generates an optimal construction plan and proposes the timing of repairs and maintenance work. The output is a repair proposal.

[1403] Step 7:

[1404] The server uses an emotion recognition engine to recognize emotions from the user's facial expressions and tone of voice using the camera and microphone installed on the user's device. The input is camera video and audio data. The server determines and records the user's emotional state (e.g., stress, anxiety). The output is the user's emotional data.

[1405] Step 8:

[1406] The server adjusts the content and presentation method of repair suggestions based on the user's emotional state, obtained through emotion recognition. The input is the user's emotional data and repair suggestions. If the user is feeling stressed, the server presents repair suggestions with detailed explanations. The output is an adjusted repair suggestion.

[1407] Step 9:

[1408] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. The input is the adjusted repair proposal. The user reviews the proposal, selects the optimal plan, and executes it. The output is the implementation of the selected repair plan.

[1409] By following the above process, it is possible to predict the deterioration of a building over time in advance and provide an optimal repair plan that takes into account the user's emotional state.

[1410] (Application example 2)

[1411] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1412] In recent years, there has been an increasing demand for efficient management of building deterioration over time. However, conventional systems are limited to predicting physical deterioration and do not take into account the psychological burden on users. As a result, unexpected repair costs and prolonged construction periods have led to increased stress and anxiety among users. Furthermore, data analysis and predictive modeling to determine the appropriate timing for repairs are not fully functional. Therefore, a new system that can simultaneously effectively manage deterioration over time and reduce the psychological burden on users is needed.

[1413] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: data collection means for collecting data from multiple sensors installed in the building; data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; warning generation means for identifying problem areas based on the prediction results and generating warnings; proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; emotion engine means for recognizing emotions from the user's facial expressions and tone of voice using a camera and microphone installed in the user terminal; emotion-based proposal adjustment means for adjusting the content and presentation method of repair proposals based on the recognized user's emotional state; and user interface means for displaying proposed repair plans on the user terminal and allowing the user to select and implement the optimal plan. This makes it possible to predict building deterioration and perform effective repairs while reducing user stress and anxiety.

[1414] A "data collection means" is a device or method for collecting data from multiple sensors installed in a building.

[1415] "Data preprocessing means" refers to a device or method that preprocesses collected data and detects and removes outliers and missing values.

[1416] "Data analysis means" refers to a device or method that analyzes the pre-processed data to detect signs of aging.

[1417] The "deterioration prediction model generating means" is a device or method that models the deterioration progression pattern and predicts future deterioration.

[1418] The "warning generation means" is a device or method that identifies problem areas based on the prediction results and generates a warning.

[1419] The "proposal generation means" is a device or method that optimizes the timing of repair and maintenance work and generates an optimal construction plan that takes into account the budget and construction period.

[1420] The "emotion engine means" is a device or method that uses a camera or microphone mounted on the user terminal to recognize emotions from the user's facial expressions and tone of voice.

[1421] An "emotion-based suggestion adjustment means" is a device or method that adjusts the content and presentation of repair suggestions based on the recognized emotional state of the user.

[1422] The "user interface means" is a device or method for displaying proposed repair plans on a user terminal and allowing the user to select and execute the most suitable plan.

[1423] This invention is a system that predicts building deterioration and provides repair plans based on the user's emotional state. The system includes four main components: a server, sensors, a user terminal, and an emotion engine.

[1424] Data collection

[1425] The server collects data in real time from multiple sensors installed in the building, such as temperature sensors, humidity sensors, and vibration sensors, allowing the server to understand the current state of the building and secure the necessary data.

[1426] Data Preprocessing

[1427] The server performs preprocessing on the collected data, detecting outliers and missing values ​​and correcting or removing data as needed. This process results in a clean dataset suitable for data analysis.

[1428] Data analysis

[1429] The server analyzes the preprocessed data to detect signs of building deterioration over time, using time series analysis and clustering algorithms to identify patterns of fluctuations and anomalies in the data.

[1430] Deterioration prediction model generation

[1431] Based on the analysis results, the server runs a machine learning algorithm to model the progression of deterioration, which can then be used to predict future deterioration locations.

[1432] Warning generation

[1433] The server analyzes the results of the degradation prediction model and identifies areas where problems are likely to occur. It generates warnings for problem areas and records them in a database. Users are notified in real time.

[1434] Proposal generation

[1435] The server uses a multivariate optimization algorithm to optimize the timing of repair and maintenance work, taking into account factors such as budget, construction time, and materials used to create the optimal repair plan.

[1436] Emotion recognition by emotion engine

[1437] The system uses a camera and microphone installed on the user's device to recognize emotions from the user's facial expressions and tone of voice. The server's emotion engine analyzes this data and determines whether the user is feeling stressed.

[1438] Emotion-based suggestion adjustment

[1439] The server adjusts the repair suggestions and how they are presented based on the results of the emotion engine, for example, providing more detailed and thoughtful explanations if the user indicates high stress levels.

[1440] User Interface

[1441] The user's device displays the warnings and repair proposals sent from the server. The user can view the proposals and select and implement the repair plan that best suits them. For example, if cracks are predicted in the exterior wall, a notification will appear on the user's device saying, "Cracks have been detected in the exterior wall. Repairs are required," and the user can check the repair plan in detail.

[1442] Prompt Sentence Examples

[1443] "Describe a program that uses smart glasses to display real-time emotion recognition and maintenance information within a building. The program uses a pre-trained emotion recognition model and uses OpenCV for face detection. The program should also include the ability to display repair suggestions based on the user's emotional state."

[1444] This system makes it possible to predict building deterioration and carry out appropriate repairs, while also reducing the psychological burden on users.

[1445] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1446] Step 1:

[1447] The server collects data in real time from various sensors installed in the building (e.g., temperature, humidity, and vibration sensors). The collected data is input into the server and stored in a database. This allows the current status of the building to be monitored at any time.

[1448] Step 2:

[1449] The server performs pre-processing on the collected data. During this pre-processing stage, outliers and missing values ​​are detected and corrected or removed as necessary. For example, if abnormally high temperature data is detected, it is removed to generate a clean data set. This clean data is then sent to the next analysis step.

[1450] Step 3:

[1451] The server analyzes the preprocessed data to detect signs of aging deterioration. Specifically, it uses time series analysis and clustering algorithms. For example, if vibration sensor data from a specific part of a building shows a certain pattern, that pattern can be analyzed to find signs of deterioration. The results of this analysis become the input data.

[1452] Step 4:

[1453] The server then runs a machine learning algorithm based on the analysis results to model the deterioration progression pattern. This model is then used to predict the deterioration progression of the building. For example, it predicts future deterioration progression patterns based on past data. This predicted data is then input into the next step.

[1454] Step 5:

[1455] The server analyzes the output of the deterioration prediction model and identifies areas where problems are likely to occur. At this stage, the server generates a warning, records it in a database, and sends notifications to users in real time. For example, if cracks are progressing in the exterior wall, the server will generate a warning such as "Cracks are progressing in the exterior wall."

[1456] Step 6:

[1457] The server uses a multivariate optimization algorithm to optimize the timing of repair and maintenance work. It creates an optimal repair plan taking into account conditions such as budget, construction period, and materials used. For example, it proposes a repair plan that can be carried out within the shortest construction period and budget. This proposed plan is then sent to the user's device.

[1458] Step 7:

[1459] The server's emotion engine uses the camera and microphone installed on the user's device to recognize emotions from the user's facial expressions and tone of voice. For example, when a user checks a repair plan, the camera reads their facial expressions and the microphone analyzes their tone of voice to determine whether the user is feeling stressed. The recognized emotion data is input into the next step.

[1460] Step 8:

[1461] The server adjusts the repair suggestion content and presentation method based on the recognition results of the emotion engine. For example, if the user is in a high stress state, the server will provide detailed and thorough repair instructions to reduce anxiety. The adjustment results are then sent to the user's device.

[1462] Step 9:

[1463] The user's device displays the warnings and repair proposals sent from the server. The user can view the proposed plans and select and implement the repair plan that best suits them. For example, a notification will be displayed saying, "Cracks have been detected in the exterior wall. Repairs are required." The user can check the detailed information and select a repair plan.

[1464] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1465] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1466] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1467] [Fourth embodiment]

[1468] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1469] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1470] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1471] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1472] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1473] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1474] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1475] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1476] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1477] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1478] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1479] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1480] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1481] The present invention relates to a system that predicts the deterioration of buildings over time and performs repair and maintenance work at the optimal timing. To implement the present invention, three main components are required: a server, a sensor, and a user terminal. A specific embodiment of the system is shown below.

[1482] This system collects data from multiple sensors installed in buildings and predicts deterioration over time based on the collected data. The server analyzes the data and predicts deterioration, and based on the results, sends warnings and repair proposals to the user. The user can then review the proposals and select and implement the optimal repair plan.

[1483] Data collection

[1484] server

[1485] Data is collected periodically from various sensors in the building (temperature sensors, humidity sensors, vibration sensors, etc.) The server receives the data sent from these sensors and stores it in a database.

[1486] Data Preprocessing

[1487] server

[1488] Preprocessing is performed on the collected data, specifically detecting outliers and missing values ​​and removing or correcting them as necessary. This process improves data quality.

[1489] Data analysis

[1490] server

[1491] The pre-processed data is analyzed to detect signs of aging deterioration. Various algorithms (e.g., time series analysis, clustering) are used for data analysis.

[1492] Deterioration prediction model generation

[1493] server

[1494] Based on the analysis results, a machine learning algorithm is implemented to model the deterioration progression pattern. The deterioration prediction model is used to predict future deterioration.

[1495] Warning generation

[1496] server

[1497] Based on the results of the degradation prediction model, areas where problems are likely to occur are identified and warnings are generated, which are notified to the user in real time.

[1498] Proposal generation

[1499] server

[1500] The system uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. The proposed repair plan takes into account budget, construction time, materials used, and other factors.

[1501] Check and execute on the user interface

[1502] User

[1503] Users access the system using a terminal, check the warnings sent from the server and the proposed repair plans, select the repair plan they think is most appropriate, and carry out the specific work.

[1504] Specific examples

[1505] Example 1: Predicting and repairing cracks in exterior walls

[1506] 1. Data Collection

[1507] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[1508] 2. Data Preprocessing

[1509] An anomaly detection algorithm removes false sensor detection data.

[1510] 3. Data Analysis

[1511] Time series analysis is performed to detect crack progression patterns.

[1512] 4. Deterioration prediction model generation

[1513] Machine learning algorithms are used to generate predictive models of crack progression.

[1514] 5. Warning generation

[1515] Before the cracks become serious, the server generates a warning to notify the user.

[1516] 6. Proposal generation

[1517] The server will suggest the best time and method for repairing the crack, for example, indicating that repairs are needed within the next two months.

[1518] 7. User Interface

[1519] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[1520] In this way, this system predicts the deterioration of buildings over time in advance and takes preventative measures at the appropriate time, making it possible to maintain the living environment over the long term.

[1521] The processing flow will be explained below.

[1522] Step 1:

[1523] server

[1524] Data is collected in real time from multiple sensors installed in buildings (temperature sensors, humidity sensors, vibration sensors, etc.) The data is sent via the Internet, and the server receives it and stores it in a database.

[1525] Step 2:

[1526] server

[1527] Preprocessing the collected data involves running algorithms to detect outliers and missing values ​​and remove or correct them as needed, creating a dataset suitable for analysis.

[1528] Step 3:

[1529] server

[1530] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering are performed to find patterns of fluctuations and abnormalities in the data.

[1531] Step 4:

[1532] server

[1533] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[1534] Step 5:

[1535] server

[1536] Analyzes the results of the deterioration prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records them in a database.

[1537] Step 6:

[1538] server

[1539] Optimize the timing of repair and maintenance work using a multivariate optimization algorithm. Create the optimal repair plan taking into account conditions such as budget, construction period, and materials used.

[1540] Step 7:

[1541] server

[1542] The generated repair plan is sent to the user interface, which includes the details of the repair plan for easy user review.

[1543] Step 8:

[1544] User

[1545] Users access the system from their devices (smartphones or PCs) and check the warnings and repair proposals sent from the server. Users can then view the proposed plans and select the most appropriate one.

[1546] Step 9:

[1547] User

[1548] Based on the repair plan selected by the user, the system schedules specific work, contacts repair contractors as necessary, and carries out the actual repair and maintenance work.

[1549] Example 1

[1550] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1551] In recent years, problems caused by the deterioration of buildings over time have been increasing. This often results in unexpected repair costs and the need for long-term construction work. However, with current systems, it is difficult to accurately predict the progression of deterioration in advance and carry out repair and maintenance work at the optimal time. Therefore, there is a need for a system that can accurately predict the deterioration of buildings over time and propose repair and maintenance work at the optimal time, thereby preventing problems before they occur and optimizing costs.

[1552] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1553] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in a building; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; a notification means for notifying a user terminal of the prediction results in real time based on the constructed deterioration prediction model; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; a user interface means for displaying the proposed repair plans on the user terminal and allowing the user to select and execute the optimal plan; and a deterioration progress prediction algorithm means for predicting the progression of deterioration of a building using a machine learning algorithm from the collected data. This makes it possible to accurately predict building deterioration and perform repair and maintenance work at the optimal time.

[1554] A "data collection means" is a means for collecting data from multiple sensors installed in a building.

[1555] The "data preprocessing means" is a means for preprocessing collected data to detect and remove outliers and missing values.

[1556] The "data analysis means" is a means for analyzing the pre-processed data to detect signs of aging deterioration.

[1557] The "deterioration prediction model generating means" is a means for modeling the deterioration progression pattern and predicting future deterioration.

[1558] The "notification means" is a means for notifying the user terminal of the prediction results in real time based on the constructed deterioration prediction model.

[1559] The "proposal generation means" is a means for optimizing the timing of repair and maintenance work and generating an optimal construction plan that takes into account the budget and construction period.

[1560] The "user interface means" is a means for displaying proposed repair plans on a user terminal and allowing the user to select and execute the most suitable plan.

[1561] The "deterioration progress prediction algorithm means" is a means for predicting the deterioration progress of a building using a machine learning algorithm from collected data.

[1562] The present invention relates to a system for predicting deterioration of a building over time and performing repair and maintenance work at an optimal timing. Specific embodiments for carrying out the present invention are described below.

[1563] The system consists of three main components: a server, sensors, and user terminals. The entire system runs through the following stages: data collection, data preprocessing, data analysis, degradation prediction model generation, warning generation, suggestion generation, and user interface.

[1564] Data collection

[1565] The server periodically collects data from multiple sensors installed in the building, such as temperature sensors, humidity sensors, and vibration sensors. This collected data is then stored in a database by the server. The database used is MySQL.

[1566] Data Preprocessing

[1567] The server preprocesses the collected data to detect and remove outliers and missing values. It uses the PyOD library to detect outliers and imputes missing values ​​with the most recent valid data, thereby improving data quality.

[1568] Data analysis

[1569] Based on the preprocessed data, the server analyzes signs of aging deterioration. For the analysis, it uses "scikit-learn" and "statsmodels" to perform time series analysis and clustering. For example, it applies the "ARIMA model" to vibration data from building exterior walls to detect fluctuation patterns.

[1570] Deterioration prediction model generation

[1571] The server generates a deterioration prediction model based on the results of the data analysis. This model is generated using a deep learning algorithm using TensorFlow, specifically LSTM (Long Short Term Memory). The generated model is used to predict future deterioration and can accurately capture the progression pattern of deterioration.

[1572] Warning generation

[1573] The server identifies problem areas and generates warnings based on the results of the deterioration prediction model. For example, if a crack in an exterior wall exceeds 5 mm in depth, it will generate a warning as a serious error. This warning is sent to the user in real time via email or SMS using a messaging API such as "Twilio."

[1574] Proposal generation

[1575] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. Proposed repair plans are created taking into account budget, construction time, materials used, and more. The SciPy library is used for optimization, generating specific suggestions, such as "repairs are required within the next two months."

[1576] User Interface

[1577] Users access the system using a terminal and check the warnings and proposed repair plans sent from the server. The user interface is built using React and Angular, and information can be viewed in real time. For example, users can click the "Accept repair proposal" button, which will send a notification to the repair company.

[1578] Specific examples

[1579] Example 1: Predicting and repairing cracks in exterior walls

[1580] 1. Data Collection

[1581] The server collects crack data every hour from vibration sensors installed on the exterior walls.

[1582] Example: "Sensor ID: 123, Time: 10:00, Vibration Data: 0.05"

[1583] 2. Data Preprocessing

[1584] An outlier (0.10) is detected, removed, and supplemented with the latest valid data (0.05).

[1585] 3. Data Analysis

[1586] The server applies the ARIMA model and generates a forecast using the most recent three months of data.

[1587] 4. Deterioration prediction model generation

[1588] Generate a degradation prediction model using the SaaS deep learning algorithm LSTM. Example: "LSTM model generation complete, accuracy: 95%."

[1589] 5. Warning generation

[1590] Send a warning to the user if the crack depth exceeds 5mm (10:05)

[1591] 6. Proposal generation

[1592] The proposed implementation plan is to repair within the next two months, with materials A and construction time of 3 days.

[1593] 7. User Interface

[1594] The server sends a notification to the contractor when the user confirms the proposal and clicks the "Request Repair" button (10:10)

[1595] Example of input prompt for generative AI model

[1596] "Please predict cracks in the exterior walls of buildings and suggest appropriate repair timing."

[1597] This enables the system to accurately predict deterioration of buildings over time and carry out repairs and maintenance work at the optimal time, preventing problems before they occur and optimizing costs.

[1598] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1599] Step 1: Data collection

[1600] server

[1601] The server collects data from temperature sensors, humidity sensors, vibration sensors, etc. installed in the building. This data collection is done periodically, for example, every hour, and data from the sensors is sent to the server. The collected data includes information such as the sensor ID, time, and measurement value. The server receives this data and stores it in a database. The database used is "MySQL."

[1602] Input: Measurement data from sensors (temperature, humidity, vibration)

[1603] Output: Sensor data stored in a database

[1604] Step 2: Data Preprocessing

[1605] server

[1606] The server preprocesses the collected data, including detecting and removing outliers and missing values. For example, it uses the PyOD library to detect outliers and removes outliers with data points exceeding three sigma. Additionally, if missing values ​​are found, they are imputed with the most recent valid data.

[1607] Input: Sensor data stored in a database

[1608] Output: Pre-processed high-quality sensor data

[1609] Step 3: Data analysis

[1610] server

[1611] The server analyzes the preprocessed data. For this analysis, it uses "scikit-learn" and "statsmodels" to perform time series analysis and clustering. For example, for vibration data on building exterior walls, it uses the "ARIMA model" to analyze data fluctuation patterns.

[1612] Input: Preprocessed sensor data

[1613] Output: Analysis results (deterioration progression pattern, etc.)

[1614] Step 4: Deterioration prediction model generation

[1615] server

[1616] The server generates a degradation prediction model based on the results of the data analysis. This model is generated using a deep learning algorithm using TensorFlow, specifically LSTM (Long Short Term Memory). LSTM is suitable for building prediction models based on time series data.

[1617] Input: Analysis results

[1618] Output: Deterioration prediction model (model for predicting future deterioration)

[1619] Step 5: Generate warnings

[1620] server

[1621] The server identifies problem areas and generates warnings based on the results of the deterioration prediction model. For example, if a crack in an exterior wall exceeds 5 mm in depth, it will generate a warning as a serious error. This warning is sent to the user in real time via email or SMS using a messaging API such as "Twilio."

[1622] Input: Results of the deterioration prediction model

[1623] Output: A warning notice to the user

[1624] Step 6: Proposal Generation

[1625] server

[1626] The server uses multivariate optimization algorithms to optimize the timing of repairs and maintenance work. Proposals are created taking into account budget, construction time, materials used, etc. For example, it uses the SciPy optimization algorithm to notify you that repairs are needed within the next two months.

[1627] Input: Warning Notification

[1628] Output: Optimized repair plan

[1629] Step 7: Check and execute in the user interface

[1630] User

[1631] Users access the system using a terminal and check the warnings and proposed repair plans sent from the server. The user interface is built using React and Angular, allowing users to check information in real time. By clicking the "Accept Repair Proposal" button, an automatic notification is sent to the repair company.

[1632] Input: Warning and suggestion notifications from the server

[1633] Output: Confirmation of repair execution (execution of the repair plan selected by the user)

[1634] (Application example 1)

[1635] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1636] Conventional building and equipment maintenance management systems had low accuracy in predicting deterioration, making it difficult to carry out repairs at the appropriate time. Furthermore, there was a lack of means for on-site staff to receive signs of deterioration and repair proposals in real time. This increased the risk of sudden breakdowns and malfunctions in buildings and equipment, leading to increased maintenance costs and reduced work efficiency. To solve these issues, a system that can more accurately predict deterioration and make more efficient repair proposals is needed.

[1637] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1638] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in a building or facility; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; a warning generation means for identifying problem areas based on the prediction results and generating warnings; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; a user interface means for displaying the proposed repair plan on a user terminal or wearable device and allowing the user to select and execute the optimal plan; and a notification means for sending notifications to on-site staff in real time based on the data collected from the sensors and the generated deterioration prediction model. This enables highly accurate deterioration predictions and efficient repair proposals, preventing unexpected breakdowns of buildings and facilities, reducing maintenance costs, and improving work efficiency.

[1639] "Data collection means" refers to devices or software that function to collect data from multiple sensors installed in buildings or facilities.

[1640] "Data preprocessing means" refers to devices or software that have the function of preprocessing collected data and detecting and removing outliers and missing values.

[1641] "Data analysis means" refers to devices or software that have the function of analyzing preprocessed data and detecting signs of deterioration over time.

[1642] The "deterioration prediction model generation means" refers to a device or software that uses techniques such as machine learning algorithms to model the progression pattern of deterioration and predict future deterioration.

[1643] The "warning generation means" is a device or software that has the function of identifying problem areas based on the results of deterioration prediction and generating a warning.

[1644] The "proposal generation means" refers to a device or software that has the function of optimizing the timing of repair and maintenance work and generating an optimal construction plan that takes into account the budget and construction period.

[1645] A "user interface means" is a device or software that has the function of displaying proposed repair plans on a user terminal or wearable device and allowing the user to select and execute the optimal plan.

[1646] "Notification means" refers to devices or software that have the ability to send notifications to on-site staff in real time based on data collected from sensors and the generated deterioration prediction model.

[1647] The present invention relates to a system that predicts the deterioration of buildings and facilities over time and performs repairs and maintenance work at the optimal time. Specifically, the system uses a server, sensors, user terminals, wearable devices, etc. Specific embodiments of the system are described below.

[1648] System Configuration

[1649] The system includes the following major components:

[1650] 1. Server

[1651] The server includes a data collection means, a data preprocessing means, a data analysis means, a degradation prediction model generation means, a warning generation means, a proposal generation means, and a notification means. The server plays a central role in collecting and analyzing data from sensors, generating models, etc.

[1652] 2. Sensors

[1653] Data is collected from multiple sensors (e.g., temperature, humidity, and vibration sensors) installed in buildings and facilities. These sensors are used to monitor changes in the state of the objects in real time.

[1654] 3. User Device

[1655] This device displays repair plans and warnings to the user, and has a user interface function that allows the user to select and implement the proposed repair plans.

[1656] 4. Wearable devices

[1657] A device worn by field staff (e.g., smart glasses) used to receive real-time notifications.

[1658] Data collection

[1659] The server collects data from sensors installed in buildings and facilities, including information on temperature, humidity, vibration, and other factors.

[1660] Data Preprocessing

[1661] The server performs preprocessing on the collected data, detecting and removing outliers and missing values ​​to improve data quality.

[1662] Data analysis

[1663] The pre-processed data is analyzed to detect signs of aging. The server analyzes the data using algorithms such as time series analysis and clustering.

[1664] Deterioration prediction model generation

[1665] Based on the analysis results, a machine learning algorithm is run to model the deterioration progression pattern. The generated predictive model is used to predict future deterioration.

[1666] Warning generation

[1667] Based on the results of the degradation prediction model, areas where problems are likely to occur are identified and warnings are generated, which are sent in real time via user terminals or wearable devices.

[1668] Proposal generation

[1669] Using a multivariable optimization algorithm, the timing of repairs and maintenance work is optimized. The server generates an optimal construction plan taking into account budget, construction period, materials used, etc.

[1670] User Interface

[1671] Users access the system using a terminal or wearable device, check the warnings and repair plans sent from the server, select the repair plan they think is most appropriate, and carry out the specific work.

[1672] Specific examples

[1673] Example 1: Vibration detection and repair of factory equipment

[1674] 1. Data Collection

[1675] The server periodically collects data from vibration sensors installed within the factory.

[1676] 2. Data Preprocessing

[1677] An anomaly detection algorithm removes false sensor detection data.

[1678] 3. Data Analysis

[1679] The server performs time series analysis and analyzes the vibration patterns of the equipment.

[1680] 4. Deterioration prediction model generation

[1681] A machine learning algorithm is used to generate a predictive model of vibration progression.

[1682] 5. Warning generation

[1683] The server generates a warning before vibrations become abnormally high and notifies on-site staff.

[1684] 6. Proposal generation

[1685] The server will suggest the optimal timing and method for repairs to reduce vibrations, for example, informing users that maintenance is required by the end of the next month.

[1686] 7. User Interface and Notifications

[1687] On-site staff can view warnings and suggestions through smart glasses and instruct on specific repair work.

[1688] Example prompt sentence:

[1689] text

[1690] The following data has been collected from temperature, humidity, and vibration sensors in the factory. Please analyze this data to predict equipment deterioration and propose the optimal timing for repairs.

[1691] data:

[1692] Temperature: 20 ± 2 degrees

[1693] Humidity: 30 ± 5%

[1694] Vibration: 0 ± 0.1g

[1695] Data were recorded every hour.

[1696] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1697] Step 1:

[1698] Data collection

[1699] The server periodically collects data from sensors (such as temperature sensors, humidity sensors, and vibration sensors) installed in buildings and facilities. The input data from the sensors includes temperature, humidity, and vibration values. The server receives the input data from the sensors and stores it in a database. The output of this step is the raw data stored in the database.

[1700] Step 2:

[1701] Data Preprocessing

[1702] The server performs preprocessing on the raw data stored in the database. The input data may contain missing or outliers, and an outlier detection algorithm is used to detect and remove them. Specifically, outliers are detected using statistical methods and machine learning models, and missing values ​​are imputed using statistical imputation methods. The preprocessed data is output in a form that can be used for data analysis.

[1703] Step 3:

[1704] Data analysis

[1705] The server analyzes the preprocessed data. It uses the preprocessed data as input data. It applies algorithms such as time series analysis and clustering to detect abnormal equipment behavior and signs of deterioration. The server obtains the analysis results and uses them in the next step. The output of the analysis results includes data that identifies abnormal areas and indicates signs of progressing deterioration.

[1706] Step 4:

[1707] Deterioration prediction model generation

[1708] The server generates a deterioration prediction model based on the results of the data analysis. The input data includes the analyzed signs and patterns. A machine learning algorithm (e.g., random forest, LSTM, etc.) is used to train the prediction model and create a model for predicting future deterioration. The output is a prediction model of deterioration progression.

[1709] Step 5:

[1710] Warning generation

[1711] The server identifies problem areas based on the results of the degradation prediction model and generates warnings. It uses the results of the prediction model as input data. The server generates warnings when problem areas exceed a certain degradation risk and sends notifications to user terminals or wearable devices. The output is a real-time warning sent to the user.

[1712] Step 6:

[1713] Proposal generation

[1714] The server optimizes the timing of repair and maintenance work and generates an optimal construction plan that takes into account budget and construction time. Input data includes the results of the deterioration prediction model and warning data. Using a multivariable optimization algorithm, the optimal repair plan is constructed and provided to the user via a user interface. The output is a detailed repair plan.

[1715] Step 7:

[1716] User Interface and Notifications

[1717] Users access the system using a terminal or wearable device to check the warnings and repair plans sent from the server. Input data includes warning data and repair plans. The user checks the repair plan that seems most appropriate and carries out the specific repair work. Field staff can also receive real-time notifications via the wearable device. The output is a repair plan that the user and field staff can check and carry out.

[1718] The above is the specific processing flow of this system.

[1719] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1720] This invention combines a system that predicts the deterioration of buildings over time and carries out repair and maintenance work at the optimal time with an emotion engine that recognizes the user's emotions. This invention makes it possible to provide efficient repair plans while reducing the user's stress and anxiety.

[1721] To implement the present invention, four main components are required: a server, a sensor, a user terminal, and an emotion engine. The specific embodiment is shown below.

[1722] Data collection

[1723] server

[1724] It collects data in real time from various sensors in the building (temperature sensors, humidity sensors, vibration sensors, etc.) The server receives the data sent from these sensors and stores it in a database.

[1725] Data Preprocessing

[1726] server

[1727] Preprocessing is performed on the collected data, detecting outliers and missing values ​​and removing or correcting them as necessary, to create a dataset suitable for analysis.

[1728] Data analysis

[1729] server

[1730] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering are performed to find patterns of fluctuations and abnormalities in the data.

[1731] Deterioration prediction model generation

[1732] server

[1733] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[1734] Warning generation

[1735] server

[1736] Analyzes the results of the deterioration prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records them in a database.

[1737] Proposal generation

[1738] server

[1739] Optimize the timing of repair and maintenance work using a multivariate optimization algorithm. Create the optimal repair plan taking into account conditions such as budget, construction period, and materials used.

[1740] Emotion recognition by emotion engine

[1741] server

[1742] Using the camera and microphone installed on the user's device, the system recognizes emotions from the user's facial expressions and tone of voice, and uses an emotion engine to determine whether the user is feeling stressed.

[1743] Emotion-based suggestion adjustment

[1744] server

[1745] The emotion engine recognizes the user's emotional state and adjusts the content and presentation of repair suggestions based on that. For example, if the user is feeling stressed, the suggestions will be presented with more detailed explanations.

[1746] Check and execute on the user interface

[1747] User

[1748] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. Users then view the proposed plans and select the most appropriate one.

[1749] Specific examples

[1750] Example 1: Predicting and repairing cracks in exterior walls

[1751] 1. Data Collection

[1752] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[1753] 2. Data Preprocessing

[1754] An anomaly detection algorithm removes false sensor detection data.

[1755] 3. Data Analysis

[1756] Time series analysis is performed to detect crack progression patterns.

[1757] 4. Deterioration prediction model generation

[1758] Machine learning algorithms are used to generate predictive models of crack progression.

[1759] 5. Warning generation

[1760] Before the cracks become serious, the server generates a warning to notify the user.

[1761] 6. Proposal generation

[1762] Based on a multivariate optimization algorithm, it will suggest repairs that are needed within the next two months.

[1763] 7. Emotion Engine

[1764] When a user operates the device, the emotion engine recognizes the user's emotions from their facial expressions and voice.

[1765] 8. Adjusting suggestions based on emotions

[1766] If the user is stressed, the server will adjust the way the repair plan is presented, providing more detailed instructions.

[1767] 9. User Interface

[1768] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[1769] In this way, this system can predict deterioration of buildings over time and carry out repairs at the appropriate time, not only enabling the long-term maintenance of living environments but also reducing stress and anxiety for users.

[1770] The processing flow will be explained below.

[1771] Step 1:

[1772] server

[1773] It collects data in real time from multiple sensors (temperature sensors, humidity sensors, vibration sensors, etc.) installed in buildings. The sensors send the data via the Internet, and the server receives the data and stores it in a database.

[1774] Step 2:

[1775] server

[1776] Preprocessing the collected data involves running algorithms to detect outliers and missing values ​​and remove or correct them as needed, preparing a clean dataset suitable for analysis.

[1777] Step 3:

[1778] server

[1779] The preprocessed data is analyzed to detect signs of aging. Specifically, time series analysis and clustering algorithms are used to identify patterns of fluctuation and abnormalities in the data.

[1780] Step 4:

[1781] server

[1782] Based on the analysis results, a machine learning algorithm is run to model the degradation progression pattern, which is then used to predict future degradation.

[1783] Step 5:

[1784] server

[1785] Analyzes the results of the degradation prediction model to identify areas where problems are likely to occur, generates warnings for problem areas, and records the information in a database.

[1786] Step 6:

[1787] server

[1788] The timing of repair and maintenance work is optimized using a multivariate optimization algorithm, taking into account conditions such as budget, construction period, and materials used to generate the optimal repair plan.

[1789] Step 7:

[1790] server

[1791] The generated repair plan is sent to the user interface in cooperation with the emotion engine. The emotion engine recognizes the user's emotional state, and if the user is feeling stressed, the emotion engine provides the user with a more detailed explanation of the repair plan.

[1792] Step 8:

[1793] Terminal

[1794] The user's device (smartphone or PC) receives and displays the warnings and repair plans sent from the server. The emotion engine analyzes the user's facial expressions and tone of voice to recognize the user's emotions.

[1795] Step 9:

[1796] User

[1797] Users can check the repair plans on their device and select the most suitable one. Based on the advice and information provided by the emotion engine, users can make a repair selection with peace of mind.

[1798] Step 10:

[1799] User

[1800] Schedule specific repair work based on the selected repair plan, and contact repair contractors as necessary to request repair work.

[1801] In this way, the system can predict the deterioration of buildings over time and carry out repairs at the appropriate time and in the appropriate manner, while also recognizing the user's emotions and providing appropriate information, thereby reducing the user's stress and anxiety.

[1802] Example 2

[1803] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1804] Conventional building deterioration prediction systems are limited to predicting deterioration and proposing repair timing, and do not include work plans or proposals that take the user's emotional state into account. This can cause users to feel stressed or anxious about repair proposals, which can result in delays in appropriate repairs. Therefore, there is a need for a system that provides efficient repair plans that take the user's emotional state into account and minimizes their emotional burden.

[1805] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1806] In this invention, the server includes: a data collection means for collecting data from multiple sensors installed in the building; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; a deterioration prediction model generation means for modeling the progression pattern of deterioration and predicting future deterioration; a warning generation means for identifying problem areas based on the prediction results and generating warnings; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; an emotion recognition means for recognizing the user's emotion; a proposal adjustment means for adjusting the proposal content based on the recognized emotion; and a user interface means for displaying the proposed repair plan on a user terminal and allowing the user to select and implement the optimal plan. This makes it possible to predict aging deterioration and provide an optimal repair plan while taking into account the user's emotional state.

[1807] "Data collection means" is a function for collecting data from multiple sensors installed in a building.

[1808] The "data preprocessing means" is a function that preprocesses collected data and detects and removes outliers and missing values.

[1809] The "data analysis means" is a function that analyzes the preprocessed data to detect signs of deterioration over time.

[1810] The "deterioration prediction model generation means" is a function that models the deterioration progression pattern and executes a machine learning algorithm to predict future deterioration.

[1811] The "warning generation means" is a function that identifies problem areas based on the prediction results and generates a warning.

[1812] The "proposal generation means" is a function that optimizes the timing of repair and maintenance work and generates the optimal construction plan taking into account the budget and construction period.

[1813] The "emotion recognition means" is a function for recognizing the user's emotions, and uses a camera and microphone to analyze the user's facial expressions and tone of voice.

[1814] The "proposal adjustment means" is a function that adjusts the content and presentation method of repair proposals based on the recognized emotions.

[1815] The "user interface means" is a function that displays proposed repair plans on a user terminal and allows the user to select and execute the most suitable plan.

[1816] MODE FOR CARRYING OUT THE INVENTION

[1817] The present invention combines a system that predicts the deterioration of buildings over time and performs repair and maintenance work at the optimal timing with a function that recognizes user emotions. This system makes it possible to provide efficient repair plans while reducing user stress and anxiety. Specific embodiments of the system are described below.

[1818] Hardware and software used

[1819] server

[1820] Data collection, preprocessing, analysis, model generation, warning generation, and suggestion generation

[1821] Specific software used: Python, TensorFlow, Scikit-learn, OpenCV

[1822] sensor

[1823] Temperature sensor, humidity sensor, vibration sensor

[1824] User terminal

[1825] Smartphones, PCs

[1826] Emotion Recognition Engine

[1827] Specific software used: Python, OpenCV, NVIDIA GPU (CUDA)

[1828] System configuration and functions

[1829] The server has the following functions:

[1830] 1. Data Collection Methods

[1831] The server collects data in real time from temperature, humidity, and vibration sensors installed in the building. The sensors measure environmental data from each part and send it to the server. The data is then stored in the server's database.

[1832] 2. Data preprocessing methods

[1833] The server preprocesses the collected data, detecting outliers and missing values ​​and removing or imputing them as necessary. Preprocessing involves the use of outlier detection algorithms (e.g., Z-score analysis) and data imputation algorithms (e.g., linear interpolation).

[1834] 3. Data Analysis Methods

[1835] The server analyzes the preprocessed data to detect signs of aging degradation, and uses time series analysis (e.g., ARIMA model) and clustering (e.g., K-means method) to identify patterns of fluctuation and abnormalities in the data.

[1836] 4. Deterioration prediction model generation method

[1837] Based on the analysis results, the server runs a machine learning algorithm to model the progression pattern of deterioration, using recurrent neural networks (RNNs) and random forests to generate a deterioration prediction model.

[1838] 5. Warning generation means

[1839] The server identifies problem areas based on the prediction results and generates warnings as necessary, which are stored in a database and sent to the user's device.

[1840] 6. Proposal generation means

[1841] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work, for example, generating a repair plan to be carried out within the next two months, taking into account budget and construction time.

[1842] 7. Emotion recognition means

[1843] The server uses the camera and microphone installed on the user's device to recognize the user's emotions from their facial expressions and tone of voice, and the emotion engine determines in real time whether the user is feeling stressed.

[1844] 8. Proposed adjustment measures

[1845] The server adjusts the content and presentation of repair suggestions based on the user's emotional state, acquired through emotion recognition. If the user is feeling stressed, the server displays suggestions with more detailed explanations.

[1846] User accesses the device:

[1847] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. They then view the proposed plans, select the most appropriate one, and implement it. If necessary, they can request repairs from a repair company.

[1848] Specific examples

[1849] Example 1: Predicting and repairing cracks in exterior walls

[1850] 1. Data Collection

[1851] The server periodically collects crack data from vibration sensors installed on the exterior walls.

[1852] 2. Data Preprocessing

[1853] An anomaly detection algorithm removes false sensor detection data.

[1854] 3. Data Analysis

[1855] Time series analysis is performed to detect crack progression patterns.

[1856] 4. Deterioration prediction model generation

[1857] Machine learning algorithms are used to generate predictive models of crack progression.

[1858] 5. Warning generation

[1859] Before the cracks become serious, the server generates a warning to notify the user.

[1860] 6. Proposal generation

[1861] Based on a multivariate optimization algorithm, it will suggest repairs that are needed within the next two months.

[1862] 7. Emotion recognition means

[1863] When a user operates the device, the emotion engine recognizes the user's emotions from their facial expressions and voice.

[1864] 8. Proposed adjustment measures

[1865] If the user is stressed, the server will adjust the way the repair plan is presented, providing more detailed instructions.

[1866] 9. User Interface

[1867] Users can view warnings and suggestions on their device and then either call a repair service or fix the problem themselves.

[1868] Example prompts to input to the generative AI model

[1869] "Your goal is to generate a machine learning algorithm that analyzes data from vibration sensors installed on the building's exterior walls and predicts the progression pattern of cracks. You also integrate an emotion engine to adjust the way repair plans are presented to users based on their emotional state."

[1870] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1871] Step 1:

[1872] The server collects data in real time from multiple sensors (temperature, humidity, vibration, etc.) installed in the building. The input is environmental data sent from the sensors, specifically temperature, humidity, and vibration values. The server stores this data in a database. The output is a raw dataset for preprocessing.

[1873] Step 2:

[1874] The server preprocesses the collected dataset. The input is the raw data collected in step 1. The server detects outliers and missing values, removes them using an outlier detection algorithm (e.g., Z-score analysis), and imputes missing values ​​using a missing value imputation algorithm (e.g., linear interpolation). The output is a clean dataset.

[1875] Step 3:

[1876] The server analyzes the clean dataset. The input is preprocessed data. It performs time series analysis (e.g., ARIMA model) and clustering (e.g., K-means) to detect signs of aging. Specifically, it extracts patterns of fluctuation and abnormalities in the data. The output is the analysis results.

[1877] Step 4:

[1878] The server then runs a machine learning algorithm based on the analysis results to model the deterioration progression pattern. The input is the data analysis results. A deterioration prediction model is generated using a recurrent neural network (RNN) or random forest. The output is the deterioration prediction model.

[1879] Step 5:

[1880] The server generates warnings using the results of the degradation prediction model. The input is the prediction results of the degradation prediction model. The server identifies high-risk problem areas, records warnings in a database, and sends notifications to user terminals in real time. The output is a warning message.

[1881] Step 6:

[1882] The server uses a multivariate optimization algorithm to optimize the timing of repairs and maintenance work. The inputs are warning messages and user constraints (budget, construction period, materials used, etc.). The server generates an optimal construction plan and proposes the timing of repairs and maintenance work. The output is a repair proposal.

[1883] Step 7:

[1884] The server uses an emotion recognition engine to recognize emotions from the user's facial expressions and tone of voice using the camera and microphone installed on the user's device. The input is camera video and audio data. The server determines and records the user's emotional state (e.g., stress, anxiety). The output is the user's emotional data.

[1885] Step 8:

[1886] The server adjusts the content and presentation method of repair suggestions based on the user's emotional state, obtained through emotion recognition. The input is the user's emotional data and repair suggestions. If the user is feeling stressed, the server presents repair suggestions with detailed explanations. The output is an adjusted repair suggestion.

[1887] Step 9:

[1888] Users access the system from their devices (smartphones, PCs, etc.) and check the warnings and repair proposals sent from the server. The input is the adjusted repair proposal. The user reviews the proposal, selects the optimal plan, and executes it. The output is the implementation of the selected repair plan.

[1889] By following the above process, it is possible to predict the deterioration of a building over time in advance and provide an optimal repair plan that takes into account the user's emotional state.

[1890] (Application example 2)

[1891] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1892] In recent years, there has been an increasing demand for efficient management of building deterioration over time. However, conventional systems are limited to predicting physical deterioration and do not take into account the psychological burden on users. As a result, unexpected repair costs and prolonged construction periods have led to increased stress and anxiety among users. Furthermore, data analysis and predictive modeling to determine the appropriate timing for repairs are not fully functional. Therefore, a new system that can simultaneously effectively manage deterioration over time and reduce the psychological burden on users is needed.

[1893] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: data collection means for collecting data from multiple sensors installed in the building; data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; data analysis means for analyzing the preprocessed data to detect signs of aging deterioration; deterioration prediction model generation means for modeling the deterioration progression pattern and predicting future deterioration; warning generation means for identifying problem areas based on the prediction results and generating warnings; proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into account the budget and construction period; emotion engine means for recognizing emotions from the user's facial expressions and tone of voice using a camera and microphone installed in the user terminal; emotion-based proposal adjustment means for adjusting the content and presentation method of repair proposals based on the recognized user's emotional state; and user interface means for displaying proposed repair plans on the user terminal and allowing the user to select and implement the optimal plan. This makes it possible to predict building deterioration and perform effective repairs while reducing user stress and anxiety.

[1894] A "data collection means" is a device or method for collecting data from multiple sensors installed in a building.

[1895] "Data preprocessing means" refers to a device or method that preprocesses collected data and detects and removes outliers and missing values.

[1896] "Data analysis means" refers to a device or method that analyzes the pre-processed data to detect signs of aging.

[1897] The "deterioration prediction model generating means" is a device or method that models the deterioration progression pattern and predicts future deterioration.

[1898] The "warning generation means" is a device or method that identifies problem areas based on the prediction results and generates a warning.

[1899] The "proposal generation means" is a device or method that optimizes the timing of repair and maintenance work and generates an optimal construction plan that takes into account the budget and construction period.

[1900] The "emotion engine means" is a device or method that uses a camera or microphone mounted on the user terminal to recognize emotions from the user's facial expressions and tone of voice.

[1901] An "emotion-based suggestion adjustment means" is a device or method that adjusts the content and presentation of repair suggestions based on the recognized emotional state of the user.

[1902] The "user interface means" is a device or method for displaying proposed repair plans on a user terminal and allowing the user to select and execute the most suitable plan.

[1903] This invention is a system that predicts building deterioration and provides repair plans based on the user's emotional state. The system includes four main components: a server, sensors, a user terminal, and an emotion engine.

[1904] Data collection

[1905] The server collects data in real time from multiple sensors installed in the building, such as temperature sensors, humidity sensors, and vibration sensors, allowing the server to understand the current state of the building and secure the necessary data.

[1906] Data Preprocessing

[1907] The server performs preprocessing on the collected data, detecting outliers and missing values ​​and correcting or removing data as needed. This process results in a clean dataset suitable for data analysis.

[1908] Data analysis

[1909] The server analyzes the preprocessed data to detect signs of building deterioration over time, using time series analysis and clustering algorithms to identify patterns of fluctuations and anomalies in the data.

[1910] Deterioration prediction model generation

[1911] Based on the analysis results, the server runs a machine learning algorithm to model the progression of deterioration, which can then be used to predict future deterioration locations.

[1912] Warning generation

[1913] The server analyzes the results of the degradation prediction model and identifies areas where problems are likely to occur. It generates warnings for problem areas and records them in a database. Users are notified in real time.

[1914] Proposal generation

[1915] The server uses a multivariate optimization algorithm to optimize the timing of repair and maintenance work, taking into account factors such as budget, construction time, and materials used to create the optimal repair plan.

[1916] Emotion recognition by emotion engine

[1917] The system uses a camera and microphone installed on the user's device to recognize emotions from the user's facial expressions and tone of voice. The server's emotion engine analyzes this data and determines whether the user is feeling stressed.

[1918] Emotion-based suggestion adjustment

[1919] The server adjusts the repair suggestions and how they are presented based on the results of the emotion engine, for example, providing more detailed and thoughtful explanations if the user indicates high stress levels.

[1920] User Interface

[1921] The user's device displays the warnings and repair proposals sent from the server. The user can view the proposals and select and implement the repair plan that best suits them. For example, if cracks are predicted in the exterior wall, a notification will appear on the user's device saying, "Cracks have been detected in the exterior wall. Repairs are required," and the user can check the repair plan in detail.

[1922] Prompt Sentence Examples

[1923] "Describe a program that uses smart glasses to display real-time emotion recognition and maintenance information within a building. The program uses a pre-trained emotion recognition model and uses OpenCV for face detection. The program should also include the ability to display repair suggestions based on the user's emotional state."

[1924] This system makes it possible to predict building deterioration and carry out appropriate repairs, while also reducing the psychological burden on users.

[1925] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1926] Step 1:

[1927] The server collects data in real time from various sensors installed in the building (e.g., temperature, humidity, and vibration sensors). The collected data is input into the server and stored in a database. This allows the current status of the building to be monitored at any time.

[1928] Step 2:

[1929] The server performs pre-processing on the collected data. During this pre-processing stage, outliers and missing values ​​are detected and corrected or removed as necessary. For example, if abnormally high temperature data is detected, it is removed to generate a clean data set. This clean data is then sent to the next analysis step.

[1930] Step 3:

[1931] The server analyzes the preprocessed data to detect signs of aging deterioration. Specifically, it uses time series analysis and clustering algorithms. For example, if vibration sensor data from a specific part of a building shows a certain pattern, that pattern can be analyzed to find signs of deterioration. The results of this analysis become the input data.

[1932] Step 4:

[1933] The server then runs a machine learning algorithm based on the analysis results to model the deterioration progression pattern. This model is then used to predict the deterioration progression of the building. For example, it predicts future deterioration progression patterns based on past data. This predicted data is then input into the next step.

[1934] Step 5:

[1935] The server analyzes the output of the deterioration prediction model and identifies areas where problems are likely to occur. At this stage, the server generates a warning, records it in a database, and sends notifications to users in real time. For example, if cracks are progressing in the exterior wall, the server will generate a warning such as "Cracks are progressing in the exterior wall."

[1936] Step 6:

[1937] The server uses a multivariate optimization algorithm to optimize the timing of repair and maintenance work. It creates an optimal repair plan taking into account conditions such as budget, construction period, and materials used. For example, it proposes a repair plan that can be carried out within the shortest construction period and budget. This proposed plan is then sent to the user's device.

[1938] Step 7:

[1939] The server's emotion engine uses the camera and microphone installed on the user's device to recognize emotions from the user's facial expressions and tone of voice. For example, when a user checks a repair plan, the camera reads their facial expressions and the microphone analyzes their tone of voice to determine whether the user is feeling stressed. The recognized emotion data is input into the next step.

[1940] Step 8:

[1941] The server adjusts the repair suggestion content and presentation method based on the recognition results of the emotion engine. For example, if the user is in a high stress state, the server will provide detailed and thorough repair instructions to reduce anxiety. The adjustment results are then sent to the user's device.

[1942] Step 9:

[1943] The user's device displays the warnings and repair proposals sent from the server. The user can view the proposed plans and select and implement the repair plan that best suits them. For example, a notification will be displayed saying, "Cracks have been detected in the exterior wall. Repairs are required." The user can check the detailed information and select a repair plan.

[1944] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1945] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1946] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1947] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1948] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1949] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1950] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1951] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1952] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1953] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1954] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1955] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1956] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1957] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1958] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1959] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1960] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1961] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1962] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1963] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1964] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1965] The following is further disclosed regarding the above embodiment.

[1966] (Claim 1)

[1967] a data collection means for collecting data from a plurality of sensors installed in the building;

[1968] a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values;

[1969] a data analysis means for analyzing the pre-processed data to detect signs of aging;

[1970] a degradation prediction model generation means for modeling a degradation progression pattern and predicting future degradation;

[1971] a warning generation means for identifying a problem location based on the prediction result and generating a warning;

[1972] a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into consideration the budget and construction period;

[1973] a user interface means for displaying the proposed repair plans on a user terminal and allowing the user to select and execute an optimal plan;

[1974] A system including:

[1975] (Claim 2)

[1976] 10. The system of claim 1, wherein the aging progression pattern is modeled using a machine learning algorithm.

[1977] (Claim 3)

[1978] 2. The system of claim 1, wherein the warning generating means sends a notification to the user in real time when it detects a high-risk symptom for a specific substructure or part.

[1979] "Example 1"

[1980] (Claim 1)

[1981] a data collection means for collecting data from a plurality of sensors installed in the building;

[1982] a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values;

[1983] a data analysis means for analyzing the pre-processed data to detect signs of aging;

[1984] a degradation prediction model generation means for modeling a degradation progression pattern and predicting future degradation;

[1985] a warning generation means for identifying a problem location based on the prediction result and generating a warning;

[1986] a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into consideration the budget and construction period;

[1987] a user interface means for displaying the proposed repair plans on a user terminal and allowing the user to select and execute an optimal plan;

[1988] a deterioration progress prediction algorithm means for predicting the deterioration progress of a building using a machine learning algorithm from the collected data;

[1989] a notification means for notifying a user terminal of a prediction result in real time based on the constructed deterioration prediction model;

[1990] A system including:

[1991] (Claim 2)

[1992] 2. The system according to claim 1, wherein the model generated by the deterioration progress prediction algorithm means is one that uses time series analysis and a clustering algorithm.

[1993] (Claim 3)

[1994] 2. The system according to claim 1, wherein the notification means sends a warning to the user in real time when a high-risk sign of deterioration is detected in a specific structural part of the building.

[1995] "Application Example 1"

[1996] (Claim 1)

[1997] ...

Claims

1. a data collection means for collecting data from a plurality of sensors installed in the building; a data preprocessing means for preprocessing the collected data to detect and remove outliers and missing values; a data analysis means for analyzing the pre-processed data to detect signs of aging; a degradation prediction model generation means for modeling a degradation progression pattern and predicting future degradation; a warning generation means for identifying a problem location based on the prediction result and generating a warning; a proposal generation means for optimizing the timing of repair and maintenance work and generating an optimal construction plan taking into consideration the budget and construction period; a user interface means for displaying the proposed repair plans on a user terminal and allowing the user to select and execute an optimal plan; A system including:

2. The system of claim 1 , wherein the aging progression pattern is modeled using a machine learning algorithm.

3. 2. The system of claim 1, wherein the warning generating means sends a notification to the user in real time when it detects a high-risk symptom for a specific substructure or part.

Images