In-room damage estimation system
The indoor damage estimation system uses deep learning models to predict furniture behavior and indoor damage by accounting for floor interaction, enhancing prediction accuracy and duration.
Patent Information
- Application Number
- JP2023216291
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-12-21
- Publication Date
- 2025-07-03
AI Technical Summary
Existing systems fail to accurately estimate indoor damage during earthquakes due to the lack of consideration for the interaction between furniture and floor movement, especially when seismic motion varies.
An indoor damage estimation system that utilizes a furniture recognition model and a floor response estimation model, both trained with deep learning techniques, to predict furniture behavior and movement based on seismic data, incorporating dynamic frictional forces and periodic characteristics.
Accurately predicts furniture movement and indoor damage by considering floor interaction, enabling precise estimation of indoor conditions during earthquakes, extending prediction time from 0.1 second to 10 seconds using an improved LSTM network.
Smart Images
Figure 2025099551000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an indoor damage estimation system for estimating the indoor damage situation in a building due to an earthquake.
Background Art
[0002] There is known a structural health monitoring system for quickly evaluating the soundness of a building immediately after an earthquake. The structural health monitoring system is being popularized and deployed under the leadership of general construction contractors, and its introduction to companies and the like has started. According to the structural health monitoring system, earthquake observation and evaluation of the soundness of a building are quickly performed, which helps business continuity planning (BCP) of companies immediately after an earthquake. However, even if the structure is safe, the safety of the occupants is not always guaranteed. For example, furniture such as bookshelves and desks placed indoors is not fixed. When furniture is not fixed, there is a high risk of movement and tipping due to the shaking of an earthquake, and in addition to direct harm to people, there is also a high possibility of obstructing evacuation by blocking the entrances and exits.
[0003] As a past study, in the 1990s, there was a paper on the relationship between the tipping rate and the floor response from an investigation of furniture tipping (see Non-Patent Document 1). In addition, due to a questionnaire survey on the Great East Japan Earthquake in 2011 (see Non-Patent Document 2) and the like, tipping of furniture in super high-rise buildings was confirmed. Therefore, until around 2015, papers on damage estimation due to furniture tipping (see Non-Patent Document 3) and papers on simulation of the behavior of furniture during an earthquake by experiments (see Non-Patent Documents 4 and 5) were mainstream. As a subsequent study, there is a study on indoor damage by CNN by learning 3D simulation of furniture during an earthquake (see Non-Patent Document 6).
Prior Art Documents
Non-Patent Documents
[0004]
Non-Patent Document 1
[0005] Conventionally, only the movement of furniture has been considered for indoor damage. Therefore, when the seismic motion is different, there is a problem that the movement of furniture cannot be predicted. That is, since the interaction with the movement of the floor is not considered, it has been difficult to accurately estimate indoor damage based on seismic motion.
[0006] From such a perspective, the present invention provides an indoor damage estimation system capable of accurately estimating indoor damage based on seismic motion.
Means for Solving the Problem
[0007] The present invention is an indoor damage estimation system for estimating the indoor damage situation in a building due to an earthquake. The input data is a furniture recognition model (furniture information data) representing the position and three-dimensional shape of furniture, and indoor image data representing the indoor damage situation including furniture before and after an earthquake. As teacher data corresponding to the input data, indoor damage situation data including the behavior of furniture after an earthquake is used as learning data, and a furniture behavior estimation unit deeply learned based on a convolutional neural network. The input data is a mass point system model (building information data) representing the position of the building, and the structural form and number of floors, and time history waveform data of a single floor (acceleration and displacement) in the past. As teacher data corresponding to the input data, time history waveform data of a single floor after an earthquake is used as learning data, and a floor response estimation model deeply learned based on the LSTM network method. The furniture behavior estimation unit and the floor response estimation model are provided with furniture information data and earthquake information data to be estimated for the indoor damage situation, and a damage estimation main body unit for predicting the time history response of a single floor after an earthquake and the movement amount of furniture including the floor response. The movement amount of the furniture is obtained by creating a synthetic spectrum using the behavior of a single furniture obtained by the furniture behavior estimation unit and the time history response of a single floor obtained by the floor response estimation model, and then performing forward and inverse Fourier transforms on the synthetic spectrum to calculate the periodic characteristics of the susceptibility to shaking from the movement amount of the furniture including the floor response.
[0008] In the damage estimation main body, after creating a composite spectrum using the predicted time history response waveform from the first time point to the second time point and the behavior of the furniture alone, an inverse Fourier transform may be performed on the composite spectrum to calculate the amount of movement of the furniture considering the influence of the time history response waveform.
[0009] Further, in the damage estimation main body, when the load obtained by multiplying the acceleration of the time history response waveform by the mass of the furniture exceeds the dynamic frictional force acting between the floor and the furniture, the amount of slippage due to the dynamic frictional force may be added to the behavior of the furniture alone to increase the amount of movement of the furniture. By doing so, it is possible to estimate the amount of movement of the furniture that adapts to the floor response, in line with the actual phenomenon considering the influence of the frictional force between the floor and the furniture.
[0010] Moreover, the indoor damage estimation system according to the present invention is a system that estimates the indoor damage situation in the estimation process based on the results of a learning process performed in advance. The learning process includes a learning preparation process, a response analysis process, and a time history response waveform learning process. The estimation process includes an estimation preparation process, an estimation information acquisition process, an indoor furniture estimation process, a time history response waveform prediction process, and a furniture behavior estimation process.
[0011] In the learning preparation process, a mass point system model of the building is created. In the response analysis process, the time history response waveform of each floor is obtained by performing a response analysis on the mass point system model. In the time history response waveform learning process, the time history response waveform from the earthquake occurrence time to the first time point is input into the floor response estimation model, and the floor response estimation model is trained to output the time history response waveform from the first time point to the second time point, which is later than the first time point. The floor response estimation model has a network structure of LSTM (Long Short-Term Memory) that extends the Recurrent Neural Network (RNN).
[0012] In the preliminary preparation process, a furniture recognition model that detects furniture from an indoor image and outputs the type, position, and dimensions of the detected furniture, and response behavior information that associates the behavior of the furniture with the time history response waveform are prepared. In the estimation information acquisition process, an indoor image taken during an earthquake is acquired, and a time history response waveform from the time of the earthquake at the floor where the estimation is performed to the first time point is acquired.
[0013] In the indoor furniture estimation process, the furniture recognition model is used to recognize the furniture shown in the indoor image, and the type, position, and dimensions of the furniture are estimated. In the time history response waveform prediction process, the time history response waveform from the time of the earthquake to the first time point is input into the floor response estimation model that has been learned, and the time history response waveform from the first time point to the second time point is predicted. In the furniture behavior estimation process, based on the response behavior information, the estimated type, position, and dimensions of the furniture in the room, and the predicted time history response waveform from the first time point to the second time point, the behavior of the furniture at each time from the first time point to the second time point is obtained.
[0014] In the indoor damage estimation system according to the present invention, since the interaction with the movement of the floor is considered, it is possible to predict the movement of furniture in response to various ground motions. Therefore, it is possible to accurately estimate the indoor damage based on the ground motion.
Effect of the Invention
[0015] According to the present invention, it is possible to provide an indoor damage estimation system capable of accurately estimating indoor damage based on ground motion.
Brief Description of the Drawings
[0016]
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Mode for Carrying Out the Invention
[0017] The present invention is an indoor damage estimation system for estimating the behavior of furniture installed indoors, including the floor response several seconds after an earthquake occurs. In the indoor damage estimation system, it is possible to estimate the time history response of the floor response obtained by the floor response estimation model (LSTM method) and the behavior of each piece of furniture obtained by the furniture behavior estimation unit (convolutional neural network). Further, after creating a composite spectrum from these respective values, the composite spectrum is subjected to forward and inverse Fourier transforms to predict the periodic characteristics of the furniture including the floor response. Hereinafter, embodiments for carrying out the present invention will be described in detail with appropriate reference to the drawings. Each drawing only schematically shows to such an extent that the present invention can be sufficiently understood. Therefore, the present invention is not limited only to the illustrated examples. In each drawing, for common components and similar components, the same reference numerals are given, and their repeated descriptions are omitted.
[0018] <Configuration of Indoor Damage Estimation System According to Embodiment> With reference to FIG. 1, the configuration of the indoor damage estimation system 1 according to the embodiment will be described. FIG. 1 is a schematic configuration diagram of the indoor damage estimation system 1 according to the embodiment. The indoor damage estimation system 1 is a system for estimating indoor damage caused by seismic motion. The damage in this embodiment is that furniture (an example is furniture) placed indoors moves or topples. The indoor damage estimation system 1 predicts the behavior of furniture with respect to the future floor response caused by seismic motion, and can particularly predict the movement of furniture several seconds after an earthquake occurs.
[0019] As shown in FIG. 1, the indoor damage estimation system 1 according to the embodiment includes an information acquisition unit (sensor 2, imaging device 3) that detects vibration or captures moving images and still images, and an indoor damage estimation device 4 that estimates the response of the floor slab and the behavior of furniture that occur with the occurrence of an earthquake. The sensor 2 and the imaging device 3 are installed inside the rooms of the building. The building has, for example, a plurality of floors, and rooms are provided on any of the floors. Furniture (an example is furniture) is placed indoors. The installation location of the indoor damage estimation device 4 is not particularly limited, and it can communicate with the sensor 2 and the imaging device 3 via data.
[0020] The sensor 2 is a sensor that detects vibration, and outputs, for example, the time history response waveforms of acceleration and displacement. The sensor 2 is installed, for example, on the ground surface or on the floor where the room is provided. The detected time history response waveform is transmitted to the indoor damage estimation device 4.
[0021] The imaging device 3 captures a moving image or a still image and outputs the captured moving image or still image. When capturing a still image, the imaging device 3 can perform continuous shooting at a predetermined time interval (e.g., several milliseconds). The captured moving image or still image (which may simply be referred to as an "image" without distinction) is transmitted to the indoor damage estimation device 4.
[0022] The indoor damage estimation device 4 estimates the indoor damage caused by seismic motion and is a device that estimates the response of the floor slab and the behavior of furniture occurring with the occurrence of an earthquake. The indoor damage estimation device 4 acquires the time history response waveform related to the room and the indoor image at the time of earthquake occurrence and predicts the movement of furniture several seconds later due to seismic motion. The indoor damage estimation device 4 is a personal computer (PC: Personal Computer) or an application server communicably connected to the personal computer. The indoor damage estimation device 4 may be a device constituting a cloud system. Note that the personal computer and the application server are examples of a computer.
[0023] The indoor damage estimation device 4 includes, for example, a learned model storage unit 10 that stores each information regarding the learned model for estimating indoor damage and the parameters used in the learned model, a control unit 20 that estimates indoor damage, and an estimation result output unit 30. In the present embodiment, it is assumed that the indoor damage estimation device 4 is a single device, but it may be configured by a plurality of devices. For example, the indoor damage estimation device 4 may have only the control unit 20, and the learned model storage unit 10 and the estimation result output unit 30 may be provided in another device.
[0024] The learned model storage unit 10 is a component that stores information necessary for estimating indoor damage. The learned model storage unit 10 is, for example, a storage medium such as a RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory. The learned model storage unit 10 transmits the stored information to the control unit 20 as necessary.
[0025] The control unit 20 is a component that estimates indoor damage through arithmetic processing. The control unit 20 realizes a learned model unit (response analysis unit 21, floor response estimation model learning processing unit 22, response behavior information creation processing unit 23) and a damage estimation main body unit (information acquisition unit for estimation 24, indoor furniture estimation unit 25, time history response waveform prediction unit 26, furniture behavior estimation unit 27, etc.). When the control unit 20 realizes various functions through program execution processing, a program (indoor damage estimation program) for realizing various functions is stored in the learning model storage unit 10. The control unit 20 acquires information necessary for estimating indoor damage from the learning model storage unit 10 as needed.
[0026] The estimation result output unit 30 is a component that outputs the result of the estimated indoor damage. The estimation result output unit 30 makes the time history waveform of the floor response after the earthquake, the behavior of the furniture, and the moving image of the furniture representing the behavior of the furniture at the time of the earthquake into a display or a generated file. For example, it is possible to display this information on a display by the display method described in Japanese Patent Application Laid-Open No. 2011-163837 "Building Condition Display System". An example of the display method by the estimation result output unit 30 will be described. The estimation result output unit 30, for example, displays virtual light emitting means corresponding to the floor within the screen, and changes the light emitting state of the light emitting means according to the physical quantity related to the floor response to notify the result of the indoor damage. Further, the estimation result output unit 30, for example, displays virtual light emitting means corresponding to each piece of furniture within the screen, and changes the light emitting state of the light emitting means according to the physical quantity related to the behavior of the furniture to notify the result of the indoor damage.
[0027] The learning model storage unit 10 stores, for example, a mass point system model, an indoor model, a furniture recognition model, a floor response estimation model, response behavior information, an indoor parsing diagram, and the like.
[0028] The mass point system model is a model of a building having rooms. Specifically, the mass point system model is building information data representing the position of the building, and the structural form and the number of floors, which are input data of the furniture behavior estimation unit 27 constituting the damage estimation main body unit. The indoor model is a model of an indoor space where furniture is arranged. The indoor model is, for example, a three-dimensional model created using 3D CAD or the like. The indoor model includes three-dimensional data of the furniture.
[0029] The furniture recognition model is an artificial intelligence (AI) that detects furniture from an indoor image and outputs the type, position, and dimensions of the detected furniture. The furniture recognition model can be realized, for example, by a method of instance segmentation (= object detection × semantic segmentation), and is constructed as a convolutional neural network (CNN). Note that the furniture recognition model may be configured as an artificial intelligence (AI) that separately estimates each of the type, position, and dimensions of the furniture. Specifically, the furniture recognition model is furniture information representing the position and three-dimensional shape of the furniture, which is the input data of the furniture behavior estimation unit that constitutes the damage estimation main body.
[0030] The floor response estimation model is an artificial intelligence (AI) that predicts the future floor response (for example, the time history response waveform of acceleration or displacement, such as several tens of seconds later) based on the past floor response (an example is the time history response waveform of acceleration or displacement) several seconds ago. The floor response estimation model can be realized by a recurrent neural network (RNN), and is preferably constructed as an LSTM (Long Short-Term Memory). LSTM is an extension of the recurrent neural network (RNN) and predicts the waveform of several seconds after ta from the waveform at time t of the time history waveform. Specifically, in the floor response estimation model, the input data is building information data representing the position of the building and its structural form and floor number, and the time history waveform data of the past floor response (acceleration and displacement). As the teacher data corresponding to the input data, the time history waveform data of the floor response after an earthquake is used as learning data, and it is constructed as an NN (neural network) as an example based on the LSTM network method.
[0031] In the RNN model, for a short-term and periodically close floor response (time history waveform), it is a method of predicting the values of the waveform in the subsequent ta seconds from the waveform in the previous tb seconds with the time point t as the boundary. However, since it is not suitable for long-term random waveforms such as seismic waves (waveforms of about several minutes), in this embodiment, an LSTM that can be applied to long-term random waveforms is adopted by expanding the RNN.
[0032] Referring to FIGS. 2 to 5, the configuration of the floor response estimation model will be described. FIG. 2 is a basic network configuration diagram. FIG. 3 is a configuration diagram of the RNN. FIG. 4 is the network structure of the LSTM. FIG. 5 is an improved version of the network structure of the LSTM. In this embodiment, except for some mathematical formulas (Formulas (3) to (7)), vector notation is used as scalar notation.
[0033] As shown in FIG. 2, the basic network is represented by a recurrent connection. Here, "f" and "g" represent functions, and since they are highly non-linear, they are represented by activation functions. The equation of this neural network can be expressed as Formula (1) below.
Equation
[0034] Thus, it is represented by two equations, and by eliminating the value "t" of the intermediate layer from Formula (1), it can be expressed as a composite function as in Formula (2). i * 」
Equation
[0035] On the other hand, since the equation of the RNN network shown in FIG. 3 needs to consider time, when expressed using time t, it becomes as in Formula (3).
Equation
[0036] Although the first equation of Equation (3) is expressed as a linear combination, considering that non-linear combinations are common in actual phenomena, the expression of Equation (4) can be obtained. Also, when this is arranged into a simplified equation, the equation of the RNN is expressed as in Equation (5).
Number
[0037] Here, the intermediate layer Z(t) is eliminated to form a composite function. Let the time from the current time to the previous time be t b and as the time step is gradually increased, Equation (4) can be expressed as in Equation (6).
Number
[0038] When the intermediate layer Z(t) is sequentially eliminated, it becomes as in Equation (7). From this Equation (7), if t of X(t) b and Z(t - t b ) are known, the next time step can be calculated.
Number
[0039] Next, referring to Figure 4, the LSTM that compensates for the demerits of the RNN will be described. The internal elements of the LSTM consist of four: "Input gate (Input) i t ", "Forget gate (Forget) f t ", "Output gate (Output) o t ", and "Memory area". In addition, the other symbols in Figure 4 are as follows. X t : Response data at time t, h t-1 , h t : Short-term memory data at time (t - 1) and time t, C with "~ (tilde)" t : New response candidate vector, C t-1 , C t: Old response vector and new response vector, σ: Sigmoid function, symbol combining ○ and ×: Tensor product, symbol combining ○ and +: Direct sum
[0040] Input gate i t is a function that determines how the output is reflected from the input and the data at the previous time. Forget gate f t is a function that adjusts to what extent the content of past memories is retained. Output gate o t is a function that feeds back the past output to the input. The memory area is a function that retains the past memories of the time history. The feature of this method is that the input gate i t and the forget gate f t are constructed, so that the data used can be minimized compared to the RNN shown in Figure 3. Also, it has a recursive structure in terms of input and forgetting, and the calculation time is also faster.
[0041] Subsequently, referring to Figure 4, each component of the LSTM will be described more specifically. (Input gate) Input gate i t processes the new input (x t ). The input gate i t takes h t-1 and the new input (x t ) and applies them to the sigmoid function σ and the tanh function respectively, and calculates a vector of new candidate values by XOR. Then, this new candidate value is added to the product of the forget gate f t and the old candidate value C t-1 .
Equation
[0042] (Forget gate) Forget gate f tis a gate for "not storing" unnecessary data, and when the time history waveform changes significantly, it performs a "not storing" process on the content once stored in the memory cell.
Number
[0043] (Memory area) The memory area performs a process for storing in a temporary memory the one calculated from the input gate i t and the forget gate f t from.
Number
[0044] (Output gate) The output gate o t is for processing the output. After performing sigmoid processing on the input in the same way as other gates, it has a structure of multiplying the cell state processed by the tanh function.
Number
[0045] However, this method is not a recursive type of the value calculated by the output gate o t and the reliability of the output is not always ensured. Therefore, as shown in FIG. 5, this patent constructs a network for ensuring prediction accuracy by returning the one calculated by the output gate o t back to the tanh of the input gate i t again. At the same time, by applying the recurrence of the memory area not only to forgetting but also to the input, it memorizes the feature amount of random waves such as seismic motions (see the dashed arrow in FIG. 5).
[0046] Referring to FIG. 5, each component of the LSTM (improved version) will be described more specifically. (Input gate) The input gate i t is, as in the case shown in FIG. 4, the new input (xt ) processes it. Input gate i t takes h t-1 and the new input (x t ) and applies them to the sigmoid function σ and the tanh function respectively, and XORs them to calculate a vector of new candidate values. However, by adding the past calculation result C t-1 stored in the memory area to the network function, the input features (such as amplitude and periodicity) are emphasized.
Number
[0047] (Forget gate) Forget gate f t is a gate for "not memorizing" unnecessary data, similar to the case shown in Figure 4. Contrary to the input gate i t , this process is added because it is considered applicable when deleting noise etc. by adding a process that does not memorize features among the past calculation results C t-1 stored in this memory area.
Number
[0048] (Memory area) The memory area performs a process of storing in a temporary memory what is calculated from the input gate i t and the forget gate f t .
Number
[0049] (Output gate) Output gate o t is for processing the output, similar to the case shown in Figure 4. The feature of the output gate o t is to calculate the amplitude of the output at the current time according to the weight of the memory area C t . Then, the input gate i t and the forget gate f tCalculate the optimal output from the characteristic quantities.
Number
[0050] The response behavior information shown in the learning model storage unit 10 of FIG. 1 is information that links the behavior of the furniture with the time history response waveform. The response behavior information is constructed, for example, as a function or artificial intelligence (AI). The response behavior information may be, for example, the application of the LightGBM (Light Gradient Boosting Machine) method of decision trees, and may be the shaking of the furniture according to the floor response of the earthquake represented by the mean squared deviation of the regression formula. Further, the response behavior information may be, for example, a model that has learned the relationship between the time history response waveform and the behavior of the furniture at each time corresponding to the type, position, and dimensions of the furniture.
[0051] The indoor parsing diagram is an indoor parsing diagram. The indoor parsing diagram is, for example, a diagram in which a viewpoint is set in an indoor model and shows the state during an earthquake as seen from the viewpoint, and is created continuously in time series.
[0052] The control unit 20 shown in FIG. 1 has a learned model unit (response analysis unit 21, floor response estimation model learning processing unit 22, response behavior information creation processing unit 23) and a damage estimation main body unit (estimation information acquisition unit 24, indoor furniture estimation unit 25, time history response waveform prediction unit 26, furniture behavior estimation unit 27, etc.). Here, the outline of each function will be described.
[0053] The response analysis unit 21 is a function for performing response analysis on the mass point system model. The floor response estimation model learning processing unit 22 is a function for learning the floor response estimation model. The response behavior information creation processing unit 23 is a function for creating response behavior information. The estimation information acquisition unit 24 is a function for acquiring information necessary for estimating indoor damage.
[0054] The indoor furniture estimation unit 25 is a function for recognizing the furniture shown in the indoor image and estimating the type, position, and dimensions of the furniture. The time history response waveform prediction unit 26 has a function of predicting the time history response waveform from the first time point to the second time point based on the time history response waveform from the occurrence of an earthquake to the first time point. The furniture behavior estimation unit 27 has a function of obtaining the behavior of the furniture at each time from the first time point to the second time point.
[0055] <Indoor damage estimation method by the indoor damage estimation system according to the embodiment> With reference to FIGS. 6 and 7 (and appropriately referring to FIG. 1), the indoor damage estimation method by the indoor damage estimation system 1 according to the embodiment will be described. The indoor damage estimation method according to the embodiment mainly includes a learning step (S10) of constructing a learned model and an estimation step (S20) of constructing a damage estimation main body part. FIG. 6 is an example of a flowchart showing the processing flow related to the learning step. FIG. 7 is an example of a flowchart showing the processing flow related to the estimation step. The learning step (S10) is performed prior to the estimation step (S20).
[0056] (Learning step (S10) of constructing a learned model) The learning step S10 mainly includes a learning preparation step S11, a response analysis step S12, a time history response waveform learning step S13, and a response behavior association step S14. The learning step S10 is performed in advance before an earthquake occurs.
[0057] In the learning preparation step S11, a mass point system model of the building and an indoor model of the room for estimating indoor damage are created. Furniture is arranged in the indoor model. In the response analysis step S12, the time history response waveform of each floor is obtained by performing response analysis on the mass point system model.
[0058] In the time history response waveform learning step S13, by inputting the floor time history response waveform from the earthquake occurrence time to the first time point into the floor response estimation model, the floor response estimation model is trained to output the floor time history response waveform from the first time point to the second time point that is after the first time point. In the time history response waveform learning step S13, not only a single time history response waveform (for example, an acceleration waveform) but also a plurality of time history response waveforms are read and trained in the floor response estimation model. Specifically, in the floor response estimation model, the input data is the building location, building information data representing the structural form and number of floors, and the time history waveform data of past floor responses (acceleration and displacement). Using the time history waveform data of the floor response after the earthquake as the teacher data corresponding to the input data, a deep learning model is constructed based on the LSTM network method.
[0059] In the response behavior association step S14, earthquake simulation is performed using the floor time history response waveform and the indoor model, and response behavior information is created based on the results of the earthquake simulation. The response behavior association step S14 includes, for example, an indoor model furniture estimation step S14a, a behavior determination step S14b, and a response behavior information creation step S14c.
[0060] In the indoor model furniture estimation step S14a, a viewpoint is set in the indoor model, and indoor perspective diagrams showing the state during the earthquake as seen from the viewpoint are continuously created in time series. Also, in the indoor model furniture estimation step S14a, the furniture shown in the indoor perspective diagram is recognized using a furniture recognition model, and the type, position, and dimensions of the furniture are estimated. In the indoor model furniture estimation step S14a, a depth estimation technique may be used to calculate the dimensions of the furniture considering the sense of distance. Also, the furniture may be recognized using the information registered in the three-dimensional data of the furniture constituting the indoor model, and the type, position, and dimensions of the furniture may be specified. It is also possible to create response behavior information using images taken indoors when an earthquake occurred in the past. In that case, indoor images taken during the earthquake are continuously prepared in time series, the furniture shown in the indoor images is recognized, and the type, position, and dimensions of the furniture are estimated.
[0061] In the behavior determination step S14b, the behavior of the furniture is derived from the difference in the context of the indoor floor plan diagrams that are continuous in time series using the Optical Flow technique. Here, Optical Flow is the displacement vector calculated for the movement of an object for each frame rate (sampling interval) of a moving image. When using the information registered in the three-dimensional data of the furniture that constitutes the indoor model, it is possible to derive the behavior of the furniture from the results of the earthquake simulation.
[0062] In the response behavior information creation step S14c, response behavior information is created based on the behavior of each piece of furniture over time and the time history response waveform of the floor. For example, the value at each time of the time history response waveform of the floor and the amount of movement of each piece of furniture at each time are obtained, and response behavior information is created based on the correlation relationship such as how much the furniture moves when the floor moves this much. The response behavior information may be, for example, the shaking of the furniture in response to the floor response of the earthquake represented by the mean squared deviation of the regression equation. It is also possible to create a response behavior model that takes the time history response waveform and the type, position, and dimensions of the furniture as inputs and outputs the behavior of the furniture at each time through learning.
[0063] (Estimation step (S20) that constitutes the damage estimation main body part) The estimation step S20 mainly includes an estimation preparation step S21, an estimation information acquisition step S22, an indoor furniture estimation step S23, a time history response waveform prediction step S24, a furniture behavior estimation step S25, and an estimation result display step S26. The estimation step S20 is carried out immediately after the earthquake occurs (during the period when the seismic motion continues), and estimates the indoor damage several seconds ahead in the earthquake that is occurring.
[0064] In the estimation preparation step S21, a furniture recognition model, a floor response estimation model, and response behavior information are prepared. The furniture recognition model used in the estimation step S20 may be the same as the furniture recognition model used in the learning step S10. The floor response estimation model and the response behavior information are those learned and created in the learning step S10.
[0065] In the process S22 of acquiring information for estimation, an indoor image captured by the imaging device 3 at the time of earthquake occurrence is acquired. Further, in the process S22 of acquiring information for estimation, a period from the time of earthquake occurrence to the first time point on the floor where the estimation is to be performed is obtained, and a time history response waveform detected by the sensor 2 is acquired.
[0066] In the indoor furniture estimation process S23, using a furniture recognition model, the furniture shown in the indoor image is recognized, and the type, position, and dimensions of the furniture are estimated. In the indoor furniture estimation process S23, a depth estimation technique may be used to calculate the dimensions of the furniture considering the sense of perspective.
[0067] In the time history response waveform prediction process S24, by inputting the time history response waveform of the floor of a specific floor from the time of earthquake occurrence to the first time point into a learned floor response estimation model, the time history response waveform of the floor of the specific floor from the first time point to the second time point is predicted.
[0068] In the furniture behavior estimation process S25, for example, based on the response behavior information, the estimated type, position, and dimensions of the indoor furniture, and the predicted time history response waveform from the first time point to the second time point, the behavior of the furniture at each time from the first time point to the second time point is obtained. Specifically, in the furniture behavior estimation unit 27, the input data is regarded as furniture information data representing the position and three-dimensional shape of the furniture and indoor image data representing the indoor damage situation including the furniture before and after the earthquake. As teacher data corresponding to the input data, indoor damage situation data including the behavior of the furniture after the earthquake is used as learning data, and a deep learning model is constructed based on a convolutional neural network.
[0069] In the furniture behavior estimation process S25, a composite spectrum is created using the predicted time history response waveform of the floor of a specific floor from the first time point to the second time point and the behavior of the furniture at each time for each individual furniture. Further, an inverse Fourier transform is performed on the composite spectrum, and by calculating the periodic characteristics of the furniture considering the influence of the time history response waveform, it can also be utilized for countermeasures against locking under the feet of the furniture. Specifically, as the periodic characteristics of the ease of shaking, in the building at the time of earthquake occurrence, calculate how the furniture installed in the building shakes or to what period of seismic motion the behavior of the furniture is sensitive.
[0070] In addition, in the furniture behavior estimation step S25, when the load obtained by multiplying the acceleration of the time history response waveform by the mass of the furniture exceeds the dynamic frictional force acting between the floor and the furniture, the amount of slip due to the dynamic frictional force may be added to the behavior of the furniture alone to increase the movement amount of the furniture.
[0071] In the estimation result display step S26, the behavior of the furniture at each time from the first time point to the second time point is displayed on the estimation result output unit 30. The display method of the estimation result is not particularly limited. In the estimation result display step S26, an indoor perspective view corresponding to the arrangement of the furniture and the behavior of the furniture at each time from the first time point to the second time point may be displayed on the estimation result output unit 30. In that case, for example, in the estimation preparation step S21, the indoor perspective view is stored in the learning model storage unit 10 in association with the arrangement and behavior of the furniture, and the indoor perspective view corresponding to the estimation result is acquired from the learning model storage unit 10 and displayed.
[0072] As described above, according to the indoor damage estimation system 1 according to the present embodiment, since the interaction with the movement of the floor is considered, it is possible to predict the movement of furniture against various seismic motions. Therefore, it is possible to accurately estimate the indoor damage based on the seismic motion. In addition, when applying a standard RNN, the prediction up to 0.1 second ahead was limited, but by using the improved version of the LSTM network structure shown in FIG. 5, it has become possible to predict 10 seconds ahead.
[0073] In order to verify the effect of the indoor damage estimation system 1 according to the present embodiment, a damage prediction test using actual data will be described. The flow of the prediction test is shown in FIG. 8. FIG. 8 is a flowchart showing the flow of the damage prediction test using actual data.
[0074] (1) Construction of a CNN for estimating the types of furniture (such as desks, chairs, and bookshelves) shown in the indoor image. (2) Construct a CNN for estimating the position and size of fixtures. Note that Fig. 9 is an illustration of an indoor model, and Fig. 10 is an image of the result by instance segmentation processing.
[0075] (3) Apply Optical Flow to grasp the behavior (amount of movement) of fixtures from the difference between images before and after movement. Note that Fig. 11 is an image of the ID addition process in Optical Flow. Fig. 12 is an image of the process to which Optical Flow is applied. (4) Since the size and amount of movement of fixtures may be different in actual size and amount of movement even if these values are the same on the image, a depth estimation AI that corrects the perspective difference is applied and calculated even with a monocular imaging device.
[0076] (5) The time history response waveform of the floor for reading is calculated based on the following conditions. As the input ground motion, those of three locations in Miyagi Prefecture, Ibaraki Prefecture, and a certain place in Tokyo due to earthquakes with the Tohoku region as the epicenter, and three locations in Hokkaido due to earthquakes with the Iburi region as the epicenter were used. As the target buildings, high-rise buildings (20 floors) and low-rise buildings (4 floors) were assumed, and the floor responses of the 1st, 5th, 10th, 15th, 20th floors of the high-rise building and each floor of the low-rise building were used as outputs.
[0077] (6) The time history waveform time was downsampled. In this process, since the sampling rate of the image is usually 30fps (equivalent to 30Hz) and the sampling time of the analysis in (5) is 100Hz, the greatest common divisor was applied to downsample to 5Hz. However, if it is technically possible to obtain the image at 100Hz instead of 30Hz in the future, it is also possible at 100Hz without downsampling.
[0078] (7) Set the time t in the time history response waveform of the floor, and determine the several seconds tb before and the several seconds ta after that. As an example, tb = 5 seconds before and ta = 10 seconds after were set. (8) The waveform several seconds after t seconds of the time history waveform is predicted by LSTM (Long Short-Term Memory) which is an extension of the recursive network (RNN) (see Fig. 5). An example of the prediction result of the floor time history response waveform is shown in Fig. 13. The horizontal axis of the graph shown in Fig. 13 is time (number of STEPs), and the vertical axis is acceleration. In Fig. 13, the measured value is shown by a thin line, and the predicted value is shown by a thick line.
[0079] (9) When determining the two learning models of the CNN model and the RNN model into one, the LightGBM method of the decision tree was applied. The situation for each time section of each instrument is predicted by the two learning models with the mean squared deviation of the regression formula for the shaking of the instrument according to the floor response of the earthquake. An example of the result of predicting the movement amount of instruments (desk, bookshelf, chair) due to an earthquake is shown in Fig. 14. The horizontal axis of the graph shown in Fig. 14 is time, and the vertical axis is the absolute value of the movement amount. In Fig. 14, the analytical value is shown by a thin line, and the estimated value by machine learning is shown by a thick line.
[0080] As a result of the damage prediction test using actual data, according to the indoor damage estimation system 1, it was confirmed that the periodic characteristics and the shape of the waveform representing the shaking method of the instruments installed in the building at the time of an earthquake can be reproduced (see Figs. 13 and 14). As described above, the embodiments of the present invention have been described, but the present invention is not limited thereto, and can be implemented without changing the gist of the claims.
Explanation of Signs
[0081] 1 Indoor damage estimation system 2 Sensor 3 Photographing device (camera, video) 4 Indoor damage estimation device 10 Learning model storage unit 20 Control unit 21 Response analysis unit 22 Floor response estimation model learning processing unit 23 Response behavior information creation processing unit 24 Estimation information acquisition unit 25 Indoor furniture estimator 26 Time history response waveform predictor 27 Furniture behavior estimator 30 Estimation result output unit
Claims
1. An indoor damage estimation system for estimating the indoor damage situation in a building due to an earthquake, comprising: input data as furniture information data representing the position and three-dimensional shape of furniture, and indoor image data representing the indoor damage situation including furniture before and after an earthquake; using indoor damage situation data including the behavior of furniture after an earthquake as learning data for teacher data corresponding to the input data, a furniture behavior estimation unit deeply learned based on a convolutional neural network; input data as building information data representing the position of the building, and the structural form and number of floors, and time history waveform data of a single floor in the past; using time history waveform data of a single floor after an earthquake as learning data for teacher data corresponding to the input data, a floor response estimation model deeply learned based on the LSTM network method; a damage estimation main body unit that inputs furniture information data and earthquake information data to be estimated for the indoor damage situation into the furniture behavior estimation unit and the floor response estimation model, and predicts the time history response of a single floor after an earthquake and the movement amount of furniture including the floor response; The movement amount of the furniture is characterized in that after creating a synthetic spectrum using the behavior of a single piece of furniture obtained by the furniture behavior estimation unit and the time history response of a single floor obtained by the floor response estimation model, forward and inverse Fourier transforms are performed on the synthetic spectrum, and the periodic characteristics of the susceptibility to shaking are calculated from the movement amount of the furniture including the floor response. An indoor damage estimation system.
2. In the damage estimation main body unit, after creating a synthetic spectrum using the predicted time history response waveform from the first time point to the second time point and the behavior of a single piece of furniture, an inverse Fourier transform is performed on the synthetic spectrum, and the movement amount of the furniture considering the influence of the time history response waveform is calculated. The indoor damage estimation system according to claim 1, characterized by the above.
3. In the damage estimation main body unit, when the load obtained by multiplying the acceleration of the time history response waveform by the mass of the furniture exceeds the dynamic frictional force acting between the floor and the furniture, the amount of slip due to the dynamic frictional force is added to the behavior of a single piece of furniture to increase the movement amount of the furniture. The indoor damage estimation system according to claim 1, characterized by the above.