House structure safety detection method and system based on multi-modal sensing fusion

Abstract

The invention provides a house structure safety detection method and system based on multi-modal sensing fusion, and relates to the technical field of house structure safety detection. The method comprises the following steps: acquiring multi-modal sensing data such as vibration, acoustic emission and temperature of a key part of a house structure; performing noise reduction, normalization processing and time sequence alignment on the data; extracting dynamic response features and environmental response features; calculating a dynamic stress index (DSI) and a composite stability factor (CSF); generating a real-time safety score of the house structure based on a multi-modal data fusion algorithm; judging whether potential safety hazards exist or not according to the safety score; after the hidden danger is confirmed, alarm information is generated and pushed to a user terminal. Through multi-modal data fusion, the safety of the house structure can be accurately evaluated, real-time monitoring and early warning are provided, and the long-term stability and safety of the house structure are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of housing structure safety detection, and particularly to a housing structure safety detection method and system based on multi-modal sensing fusion. Background Art

[0002] With the continuous development of the construction industry, the safety issues of housing structures have received increasing attention. Traditional housing structure safety detection methods mainly rely on manual inspections and single detection means, such as visual inspections, manual measurements, and the use of single sensors. These traditional methods are often limited by the experience and subjective judgment of the inspectors, and cannot achieve comprehensive and real-time monitoring of housing structures, presenting significant potential safety hazards and risks of missed detections.

[0003] In recent years, with the rapid development of sensing technology, Internet of Things technology, and artificial intelligence technology, multi-modal sensing fusion technology has begun to be applied in the field of housing structure safety detection. By simultaneously using multiple sensors (such as vibration sensors, acoustic emission sensors, and temperature sensors), more comprehensive and accurate structural health data can be obtained. However, existing multi-modal sensing data fusion technologies still face many challenges, mainly including the following points:

[0004] Complexity of multi-modal data processing: The data formats, frequencies, and precisions collected by different types of sensors vary greatly, making it a complex task to effectively fuse multi-modal data. Existing technologies often cannot achieve efficient and accurate data fusion, resulting in certain errors in the data processing results and affecting the accuracy of housing structure safety assessment.

[0005] Data noise and interference problems: In housing structure safety monitoring, the data collected by sensors often contains noise and interference. How to effectively remove this noise and extract useful information is a difficult problem in current technologies. Traditional noise reduction methods often rely on manual experience and lack pertinence, resulting in unstable results.

[0006] Lack of real-time performance and precision: The analysis results after multi-modal data fusion in existing technologies often cannot meet the real-time requirements. For the detection of structural hidden dangers, most methods rely on post-event analysis and cannot provide real-time warnings, thus missing the opportunity for timely intervention and repair.

[0007] Single evaluation index: Existing housing structure safety evaluation methods usually only rely on a single index, such as stress, deformation, etc., lacking consideration of comprehensive factors such as environmental factors and long-term wear and tear. Therefore, traditional safety evaluation methods cannot comprehensively reflect the overall stability of housing structures and may lead to missed and misjudged hidden dangers.

[0008] To solve the above problems, combining multi-modal sensing fusion technology with intelligent analysis methods has become an effective solution. By introducing parameters such as the Dynamic Stress Index (DSI) and the Composite Stability Factor (CSF), based on the multi-modal data fusion model, the present invention can achieve real-time monitoring, comprehensive evaluation of the housing structure, and provide accurate safety warnings. Through the application of this technology, the accuracy, real-time performance, and intelligent level of housing structure safety detection can be improved, and the potential safety risks brought by traditional detection methods can be significantly reduced. Summary of the Invention

[0009] To solve the technical problems of the complexity of multi-modal sensing data processing, data noise and interference problems, insufficient real-time performance and accuracy, and single evaluation index in the prior art, the present invention provides a housing structure safety detection method and system based on multi-modal sensing fusion.

[0010] The technical solutions provided by the present invention are as follows:

[0011] First aspect:

[0012] A housing structure safety detection method based on multi-modal sensing fusion provided by the present invention includes:

[0013] S1. Obtain multi-modal sensing data of key parts of the housing structure, where the sensing data includes vibration signals, acoustic emission signals, and temperature signals;

[0014] S2. Perform data preprocessing on the multi-modal sensing data, where the data preprocessing includes signal denoising, normalization processing, and multi-source time series alignment;

[0015] S3. Through a feature extraction module, extract the dynamic response features and environmental response features of the multi-modal sensing data, where the dynamic response features are used to describe the behavior of the housing structure under external forces, and the environmental response features are used to reflect the long-term impact of environmental factors on the housing structure;

[0016] S4. Introduce the Dynamic Stress Index DSI and the Composite Stability Factor CSF as parameters for safety evaluation. The Dynamic Stress Index is used to quantify the stress change rate of the housing structure under dynamic loads, and the Composite Stability Factor is used to evaluate the overall stability of the housing structure;

[0017] S5. Input the features into a fusion analysis model, which is based on a multi-modal data fusion algorithm and combines a historical safety threshold library to generate a real-time safety score of the housing structure;

[0018] S6. According to the real-time safety score, combine a specific threshold to determine whether there are safety hazards in the housing structure;

[0019] S7. After confirming potential safety hazards, generate corresponding alarm messages and push them to the user terminal, while recording the characteristics and locations of the hazards.

[0020] Second aspect:

[0021] A building structure safety detection system based on multi-modal sensing fusion provided by the present invention includes:

[0022] A multi-modal sensing unit for collecting vibration, acoustic emission, and temperature signals;

[0023] A data processing module for preprocessing multi-modal sensing data;

[0024] A feature extraction module for extracting dynamic response features and environmental response features;

[0025] A fusion analysis module for performing fusion analysis on the extracted features;

[0026] An alarm module for generating alarm messages after determining potential hazards;

[0027] A user terminal for receiving alarm messages and providing a visual analysis interface.

[0028] Through the collaborative work of the modules, the system realizes real-time detection and evaluation of the safety of building structures.

[0029] The beneficial effects brought by the technical solution provided by the present invention at least include:

[0030] (1) In the present invention, through the fusion of multi-modal sensor data, the advantages of different sensors can be fully utilized to comprehensively improve the accuracy and reliability of building structure safety monitoring. The comprehensive analysis of various sensing data such as vibration, acoustic emission, and temperature signals can provide a more comprehensive and accurate basis for the real-time evaluation of building structures, overcoming the detailed information that may be overlooked during single-sensor detection;

[0031] (2) In the present invention, by introducing parameters such as the dynamic stress index (DSI) and the composite stability factor (CSF), not only can the short-term stress response of the structure be reflected in real time, but also the influence of environmental factors can be comprehensively considered, thereby comprehensively evaluating the long-term stability and safety of building structures. The introduction of these parameters makes the present invention have higher sensitivity and adaptability, and can better meet the safety detection requirements of different types of building structures;

[0032] (3) In the present invention, by introducing parameters such as the dynamic stress index (DSI) and the composite stability factor (CSF), not only can the short-term stress response of the structure be reflected in real time, but also the influence of environmental factors can be comprehensively considered, so as to comprehensively evaluate the long-term stability and safety of the housing structure. The introduction of these parameters makes the present invention have higher sensitivity and adaptability, and can better meet the safety detection requirements of different types of housing structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0034] Figure 1 It is a schematic flowchart of a method for detecting the safety of a housing structure based on multi-modal sensing fusion provided by an embodiment of the present invention;

[0035] Figure 2 It is a schematic structural diagram of a system for detecting the safety of a housing structure based on multi-modal sensing fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] The following will describe the technical solutions in the present invention with reference to the drawings.

[0037] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "example" in the present invention should not be construed as more preferred or more advantageous than other embodiments or design solutions. Exactly speaking, the use of the word "example" is intended to present concepts in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one of the two can be selected.

[0038] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when their differences are not emphasized, the meanings they express are the same. "(of)", "corresponding", and "corresponding" can sometimes be used interchangeably. It should be noted that when their differences are not emphasized, the meanings they express are the same.

[0039] In the embodiments of the present invention, sometimes subscripts such as W1 may be miswritten as non-subscript forms such as W1. When their differences are not emphasized, the meanings they express are the same.

[0040] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0041] Refer to the attached Figure 1 figures, which show a schematic flow chart of a method for detecting the safety of a building structure based on multi-modal sensing fusion provided by an embodiment of the present invention.

[0042] An embodiment of the present invention provides a method for detecting the safety of a building structure based on multi-modal sensing fusion. This method can be implemented by a device for detecting the safety of a building structure based on multi-modal sensing fusion. This device for detecting the safety of a building structure based on multi-modal sensing fusion can be a terminal or a server. The processing flow of a method for detecting the safety of a building structure based on multi-modal sensing fusion can include the following steps:

[0043] S1. Obtain multi-modal sensing data of key parts of the building structure. The sensing data includes vibration signals, acoustic emission signals, and temperature signals.

[0044] It can be understood that the main purpose of this step is to comprehensively collect the performance data of the building structure under different environmental and load conditions. Vibration signals can reflect the dynamic response of the structure, acoustic emission signals can capture micro-cracks and damage signals in the structural materials, and temperature signals can provide long-term stability information related to environmental changes. The acquisition of multi-modal sensing data is the basis for subsequent analysis and evaluation, which can ensure the comprehensiveness and accuracy of monitoring information and provide multi-angle inputs for structural safety assessment.

[0045] S2. Perform data preprocessing on the multi-modal sensing data. The data preprocessing includes signal denoising, normalization processing, and multi-source time series alignment.

[0046] It can be understood that multi-modal sensing data often has problems such as noise, deviation, and time alignment. Direct analysis may lead to inaccurate results. Therefore, in the data preprocessing stage, irrelevant noise is removed through signal denoising algorithms to ensure the purity of the signal; different sensor data is converted into comparable standards through normalization processing to reduce the influence caused by dimensional differences; through multi-source time series alignment technology, the data from different sensors is ensured to be consistent in the time dimension, thereby improving the accuracy and credibility of subsequent analysis.

[0047] S3. Through a feature extraction module, extract the dynamic response features and environmental response features of the multi-modal sensing data. Among them, the dynamic response features are used to describe the behavior of the building structure under external forces, and the environmental response features are used to reflect the long-term impact of environmental factors on the building structure.

[0048] It is understandable that this step extracts key dynamic response features and environmental response features from multi-modal data through feature extraction techniques. The dynamic response features can help evaluate the stability of the building structure under different external forces and determine whether there are potential damages or safety hazards; the environmental response features can reflect the changing trends of the long-term impacts of environmental factors such as temperature and humidity on the building, thus helping to predict possible fatigue or aging problems of the structure. Feature extraction is a refined analysis process of multi-modal data and provides key information for subsequent safety assessments.

[0049] S4. Introduce the Dynamic Stress Index (DSI) and the Composite Stability Factor (CSF) as parameters for safety assessment. The Dynamic Stress Index is used to quantify the stress change rate of the building structure under dynamic loads, and the Composite Stability Factor is used to evaluate the overall stability of the building structure.

[0050] It is understandable that in traditional structural safety assessments, static stress and stability analyses are often relied on, but the dynamic response and long-term stability of building structures are equally crucial. The embodiments of the present invention introduce the Dynamic Stress Index (DSI) and the Composite Stability Factor (CSF). The former can quantify the stress change rate of the structure under dynamic loads and promptly reflect the impacts of factors such as external vibrations or wind loads on the structure; the latter comprehensively considers various factors and provides an assessment of the overall stability of the building. These two parameters effectively make up for the deficiencies of traditional assessment methods and can achieve more comprehensive and accurate safety assessments.

[0051] S5. Input the features into the fusion analysis model. The model, based on the multi-modal data fusion algorithm and combined with the historical safety threshold library, generates a real-time safety score for the building structure.

[0052] It is understandable that inputting the preprocessed and feature-extracted multi-modal data into the fusion analysis model for processing can comprehensively consider various information (such as dynamic responses, environmental factors, historical data, etc.) and achieve multi-dimensional safety assessments. Based on the multi-modal data fusion algorithm, the accuracy of the assessment can be effectively improved, and by comparing with the historical safety threshold library, a real-time safety score is generated. This score provides a quantitative indicator for the safety of the building and facilitates assessors and users to understand the current state of the structure.

[0053] S6. According to the real-time safety score, combined with specific thresholds, determine whether there are safety hazards in the building structure.

[0054] It is understandable that this step compares the real-time generated safety score with a preset safety threshold to determine whether there are potential safety hazards in the building structure. By comparing with the threshold, it can accurately judge whether the structure is within the safe range and timely detect potential risks in the structure. This judgment provides a basis for subsequent risk early warning and repair decision-making, helping to detect potential hazards at an early stage and carry out effective intervention.

[0055] S7. After confirming the safety hazards, generate corresponding alarm information and push it to the user terminal, and record the characteristics and location of the hazards at the same time.

[0056] It is understandable that when the system detects safety hazards in the building structure, it will notify the user through alarm information and provide the specific characteristics and location of the hazards. This function can timely remind building managers or residents to pay attention to potential dangers, and can accurately locate the location of the hazards, facilitating subsequent inspections and repairs. In addition, the data recording the characteristics and location of the hazards also provides valuable historical data for the later structural safety assessment, further improving the scientificity and efficiency of building structure health management.

[0057] In a possible implementation manner, the multimodal data fusion algorithm is based on the combination of a deep learning model and an expert rule model, where the deep learning model is used to identify complex non-linear relationships, and the expert rule model is used to manually correct the detection results based on experience.

[0058] Specifically, the multimodal data fusion algorithm adopts the combination of a deep learning model and an expert rule model. This fusion method can give full play to the respective advantages of the two, improving the accuracy and stability of data processing and analysis.

[0059] The role of the deep learning model: The deep learning model, especially models based on neural networks (such as convolutional neural network CNN, long short-term memory network LSTM, etc.), can automatically learn complex non-linear relationships from large-scale and multi-dimensional sensing data. These relationships may include the interaction between sensing data, the impact of environmental changes on structural responses, and the mutual dependence between different types of sensor data. Since the health state of a building structure is often affected by multiple factors, including the physical characteristics of the structure, environmental changes, external loads, etc., the deep learning model can capture these complex patterns and trends, and even discover potential laws that are difficult for human experts to identify in some cases. Through end-to-end training, the deep learning model can find key features related to the safety of the building structure in the high-dimensional data space and deeply mine the data, thereby improving the accuracy of structural safety assessment.

[0060] Role of the expert rule model: Although deep learning models have powerful learning capabilities, they still have certain limitations. Especially when facing certain specific situations, the prediction ability of the model may be affected by scarce data or complex environments. At this time, the expert rule model plays an important role. The expert rule model relies on the knowledge and experience of domain experts and combines industry standards for the safety assessment of building structures to formulate a series of clear rules. For example, for certain structural types, experts can formulate specific safety judgment rules based on past experience, or establish a simplified safety assessment framework based on known building material performance data. The role of the expert rule model is to correct the results of the deep learning model to ensure that the detection results are more in line with engineering practice and actual needs in the case of data loss or anomalies. In addition, the expert rule model can quickly respond to judgments in specific scenarios, avoid the model's over-reliance on data, and enhance the flexibility and robustness of the system.

[0061] Combination of deep learning and expert rule model:

[0062] The combination of the deep learning model and the expert rule model can not only enhance the automation ability of the system, but also use the expert rule model to "post-process" the results output by the deep learning model, improving the reliability and practicality of the results. Specifically, the deep learning model will automatically generate safety assessment results for the building structure based on the training set and real-time sensing data, while the expert rule model will manually correct these assessment results according to the actual situation to ensure compliance with the standards and common sense of structural safety. For example, when the assessment result given by the deep learning model is close to a certain critical value, the expert rule can automatically adjust this result according to experience to make the assessment more in line with the actual safety judgment standard. In addition, the expert rule model can also make additional adjustments for specific environmental changes or special situations, thereby improving the adaptability of the system.

[0063] Combining the advantages of deep learning and expert rule models, the multi-modal data fusion algorithm of the embodiments of the present invention not only improves the intelligent level of data analysis, but also enhances the accuracy of the results and the feasibility of engineering applications. This dual combination of intelligence and experience effectively solves the problem of processing and interpreting multi-modal sensing data in the safety assessment of building structures, realizing more efficient, accurate and real-time structural health monitoring.

[0064] In a possible implementation manner, the calculation method of the dynamic stress index DSI includes the following formula:

[0065]

[0066] where DSI represents the dynamic stress index, σ iThe stress value of sampling point i is denoted as, T represents the total length of the time period, and n represents the number of sampling points. The above formula is used to quantitatively evaluate the dynamic change amplitude of the structural stress and reflect the short-term response ability of the structure to external forces.

[0067] It should be noted that the Dynamic Stress Index (DSI) is used to quantify the stress change rate of the building structure under dynamic loads. By calculating the stress differences between each sampling point and accumulating them, DSI reflects the stress change amplitude of the structure under the action of external forces.

[0068] In a possible implementation, the calculation method of the Composite Stability Factor (CSF) includes the following formula:

[0069] CSF = α·S env +β·S dyn

[0070] where CSF represents the Composite Stability Factor, S env represents the environmental response score, which is quantified by environmental characteristics, and S dyn represents the dynamic response score, which is quantified by dynamic characteristics. The environmental characteristics include temperature and humidity, and the dynamic characteristics include the dynamic stress change rate. α and β respectively represent the weights of the environmental characteristics and dynamic characteristics, which are adaptively updated through historical data. The weight update formula is:

[0071]

[0072] where E env and E dyn respectively represent the cumulative errors of the environmental characteristics and dynamic characteristics.

[0073] It should be noted that the Composite Stability Factor (CSF) comprehensively considers the impacts of environmental factors and dynamic factors on the stability of the building structure. The weight update formula can dynamically adjust the influence weights of environmental and dynamic factors, thereby optimizing the stability assessment of the building structure according to the actual situation.

[0074] In a possible implementation, the fusion analysis model includes an improved clustering analysis algorithm for detecting abnormal states based on the historical data of the building and the current multi-modal characteristics, including the following steps:

[0075] S501. Reduce the dimensionality of the multi-modal feature space to reduce the interference of redundant information.

[0076] Specifically, the purpose of this step is to reduce the complexity of the data and remove redundant information by performing dimensionality reduction on multimodal sensor data, thereby improving the efficiency and accuracy of subsequent clustering analysis. Multimodal data comes from different sensors, such as vibration, acoustic emission, and temperature sensors. These data have different dimensions and correlations and may contain a large amount of redundant information. In the dimensionality reduction process, algorithms such as principal component analysis (PCA) or t-SNE are used to compress high-dimensional data into a lower-dimensional feature space while retaining the key information and features of the data as much as possible. Dimensionality reduction can not only improve the data processing speed, but also make it easier for clustering algorithms to identify potential patterns and anomalies in the data, thereby reducing the interference of redundant information on subsequent analysis results. This step is an important part of data cleaning and lays the foundation for accurate abnormal pattern detection.

[0077] S502: Based on the density peak clustering algorithm, identify abnormal patterns in different feature categories.

[0078] Specifically, in the feature space after dimensionality reduction, the Density Peaks Clustering (DPC) algorithm is used to identify abnormal patterns. The DPC algorithm divides data points into different clusters by calculating the local density and distance of data points, and determines points with higher density and far away from other clusters as abnormal points. In this case, the normal state of the house structure will cluster together to form a high-density area, while the abnormal pattern is manifested as isolated points with lower density and far away from other data points. Through this clustering process, potential problems in the house structure, such as local damage, fatigue or overload, can be effectively discovered. The advantage of this method is that it does not rely on the pre-set number of clusters, but can adaptively and automatically identify anomalies based on the distribution of data. Therefore, the DPC algorithm can flexibly identify various potential abnormal states in dynamically changing house structure data.

[0079] S503. Combined with time series trend analysis, perform secondary verification on abnormal patterns to eliminate false positives.

[0080] Specifically, after initially identifying abnormal patterns based on the density peak clustering algorithm, a secondary verification is further carried out through time series trend analysis to eliminate possible false alarms. Since the safety of the building structure is affected by various factors, there may occasionally be large fluctuations in certain sensing data, but these fluctuations do not necessarily mean that there are safety hazards in the structure. Therefore, relying solely on a single clustering analysis may lead to false alarms. To avoid this situation, this step further confirms the trend of the detected abnormal patterns in the time series by combining time series analysis (such as autoregressive model AR, moving average model MA, long short-term memory network LSTM, etc.). Specifically, if a similar fluctuation trend does not appear in the historical data for a certain abnormal pattern, or its fluctuation fails to match the long-term trend, then this abnormal pattern may be just short-term noise, and the system will exclude it. On the contrary, if this abnormal pattern shows a gradually intensifying trend over time or is consistent with the historical pattern, then its status as an actual safety hazard can be further confirmed. Through this secondary verification mechanism, this step can significantly improve the accuracy of the system, reduce the possibility of false alarms, and ensure the reliability of the monitoring results.

[0081] It should be noted that this series of steps together constitute an abnormal state detection process based on multi-modal data. S501 reduces the dimension of the data through dimensionality reduction, making subsequent clustering and trend analysis more efficient and accurate; S502 uses the density peak clustering algorithm to identify potential abnormal patterns, which not only improves the flexibility of data processing but also reduces manual intervention; S503 further verifies the initially identified abnormal patterns by combining time series trend analysis to ensure that the final detection results are accurate and eliminate the possibility of false alarms.

[0082] Through this multi-level detection method, it is possible to more accurately identify potential safety hazards in the building structure, especially those minor abnormalities that are difficult to detect by traditional monitoring means. This intelligent algorithm that comprehensively applies deep learning and clustering analysis can effectively improve the intelligence and automation level of building structure health monitoring and provide strong support for real-time structural safety assessment.

[0083] In a possible implementation, the update formula for the dynamic stress index is:

[0084]

[0085] where DSI updated represents the updated dynamic stress index, ΔF represents the change value of the external load, and F max represents the maximum load allowed for the building structure. The above formula is used to dynamically adjust the dynamic stress index to adapt to the impact of sudden external forces.

[0086] It should be noted that this formula is used to dynamically adjust the Dynamic Stress Index (DSI) to adapt to the impact of sudden external forces. The updated Dynamic Stress Index DSI pdated considers the change in external load ΔF and the maximum load-bearing capacity F of the building structure max ratio. By adjusting the dynamic response to changes in external load, this formula can help accurately reflect the stress changes in the building structure under sudden loads and provide real-time data for subsequent safety assessments.

[0087] In one possible implementation, the update formula for the composite stability factor is:

[0088]

[0089] where CSF updated represents the updated composite stability factor, ΔT represents the temperature change, and T max represents the maximum temperature difference allowed for the building structure. The above formula considers the dynamic impact of temperature changes on structural stability.

[0090] It should be noted that this formula is used to update the composite stability factor (CSF) based on the temperature change ΔT. By considering the impact of temperature changes on the stability of the building structure in an exponential decay manner. Dynamically adjusting the composite stability factor through temperature changes can reflect the impact of temperature fluctuations on the long-term stability of the structure. This formula helps identify structural stress changes caused by temperature differences and issue timely safety warnings.

[0091] In one possible implementation, the historical safety threshold library is established based on time, space, and material properties and continuously optimized through artificial intelligence methods to meet the diverse needs of different types of building structures.

[0092] Specifically, during the implementation of the embodiments of the present invention, the historical safety threshold library plays a very crucial role. It provides a reliable reference benchmark for the safety assessment of building structures. To ensure the accuracy and adaptability of this library, the historical safety threshold library is not only based on traditional safety assessment criteria but also combines the following three important factors: time, space, and material properties, and is continuously optimized through artificial intelligence (AI) methods to meet the diverse needs of different types of building structures.

[0093] The time factor is an important variable in evaluating the health of a house structure because the stability of the structure changes over time. As the house ages, environmental conditions change, and the frequency of use increases, the safety threshold of the house may gradually change. Therefore, the historical safety threshold database first records the house structure data collected at different time points and combines the changes in the health status of the structure to form dynamic safety thresholds. For example, the safety thresholds of a certain house may vary after 5 years, 10 years, and 20 years of use. These historical data provide a continuously updated safety assessment benchmark for the system, which can reflect the gradual changes in the safety of the structure over time.

[0094] The spatial factor means that the house structures at different locations may face different load and stress conditions, and this spatial difference will affect the setting of safety thresholds. For example, the load-bearing capacities of the upper and lower floors, the exterior and interior walls of the house may be different. Therefore, in the historical safety threshold database, the information in the spatial dimension will be recorded in detail. For the structural components at different locations, different safety thresholds will be determined based on the data collected by sensors (such as vibration, temperature, etc.). For example, a house located on a high floor may be affected by factors such as wind load and temperature change. These data will be recorded in the historical database and used as a benchmark for different spatial areas to provide safety assessment support.

[0095] Material properties have an important impact on the safety of house structures. Different building materials (such as concrete, steel, wood, etc.) have different load-bearing capacities and fatigue characteristics under different environmental conditions. In the historical safety threshold database, in addition to the time and space factors, it is also necessary to combine the material properties of the house structure for safety assessment. For example, for steel structures and concrete structures under the same load, their stress responses and compressive strengths may be different. Therefore, the historical safety threshold database will formulate safety threshold standards suitable for the material properties for each type of material by combining experimental data and on-site monitoring. These standards help to accurately evaluate the stability of house structures composed of different materials under different environments.

[0096] To enable the historical safety threshold library to continuously adapt to the diverse needs of different types of building structures and provide efficient and accurate safety assessments in building structure health monitoring, artificial intelligence methods play an important role in the construction and optimization of this library. Through AI methods such as machine learning and deep learning, based on the collected real-time data and historical data, the performance of building structures under different conditions can be automatically analyzed, thereby dynamically adjusting and optimizing the safety thresholds. For example, based on the supervised learning method, the system can learn the changes in the safety status of building structures under specific environmental conditions (such as high temperature, humidity, or earthquakes), and thus update the safety threshold library in real time to adapt to new environmental and safety requirements. In addition, unsupervised learning methods can also help identify potential risks that have not been seen before, and automatically adjust the threshold criteria based on the abnormal patterns existing in historical data. This AI-based optimization mechanism can not only automatically adjust the safety thresholds according to the real-time collected data, but also predict the possible future trends of structural damage, and then give early warnings of potential safety hazards.

[0097] With the continuous development of building design and material applications, the types of building structures are becoming increasingly diverse, including traditional reinforced concrete structures, steel structures, wood structures, and new composite material structures, etc. Through the optimization and adaptive adjustment of AI, the historical safety threshold library can optimize the safety thresholds for different structural types in a personalized manner. In each different type of building structure, the AI model gradually adjusts the applicable safety standards by analyzing a large amount of historical data, environmental changes, material properties, and load conditions to ensure that the safety of various structural types in different situations can be accurately evaluated.

[0098] By considering multiple dimensions of time, space, and material properties, combined with the optimization mechanism of artificial intelligence methods, the historical safety threshold library can adaptively optimize the safety assessments of different types of building structures, thereby providing more accurate and dynamic safety monitoring results. This dynamic optimization mechanism not only enhances the intelligent level of the system, but also provides a solid foundation for the long-term safety assessment of diverse building structures.

[0099] In a possible implementation manner, the alarm information specifically includes: the three-dimensional positioning results of the specific hidden danger parts of the building structure; safety repair suggestions provided according to the hidden danger types; and visual charts of historical records for assisting users in trend analysis.

[0100] Specifically, in the implementation of the embodiments of the present invention, when the system detects potential safety hazards in the building structure and confirms their danger, alarm information will be generated and pushed to the user terminal so that corresponding measures can be taken in a timely manner. To ensure the efficiency, intuitiveness, and operability of the alarm information, the content of the alarm information includes three important components: the three-dimensional positioning result of the specific hidden danger part of the building structure, the safety repair suggestions provided according to the type of hidden danger, and the visual chart of the historical records, which will be described in detail as follows:

[0101] When it is detected that there are safety hazards in the building structure, the first step is to accurately determine the three-dimensional positioning result of the hidden danger part. By integrating three-dimensional modeling technologies of the building structure (such as laser scanning, three-dimensional photogrammetry, or structural sensor data fusion, etc.), the system can provide the spatial coordinates of the hidden danger location and generate a three-dimensional visualization image of the location where the hidden danger is located. This positioning result can include the specific X, Y, Z coordinates of the part where the hidden danger occurs, as well as the relative position with other components of the building. This three-dimensional positioning information has the following advantages: helping users to clearly define the exact location of the hidden danger, avoiding misunderstandings and response delays caused by ambiguous positions; presenting intuitively, through the three-dimensional model, users can more intuitively understand the relationship between the area where the hidden danger is located and other parts of the building; on-site guidance: for engineering and technical personnel, the three-dimensional positioning information can directly guide them to the area where the hidden danger occurs, thereby improving the repair efficiency.

[0102] For example, if there is a crack or material aging in a certain part of the building, the system can not only give its approximate location, but also accurately display the spatial positions of the floor, wall, and other structural elements where this part is located. This information can be displayed through a three-dimensional visualization interface on terminal devices such as mobile phones or tablets.

[0103] According to the identified type of hidden danger, the system will automatically generate targeted safety repair suggestions. These suggestions are generated based on the safety history, current state of the building, and expert experience. The repair suggestions can include the following content:

[0104] Repair plan: For specific hidden dangers, the system will provide different repair plans, which may include material replacement, local reinforcement, structural re-design, etc. Each plan will evaluate different repair effects, time requirements, and required resources to help users make choices.

[0105] Emergency treatment measures: If the hidden danger is so serious that it may lead to structural instability or damage, the system will provide emergency treatment measures, such as evacuating personnel, strengthening supports, suspending use, etc., to ensure that safety risks are minimized as much as possible before repair.

[0106] Repair Priority: Based on the severity, location, and scope of influence of the potential hazards, the system will recommend the repair priority. Some potential hazards may require immediate attention, while others can be addressed based on further inspection results or scheduling.

[0107] Such repair recommendations based on the type of potential hazards can help building managers or technicians quickly understand the problem and provide scientific solutions, avoiding the cost and time waste caused by over-repair or incorrect repair methods.

[0108] To facilitate long-term monitoring and decision-making by building managers, the system will also provide visual charts of historical records to assist users in trend analysis. These charts usually include the following types of content:

[0109] Trend Chart of Potential Hazard Development: It shows the development trend of potential hazards in the building structure through charts, including the time of appearance of potential hazards, the amplitude of changes, and their frequencies. For example, if some potential hazards gradually worsen over time, the trend chart can clearly show these changes, helping users judge the severity of potential hazards.

[0110] Trend Chart of Multimodal Data: Based on multimodal sensing data such as vibration, acoustic emission, and temperature, it shows the historical data of the building structure state. These charts can display the fluctuations of monitoring data at different time points and different locations, helping users identify the structural change trends. For example, by analyzing the changes in vibration signals, users can determine whether the building is affected by external loads or shows signs of structural fatigue.

[0111] Evolution Chart of Safety Score: The safety score is a quantitative indicator generated based on multimodal data and historical safety thresholds. By showing the curve of the safety score changing over time through a chart, it helps users evaluate the overall health status of the building structure at different time points.

[0112] Through these visual charts, users can clearly see the safety status of the building structure at different time periods, conduct trend analysis, predict potential future risks, and formulate long-term maintenance plans. These historical record charts provide strong decision-making support for building managers and can help them make more accurate judgments in preventive maintenance.

[0113] The content of the alarm information not only includes the three-dimensional positioning results of potential hazards to ensure accurate location of the problem area, but also provides visual charts of safety repair suggestions and historical records, enabling users to quickly and intuitively understand the nature, severity, and solutions of potential hazards. This information can help building managers take effective measures in a timely manner to prevent the further expansion of safety hazards, and also provide data support for long-term monitoring and management to ensure the safety of the building structure during use.

[0114] Through this comprehensive alarm information transmission solution, the embodiments of the present invention can provide more comprehensive and intelligent house structure health management, greatly improving the efficiency of emergency response and the scientific nature of preventive maintenance.

[0115] The operation process of this embodiment is as follows:

[0116] Step 1: Obtain multi-modal sensing data: Step S1: Through vibration sensors, acoustic emission sensors, and temperature sensors installed at key parts of the house, the system collects multi-modal data of the house structure. Vibration signals are mainly used to detect the dynamic response of the structure under external loads, acoustic emission signals can reflect possible cracks or damages in the house structure, and temperature signals are used to evaluate the impact of environmental changes on the house structure.

[0117] Step 2: Data preprocessing: Step S2: Input the collected original multi-modal data into the data processing module for the following preprocessing operations: Signal denoising: Remove noise signals through filtering algorithms to ensure data accuracy; Normalization processing: Perform unified dimension conversion on different types of data for effective comparison; Multi-source time series alignment: Align the data collected by different sensors to ensure their synchronization in the time dimension.

[0118] Step 3: Feature extraction: Step S3: Through the feature extraction module, extract key features from the processed data: Dynamic response features: Such as vibration frequency, acceleration, etc., which are used to describe the behavior of the house structure under external forces and reflect the dynamic safety of the structure; Environmental response features: Such as temperature changes, humidity fluctuations, etc., which are used to reflect the long-term impact of environmental factors on the structure and identify possible long-term stability problems.

[0119] Step 4: Calculate the Dynamic Stress Index (DSI) and the Composite Stability Factor (CSF): Step S4: Based on the extracted dynamic response features and environmental response features, calculate the Dynamic Stress Index (DSI) and the Composite Stability Factor (CSF). The specific calculation methods and formulas are as described above and will not be elaborated here.

[0120] Step 5: Fusion analysis: Step S5: Input the extracted features (including DSI, CSF, etc.) into the fusion analysis module. This module uses an algorithm that combines deep learning and an expert rule model to fuse the features and generate a real-time safety score for the house structure. The deep learning model is used to identify complex non-linear relationships, and the expert rule model is used to correct the detection results based on practical experience, thereby improving the detection accuracy.

[0121] Step Six: Safety Hazard Judgment and Alarm: Step S6: Compare the generated real-time safety score with the safety standards in the historical safety threshold library. If the safety score is lower than the preset safety threshold, it is determined that there are safety hazards in the building structure. Step S7: After the hazards are confirmed, the system automatically generates an alarm message, which includes the following content: the 3D positioning result of the hazard location in the building structure; safety repair suggestions for the hazard type; a visual trend chart of historical data for assisting users in long-term monitoring and trend analysis.

[0122] Step Seven: Push the Alarm Message: Step S8: The alarm message is pushed to the user terminal in real time through the network. The user can view the hazard location, repair suggestions, and historical trend chart through the visual interface and take appropriate countermeasures according to the alarm message.

[0123] In this embodiment, through the collection, feature extraction, and fusion analysis of multi-modal sensing data, it is possible to effectively identify and evaluate potential safety hazards of the building structure, and timely notify the user through the alarm system to ensure the safety of the building. In addition, by combining historical data and environmental change factors, this method can not only reflect the current state in a timely manner but also provide long-term trend analysis, providing more comprehensive decision-making support for building managers. Through this technical solution, the real-time monitoring and early warning capabilities of the building structure are significantly improved, which can greatly reduce the occurrence of sudden safety accidents and provide a scientific basis for building maintenance and management.

[0124] The beneficial effects brought by the technical solution provided by the embodiment of the present invention at least include:

[0125] (1) In the present invention, through the fusion of multi-modal sensor data, the advantages of different sensors can be fully utilized to comprehensively improve the accuracy and reliability of building structure safety monitoring. The comprehensive analysis of various sensing data such as vibration, acoustic emission, and temperature signals can provide a more comprehensive and accurate basis for the real-time evaluation of the building structure, overcoming the detailed information that may be overlooked during single-sensor detection;

[0126] (2) In the present invention, by introducing parameters such as the dynamic stress index (DSI) and the composite stability factor (CSF), not only can the short-term stress response of the structure be reflected in real time, but also the influence of environmental factors can be comprehensively considered, so as to comprehensively evaluate the long-term stability and safety of the building structure. The introduction of these parameters makes the present invention have higher sensitivity and adaptability, and can better meet the safety detection requirements of different types of building structures;

[0127] (3) In the present invention, by introducing parameters such as the dynamic stress index (DSI) and the composite stability factor (CSF), not only can the short-term stress response of the structure be reflected in real time, but also the influence of environmental factors can be comprehensively considered, so as to comprehensively evaluate the long-term stability and safety of the housing structure. The introduction of these parameters makes the present invention have higher sensitivity and adaptability, and can better meet the safety detection requirements of different types of housing structures.

[0128] Refer to the attached Figure 2 figures, which show the structural schematic diagram of a housing structure safety detection system provided by an embodiment of the present invention.

[0129] The present invention also provides a housing structure safety detection system 20 based on multi-modal sensing fusion, which is applied to the above-mentioned housing structure safety detection method based on multi-modal sensing fusion, and includes:

[0130] A multi-modal sensing unit 201, which is used to collect vibration, acoustic emission and temperature signals;

[0131] A data processing module 202, which is used to preprocess the multi-modal sensing data;

[0132] A feature extraction module 203, which is used to extract dynamic response features and environmental response features;

[0133] A fusion analysis module 204, which is used to perform fusion analysis on the extracted features;

[0134] An alarm module 205, which is used to generate alarm information after determining potential hazards;

[0135] A user terminal 206, which is used to receive alarm information and provide a visual analysis interface.

[0136] Through the collaborative work of the above modules, the system realizes the real-time detection and evaluation of the safety of the housing structure.

[0137] The multi-modal sensing unit 201 is responsible for collecting vibration, acoustic emission and temperature signals of the housing structure, and these original sensing data are transmitted to the data processing module 202 through a data transmission interface. The data processing module 202 preprocesses the original data, including signal denoising, normalization processing and multi-source time series alignment, etc. The processed data will be passed as input to the subsequent feature extraction module 203.

[0138] The preprocessed data is transmitted to the feature extraction module 203. In the feature extraction module 203, through calculation and analysis, dynamic response features and environmental response features are extracted. The extracted features will be passed as input to the fusion analysis module 204 for subsequent processing.

[0139] The feature extraction module 203 transmits the extracted dynamic response features and environmental response features to the fusion analysis module 204. In the fusion analysis module 204, through a multimodal data fusion algorithm (including the combination of a deep learning model and an expert rule model), the features are comprehensively analyzed to obtain the real-time safety score of the building structure.

[0140] According to the safety score generated by the fusion analysis module 204, when the alarm module 205 detects a safety hazard in the building structure, it will generate an alarm message. The alarm message includes content such as the location of the hazard, repair suggestions, and historical trend charts, and is then transmitted to the user terminal 206.

[0141] The alarm module 205 transmits the alarm message to the user terminal 206 through a network connection. The user terminal 206 receives the alarm message and displays the safety status of the building structure through a visual analysis interface to assist the user in making decisions and responses. The user terminal 206 can also receive historical data analysis and long-term trend charts for assisting in building maintenance management.

[0142] The data flow sequence is: multimodal sensing unit 201 → data processing module 202 → feature extraction module 203 → fusion analysis module 204 → alarm module 205 → user terminal 206.

[0143] Each module is connected and data is transmitted through standard communication protocols (such as wired networks, wireless communications, etc.) to ensure the efficient flow of information within the system.

[0144] A building structure safety detection system based on multimodal sensing fusion provided by the present invention can execute the described building structure safety detection method based on multimodal sensing fusion and achieve the same or similar technical effects. To avoid repetition, the present invention will not be elaborated further.

[0145] The beneficial effects brought by the technical solution provided in the embodiments of the present invention at least include:

[0146] (1) In the present invention, through the fusion of multimodal sensor data, the advantages of different sensors can be fully utilized to comprehensively improve the accuracy and reliability of building structure safety monitoring. The comprehensive analysis of various sensing data such as vibration, acoustic emission, and temperature signals can provide a more comprehensive and accurate basis for the real-time assessment of the building structure, overcoming the detailed information that may be overlooked during single-sensor detection;

[0147] (2) In the present invention, by introducing parameters such as the Dynamic Stress Index (DSI) and the Composite Stability Factor (CSF), not only can the short-term stress response of the structure be reflected in real time, but also the influence of environmental factors can be comprehensively considered, so as to comprehensively evaluate the long-term stability and safety of the housing structure. The introduction of these parameters makes the present invention have higher sensitivity and adaptability, and can better meet the safety detection requirements of different types of housing structures;

[0148] (3) In the present invention, by introducing parameters such as the Dynamic Stress Index (DSI) and the Composite Stability Factor (CSF), not only can the short-term stress response of the structure be reflected in real time, but also the influence of environmental factors can be comprehensively considered, so as to comprehensively evaluate the long-term stability and safety of the housing structure. The introduction of these parameters makes the present invention have higher sensitivity and adaptability, and can better meet the safety detection requirements of different types of housing structures.

[0149] The above content is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

[0150] The following points need to be explained:

[0151] (1) The attached drawings of the embodiments of the present invention only relate to the structures involved in the embodiments of the present invention, and other structures can refer to the general design.

[0152] (2) For clarity, in the attached drawings used to describe the embodiments of the present invention, the thickness of the layer or region is enlarged or reduced, that is, these drawings are not drawn according to the actual scale. It can be understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element can be "directly" on or under another element or there can be an intermediate element.

[0153] (3) Without conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0154] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for detecting the safety of a building structure based on multi-modal sensing fusion, characterized in that Including: S1. Obtain multi-modal sensing data of key parts of the house structure, where the sensing data includes vibration signals, acoustic emission signals, and temperature signals; S2. Perform data preprocessing on the multi-modal sensing data, where the data preprocessing includes signal denoising, normalization processing, and multi-source time series alignment; S3. Through a feature extraction module, extract the dynamic response features and environmental response features of the multi-modal sensing data, where the dynamic response features are used to describe the behavior of the house structure under external forces, and the environmental response features are used to reflect the long-term impact of environmental factors on the house structure; S4. Introduce the dynamic stress index DSI and the composite stability factor CSF as parameters for safety assessment. The dynamic stress index is used to quantify the stress change rate of the house structure under dynamic loads, and the composite stability factor is used to evaluate the overall stability of the house structure; S5. Input the features into a fusion analysis model. The model is based on a multi-modal data fusion algorithm and combines a historical safety threshold library to generate a real-time safety score of the house structure; S6. According to the real-time safety score, combine a specific threshold to determine whether there are safety hazards in the house structure; S7. After confirming the safety hazards, generate corresponding alarm information and push it to the user terminal, and record the hazard characteristics and locations at the same time.

2. The method for detecting the safety of a house structure based on multi-modal sensing fusion according to claim 1, characterized in that: The multi-modal data fusion algorithm is based on the combination of a deep learning model and an expert rule model, where the deep learning model is used to identify complex non-linear relationships, and the expert rule model is used to manually correct the detection results based on experience.

3. The method for detecting the safety of a building structure based on multimodal sensing fusion according to claim 2, wherein, Including: The calculation method of the dynamic stress index DSI includes the following formula: Among them, DSI represents the dynamic stress index, and σ i represents the stress value at sampling point i, T represents the total length of the time period, and n represents the number of sampling points. The above formula is used to quantitatively evaluate the dynamic change range of structural stress and reflect the short-term response ability of the structure to external forces.

4. A method for detecting the safety of a building structure based on multimodal sensing fusion according to claim 3, characterized in that Including: The calculation method of the composite stability factor CSF includes the following formula: CSF = α·S env + β·S dyn Among them, CSF represents the composite stability factor, and S env represents the environmental response score, which is quantified by environmental characteristics, and S dyn represents the dynamic response score, which is quantified by dynamic characteristics. The environmental characteristics include temperature and humidity, and the dynamic characteristics include the dynamic stress change rate. α and β respectively represent the weights of the environmental characteristics and the dynamic characteristics, which are adaptively updated through historical data. The weight update formula is: Among them, E env and E dyn represent the cumulative errors of environmental characteristics and dynamic characteristics respectively.

5. A method for detecting the safety of a building structure based on multi-modal sensing fusion according to claim 4, characterized in that, Including: The fusion analysis model includes an improved clustering analysis algorithm for detecting abnormal states according to historical house data and current multi-modal features, including the following steps: S501. Reduce the dimension of the multi-modal feature space to reduce the interference of redundant information; S502. Based on the density peak clustering algorithm, identify abnormal patterns in different feature categories; S503. Combine time series trend analysis to perform secondary verification on the abnormal patterns to eliminate false alarms.

6. A method for detecting the safety of a building structure based on multi-modal sensing fusion according to claim 5, characterized in that, Including: The update formula of the dynamic stress index is: Among them, DSI updated represents the updated dynamic stress index, ΔF represents the value of external load change, and F max represents the maximum load allowed for the housing structure. The above formula is used to dynamically adjust the dynamic stress index to adapt to the influence of sudden external forces.

7. The method for detecting the safety of a building structure based on multi-modal sensing fusion according to claim 6, wherein, Including: The update formula of the composite stability factor is: Among them, CSF updated represents the updated composite stability factor, ΔT represents the temperature change, and T max represents the maximum temperature difference allowed for the building structure. The above formula considers the dynamic impact of temperature changes on structural stability.

8. A method for detecting the safety of a building structure based on multi-modal sensing fusion according to claim 7, characterized in that, Including: The historical safety threshold library is established based on time, space, and material properties, and is continuously optimized through artificial intelligence methods to meet the diverse needs of different types of house structures.

9. A method for detecting the safety of a building structure based on multi-modal sensing fusion according to claim 1, characterized in that, The alarm information specifically includes: The three-dimensional positioning result of the specific hidden danger part of the house structure; Safety repair suggestions provided according to the type of hidden danger; Visualization charts of historical records for assisting users in trend analysis.

10. A house structure safety detection system based on multi-modal sensing fusion, characterized in that, Including: A multi-modal sensing unit for collecting vibration, acoustic emission, and temperature signals; A data processing module for preprocessing multi-modal sensing data; A feature extraction module for extracting dynamic response features and environmental response features; A fusion analysis module for performing fusion analysis on the extracted features; An alarm module, which is used to generate alarm information after determining potential hazards; A user terminal, which is used to receive the alarm information and provide a visual analysis interface. Through the collaborative work of the modules, the system realizes the real-time detection and evaluation of the structural safety of the house.

Images