House structure flatness detection method and device based on improved multilayer neural network

Abstract

The invention discloses a building structure flatness detection method and device based on an improved multilayer neural network. The method comprises the following steps: synchronously acquiring a building structure image data set and a building physical environment data set; determining the characteristics of a bump area; storing the historical moment house structure image data set of the same position bulge and the corresponding house physical environment data set as time sequence data; constructing a twin neural network model containing a shared convolutional layer; calculating the similarity between the current bump and the historical bump; and judging whether the change speed of the current bump reaches a preset risk threshold value or not. According to the invention, fusion analysis is carried out on multi-source data through the twin neural network model, the limitation of a single data source is made up, so that the system can still accurately judge the risk of the bump under a complex environment condition, and the small change of the bump along with time evolution can be captured, so that the sensitivity and accuracy of detection are improved; and the consistency and robustness in multiple times of detection are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent house detection, and in particular to a house structure flatness detection method and device based on an improved multi-layer neural network. Background Art

[0002] Currently, bulge detection in building structures primarily relies on manual inspections or data collection from a single sensor. This results in significant limitations in detecting abnormal surface changes. Manual inspections are time-consuming and labor-intensive, and susceptible to subjective judgment, while a single sensor struggles to fully capture changes in the structure's state under complex environmental conditions. Furthermore, bulges in structures made of diverse materials and with uneven surfaces often exhibit small and irregular shapes, making existing technologies prone to missed detections or false positives.

[0003] Therefore, some existing methods incorporate computer vision technology, using image processing algorithms to detect bulges on building surfaces. This can improve detection efficiency to a certain extent, but there are still shortcomings. Specifically, traditional computer vision methods often rely on preset image feature extraction algorithms, which are difficult to adapt to the complexity of bulge morphology. Because the performance of building surfaces varies greatly under different materials and environmental conditions, image noise, lighting changes, and occlusion issues can significantly affect detection accuracy. In addition, single image detection cannot determine the development trend of bulges, and cannot provide effective change analysis and early warning for building structures that require long-term monitoring.

[0004] Other technical solutions use environmental sensors to collect physical data to detect structural changes. However, this data can only reflect the environmental state at a specific point in time, and its relationship with the bulge is complex and nonlinear. This makes it difficult for existing systems to effectively integrate multi-source data, resulting in insufficient robustness in detection results. Consequently, when subtle and gradual changes occur in the surface condition of a building structure, traditional sensor systems can easily miss early warning signals, missing the optimal opportunity for maintenance.

[0005] In summary, existing technologies for detecting bulges in building structures have the following main drawbacks: First, manual inspections and single-sensor detection methods cannot efficiently and accurately identify bulges, and there is a risk of missed detections and false alarms; second, existing image detection methods cannot cope with changes in different materials and complex environments, and lack robustness in detection results; finally, existing technologies lack the ability to analyze bulge change trends, unable to detect potential risks at an early stage and generate early warning reports in a timely manner. These drawbacks are particularly prominent when a building is aging or material stress gradually accumulates, which may pose a hidden danger to the safety of the building. In view of this, it is necessary to provide a method and device for detecting the flatness of building structures based on an improved multi-layer neural network to overcome the above-mentioned drawbacks. Summary of the Invention

[0006] The purpose of the present invention is to provide a method and device for detecting the flatness of a house structure based on an improved multi-layer neural network, aiming to solve the problems that existing detection methods cannot identify bulges efficiently and accurately, there is a risk of missed detection and false alarms, cannot cope with changes in different materials and complex environments, lack robustness of detection results, and lack the ability to analyze the changing trends of bulges. It can detect potential risks at an early stage and generate early warning reports in a timely manner.

[0007] In order to achieve the above-mentioned object, the first aspect of the present invention provides a method for detecting the flatness of a building structure based on an improved multi-layer neural network, comprising:

[0008] Step S1: Synchronously collect the house structure image dataset and the house physical environment dataset through the visual sensors and environmental sensors pre-installed on the surface of the house structure;

[0009] Step S2: pre-processing the house structure image dataset to determine the bulge area characteristics, and standardizing and time-synchronizing the house physical environment dataset;

[0010] Step S3: storing the processed current moment house structure image dataset and current moment house physical environment dataset into a database, and storing the historical moment house structure image dataset with bulge at the same location and the corresponding historical moment house physical environment dataset as time series data;

[0011] Step S4: Constructing a twin neural network model including a shared convolutional layer; the twin neural network model includes two networks with the same structure, which are respectively used to process the current moment house structure image dataset and the historical moment house structure image dataset; wherein, the annotated historical bulge data pairs are input into the twin neural network model as training samples for training, so that the twin neural network model learns the trend of bulge morphological changes at different time points by calculating the similarity between the two sets of house structure image datasets at the current moment and the historical moment;

[0012] Step S5: Pairing the real-time house structure image dataset with the historical house structure image dataset at the same location and inputting them into the trained twin neural network model, extracting the features of the bulge through a shared convolutional layer, and calculating the similarity between the current bulge and the historical bulge. The changes of the bulge under different environmental conditions are analyzed in combination with the corresponding house physical environment dataset to evaluate the morphological change trend of the bulge;

[0013] Step S6: Based on the output results of the twin neural network model, determine whether the current bulge change speed reaches the preset risk threshold. If the result is yes, trigger the alarm mechanism, generate a detection report and push it to the maintenance personnel's terminal device; if the result is no and the set maintenance threshold is not reached, record the detection result of the location, and optimize the judgment criteria for the next detection by continuously monitoring the bulge change trend.

[0014] In a preferred embodiment, step S1 includes:

[0015] Step S11: Acquire a house structure image dataset I(t) using a visual sensor pre-installed on the surface of the house structure:

[0016] I(t)={I1,I2,...,I n};

[0017] Among them, I i is the house surface image frame collected at time t, including the three-dimensional information of the width, length and color channel of the house. Each frame image I i corresponding to different spatial regions or perspectives;

[0018] Step S12: Acquire the house physical environment dataset E(t) through the environmental sensors pre-installed on the surface of the house structure:

[0019] E(t)={T(t), H(t), S(t)};

[0020] Wherein, T(t) represents the temperature data collected at time t, H(t) represents the humidity data collected at time t, and S(t) represents the stress data collected at time t. Environmental parameters are recorded synchronously with time t.

[0021] In a preferred embodiment, step S2 includes:

[0022] Step S21: Denoising the collected house structure image dataset I(t) to remove image noise caused by environmental interference or sensor noise, thereby obtaining a denoised house structure image dataset I'(t);

[0023] Step S22: grayscale the denoised house structure image dataset I'(t), converting the color information of each frame into a grayscale value representation to obtain the grayscale house structure image dataset I g (t);

[0024] Step S23: grayscale the house structure image dataset I g(t) Perform edge detection and use the preset edge detection algorithm to extract edge information in the image to obtain the house structure image dataset I e (t);

[0025] Step S24: Based on the edge detection result, the boundary features of the bulge area are extracted by morphological operation to obtain the bulge area feature house structure image dataset I. e′ (t), bulge area A i (t) and shape complexity S i (t);

[0026] Step S25: Use histogram equalization to equalize the area A of the bulge region i (t) and shape complexity S i The results of (t) are combined with the enhancement processing to extract the bulge area feature of the house structure image dataset I e′ (t) Perform feature enhancement to obtain the enhanced house structure image dataset I f (t);

[0027] Step S26: Use the maximum and minimum normalization method to normalize the house physical environment dataset E(t) to obtain the standardized house physical environment dataset E n (t):

[0028] E n (t) = {T n (t), H n (t), S n (t)};

[0029] Step S27: the house structure image dataset I f (t) and the standardized house physical environment dataset E n (t) Perform time synchronization processing to ensure the consistency of all data at the same time point t.

[0030] In a preferred embodiment, step S24 includes:

[0031] Step S241: dilate the boundary features of the bulge area through morphological operation. The dilation operation is:

[0032]

[0033] Among them, I d (t) is the expanded house structure image dataset, b is the structural element, B is the set of structural elements, is the expansion operator;

[0034] Step S242: Perform an erosion operation to refine and exclude areas that are not bulges.

[0035]

[0036] Among them, I e′ (t) House structure image dataset after corrosion processing, is the corrosion operator;

[0037] Step S243: After obtaining the processed house structure image dataset I e′ After (t), the shape and area features of the bulge are extracted using the preset contour analysis algorithm; the extraction formula is as follows:

[0038]

[0039] Among them, C i (t) represents the set of contour points extracted at time t, represents the image gradient, and p is the boundary point on the contour;

[0040] Step S244: Calculate the area A of the bulge area i (t) and shape complexity S i (t), the calculation formula is as follows:

[0041]

[0042] Among them, A i (t) is the area of ​​the bulge, S i (t) is the shape complexity, L i (t) is the perimeter of the bulge area.

[0043] In a preferred embodiment, step S4 includes:

[0044] Step S41: Construct a twin neural network model N containing a shared convolutional layer. The twin neural network model consists of two networks N1 and N2 with the same structure and shared parameters. Each network structure includes multiple convolutional layers, pooling layers, and fully connected layers, which are used to process the enhanced house structure image dataset I at the current moment. f (t) and historical moments of enhanced house structure image dataset I f (t-Δt); where N is used to learn the bulge characteristics of the house structure at different time points;

[0045] Step S42: The enhanced current moment house structure image dataset I f (t) is input to the network N1, and the bulge features are extracted through multiple convolutional layers to obtain the current bulge feature representation F(t):

[0046] F(t)=f θ (I f (t))=φ(Wn *φ(W n-1 *…φ(W1*I f (t)+b1)+b n-1 )+b n );

[0047] Among them, * represents the convolution operation, W i and b i are the convolution kernel and bias term of the i-th layer, i=1,2,...,n, n is the number of convolution layers, φ(·) is the activation function, f θ (·) represents the feature extraction function of the twin neural network, θ is the shared parameter set of the twin neural network model, and F(t) is the bulge feature vector at the current time t, which reflects the bulge morphological characteristics of the current house structure;

[0048] Step S43: The enhanced historical house structure image dataset I f (t-Δt) is input to network N2, and the bulge feature is extracted through the same multi-layer convolution layer as N1 to obtain the historical bulge feature representation f(t-Δt): F(t-Δt)=f θ (I f (t-Δt))=φ(W n *φ(W n-1 *…φ(W1*I f (t-Δt)+b1)+b n-1 )+b n );

[0049] Among them, I f (t-Δt) is the enhanced house structure image dataset at time t-Δt, Δt is the time interval, and F(t-Δt) is the bulge feature vector at the historical moment, which reflects the bulge morphological characteristics of the historical house structure;

[0050] Step S44: Calculate the difference feature vector ΔF(t) between the current bulge feature representation F(t) and the historical bulge feature representation F(t-Δt):

[0051] ΔF(t)=|F(t)-F(t-Δt)|;

[0052] Here, |·| represents the element-by-element absolute value operation of a vector, and ΔF(t) is the bulge feature change vector, which reflects the degree of change in the bulge morphology of the building structure between time t and t-Δt.

[0053] Step S45: Input the difference feature vector ΔF(t) to the fully connected layer and calculate the similarity score S(t). The calculation formula of the similarity score S(t) is:

[0054]

[0055] Where Ws is the weight vector of the fully connected layer, bs is the bias term, σ(·) is the Sigmoid activation function, and S(t) ranges from (0 to 1), which is used to represent the similarity score of the bulge change. The closer the value is to 1, the smaller the bulge change is, and the closer it is to 0, the larger the bulge change is.

[0056] Step S46: compare the annotated historical bulge data to {(I f (t i ), I f (t i -Δt), y i )} is input as a training sample to the twin neural network model N, where y i is the true label of the degree of bulge change, with a value of 0 or 1, where 0 indicates no obvious bulge change and 1 indicates a significant bulge change;

[0057] Step S47: Define the loss function L to measure the similarity score S(t i ) and the true label y i the differences between;

[0058] Step S48: Optimize the parameters θ of the twin neural network model by minimizing the loss function L, and iteratively update the twin neural network model parameters using a preset optimization algorithm so that the twin neural network model learns the trend of bulge morphology changes at different time points.

[0059] In a preferred embodiment, step S47 includes:

[0060] Step S471: Define the structural region imbalance loss function L for optimizing the parameters of the twin neural network model SRF , the structural regional imbalance loss function L SRF Used to measure the performance of the twin neural network model on unbalanced data, focusing on the error handling of the twin neural network model on the difficult-to-detect bulge area:

[0061]

[0062] Among them, n is the number of training samples, y i ∈{0,1} is the true label of sample i, which indicates the change of the bulge of the house structure. 0 means no obvious bulge change, and 1 means significant bulge change. S(t i )∈(0,1) is the similarity score output by the model, reflecting the bulge at the current time t i and historical moments i-Δt; γ> is a hyperparameter used to control the degree of emphasis the model places on difficult-to-detect areas; λ is the regularization coefficient, θ j is the jth parameter of the twin neural network model, with a total of m parameters;

[0063] Step S472: similarity score S(t i ) is defined, and the twin neural network model outputs a similarity score through the fully connected layer in step S45, which is used to represent the morphological change trend between the current moment house structure image dataset and the historical moment house structure image dataset;

[0064] Step S473: Introduce the imbalance weight term (1-S(t i )) γ , where γ> is used to control the weight of the bulge area that is difficult to detect. If the similarity score S(t i ) is smaller, indicating that the current detection sample is more difficult, and the corresponding weight is higher; if S(t i ) approaches 1, it means that the twin neural network model is easier to detect the sample, and the corresponding weight will be reduced;

[0065] Step S474: transform the structural area imbalance loss function L SRF Applied to the entire training dataset, the overall loss value is obtained by calculating the loss value of each sample and summing them up:

[0066]

[0067] Where λ is the regularization coefficient, is the L2 norm of the twin neural network model parameters;

[0068] Step S475: Adopting the adaptive moment estimation algorithm to define the optimization process of the loss function, and calculating the gradient of the structural region imbalance loss function with respect to the model parameters by back propagation:

[0069]

[0070] Among them, θ (k) and θ (k+1) They represent the model parameters after the kth and k+1th iterations respectively, and η is the learning rate to control the step size of each parameter update. is the structural area imbalance loss function L SRF Gradient with respect to the parameter θ;

[0071] Step S476: Through a predetermined number of iterative trainings, the model parameters gradually converge to the optimal level, so that the twin neural network model can identify the bulge area of ​​the house structure.

[0072] In a preferred embodiment, step S6 includes:

[0073] Step S61: similarity score S(t i ) and the bulge feature change vector ΔF(t) to determine the risk level of the bulge area; pre-set the risk threshold T risk and maintenance threshold T maint , respectively used to distinguish between emergency situations and normal detection situations;

[0074] Step S62: Calculate the bulge change speed V(t i ), the evolution trend of the bulge is measured by the hourly change of the bulge feature change vector:

[0075]

[0076] Among them, V(t i ) represents time t i The bulge change rate at the moment, Δt is the time interval between two adjacent detections, and ‖·‖2 represents the Euclidean distance norm;

[0077] Step S63: Change the bulge velocity V(t i ) and risk threshold T risk For comparison, if V(t i )≥T risk , it is determined that the current bulge area has reached the emergency treatment standard, the alarm mechanism is triggered, and a detection report is generated and pushed to the maintenance personnel terminal device; wherein, the generated detection report includes the location coordinates P of the bulge area i (x,y,z), change speed V(t i ), current and historical similarity scores S(t i ) and S(t i -Δt), and the area of ​​the bulge A i (t i ) and shape complexity S i (t i );

[0078] Step S64: If the bulge change speed V(t i )<T risk And the maintenance threshold T is not reached maint , then record the detection result of the position to the database as historical data for future detection calls; the recorded content includes the bulge feature vector ΔF(t i ) and physical environment data E n (t i ), used to optimize the judgment criteria of the model;

[0079] Step S65: Based on the monitoring data within the preset time period, by analyzing the time series of the bulge change speed {V(t1), V(t2), ..., V(t n )} to build a trend prediction model to predict the risk of bulge change in the future in advance; wherein, the trend prediction model uses linear regression to fit the bulge change speed:

[0080] V(t)=α·t+β;

[0081] Among them, α is the slope, which represents the growth trend of the bulge change rate, and β is the intercept, which is used to adjust the initial value of the fitting;

[0082] Step S66: Update the preset risk threshold T in real time according to the output result of the trend prediction model. risk and maintenance threshold T maint .

[0083] In a preferred embodiment, the step S6 further includes:

[0084] Step S7: Obtain feedback information on the test results from maintenance personnel after each test is completed, and then fine-tune the twin neural network model based on the feedback information. When the accumulated test data reaches a predetermined amount, the judgment strategy of the twin neural network model is optimized using the reinforcement learning mechanism.

[0085] A second aspect of the present invention provides a device for detecting the flatness of a building structure based on an improved multi-layer neural network, comprising:

[0086] A data acquisition module is used to synchronously acquire a house structure image dataset and a house physical environment dataset through visual sensors and environmental sensors pre-installed on the surface of the house structure;

[0087] A data processing module is used to pre-process the house structure image dataset to determine the characteristics of the bulge area, and to standardize and time-synchronize the house physical environment dataset;

[0088] A data storage module is used to store the processed current moment house structure image dataset and current moment house physical environment dataset in a database, and store the historical moment house structure image dataset with bulge at the same position and the corresponding historical moment house physical environment dataset as time series data;

[0089] A model construction module is used to construct a twin neural network model containing a shared convolutional layer; the twin neural network model includes two networks with the same structure, one for processing a current moment housing structure image dataset and a historical moment housing structure image dataset, respectively; wherein the annotated historical bulge data pairs are input into the twin neural network model as training samples for training, so that the twin neural network model learns the trend of bulge morphological changes at different time points by calculating the similarity between the two sets of housing structure image datasets at the current moment and the historical moment;

[0090] A model training module is used to pair a real-time house structure image dataset with a historical house structure image dataset of the same location and input them into the trained twin neural network model. The module extracts the features of the bulge through a shared convolutional layer, calculates the similarity between the current bulge and the historical bulge, and analyzes the changes of the bulge under different environmental conditions in combination with the corresponding house physical environment dataset to evaluate the morphological change trend of the bulge.

[0091] The result judgment module is used to judge whether the current bulge change speed reaches the preset risk threshold based on the output results of the twin neural network model. If the result is yes, the alarm mechanism is triggered, and a detection report is generated and pushed to the maintenance personnel's terminal device; if the result is no and the set maintenance threshold is not reached, the detection result of the location is recorded, and the judgment criteria for the next detection are optimized by continuously monitoring the bulge change trend.

[0092] A third aspect of the present invention provides a terminal, which includes a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the various steps of the house structure flatness detection method based on the improved multi-layer neural network as described in any of the above embodiments.

[0093] A fourth aspect of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the various steps of the house structure flatness detection method based on the improved multi-layer neural network as described in any of the above embodiments.

[0094] A fifth aspect of the present invention provides a computer program product, comprising a computer program or instructions, which, when processed and executed, implement the various steps of the method for detecting the flatness of a building structure based on an improved multi-layer neural network as described in any one of the above-mentioned embodiments.

[0095] The method and device for detecting the flatness of a building structure based on an improved multi-layer neural network provided by the present invention have the following beneficial effects:

[0096] (1) A twin neural network structure with a shared convolutional layer is used to analyze the change trend of the bulge by comparing the bulge feature vectors at the current moment and the historical moment. The model can calculate the similarity between image features at different time points and judge the changes in the bulge area based on the feature differences. Compared with the traditional single-moment detection method, the twin neural network can capture the subtle changes of the bulge over time, improving the sensitivity and accuracy of detection. The model reduces the computational complexity during training by sharing the parameter structure, ensuring consistency and robustness in multiple detections.

[0097] (2) The building structure image data and physical environment data are processed in time synchronization, and multi-source data are integrated and analyzed through a twin neural network model to make up for the limitations of a single data source so that the system can still accurately judge the risk of bulging under complex environmental conditions. At the same time, the present invention introduces a trend prediction model in the preferred embodiment. By analyzing the time series of the bulging change speed, it predicts possible structural problems in the future in advance. The system dynamically adjusts the risk threshold according to the trend analysis results to realize an adaptive early warning mechanism and intelligent maintenance plan, helping users to carry out structural repairs in a timely manner, reducing maintenance costs and improving the safety of buildings.

[0098] (3) In some preferred embodiments, a special structural area imbalance loss function is designed to solve the problem of false alarms and missed detections in difficult-to-detect areas during bulge detection. The loss function adjusts the weight of samples according to the difficulty of detecting the bulge change area by introducing a dynamic weighting term. For bulge areas with slower changes or complex shapes, the loss function will increase their weights during the training process to ensure that the model does not ignore these key areas. Compared with the traditional binary cross entropy loss function, the loss function of the present invention can significantly reduce the false alarm rate and missed detection rate, further improving the robustness of the model in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0100] Figure 1 A flow chart of a method for detecting the flatness of a building structure based on an improved multi-layer neural network provided by the present invention;

[0101] Figure 2 A schematic diagram of the twin neural network model architecture in the house structure flatness detection method based on the improved multi-layer neural network provided by the present invention;

[0102] Figure 3 This is a framework diagram of the house structure flatness detection device based on the improved multi-layer neural network provided by the present invention. DETAILED DESCRIPTION

[0103] In order to make the purpose, technical solution and beneficial technical effects of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described in this specification are only for the purpose of explaining the present invention and are not intended to limit the present invention.

[0104] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0105] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0106] Example 1

[0107] In an embodiment of the present invention, a house structure flatness detection method based on an improved multi-layer neural network is provided, which is used to automatically detect bulges on the surface of the house structure and determine whether the current bulges have reached preset risk thresholds and maintenance thresholds so that maintenance personnel can take corresponding measures.

[0108] like Figure 1 and Figure 2 As shown, the house structure flatness detection method based on the improved multi-layer neural network includes steps S1-S6.

[0109] Step S1: Synchronously collect a house structure image dataset and a house physical environment dataset using visual sensors and environmental sensors pre-installed on the surface of the house structure. Step S1 includes steps S11-S12.

[0110] Step S11: Acquire a house structure image dataset I(t) using a visual sensor pre-installed on the surface of the house structure:

[0111] I(t)={I1,I2,...,I n};

[0112] Among them, I i is the house surface image frame collected at time t, including the three-dimensional information of the width, length and color channel of the house. Each frame image Ii Corresponding to different spatial areas or perspectives.

[0113] Step S12: Acquire the house physical environment dataset E(t) through the environmental sensors pre-installed on the surface of the house structure:

[0114] E(t)={T(t), H(t), S(t)};

[0115] Wherein, T(t) represents the temperature data collected at time t, H(t) represents the humidity data collected at time t, and S(t) represents the stress data collected at time t. Environmental parameters are recorded synchronously with time t.

[0116] Step S2: pre-processing the house structure image dataset to determine the bulge area characteristics, and normalizing and time-synchronizing the house physical environment dataset. Step S2 includes steps S21-S27.

[0117] Step S21 , performing denoising processing on the collected house structure image dataset I(t) to remove image noise caused by environmental interference or sensor noise, thereby obtaining a denoised house structure image dataset I'(t).

[0118] Step S22: grayscale the denoised house structure image dataset I'(t), converting the color information of each frame into a grayscale value representation to obtain the grayscale house structure image dataset I g (t).

[0119] Step S23: grayscale the house structure image dataset I g (t) Perform edge detection and use a preset edge detection algorithm (e.g., Canny edge detection algorithm) to extract edge information in the image to obtain the house structure image dataset I e (t);

[0120] Step S24: Based on the edge detection result, the boundary features of the bulge area are extracted by morphological operation to obtain the bulge area feature house structure image dataset I. e′ (t), bulge area A i (t) and shape complexity S i (t). Specifically, step S24 includes steps S241-S244.

[0121] Step S241: dilate the boundary features of the bulge area through morphological operation. The dilation operation is:

[0122]

[0123] Among them, I d(t) is the expanded house structure image dataset, b is the structural element, B is the set of structural elements, is the expansion operator.

[0124] Step S242: Perform an erosion operation to refine and exclude areas that are not bulges.

[0125]

[0126] Among them, I e′ (t) House structure image dataset after corrosion processing, is the corrosion operator.

[0127] Step S243: After obtaining the processed house structure image dataset I e′ After (t), a preset contour analysis algorithm (such as a geometric distance algorithm, OpenCV contour detection algorithm, Canny edge detection and contour search algorithm, etc.) is used to extract the shape and area features of the bulge; the extraction formula is as follows:

[0128]

[0129] Among them, C i (t) represents the set of contour points extracted at time t, represents the image gradient, and p is the boundary point on the contour.

[0130] Step S244: Calculate the area A of the bulge area i (t) and shape complexity S i (t), the calculation formula is as follows:

[0131]

[0132] Among them, A i (t) is the area of ​​the bulge, S i (t) is the shape complexity, L i (t) is the perimeter of the bulge area.

[0133] Step S25: Use histogram equalization to equalize the area A of the bulge region i (t) and shape complexity S i The results of (t) are combined with the enhancement processing to extract the bulge area feature of the house structure image dataset I e′ (t) Perform feature enhancement to obtain the enhanced house structure image dataset I f (T).

[0134] Step S26: Use the maximum and minimum normalization method to normalize the house physical environment dataset E(t) to obtain the standardized house physical environment dataset En (t):

[0135] E n (t) = {T n (t), H n (t), S n (t)}.

[0136] Step S27: the house structure image dataset I f (t) and the standardized house physical environment dataset E n (t) Perform time synchronization processing to ensure the consistency of all data at the same time point t.

[0137] Step S3: The processed current moment house structure image dataset and current moment house physical environment dataset are stored in the database, and the historical moment house structure image dataset with bulge at the same location and the corresponding historical moment house physical environment dataset are stored as time series data.

[0138] Step S4: Construct a twin neural network model containing a shared convolutional layer. The twin neural network model consists of two identical networks, one for processing the current housing structure image dataset and the other for processing historical housing structure image datasets. Labeled historical bulge data pairs are input into the twin neural network model as training samples. By calculating the similarity between the current and historical housing structure image datasets, the twin neural network model learns the trends of bulge morphological changes at different time points.

[0139] Therefore, a twin neural network based on shared convolutional layers analyzes bulge features at different time points, calculates similarities and trends between features, effectively capturing subtle changes in the bulge and improving detection sensitivity and accuracy. The shared parameter design also reduces computational complexity and enhances model consistency and robustness.

[0140] In this step, step S4 includes steps S41-S48.

[0141] Step S41: Construct a twin neural network model N containing a shared convolutional layer. The twin neural network model consists of two networks N1 and N2 with the same structure and shared parameters. Each network structure includes multiple convolutional layers, pooling layers, and fully connected layers, which are used to process the enhanced house structure image dataset I at the current moment. f (t) and historical moments of enhanced house structure image dataset I f (t-Δt); where N is used to learn the bulge features of the house structure at different time points.

[0142] Step S42: The enhanced current moment house structure image dataset If (t) is input to the network N1, and the bulge features are extracted through multiple convolutional layers to obtain the current bulge feature representation F(t):

[0143] F(t)=f θ (I f (t))=φ(W n *φ(W n-1 *…φ(W1*I f (t)+b1)+b n-1 )+b n );

[0144] Among them, * represents the convolution operation, W i and b i are the convolution kernel and bias term of the i-th layer, i=1,2,...,n, n is the number of convolution layers, φ(·) is the activation function, f θ (·) represents the feature extraction function of the twin neural network, θ is the shared parameter set of the twin neural network model, and F(t) is the bulge feature vector at the current time t, which reflects the bulge morphological characteristics of the current house structure.

[0145] Step S43: The enhanced historical house structure image dataset I f (t-Δt) is input to network N2, and the bulge features are extracted through the same multi-layer convolutional layer as N1 to obtain the historical bulge feature representation f(t-Δt):

[0146] F(t-Δt)=f θ (I f (t-Δt))=φ(W n *φ(W n-1 *…φ(W1*I f (t-Δt)+b1)+b n-1 )+b n );

[0147] Among them, I f (t-Δt) is the enhanced house structure image dataset at time t-Δt, Δt is the time interval, and F(t-Δt) is the bulge feature vector at the historical moment, which reflects the bulge morphological characteristics of the historical house structure.

[0148] Step S44: Calculate the difference feature vector ΔF(t) between the current bulge feature representation F(t) and the historical bulge feature representation F(t-Δt):

[0149] ΔF(t)=|F(t)-F(t-Δt)|;

[0150] Here, |·| represents the element-by-element absolute value operation of a vector, and ΔF(t) is the bulge feature change vector, which reflects the degree of change in the bulge morphology of the house structure between time t and t-Δt.

[0151] Step S45: Input the difference feature vector ΔF(t) to the fully connected layer and calculate the similarity score S(t). The calculation formula of the similarity score S(t) is:

[0152]

[0153] Where Ws is the weight vector of the fully connected layer, bs is the bias term, σ(·) is the Sigmoid activation function, and S(t) ranges from (0, 1), which is used to represent the similarity score of the bulge change. The closer the value is to 1, the smaller the bulge change is, and the closer it is to 0, the larger the bulge change is.

[0154] Step S46: compare the annotated historical bulge data to {(I f (t i ), I f (t i -Δt), y i )} is input as a training sample to the twin neural network model N, where y i is the true label of the degree of bulge change, with a value of 0 or 1, where 0 indicates no obvious bulge change and 1 indicates a significant bulge change.

[0155] Step S47: Define the loss function L to measure the similarity score S(t i ) and the true label y i Specifically, step S47 includes steps S471-S476.

[0156] Step S471: Define the structural region imbalance loss function L for optimizing the parameters of the twin neural network model SRF , the structural regional imbalance loss function L SRF Used to measure the performance of the twin neural network model on unbalanced data, focusing on the error handling of the twin neural network model on the difficult-to-detect bulge area:

[0157]

[0158] Among them, n is the number of training samples, y i ∈{0,1} is the true label of sample i, which indicates the change of the bulge of the house structure, 0 means no obvious bulge change, and 1 means there is a significant bulge change; S(t i )∈(0,1) is the similarity score output by the model, reflecting the bulge at the current time t i and historical momentsi -Δt; γ>0 is a hyperparameter used to control the model's emphasis on difficult-to-detect areas; λ is the regularization coefficient, θ j is the jth parameter of the twin neural network model, and there are m parameters in total.

[0159] Step S472: similarity score S(t i ) is defined, and the twin neural network model outputs a similarity score through the fully connected layer in step S45, which is used to represent the morphological change trend between the house structure image dataset at the current moment and the house structure image dataset at the historical moment.

[0160] Step S473: Introduce the imbalance weight term (1-S(t i )) γ , where γ>0 is used to control the weight of the bulge area that is difficult to detect. If the similarity score S(t i ) is smaller, indicating that the current detection sample is more difficult, and the corresponding weight is higher; if S(t i ) approaches 1, it means that the twin neural network model is easier to detect the sample, and the corresponding weight will be reduced.

[0161] Step S474: transform the structural area imbalance loss function L SRF Applied to the entire training dataset, the overall loss value is obtained by calculating the loss value of each sample and summing them up:

[0162]

[0163] Where λ is the regularization coefficient, is the L2 norm of the twin neural network model parameters.

[0164] Step S475: Adopting the adaptive moment estimation algorithm to define the optimization process of the loss function, and calculating the gradient of the structural region imbalance loss function with respect to the model parameters by back propagation:

[0165]

[0166] Among them, θ (k) and θ (k+1) They represent the model parameters after the kth and k+1th iterations respectively, and η is the learning rate to control the step size of each parameter update. is the structural area imbalance loss function L SRF The gradient of the parameter θ.

[0167] Step S476: Through a predetermined number of iterative trainings, the model parameters gradually converge to the optimal level, so that the twin neural network model can identify the bulge area of ​​the house structure.

[0168] Therefore, the structural area imbalance loss function designed by the present invention adjusts the weight of the samples according to the detection difficulty of the bulge change area by introducing a dynamic weighting term. For bulge areas with slower change speed or complex morphology, the loss function will increase its weight during the training process to ensure that the model does not ignore these key areas. Compared with the traditional binary cross entropy loss function, the loss function of the present invention can significantly reduce the false alarm rate and missed detection rate, and further improve the robustness of the model in complex environments. That is, as an improved dynamically weighted loss function, it solves the problem of false alarms and missed detections in detection. The function adjusts the weight according to the complexity and change speed of the bulge area, pays special attention to areas that are difficult to detect, effectively reduces the false alarm rate and missed detection rate, and improves the detection performance in complex environments.

[0169] Step S48: Optimize the parameters θ of the twin neural network model by minimizing the loss function L, and iteratively update the twin neural network model parameters using a preset optimization algorithm (such as gradient descent, momentum method, Nesterov accelerated gradient (NAG), adaptive learning rate algorithm, etc.) to enable the twin neural network model to learn the trend of bulge morphology changes at different time points.

[0170] Step S5: Pair the real-time collected house structure image dataset with the historical house structure image dataset at the same location and input them into the trained twin neural network model. Extract the features of the bulge through the shared convolutional layer, and calculate the similarity between the current bulge and the historical bulge. Combined with the corresponding house physical environment dataset, analyze the changes of the bulge under different environmental conditions to evaluate the morphological change trend of the bulge.

[0171] Step S6: Based on the output results of the twin neural network model, determine whether the current bulge change speed reaches the preset risk threshold. If the result is yes, trigger the alarm mechanism, generate a detection report and push it to the maintenance personnel's terminal device; if the result is no and the set maintenance threshold is not reached, record the detection result of the location, and optimize the judgment criteria for the next detection by continuously monitoring the bulge change trend.

[0172] Specifically, step S6 includes steps S61-S66.

[0173] Step S61: similarity score S(t i) and the bulge feature change vector ΔF(t) to determine the risk level of the bulge area; pre-set the risk threshold T risk and maintenance threshold T maint , which are used to distinguish between emergency situations and normal detection situations.

[0174] Step S62: Calculate the bulge change speed V(t i ), the evolution trend of the bulge is measured by the hourly change of the bulge feature change vector:

[0175]

[0176] Among them, V(t i ) represents time t i The bulge change rate at the moment, Δt is the time interval between two adjacent detections, and ||·||2 represents the Euclidean distance norm.

[0177] Step S63: Change the bulge velocity V(t i ) and risk threshold T risk For comparison, if V(t i )≥T risk , it is determined that the current bulge area has reached the emergency treatment standard, the alarm mechanism is triggered, and a detection report is generated and pushed to the maintenance personnel terminal device; wherein, the generated detection report includes the location coordinates P of the bulge area i (x,y,z), change speed V(t i ), current and historical similarity scores S(t i ) and S(t i -Δt), and the area of ​​the bulge A i (t i ) and shape complexity S i (t i ).

[0178] Step S64: If the bulge change speed V(t i )<T risk And the maintenance threshold T is not reached maint , then record the detection result of the position to the database as historical data for future detection calls; the recorded content includes the bulge feature vector ΔF(t i ) and physical environment data E n (t i ), used as the judgment criteria for optimizing the model.

[0179] Step S65: Based on the monitoring data within the preset time period, by analyzing the time series of the bulge change speed {V(t1), V(t2), ..., V(t n)} to build a trend prediction model to predict the risk of bulge changes in the future. The trend prediction model uses linear regression to fit the bulge change speed:

[0180] V(t)=α·t+β;

[0181] Among them, α is the slope, which represents the growth trend of the bulge change rate, and β is the intercept, which is used to adjust the initial value of the fitting.

[0182] Step S66: Update the preset risk threshold T in real time according to the output result of the trend prediction model risk and maintenance threshold T maint .

[0183] Further, combined with Figure 1 As shown, step S6 also includes: step S7: obtaining feedback information on the detection results from the maintenance personnel after each detection is completed, and then fine-tuning the twin neural network model based on the feedback information. When the accumulated detection data reaches a predetermined amount, the judgment strategy of the twin neural network model is optimized using the reinforcement learning mechanism.

[0184] It should be noted that in order to verify the feasibility and superiority of the present invention in practical applications, the experimenters conducted a systematic experiment on the detection of the flatness of the house structure based on a combination of public data sets and simulation experiments. This exemplary experiment used an actual building bulge monitoring case and analyzed and compared it with the help of a public building defect detection data set to demonstrate the performance advantages of the present invention over traditional methods.

[0185] This experiment selected an office building built in 2008 for bulging detection experiments. The office building is located in City B, where climate change is obvious. Due to long-term exposure to high humidity, drastic temperature changes and wind erosion conditions, the exterior walls and load-bearing walls of the building are at risk of bulging and cracking. To ensure the safety of the building, the experimenters continuously monitored the structure of the building from January to June 2023, and used the monitoring data for model training and detection verification.

[0186] In the experiment, researchers selected 50 inspection areas on the south and west walls of the building and installed high-resolution cameras and various environmental sensors, including temperature, humidity, and stress sensors, to collect images and environmental data in real time. The dataset used in this experiment was derived from self-generated data and the publicly available CrackForest dataset for building defect detection. The data was divided into training and test sets in an 8:2 ratio.

[0187] During the experiment, image data and environmental data from 50 housing areas were collected once a week for a total of 6 months, with a cumulative data volume of 3,000 sets of images and multi-source environmental parameters. The resolution of each set of images was 1920×1080. The CrackForest dataset contains approximately 5,000 labeled building defect images with a resolution of 480×320, which are mainly used for preliminary model training and extraction of common building defect features.

[0188] During the image processing process, the experimenters used Gaussian filtering for denoising, grayscale processing to reduce redundant information, and the Canny edge detection algorithm to extract edge features. In terms of environmental data processing, maximum and minimum normalization was used to standardize the temperature, humidity and stress data.

[0189] The twin neural network model was trained using the two datasets mentioned above. The Adam optimizer was used with a learning rate set to 0.001 and a mini-batch size of 64. The model was trained for a total of 300 epochs. Initially, the CrackForest dataset was used to pre-train the model weights, and then the self-built dataset was input into the network for fine-tuning to improve the model's detection performance in complex environments.

[0190] During training, the model uses shared convolutional layers to process the bulge region features at both the current moment and the previous moment, calculating the similarity between the two. To address errors in hard-to-detect areas, the researchers introduced a structural region imbalance loss function and used hyperparameters to prioritize slowly changing regions. Finally, after model training, the detection areas were monitored in real time and trend analysis was performed.

[0191] Under the same experimental conditions, the experimenters used traditional manual inspection, single image detection, and the method of the present invention to monitor the building and recorded their respective detection data. Tables 1 and 2 show the performance comparison on different data sets:

[0192] Table 1: Comparison of detection results on self-built datasets

[0193]

[0194] Table 2: Comparison of detection results on the CrackForest public dataset

[0195] Detection method Detection accuracy Accuracy Recall F1 score Traditional convolutional neural network 81.5％ 80.2％ 78.8％ 79.5％ Multimodal detection (image + sensor) 85.4％ 84.8％ 82.1％ 83.4％ Method of the present invention 92.3％ 91.7％ 91.2％ 91.5％

[0196] From the above comparison results, it can be seen that the method of the present invention outperforms the traditional method in both the self-built dataset and the public dataset. In the self-built dataset, the method of the present invention missed detection only once, which is much lower than the 5 times of traditional manual inspection and the 3 times of single image detection. The detection time is greatly shortened, and a comprehensive detection is completed in only 20 minutes. In addition, through the trend analysis function of the twin neural network model, the system successfully triggered an early warning when the bulge change rate reached 0.008 square meters / week, avoiding maintenance delays caused by detection lags.

[0197] The experimental results on the CrackForest public dataset show that the method of the present invention is significantly superior to traditional convolutional neural networks and multimodal detection methods in detection accuracy, precision, recall rate and F1 score, demonstrating its advantages in dealing with complex building defects.

[0198] This exemplary experiment fully demonstrated the feasibility and effectiveness of the present invention by simulating the bulge detection process of an actual building and conducting comparative analysis in combination with a public dataset. Experimental data show that the method of the present invention not only improves the accuracy and efficiency of bulge detection, but also realizes intelligent early warning and maintenance optimization through multi-source data fusion and trend prediction functions. Compared with traditional methods, the present invention has significantly improved the missed detection rate, false alarm rate and detection time, providing strong technical support for the long-term monitoring and safety maintenance of buildings.

[0199] Example 2

[0200] The present invention provides a housing structure flatness detection device 100 based on an improved multi-layer neural network, which is used to automatically detect bulges on the surface of a housing structure and determine whether the current bulge has reached preset risk thresholds and maintenance thresholds, so that maintenance personnel can take corresponding measures. It should be noted that the implementation principle and specific implementation method of the housing structure flatness detection device 100 based on the improved multi-layer neural network can be referred to the above-mentioned housing structure flatness detection method based on the improved multi-layer neural network, and will not be repeated below.

[0201] like Figure 3 As shown, the house structure flatness detection device 100 based on the improved multi-layer neural network includes:

[0202] The data acquisition module 10 is used to synchronously acquire a house structure image dataset and a house physical environment dataset through visual sensors and environmental sensors pre-installed on the surface of the house structure;

[0203] A data processing module 20 is used to pre-process the house structure image dataset to determine the characteristics of the bulge area, and to standardize and time-synchronize the house physical environment dataset;

[0204] The data storage module 30 is used to store the processed house structure image dataset and the house physical environment dataset at the current moment in a database, and store the house structure image dataset and the corresponding house physical environment dataset at the same historical moment with a bulge at the same location as time series data;

[0205] A model construction module 40 is used to construct a twin neural network model including a shared convolutional layer; the twin neural network model includes two networks with the same structure, one for processing a current housing structure image dataset and a historical housing structure image dataset, respectively; wherein the annotated historical bulge data pairs are input as training samples into the twin neural network model for training, so that the twin neural network model learns the trend of bulge morphological changes at different time points by calculating the similarity between the two sets of housing structure image datasets at the current time and the historical time;

[0206] The model training module 50 is used to pair the real-time collected housing structure image dataset with the historical housing structure image dataset of the same location and input them into the trained twin neural network model, extract the characteristics of the bulge through the shared convolution layer, calculate the similarity between the current bulge and the historical bulge, and analyze the changes of the bulge under different environmental conditions in combination with the corresponding housing physical environment dataset to evaluate the morphological change trend of the bulge;

[0207] The result judgment module 60 is used to judge whether the current bulge change speed reaches the preset risk threshold based on the output results of the twin neural network model. If the result is yes, the alarm mechanism is triggered, and a detection report is generated and pushed to the terminal device of the maintenance personnel; if the result is no and the set maintenance threshold is not reached, the detection result of the location is recorded, and the judgment criteria for the next detection are optimized by continuously monitoring the bulge change trend.

[0208] Example 3

[0209] The present invention provides a terminal, which includes a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the various steps of the house structure flatness detection method based on the improved multi-layer neural network as described in any of the above-mentioned embodiments.

[0210] Example 4

[0211] The present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the various steps of the house structure flatness detection method based on the improved multi-layer neural network as described in any of the above-mentioned embodiments.

[0212] Example 5

[0213] The present invention provides a computer program product, including a computer program or instructions, which, when processed and executed, implement the various steps of the building structure flatness detection method based on the improved multi-layer neural network as described in any of the above-mentioned embodiments.

[0214] In summary, the method and device for detecting the flatness of a building structure based on an improved multi-layer neural network provided by the present invention have the following beneficial effects:

[0215] (1) A twin neural network structure with a shared convolutional layer is used to analyze the change trend of the bulge by comparing the bulge feature vectors at the current moment and the historical moment. The model can calculate the similarity between image features at different time points and judge the changes in the bulge area based on the feature differences. Compared with the traditional single-moment detection method, the twin neural network can capture the subtle changes of the bulge over time, improving the sensitivity and accuracy of detection. The model reduces the computational complexity during training by sharing the parameter structure, ensuring consistency and robustness in multiple detections.

[0216] (2) The house structure image data and the physical environment data are processed in time synchronization, and the multi-source data are integrated and analyzed through the twin neural network model to make up for the limitations of a single data source so that the system can still accurately judge the risk of bulging under complex environmental conditions. At the same time, the present invention introduces a trend prediction model to predict possible structural problems in the future by analyzing the time series of the bulging change speed. The system dynamically adjusts the risk threshold according to the trend analysis results to realize an adaptive early warning mechanism and intelligent maintenance plan, helping users to carry out structural repairs in a timely manner, reducing maintenance costs and improving the safety of buildings.

[0217] (3) In some preferred embodiments, a special structural area imbalance loss function is designed to solve the problem of false alarms and missed detections in difficult-to-detect areas during bulge detection. The loss function adjusts the weight of samples according to the difficulty of detecting the bulge change area by introducing a dynamic weighting term. For bulge areas with slower changes or complex shapes, the loss function will increase their weights during the training process to ensure that the model does not ignore these key areas. Compared with the traditional binary cross entropy loss function, the loss function of the present invention can significantly reduce the false alarm rate and missed detection rate, further improving the robustness of the model in complex environments.

[0218] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0219] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0220] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0221] In the embodiments provided by the present invention, it should be understood that the disclosed systems or devices / terminal devices and methods can be implemented in other ways. For example, the system or device / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the system or unit can be electrical, mechanical or other forms.

[0222] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0223] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0224] The present invention is not limited to what is described in the specification and embodiments, and additional advantages and modifications will be readily apparent to those skilled in the art. Therefore, the present invention is not limited to the specific details, representative devices, and illustrative examples shown and described herein without departing from the spirit and scope of the general concept defined by the claims and their equivalents.

Claims

1. A method for detecting the flatness of a house structure based on an improved multi-layer neural network, characterized in that: include: Step S1: synchronously collect the house structure image dataset and the house physical environment dataset through the visual sensor and environmental sensor pre-installed on the surface of the house structure; Step S2: preprocessing the house structure image dataset to determine the bulge area characteristics, and standardizing and time-synchronizing the house physical environment dataset; Step S3: storing the processed current moment house structure image dataset and current moment house physical environment dataset in a database, and storing the historical moment house structure image dataset with bulge at the same position and the corresponding historical moment house physical environment dataset as time series data; Step S4: constructing a twin neural network model including a shared convolutional layer; the twin neural network model includes two networks with the same structure, which are respectively used to process the house structure image dataset at the current moment and the house structure image dataset at the historical moment; wherein the annotated historical bulge data pairs are input into the twin neural network model as training samples for training, so that the twin neural network model learns the trend of bulge morphology changes at different time points by calculating the similarity between the two sets of house structure image datasets at the current moment and the historical moment; Step S5: Pair the real-time collected house structure image dataset with the historical house structure image dataset at the same location and input them into the trained twin neural network model, extract the features of the bulge through the shared convolution layer, calculate the similarity between the current bulge and the historical bulge, and analyze the changes of the bulge under different environmental conditions in combination with the corresponding house physical environment dataset to evaluate the morphological change trend of the bulge; Step S6: Based on the output results of the twin neural network model, determine whether the current bulge change speed reaches the preset risk threshold. If the result is yes, trigger the alarm mechanism, generate a detection report and push it to the maintenance personnel's terminal device; if the result is no and the set maintenance threshold is not reached, record the detection result of the location, and optimize the judgment criteria for the next detection by continuously monitoring the bulge change trend.

2. The method for detecting the flatness of a building structure based on an improved multi-layer neural network as claimed in claim 1, characterized in that: The step S1 comprises: Step S11, obtaining a house structure image dataset I(t) through a visual sensor pre-installed on the surface of the house structure: I(t)={I1,I2,...,I n }; Among them, I i is the house surface image frame collected at time t, including the three-dimensional information of the width, length and color channel of the house. Each frame image I of the house surface i Corresponding to different spatial regions or perspectives; Step S12: Obtain the house physical environment data set E(t) through the environmental sensors pre-installed on the surface of the house structure: E(t)={T(t), H(t), S(t)}; Among them, T(t) represents the temperature data collected at time t, H(t) represents the humidity data collected at time t, S(t) represents the stress data collected at time t, and the environmental parameters are recorded synchronously with time t.

3. The method for detecting the flatness of a building structure based on an improved multi-layer neural network as claimed in claim 2, characterized in that: The step S2 comprises: Step S21, denoising the collected house structure image dataset I(t) to remove image noise points caused by environmental interference or sensor noise, and obtain a denoised house structure image dataset I'(t); Step S22, graying the denoised house structure image dataset I'(t), converting the color information of each frame of the image into a gray value representation, and obtaining the grayed house structure image dataset Ig(t); Step S23: perform edge detection on the grayscaled house structure image dataset Ig(t), and use a preset edge detection algorithm to extract edge information in the image to obtain the house structure image dataset Ig(t). e (t); Step S24: Based on the edge detection result, the boundary features of the bulge area are extracted by morphological operation to obtain the bulge area feature house structure image dataset I. e′ (t), bulge area A i (t) and shape complexity S i (t); Step S25: Use histogram equalization to equalize the area A of the bulge region i (t) and shape complexity S i The results of (t) are combined with the enhancement processing to extract the bulge area feature house structure image dataset I e′ (t) Perform feature enhancement to obtain the enhanced house structure image dataset I f (t); Step S26: Use the maximum and minimum normalization method to standardize the house physical environment dataset E(t) to obtain a standardized house physical environment dataset E n (t): E n (t)={T n (t),H n (t),S n (t)}; Step S27: The house structure image dataset I f (t) and the standardized house physical environment dataset E n (t) Perform time synchronization to ensure the consistency of all data at the same time point t.

4. The method for detecting the flatness of a building structure based on an improved multi-layer neural network as claimed in claim 3, characterized in that: The step S24 comprises: Step S241: dilate the boundary features of the bulge area through morphological operation, and the dilation operation is: Among them, I d (t) is the expanded house structure image dataset, b is the structural element, B is the set of structural elements, is the expansion operator; Step S242: perform corrosion operation to refine and exclude areas that are not bulges: Among them, I e′ (t) House structure image dataset after corrosion processing, is the corrosion operator; Step S243: After obtaining the processed house structure image dataset I e′ (t), the shape and area features of the bulge are extracted using the preset contour analysis algorithm; the extraction formula is as follows: Among them, C i (t) represents the set of contour points extracted at time t, represents the image gradient, and p is the boundary point on the contour; Step S244: Calculate the area A of the bulge area i (t) and shape complexity S i (t), the calculation formula is as follows: Among them, A i (t) is the area of ​​the bulge, S i (t) is the shape complexity, L i (t) is the circumference of the bulge area.

5. The method for detecting the flatness of a building structure based on an improved multi-layer neural network as claimed in claim 3, characterized in that: The step S4 comprises: Step S41, construct a twin neural network model N containing a shared convolutional layer, the twin neural network model includes two networks N1 and N2 with the same structure and shared parameters, each network structure includes multiple convolutional layers, pooling layers and fully connected layers, which are used to process the enhanced house structure image dataset I at the current moment respectively. f (t) and historical moments of enhanced house structure image dataset I f (t-Δt); where N is used to learn the bulge characteristics of the house structure at different time points; Step S42: The enhanced current moment house structure image dataset I f (t) is input to the network N1, and the bulge features are extracted through multiple convolutional layers to obtain the current bulge feature representation F(t): F(t)=f θ (I f (t))=φ(W n *φ(W n-1 *…φ(W1*I f (t)+b1)+b n-1 )+b n ); Among them, * represents the convolution operation, W i and b i are the convolution kernel and bias term of the i-th layer, i = 1, 2, ..., n, n is the number of convolution layers, φ(·) is the activation function, f θ (·) represents the feature extraction function of the twin neural network, θ is the shared parameter set of the twin neural network model, and F(t) is the bulge feature vector at the current time t, which reflects the bulge morphological characteristics of the current house structure; Step S43: The enhanced historical house structure image dataset I f (t-Δt) is input to network N2, and the bulge features are extracted through the same multi-layer convolutional layer as N1 to obtain the historical bulge feature representation F(t-Δt): F(t-Δt) = f θ (If(t-Δt))=φ(W n-1 *...φ(W1*I f (t-Δt)+b1)+b n-1 )+b n ); Among them, I f (t-Δt) is the enhanced house structure image dataset at time t-Δt, Δt is the time interval, and F(t-Δt) is the bulge feature vector at the historical moment to reflect the bulge morphological characteristics of the historical house structure; Step S44, calculate the difference feature vector ΔF(t) between the current bulge feature representation F(t) and the historical bulge feature representation F(t-Δt): ΔF(t)=|F(t)-F(t-Δt)|; Among them, |·| represents the element-by-element absolute value operation of the vector, ΔF(t) is the bulge feature change vector, which reflects the degree of change of the bulge morphology of the house structure between time t and t-Δr; Step S45: input the difference feature vector ΔF(t) into the fully connected layer to calculate the similarity score S(t). The calculation formula of the similarity score S(t) is: Where Ws is the weight vector of the fully connected layer, bs is the bias term, σ(·) is the Sigmoid activation function, and S(t) ranges from (0,1) to represent the similarity score of the bulge change. The closer the value is to 1, the smaller the bulge change is, and the closer it is to 0, the larger the bulge change is. Step S46: compare the annotated historical bulge data to {(I f (t i ), I f (t i -Δt), y i } is used as a training sample to input into the twin neural network model N, where y i is the true label of the degree of bulge change, with a value of 0 or 1, where 0 indicates no obvious bulge change and 1 indicates a significant bulge change; Step S47, define the loss function L to measure the similarity score S(t i ) and the true label y i The difference between Step S48, optimize the parameters θ of the twin neural network model by minimizing the loss function L, and iteratively update the parameters of the twin neural network model using a preset optimization algorithm so that the twin neural network model learns the trend of the bulge morphology change at different time points.

6. The method for detecting the flatness of a building structure based on an improved multi-layer neural network as claimed in claim 5, characterized in that: The step S47 comprises: Step S471: define a structural region imbalance loss function L for optimizing the parameters of the twin neural network model SRF , the structural area imbalance loss function L SRF Used to measure the performance of the twin neural network model on unbalanced data to focus on the error handling of the twin neural network model on the difficult-to-detect bulge area: Where n is the number of training samples, y i ∈{0,1} is the true label of sample i, indicating the change of the bulge of the house structure, 0 means no obvious bulge change, and 1 means significant bulge change; S(t i )∈(0,1) is the similarity score output by the model, reflecting the bulge at the current time t i and the morphological similarity between the historical moment t1-Δt; γ>0 is a hyperparameter used to control the model’s emphasis on difficult-to-detect areas; λ is the regularization coefficient, θ j is the jth parameter of the twin neural network model, with a total of m parameters; Step S472: The similarity score S(t i ) is defined, and the twin neural network model outputs a similarity score through the fully connected layer in step S45, which is used to represent the morphological change trend between the house structure image dataset at the current moment and the house structure image dataset at the historical moment; Step S473: Introduce an imbalance weight term (1-S(t i )) γ , where γ>0 is used to control the weight of the bulge area that is difficult to detect. If the similarity score S(t i ) is smaller, the more difficult the current detection sample is, and the higher the corresponding weight is; if S(t i ) is close to 1, it means that the twin neural network model is easier to detect the sample, and the corresponding weight will be reduced; Step S474: transform the structural area imbalance loss function L SRF Applied to the entire training dataset, the overall loss value is obtained by calculating the loss value of each sample and summing them up: Among them, λ is the regularization coefficient, is the L2 norm of the twin neural network model parameters; Step S475: Adopting the adaptive moment estimation algorithm to define the optimization process of the loss function, the gradient of the structural region imbalance loss function to the model parameters is calculated by back propagation: Among them, θ (k) and θ (k+1) They represent the model parameters after the kth and k+1th iterations respectively, η is the learning rate to control the step size of each parameter update, is the structural area imbalance loss function L SRF The gradient of the parameter θ; Step S476: Through a predetermined number of iterative trainings, the model parameters gradually converge to the optimum, so that the twin neural network model can identify the bulging area of ​​the building structure.

7. The method for detecting the flatness of a building structure based on an improved multi-layer neural network as claimed in claim 6, characterized in that: The step S6 comprises: Step S61: similarity score S(t i ) and the bulge feature change vector ΔF(t) to determine the risk level of the bulge area; pre-set the risk threshold T risk and maintenance threshold T maint , respectively used to distinguish between emergency situations and normal detection situations; Step S62: Calculate the bulge change speed V(t i ), the evolution trend of the bulge is measured by the hourly change of the bulge feature change vector: Among them, V(t i ) represents the time t i The bulge change speed at the moment, Δt is the time interval between two adjacent detections, and ||·||2 represents the Euclidean distance norm; Step S63: Change the bulge change speed V(t i ) and risk threshold T risk For comparison, if V(t i )≥T risk , it is determined that the current bulge area has reached the emergency treatment standard, the alarm mechanism is triggered, and a test report is generated and pushed to the maintenance personnel terminal device; wherein the generated test report includes the position coordinates P of the bulge area i (x,y,z), speed of change V(t i ), current and historical similarity scores S(t i ) and S(t i -Δt), and the area of ​​the bulge A i (t i ) and shape complexity S i (t i ); Step S64: If the bulge change speed V(t i )<T risk And the maintenance threshold T is not reached maint , the detection result of the position is recorded in the database as historical data for future detection; the recorded content includes the bulge feature vector ΔF(t i ) and physical environment data E n (t i ), used to optimize the judgment criteria of the model; Step S65: Based on the monitoring data within the preset time period, by analyzing the time series of the bulge change speed {V(t1), V(t2), ..., V(t n )} to build a trend prediction model to predict the risk of bulge change that may occur in the future in advance; wherein, the trend prediction model uses linear regression to fit the bulge change speed: V(t)=a·t+β; Among them, α is the slope, which indicates the growth trend of the bulge change rate, and β is the intercept, which is used to adjust the initial value of the fitting; Step S66: Update the preset risk threshold T in real time according to the output result of the trend prediction model. risk and maintenance threshold T maint .

8. The method for detecting the flatness of a building structure based on an improved multi-layer neural network as claimed in claim 7, characterized in that: After step S6, the following steps are also included: Step S7: Obtain feedback information on the test results from maintenance personnel after each test is completed, and then fine-tune the twin neural network model based on the feedback information. When the accumulated test data reaches a predetermined amount, the judgment strategy of the twin neural network model is optimized using a reinforcement learning mechanism.

9. A house structure flatness detection device based on an improved multi-layer neural network, characterized in that: include: A data acquisition module, used for synchronously acquiring a house structure image dataset and a house physical environment dataset through a visual sensor and an environmental sensor pre-installed on the surface of the house structure; A data processing module, used for preprocessing the house structure image dataset to determine the characteristics of the bulge area, and for standardizing and time synchronizing the house physical environment dataset; A data storage module, used to store the processed house structure image dataset at the current moment and the house physical environment dataset at the current moment in a database, and store the house structure image dataset at the historical moment with a bulge at the same position and the corresponding house physical environment dataset at the historical moment as time series data; A model building module, for building a twin neural network model including a shared convolutional layer; the twin neural network model includes two networks with the same structure, which are respectively used to process a house structure image dataset at a current moment and a house structure image dataset at a historical moment; wherein the annotated historical bulge data pairs are input into the twin neural network model as training samples for training, so that the twin neural network model learns the trend of bulge morphology changes at different time points by calculating the similarity between the two sets of house structure image datasets at the current moment and the historical moment; A model training module is used to pair the real-time collected house structure image dataset with the historical house structure image dataset at the same location and input them into the trained twin neural network model, extract the features of the bulge through the shared convolution layer, calculate the similarity between the current bulge and the historical bulge, and analyze the changes of the bulge under different environmental conditions in combination with the corresponding house physical environment dataset to evaluate the morphological change trend of the bulge; The result judgment module is used to judge whether the current bulge change speed reaches the preset risk threshold according to the output results of the twin neural network model. If the result is yes, the alarm mechanism is triggered, and a detection report is generated and pushed to the terminal device of the maintenance personnel; if the result is no and the set maintenance threshold is not reached, the detection result of the location is recorded, and the judgment criteria for the next detection are optimized by continuously monitoring the bulge change trend.

Images