A method and system for analyzing landslide vulnerability of a building on a slope based on reinforcement learning

Abstract

The application discloses a kind of based on reinforcement learning's cutting slope house building landslide vulnerability analysis method and system, comprising: pre-processing multi-source data obtains disaster sample data set, missing mark vector and damage grade label;Extract slope and house characteristic parameters and fuse missing mark vector to generate state vector set;Action set is constructed and according to the difference between predicted damage distribution and damage grade label difference calculation external return;In conservative Q learning, action value network and target action value network are used to calculate the conservative penalty of non-data action;In random network distillation, calculate novelty error and generate internal return;Fusion return constructs state transition sample and updates action value network to obtain conservative action value function, and outputs the vulnerability conclusion of the scene to be analyzed.The application realizes stable training and iteration of evaluation model under the condition of sample scarcity and incomplete data.

Description

Technical Field

[0001] This invention relates to the field of geological disaster risk assessment technology, and in particular to a method and system for analyzing landslide vulnerability in slope-cutting housing construction based on reinforcement learning. Background Technology

[0002] Slope-cutting construction is widely distributed in mountainous and hilly areas. Slope excavation alters the original topography and stress field, and, combined with rainfall infiltration and surface runoff, easily induces shallow landslides and local instability, impacting the safety of houses and people. For vulnerability analysis in such scenarios, existing research mainly focuses on landslide intensity factors and damage levels of the affected body, using modeling based on disaster investigation, monitoring, and engineering survey information. Common approaches include expert-weighted indicator system assessment, vulnerability function fitting based on statistical regression, damage probability calculation based on probabilistic models, and damage level classification or damage probability prediction based on machine learning. These methods have been applied in engineering planning, risk zoning, and emergency management.

[0003] At the data level, the number of past disaster samples related to landslides caused by building construction on slopes is limited by the low frequency of events and the cost of investigation. Sample labels often come from post-disaster records, resulting in limited spatiotemporal resolution. On-site monitoring data suffers from sampling gaps, sensor failures, and insufficient spatial coverage. Engineering survey data has inconsistent standards across different projects, leading to missing input variables and distribution drift. Existing supervised learning-based models are highly dependent on labeled samples and are prone to parameter instability and generalization bias under conditions of small samples and incomplete input. Model updates require retraining and are difficult to iterate continuously with new samples.

[0004] At the model level, methods using static vulnerability curves or fixed regression forms have limited ability to characterize nonlinear triggering mechanisms and differences between the affected body and the target structure. While high-capacity models can enhance fitting ability, they are prone to overestimation and outputting distorted damage probabilities when samples are scarce, affecting the reliability of engineering decisions. Existing methods are insufficient in identifying novel scenarios and struggle to provide robust exploratory assessment results in the absence of direct sample support.

[0005] Therefore, how to provide a method and system for analyzing the landslide vulnerability of houses built on slopes based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a method and system for analyzing landslide vulnerability in slope-cutting housing construction based on reinforcement learning. This invention has the beneficial effects of suppressing overestimation of action value, improving the stability of analysis under conditions of small samples and missing data, and supporting iterative model updates.

[0007] According to an embodiment of the present invention, a method for landslide vulnerability analysis based on reinforcement learning for slope-cutting housing construction includes:

[0008] We acquired landslide disaster samples, field monitoring data, and engineering survey data by cutting slopes for housing construction, and preprocessed them to obtain disaster sample datasets, missing label vectors, and damage level labels.

[0009] Relevant data were extracted from the disaster sample dataset, and a set of state vectors was constructed by fusing missing label vectors.

[0010] Construct an action set, and select vulnerability parameter vectors for each state vector set. Calculate the difference between the predicted damage distribution generated by the vulnerability parameter vectors and the damage level label as the external reward value to obtain the state-action external reward sample.

[0011] Initialize the conservative Q-learning action value network and the target action value network, sample multiple sets of non-data actions for the external reward samples of state actions and calculate the action value to obtain the conservative penalty amount;

[0012] Initialize the random network distillation target network and the random network distillation prediction network, input the set of state vectors and calculate the network output difference, obtain the novelty error value and convert it into the intrinsic reward value;

[0013] The external reward value and the internal reward value are combined according to the fusion coefficient to generate a fused reward value, and then combined with the adjacent states to construct state transition samples;

[0014] The temporal difference target value is calculated based on the state transition sample. The action value network is updated by jointly updating the conservative penalty amount and the target action value network simultaneously, thus obtaining the conservative action value function.

[0015] For the scenario of building houses on slopes to be analyzed, a scenario state vector is constructed, the novelty error value corresponding to the scenario state vector is calculated and applied to the action value constraint, the vulnerability parameter vector is selected according to the conservative action value function and the vulnerability result is output, and the landslide vulnerability analysis conclusion of building houses on slopes is obtained.

[0016] Optionally, the preprocessing specifically includes:

[0017] Receive samples of landslide disasters caused by building houses on slopes, on-site monitoring data and engineering survey data, unify field naming, units of measurement and data types, and assign sample identifiers to each record to obtain a multi-source raw record table;

[0018] Spatial benchmark unification is performed on the multi-source original record table. After transforming the geographic coordinates to a unified coordinate system, the slope unit mapping is completed, and the mapping results are written into the corresponding field of the sample identifier to obtain the spatial registration record table.

[0019] Perform numerical validity checks and cross-field consistency checks on the spatial registration record table to obtain a consistency marker vector;

[0020] Consistency is filtered based on consistency label vectors. A subset of reliable records is formed by recordings. The network output difference is calculated by inputting the subset of reliable records into the target network and the prediction network of the random network distillation. After sorting the network output difference from smallest to largest, a preset percentile threshold is selected to generate an anomaly label vector.

[0021] For each field of the spatial registration record table, a missing marker vector is generated to obtain the missing marker vector;

[0022] Damage description fields were extracted from landslide disaster samples of houses built on slopes and a unified coding system was established. Based on the damage level mapping table, the unified coding was mapped to discrete damage levels to obtain damage level labels.

[0023] The spatial registration record table, missing marker vector, and damage level label are aligned and merged according to the sample identifier, and unreasonable records are removed to obtain the disaster sample dataset, missing marker vector, and damage level label.

[0024] Optionally, the construction of the state vector set specifically involves:

[0025] Locate a single record in the disaster sample dataset by sample identifier, and read the slope geometry parameters, soil and rock parameters, rainfall triggering parameters, slope cutting construction parameters, building structure load parameters and missing marker vector to obtain the sample parameter tuple;

[0026] Perform field expansion on the sample parameter tuple to obtain parameter subvectors;

[0027] In the disaster sample dataset, each numerical field is normalized to obtain a normalized parameter sub-vector group;

[0028] Read the building structure type code from the parameter sub-vector group, perform one-hot encoding to generate a structure type vector, and write it into the normalized parameter sub-vector group to obtain the structure enhancement sub-vector group;

[0029] Calculate feature interaction vectors based on structurally enhanced sub-vector groups;

[0030] A single state vector is generated by concatenating the structure enhancement sub-vector group, feature interaction vector, and observation completeness vector in a fixed splicing order, and a set of state vectors is generated based on all sample identifiers in the disaster sample dataset.

[0031] Optionally, the external report sample of the state action specifically includes:

[0032] Read each state vector in the state vector set and read the damage level label, and count the number of discrete damage levels corresponding to the damage level label;

[0033] The structure of each vulnerable parameter vector within the action set is expanded to obtain the action parameter set;

[0034] For each state vector and action parameter group in the state vector set, a grade score is calculated to form a discrete damage grade number grade score, which is then connected in order of damage grade to obtain a grade score vector.

[0035] Perform Softmax normalization on the grade score vector to form the predicted damage distribution;

[0036] Read the observation completeness vector from each state vector and take any dimension value as the completeness scalar. Construct the target damage distribution based on the completeness scalar to obtain the target damage distribution.

[0037] Calculate the difference between the predicted damage distribution and the target damage distribution to obtain the external reward value;

[0038] Each state vector, vulnerability parameter vector, and external reward value are associated with a sample identifier and written into the sample record to form a state-action external reward sample.

[0039] Optionally, the conservative penalty amount is specifically:

[0040] During the conservative Q-learning training process, each sample record in the external reward sample of state-action is read. The state vector and the corresponding vulnerability parameter vector in the sample record are input into the action value network and the target action value network, respectively, and the data action value and the target data action value are output.

[0041] For each sample record, 10 sets of non-data action corresponding vulnerability parameter vectors, which are different from the corresponding vulnerability parameter vectors, are sampled from the action set to obtain the non-data action parameter sequence;

[0042] Input the state vector and the non-data action parameter sequence into the action value network and the target action value network, respectively, and output the non-data action value sequence and the target non-data action value sequence, respectively.

[0043] The difference action value sequence is obtained by performing item-by-item difference calculation between the non-data action value sequence and the target non-data action value sequence, and the difference data action value is obtained by performing difference calculation between the data action value and the target data action value.

[0044] Read the observation completeness vector from the state vector and take the first dimension value as the completeness scalar, and perform temperature scaling on the differential action value sequence;

[0045] Perform logarithmic exponential aggregation on the temperature-scaled differential action value sequence to obtain the aggregated differential action value, and calculate the base penalty amount;

[0046] The conservative penalty is calculated based on the differential action value sequence.

[0047] Optionally, obtaining the novelty error value and converting it into an intrinsic reward value specifically involves:

[0048] Read the set of state vectors in the order of sample identifiers and form a mini-batch state sequence with 64 state vectors to obtain the mini-batch state sequence.

[0049] Input the mini-batch state sequence one by one into the random network to distill the target network and output the target feature vector sequence; input the mini-batch state sequence one by one into the random network to distill the prediction network and output the prediction feature vector sequence.

[0050] Perform dimension-wise difference between the target feature vector sequence and the predicted feature vector sequence, and sum the result after squaring the difference result dimension-wise to form a novelty error value sequence.

[0051] The mean scalar and standard deviation scalar are calculated on the novelty error value sequence to obtain the normalized novelty error value sequence;

[0052] Read the observation completeness vector from each state vector and take the first dimension value as the completeness scalar to obtain the completeness weighted novelty error value sequence;

[0053] Perform nonnegation on the completeness-weighted novelty error value sequence to obtain the intrinsic reward value sequence;

[0054] The sample identifier, the novelty error value sequence, and the intrinsic reward value sequence are written into the reward record according to the one-to-one correspondence of the sample identifier, so as to obtain the novelty error value and the intrinsic reward value.

[0055] Optionally, the construction of the state transition sample specifically includes:

[0056] Read two adjacent state vectors in the state vector set in the order of sample identifiers, mark them as the current state vector and the successor state vector respectively, and read the external reward value and internal reward value corresponding to the current state vector.

[0057] Read the observation completeness vector from the current state vector and take the first dimension value as the completeness scalar. Calculate the fusion coefficient as the difference between the completeness scalar and 1, and limit the fusion coefficient to the interval between 0 and 1.

[0058] The external return value and the internal return value are weighted and fused according to the fusion coefficient;

[0059] Read the vulnerability parameter vector corresponding to the current state vector and use it as the action vector. Combine it with the current state vector, the fusion reward value and the subsequent state vector in a fixed order to obtain a single state transition record.

[0060] For all state vectors corresponding to consecutive sample identifiers in the state vector set, repeat the calculation of all single state transition records to form state transition samples.

[0061] Optionally, the joint conservative penalty amount updates the action value network and simultaneously updates the target action value network, specifically as follows:

[0062] Read each state transition record in the state transition sample to obtain the current state vector, action vector, fusion reward value, and successor state vector;

[0063] For the successor state vector, 10 sets of vulnerable parameter vectors are sampled from the action set to form a candidate action vector sequence, and the successor state vector and the candidate action vector sequence are combined one by one to form a successor state-action pair sequence;

[0064] The successor state-action pair sequence is input into the target action value network one by one, and the target successor action value sequence is output. The target successor action value with the highest value is selected from the target successor action value sequence as the target action value scalar.

[0065] Read the observation completeness vector from the successor state vector and take the first dimension value as the successor completeness scalar to calculate the time series difference target value;

[0066] Input the current state vector and action vector into the action value network to output the current action value, calculate the temporal difference error, and obtain the temporal difference loss value;

[0067] The conservative penalty amount is read based on the sample identifier, and the joint loss value is obtained by summing the temporal difference loss value and the conservative penalty amount.

[0068] Backpropagation is performed on the action value network based on the joint loss value to update the network parameters. The updated action value network and the target action value network are output as the conservative action value function.

[0069] Optionally, the landslide vulnerability analysis conclusions for building houses on slopes are as follows:

[0070] Receive data on the slope-cutting building scenario to be analyzed and generate a scenario state vector;

[0071] The novelty error value of the scenario is calculated based on the random network distillation target network and the random network distillation prediction network.

[0072] The action value corresponding to each vulnerability parameter vector in the action set is calculated based on the conservative action value function, and action value constraints are applied to obtain the constrained action value sequence.

[0073] Select the vulnerability parameter vector corresponding to the maximum value of the constraint action value sequence as the scenario-optimal vulnerability parameter vector;

[0074] Calculate the predicted damage distribution of the scene based on the scene's optimal vulnerability parameter vector;

[0075] Based on the scenario, the damage distribution is predicted, and the analysis results of landslide vulnerability are output for houses built on slopes.

[0076] Optionally, a reinforcement learning-based system for analyzing landslide vulnerability in slope-cutting housing construction is characterized by comprising:

[0077] The data acquisition and preprocessing unit is used to receive landslide disaster samples from slope-cutting construction, on-site monitoring data, engineering survey data, and data of slope-cutting construction scenarios to be analyzed, and output disaster sample datasets, missing label vectors, and scenario data;

[0078] The state vector construction unit is used to extract slope geometric parameters, soil and rock parameters, rainfall triggering parameters, slope cutting construction parameters, and building structure load parameters from disaster sample datasets or scene data, and to fuse missing marker vectors, feature interaction vectors, and observation completeness vectors to generate a set of state vectors or scene state vectors.

[0079] The external reward calculation unit is used to calculate the difference between the predicted damage distribution and the target damage distribution based on the state vector, vulnerability parameter vector and damage level label, and generate state action external reward samples.

[0080] The conservative penalty calculation unit is used to combine the action value network and the target action value network under the conservative Q-learning framework to perform logarithmic exponential aggregation and dispersion adjustment on the data action value and non-data action value, and calculate the conservative penalty amount.

[0081] The novelty reward calculation unit is used to calculate the novelty error value by using random network distillation of the target network and random network distillation of the prediction network, and convert the novelty error value into the intrinsic reward value.

[0082] The state transition building unit is used to fuse external reward values ​​with internal reward values ​​and combine adjacent state vectors and action vectors to construct state transition samples.

[0083] The action value update unit is used to calculate the temporal difference target value based on the state transition sample, update the action value network in conjunction with the conservative penalty amount and synchronously update the target action value network to obtain the conservative action value function.

[0084] The vulnerability analysis output unit is used to calculate the predicted damage distribution of the slope-cutting building scenario based on the conservative action value function and the random network distillation network, and output the vulnerability analysis conclusion of the slope-cutting building landslide.

[0085] The beneficial effects of this invention are:

[0086] (1) This invention improves the availability of multi-source heterogeneous data by unifying the fields and dimensions of landslide disaster samples, field monitoring data and engineering survey data for building houses on slopes, and generating missing marker vectors and damage level labels. This effectively reduces the impact of incomplete data on the consistency of input for vulnerability analysis and enhances the data reliability in the model training and inference stages.

[0087] (2) This invention extracts slope geometric parameters, soil and rock parameters, rainfall triggering parameters, slope cutting construction parameters and building structure load parameters and integrates missing marker vectors to construct a set of state vectors. At the same time, it introduces feature interaction vectors and observation completeness vectors to achieve joint characterization of triggering factors and disaster-bearing body conditions. This significantly improves the ability to distinguish state characterization under small sample conditions and shows better adaptability in the vulnerability analysis task of slope cutting building scenario.

[0088] (3) In the conservative Q-learning training process, the present invention performs differential calculation on the data action value and non-data action value through the action value network and the target action value network and forms a conservative penalty amount, which effectively suppresses the problem of overestimation of action value under the condition of scarce samples, breaks through the limitation that high-capacity models are prone to unstable output when training with few samples, and realizes the stable update of the conservative action value function. Attached Figure Description

[0089] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0090] Figure 1 This is a flowchart of a landslide vulnerability analysis method for slope-cutting housing construction based on reinforcement learning proposed in this invention;

[0091] Figure 2 This is a flowchart illustrating the state vector set construction and state transition sample construction process of a landslide vulnerability analysis method for slope-cutting housing construction based on reinforcement learning proposed in this invention. Detailed Implementation

[0092] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0093] refer to Figures 1-2 A reinforcement learning-based method for analyzing landslide vulnerability in slope-cutting housing construction includes:

[0094] We acquired landslide disaster samples, field monitoring data, and engineering survey data by cutting slopes for housing construction, and preprocessed them to obtain disaster sample datasets, missing label vectors, and damage level labels.

[0095] Relevant data were extracted from the disaster sample dataset, including slope geometry parameters, soil and rock parameters, rainfall triggering parameters, slope cutting construction parameters, and building structure load parameters, and a set of state vectors was constructed by fusing missing marker vectors.

[0096] Construct an action set, in which each action corresponds to a set of vulnerability parameter vectors, and select vulnerability parameter vectors one by one for the state vector set. Calculate the difference between the predicted damage distribution generated by the vulnerability parameter vectors and the damage level label as the external reward value to obtain the state action external reward sample.

[0097] Initialize the conservative Q-learning action value network and the target action value network, sample multiple sets of non-data actions for the external reward samples of state actions and calculate the action value to obtain the conservative penalty amount;

[0098] Initialize the random network distillation target network and the random network distillation prediction network, input the set of state vectors and calculate the network output difference, obtain the novelty error value and convert it into the intrinsic reward value;

[0099] The external reward value and the internal reward value are combined according to the fusion coefficient to generate a fused reward value, and then combined with the adjacent states to construct state transition samples;

[0100] The temporal difference target value is calculated based on the state transition sample. The action value network is updated by jointly updating the conservative penalty amount and the target action value network simultaneously, thus obtaining the conservative action value function.

[0101] For the scenario of building houses on slopes to be analyzed, a scenario state vector is constructed, the novelty error value corresponding to the scenario state vector is calculated and applied to the action value constraint, the vulnerability parameter vector is selected according to the conservative action value function and the vulnerability result is output, and the landslide vulnerability analysis conclusion of building houses on slopes is obtained.

[0102] In this embodiment, the preprocessing specifically includes:

[0103] Receive samples of landslide disasters caused by building houses on slopes, on-site monitoring data and engineering survey data, unify field naming, units of measurement and data types, and assign sample identifiers to each record to obtain a multi-source raw record table;

[0104] Spatial benchmark unification is performed on the multi-source original record table. After transforming the geographic coordinates to a unified coordinate system, the slope unit mapping is completed, and the mapping results are written into the corresponding field of the sample identifier to obtain the spatial registration record table.

[0105] Numerical validity checks and cross-field consistency checks are performed on the spatial registration record table. The numerical validity check includes slope geometric parameter angle values ​​being greater than 0 degrees and less than 90 degrees, and soil and rock parameter physical quantities being positive. The cross-field consistency check includes matching building structure load parameters with building structure type codes to obtain a consistency marker vector.

[0106] Consistency is filtered based on consistency label vectors. A subset of reliable records is formed by recordings. The network output difference is calculated by inputting the subset of reliable records into the target network and the prediction network of the random network distillation. After sorting the network output difference from smallest to largest, a preset percentile threshold is selected to generate an anomaly label vector.

[0107] The random network distillation of the target network specifically involves:

[0108] A feedforward fully connected neural network structure is adopted. The network consists of an input layer, a first fully connected layer, a first nonlinear activation layer, a second fully connected layer, a second nonlinear activation layer, and an output layer. The number of neurons in the input layer is defined as the state vector dimension d, the output dimension of the first fully connected layer is defined as 2d, the output dimension of the second fully connected layer is defined as d, and the output dimension of the output layer is defined as d.

[0109] The input layer receives a record vector formed by aligning the fields, unifying the units of measurement, and unifying the data types of a single record within the trusted record subset. The record vector is then linearly normalized field by field using the minimum and maximum values ​​of the corresponding fields in the trusted record subset and converted into a floating-point representation.

[0110] The weight matrix of the first fully connected layer is generated using zero-mean, unit-variance Gaussian random numbers and written to the storage medium. The bias vector of the first fully connected layer is written to the storage medium using an all-zero vector. The first fully connected layer performs matrix multiplication and addition operations on the record vector to obtain the first intermediate vector.

[0111] The first nonlinear activation layer performs a modified linear unit transformation on the first intermediate vector to obtain the first activation vector;

[0112] The weight matrix of the second fully connected layer is generated using zero-mean, unit-variance Gaussian random numbers and written to the storage medium. The bias vector of the second fully connected layer is written to the storage medium using all-zero vectors. The second fully connected layer performs matrix multiplication and addition on the first activation vector to obtain the second intermediate vector. The second nonlinear activation layer performs modified linear unit transformation on the second intermediate vector to obtain the second activation vector. The output layer performs matrix multiplication and addition on the second activation vector to obtain the target feature vector.

[0113] The random network distillation prediction network is specifically as follows:

[0114] A feedforward fully connected neural network structure is adopted. The random network distillation prediction network consists of an input layer, a first fully connected layer, a first nonlinear activation layer, a second fully connected layer, a second nonlinear activation layer, and an output layer. The number of neurons in the input layer is defined as the state vector dimension, the output dimension of the first fully connected layer is defined as twice the state vector dimension, the output dimension of the second fully connected layer is defined as the state vector dimension, and the output dimension of the output layer is defined as the state vector dimension.

[0115] The input layer receives a record vector formed by aligning the fields of a single record within a subset of trusted records, unifying the units of measurement, and unifying the data types. The minimum and maximum values ​​of each field within the subset of trusted records are calculated to form a normalized parameter table. The record vector is then linearly normalized field by field using the normalized parameter table and converted into a floating-point representation.

[0116] The first fully connected layer performs matrix multiplication and addition on the recording vector to obtain the first intermediate vector. The first nonlinear activation layer performs modified linear unit transformation on the first intermediate vector to obtain the first activation vector. The second fully connected layer performs matrix multiplication and addition on the first activation vector to obtain the second intermediate vector. The second nonlinear activation layer performs modified linear unit transformation on the second intermediate vector to obtain the second activation vector. The output layer performs matrix multiplication and addition on the second activation vector to obtain the predicted feature vector.

[0117] The weight matrix of the first fully connected layer is generated using zero-mean, unit-variance Gaussian random numbers and written to the storage medium. The bias vector of the first fully connected layer is written to the storage medium using an all-zero vector. The weight matrix of the second fully connected layer is generated using zero-mean, unit-variance Gaussian random numbers and written to the storage medium. The bias vector of the second fully connected layer is written to the storage medium using an all-zero vector. The weight matrix of the output layer is generated using zero-mean, unit-variance Gaussian random numbers and written to the storage medium. The bias vector of the output layer is written to the storage medium using an all-zero vector.

[0118] The target feature vector output by the random network distillation target network and the predicted feature vector output by the random network distillation prediction network are subjected to a dimension-wise difference calculation, and the sum of the squared differences is obtained to obtain the network output difference. The network output difference is used as the training error of the random network distillation prediction network.

[0119] The random network distillation prediction network uses backpropagation to calculate the gradient of the error with respect to the network parameters and update the network parameters. The network parameter update adopts the adaptive moment estimation optimization method, with a learning rate of 0.001, a first-order moment estimation decay coefficient of 0.9, a second-order moment estimation decay coefficient of 0.999, and a numerical stability term of 0.00000001.

[0120] The trusted record subset is arranged in the order of sample identifiers to form a training sequence and divided into small batches with a batch size of 64. The random network distillation prediction network iterates the training sequence for 100 rounds to update the network parameters and writes the updated network parameters into the storage medium.

[0121] For each field of the spatial registration record table, a missing marker vector is generated. The missing marker vector is assigned a value of 1 at the location where the observation exists and a value of 0 at the location where the null value is marked. The record location corresponding to the outlier marker vector is also assigned a value of 0 to obtain the missing marker vector.

[0122] Damage description fields were extracted from landslide disaster samples of houses built on slopes and a unified coding system was established. Based on the damage level mapping table, the unified coding was mapped to discrete damage levels to obtain damage level labels.

[0123] The spatial registration record table, missing marker vector, and damage level label are aligned and merged according to the sample identifier, and unreasonable records are removed. The unreasonable records are those with all missing marker vectors being 0, thus obtaining the disaster sample dataset, missing marker vector, and damage level label.

[0124] In this embodiment, constructing the state vector set specifically involves:

[0125] Locate a single record in the disaster sample dataset by sample identifier, and read the slope geometry parameters, soil and rock parameters, rainfall triggering parameters, slope cutting construction parameters, building structure load parameters and missing marker vector to obtain the sample parameter tuple;

[0126] Perform field expansion on the sample parameter tuple to obtain parameter sub-vector groups. The parameter sub-vector groups include slope geometry parameter sub-vectors, geotechnical parameter sub-vectors, rainfall trigger parameter sub-vectors, slope cutting construction parameter sub-vectors, and building structure load parameter sub-vectors generated in a fixed field order.

[0127] In the disaster sample dataset, normalization is performed on each numerical field. The minimum and maximum values ​​are used to perform linear normalization on the corresponding fields in the parameter sub-vector group. During the normalization process, the normalization value is 0 for fields where the minimum value is equal to the maximum value, thus obtaining the normalized parameter sub-vector group.

[0128] Read the building structure type code from the parameter sub-vector group, perform one-hot encoding to generate a structure type vector, and write it into the normalized parameter sub-vector group to obtain the structure enhancement sub-vector group;

[0129] The feature interaction vector is calculated based on the structural enhancement sub-vector group. The feature interaction vector includes the geometric combination quantity generated by the product of slope height and slope angle in the slope geometric parameters, the trigger combination quantity generated by the product of cumulative rainfall and rainfall duration in the rainfall trigger parameters, and the load combination quantity generated by the product of building area and unit area load in the building structural load parameters.

[0130] The calculation of the feature interaction vector is specifically as follows:

[0131] Locate the slope geometry parameter subvector, rainfall trigger parameter subvector, and building structure load parameter subvector within the structural enhancement subvector group corresponding to each sample identifier. Read the normalized values ​​of slope height, slope angle, cumulative rainfall, rainfall duration, building area, and unit area load to obtain the interactive input scalar group.

[0132] Perform floating-point multiplication on the normalized values ​​of slope height and slope angle to obtain the geometric combination quantity, and write the geometric combination quantity into the first dimension of the feature interaction vector;

[0133] A floating-point multiplication operation is performed on the normalized value of cumulative rainfall and the normalized value of rainfall duration to obtain the trigger combination value, which is then written into the second dimension of the feature interaction vector.

[0134] The load combination quantity is obtained by performing a floating-point multiplication operation on the normalized value of the building area and the normalized value of the load per unit area. The load combination quantity is written into the third dimension of the feature interaction vector.

[0135] Connect the elements in a fixed order from the first to the third dimension to form a feature interaction vector of length 3;

[0136] The observation completeness vector is calculated based on the missing label vector. The observation completeness vector contains the ratio of the number of elements with a value of 1 in the missing label vector to the length of the missing label vector to generate a completeness scalar. The completeness scalar is then copied to generate the observation completeness vector.

[0137] The calculation of the observation completeness vector is specifically as follows:

[0138] Read the missing label vector corresponding to the sample identifier. A value of 1 in each dimension of the missing label vector indicates that there is an observation in the corresponding field, and a value of 0 in each dimension of the missing label vector indicates that the corresponding field is a null value or an anomaly.

[0139] The number of observed elements is obtained by traversing all dimensions of the missing marker vector and counting the number of elements with a value of 1. The length of the missing marker vector is obtained by reading the number of dimensions of the missing marker vector.

[0140] The completeness scalar is obtained by converting the number of observed elements and the length of the missing marker vector into floating-point numbers and performing floating-point division. The completeness scalar takes a value between 0 and 1.

[0141] An observation completeness vector is generated based on the number of dimensions of the missing marker vector, and each dimension of the observation completeness vector is assigned a completeness scalar value.

[0142] A single state vector is generated by concatenating the structure enhancement sub-vector group, feature interaction vector, and observation completeness vector in a fixed splicing order, and a set of state vectors is generated based on all sample identifiers in the disaster sample dataset.

[0143] In this embodiment, the external report sample of the state action specifically refers to:

[0144] Read each state vector in the state vector set and read the damage level label, and count the number of discrete damage levels corresponding to the damage level label;

[0145] For each set of vulnerability parameter vectors within the action set, the structure is expanded. The vulnerability parameter vector is expanded into a level weight vector of discrete damage level and a level bias scalar of discrete damage level. The dimension of the level weight vector is the same as the dimension of the state vector, thus obtaining the action parameter set.

[0146] For each state vector and action parameter group in the state vector set, perform level score calculation. For each damage level, perform dimension-wise multiplication and addition on the corresponding level weight vector and state vector, and superimpose the corresponding level bias scalar to form a discrete damage level quantity level score. Connect them in order of damage level to obtain the level score vector.

[0147] The Softmax normalization operation is performed on the grade score vector. The Softmax normalization operation obtains the damage grade probability by taking the exponent of the grade score vector dimension by dimension and dividing it by the sum of the aforementioned exponents, thus forming the predicted damage distribution.

[0148] Read the observation completeness vector from each state vector and take any dimension value as the completeness scalar. Construct the target damage distribution based on the completeness scalar. Assign the completeness scalar value to the dimension corresponding to the damage level label of the target damage distribution, and assign the residual probability mass to the other damage level dimensions in an equal distribution manner to obtain the target damage distribution.

[0149] The difference between the predicted damage distribution and the target damage distribution is calculated. The difference is obtained by taking the natural logarithm of the probability of the target damage distribution in each dimension and multiplying it by the corresponding probability of the predicted damage distribution, summing the results and taking the opposite number. The external reward value is obtained by taking the opposite number of the difference.

[0150] Each state vector, vulnerability parameter vector, and external reward value are associated with a sample identifier and written into the sample record to form a state-action external reward sample.

[0151] In this embodiment, the conservative penalty amount is specifically:

[0152] During the conservative Q-learning training process, each sample record in the external reward sample of state-action is read. The state vector and the corresponding vulnerability parameter vector in the sample record are input into the action value network and the target action value network, respectively, and the data action value and the target data action value are output.

[0153] The action value network and the target action value network are specifically as follows:

[0154] The action value network adopts a feedforward fully connected neural network structure. The action value network consists of an input layer, a first fully connected layer, a first nonlinear activation layer, a second fully connected layer, a second nonlinear activation layer, and an output layer. The output layer outputs a single scalar action value.

[0155] The target action value network adopts the same feedforward fully connected neural network structure as the action value network. The target action value network consists of an input layer, a first fully connected layer, a first nonlinear activation layer, a second fully connected layer, a second nonlinear activation layer, and an output layer. The output layer outputs a single scalar target action value.

[0156] The input layer receives a concatenated vector of the state vector and the vulnerability parameter vector. The length of the concatenated vector is defined as the sum of the lengths of the state vector and the vulnerability parameter vector. The concatenation order is defined as the state vector first and the vulnerability parameter vector second.

[0157] The minimum and maximum values ​​of the action set in the same dimension are calculated dimension by dimension for the vulnerability parameter vector and linear normalization is performed. The linear normalization outputs a normalized value of 0 when the minimum value equals the maximum value. The state vector retains the normalized value output in step S23. The normalized concatenated vector is input into the first fully connected layer.

[0158] The output dimension of the first fully connected layer is defined as 4 times the length of the concatenated vector, and the output dimension of the second fully connected layer is defined as 2 times the length of the concatenated vector. The weight matrices of the first and second fully connected layers are generated using zero-mean, unit-variance Gaussian random numbers, and the bias vectors of the first and second fully connected layers are all-zero vectors.

[0159] The first and second nonlinear activation layers adopt modified linear unit transformation. The output layer is a linear fully connected layer with an output dimension of 1. The output layer weight matrix is ​​generated using zero-mean, unit-variance Gaussian random numbers. The output layer bias vector is an all-zero vector.

[0160] The action value network participates in backpropagation parameter updates during conservative Q-learning training. The parameter updates adopt the adaptive moment estimation optimization method, with a learning rate of 0.001, a first-order moment estimation decay coefficient of 0.9, a second-order moment estimation decay coefficient of 0.999, and a numerical stability term of 0.00000001.

[0161] The target action value network parameters are obtained by copying the action value network parameters during the initialization phase. The target action value network does not participate in backpropagation parameter updates during conservative Q-learning training. After each action value network parameter update, the target action value network parameters are updated using an exponential moving average method with an update coefficient of 0.005.

[0162] For each sample record, 10 sets of non-data action corresponding vulnerability parameter vectors, which are different from the corresponding vulnerability parameter vectors, are sampled from the action set to obtain the non-data action parameter sequence;

[0163] Input the state vector and the non-data action parameter sequence into the action value network and the target action value network, respectively, and output the non-data action value sequence and the target non-data action value sequence, respectively.

[0164] The difference action value sequence is obtained by performing item-by-item difference calculation between the non-data action value sequence and the target non-data action value sequence, and the difference data action value is obtained by performing difference calculation between the data action value and the target data action value.

[0165] Read the observation completeness vector from the state vector and take the first dimension value as the completeness scalar. Calculate the missing rate scalar as the difference between the completeness scalar and 1. Calculate the temperature coefficient as the sum of the missing rate scalar and 1. Perform temperature scaling on the differential action value sequence based on the temperature coefficient.

[0166] Logarithmic exponential aggregation is performed on the temperature-scaled differential action value sequence. The logarithmic exponential aggregation operation is performed by taking the exponent of each temperature scaling value in the sequence, summing them, taking the natural logarithm, and multiplying by the temperature coefficient to obtain the aggregated differential action value. The basic penalty is calculated, which is the aggregated differential action value minus the differential data action value.

[0167] The discreteness scalar is calculated based on the differential action value sequence to obtain the conservative penalty quantity. The discreteness scalar is the square root of the variance of the differential action value sequence. The scale coefficient is calculated as the sum of the discreteness scalar and 1. The conservative penalty quantity is the product of the basic penalty quantity and the scale coefficient.

[0168] In this embodiment, obtaining the novelty error value and converting it into an intrinsic reward value specifically involves:

[0169] Read the set of state vectors in the order of sample identifiers and form a mini-batch state sequence with 64 state vectors to obtain the mini-batch state sequence.

[0170] Input the mini-batch state sequence one by one into the random network to distill the target network and output the target feature vector sequence; input the mini-batch state sequence one by one into the random network to distill the prediction network and output the prediction feature vector sequence.

[0171] The initialization method for the random network distillation target network and the random network distillation prediction network is as follows:

[0172] Read any state vector from the state vector set and count the number of dimensions to obtain the state vector dimension. Write the state vector dimension into the runtime configuration and use it as the network input tensor dimension.

[0173] In memory, the parameter tensor sets of the random network distillation target network and the random network distillation prediction network are instantiated according to the network layer connection relationship. The parameter tensor names and shapes correspond one-to-one with the weight matrix and bias vector.

[0174] Based on the sample identifiers in the disaster sample dataset, the sample identifier sequence string is formed by concatenating the sample identifier sequence string in order. The sample identifier sequence string is input into the hash calculation process to obtain a 32-bit unsigned integer. The 32-bit unsigned integer is written into the initial state of the random number generator and the output sequence of the random number generator is locked.

[0175] The random number generator is called to generate zero-mean, unit-variance Gaussian random numbers in the order of the parameter tensor elements. These numbers are then used to fill all the weight matrix elements of the target network in the random network distillation process and set all the bias vector elements to 0. The filled parameter tensor set is then written to the storage medium and loaded into the runtime memory.

[0176] Continue calling the same random number generator to generate zero-mean, unit-variance Gaussian random numbers in the order of the parameter tensor elements, fill all weight matrix elements of the random network distillation prediction network in sequence, and set all bias vector elements to 0. Write the filled parameter tensor set to the storage medium and load it into the running memory.

[0177] Mark the set of parameter tensors of the target network in random network distillation as untrainable parameters and turn off gradient accumulation, and mark the set of parameter tensors of the prediction network in random network distillation as trainable parameters and turn on gradient accumulation;

[0178] An adaptive moment estimation optimization state is constructed for the random network distillation prediction network. The optimization state includes the first-order moment tensor and the second-order moment tensor corresponding to each weight matrix, as well as the first-order moment vector and the second-order moment vector corresponding to each bias vector. All elements of the first-order moment and the second-order moment are assigned the value 0. The learning rate is 0.001, the first-order moment estimation decay coefficient is 0.9, the second-order moment estimation decay coefficient is 0.999, and the numerical stability term is 0.00000001.

[0179] Perform dimension-wise difference between the target feature vector sequence and the predicted feature vector sequence, and sum the difference results after squaring them dimensionwise. For each state vector in the mini-batch state sequence, obtain a novelty error value to form a novelty error value sequence.

[0180] The mean scalar and standard deviation scalar are calculated on the novelty error value sequence to obtain the normalized novelty error value sequence;

[0181] Read the observation completeness vector from each state vector and take the first dimension value as the completeness scalar. Multiply the corresponding elements in the normalized novelty error value sequence with the completeness scalar to obtain the completeness weighted novelty error value sequence.

[0182] The completeness-weighted novelty error value sequence is nonnegated. The nonnegation process assigns the value 0 to the elements less than 0. The nonnegated elements are added to 1 and the natural logarithm is taken to obtain the intrinsic reward value sequence.

[0183] The sample identifier, the novelty error value sequence, and the intrinsic reward value sequence are written into the reward record according to the one-to-one correspondence of the sample identifier, so as to obtain the novelty error value and the intrinsic reward value.

[0184] In this embodiment, the construction of the state transition sample specifically refers to:

[0185] Read two adjacent state vectors in the state vector set in the order of sample identifiers, mark them as the current state vector and the successor state vector respectively, and read the external reward value and internal reward value corresponding to the current state vector.

[0186] Read the observation completeness vector from the current state vector and take the first dimension value as the completeness scalar. Calculate the fusion coefficient as the difference between the completeness scalar and 1, and limit the fusion coefficient to the interval between 0 and 1.

[0187] The external return value and the intrinsic return value are weighted and fused according to the fusion coefficient. The fused return value is obtained by multiplying the external return value and the intrinsic return value by complementary weights and then summing them.

[0188] Read the vulnerability parameter vector corresponding to the current state vector and use it as the action vector. Combine it with the current state vector, the fusion reward value and the subsequent state vector in a fixed order to obtain a single state transition record.

[0189] For all state vectors corresponding to consecutive sample identifiers in the state vector set, repeat the calculation of all single state transition records to form state transition samples.

[0190] In this embodiment, the joint conservative penalty update of the action value network and the synchronous update of the target action value network specifically refers to:

[0191] Read each state transition record in the state transition sample to obtain the current state vector, action vector, fusion reward value, and successor state vector;

[0192] For the successor state vector, 10 sets of vulnerable parameter vectors are sampled from the action set to form a candidate action vector sequence, and the successor state vector and the candidate action vector sequence are combined one by one to form a successor state-action pair sequence;

[0193] The successor state-action pair sequence is input into the target action value network one by one, and the target successor action value sequence is output. The target successor action value with the highest value is selected from the target successor action value sequence as the target action value scalar.

[0194] The observation completeness vector is read from the successor state vector and the first dimension value is taken as the successor completeness scalar. The temporal difference target value is calculated. The discount factor is 0.99. The temporal difference target value is obtained by summing the result of fusing the reward value and the discount factor multiplied by the successor target action value scalar.

[0195] Input the current state vector and action vector into the action value network to output the current action value, calculate the temporal difference error, and obtain the temporal difference loss value. The temporal difference error is obtained by subtracting the current action value from the temporal difference target value.

[0196] The conservative penalty amount is read based on the sample identifier, and the joint loss value is obtained by summing the temporal difference loss value and the conservative penalty amount.

[0197] Backpropagation is performed on the action value network based on the joint loss value to update the network parameters. The updated action value network and the target action value network are output as the conservative action value function.

[0198] In this embodiment, the landslide vulnerability analysis conclusion for building houses on slopes is specifically as follows:

[0199] Receive data on the slope-cutting building scenario to be analyzed and generate a scenario state vector;

[0200] The process of generating the scene state vector involves: receiving on-site monitoring data and engineering survey data of the slope-cutting building scene to be analyzed, forming a scene record table, and assigning a scene sample identifier to each record;

[0201] For each field in the scene record table, a scene missing marker vector is generated. Fields with existing observations are assigned a value of 1, and fields with null values ​​are assigned a value of 0, thus obtaining the scene missing marker vector.

[0202] Read slope geometry parameters, soil and rock parameters, rainfall triggering parameters, slope cutting construction parameters, and building structure load parameters from the scene record table and expand them into parameter sub-vector groups according to a fixed field order;

[0203] Linear normalization is performed on the corresponding fields in the parameter sub-vector group using the minimum and maximum values ​​of each numerical field in the disaster sample dataset. Fields where the minimum value equals the maximum value are output with a normalized value of 0, thus obtaining the normalized parameter sub-vector group.

[0204] Read the building structure type code from the parameter sub-vector group and perform one-hot encoding to generate a structure type vector. Write the structure type vector into the normalized parameter sub-vector group to obtain the structure enhancement sub-vector group.

[0205] The feature interaction vector is calculated based on the structural enhancement sub-vector group. The feature interaction vector includes the geometric combination quantity obtained by multiplying the normalized value of slope height and the normalized value of slope angle, the trigger combination quantity obtained by multiplying the normalized value of cumulative rainfall and the normalized value of rainfall duration, and the load combination quantity obtained by multiplying the normalized value of building area and the normalized value of load per unit area.

[0206] The number of elements with a value of 1 is counted based on the scene missing marker vector. A floating-point division is then performed with the length of the scene missing marker vector to obtain a completeness scalar. The completeness scalar is then copied to generate an observation completeness vector.

[0207] The scene state vector is obtained by concatenating the structure enhancement sub-vector group, the feature interaction vector and the observation completeness vector in a fixed concatenation order and then outputting it in association with the scene sample identifier.

[0208] The novelty error value of the scenario is calculated based on the random network distillation target network and the random network distillation prediction network.

[0209] The calculation of the scenario novelty error value is specifically as follows:

[0210] Read the scene state vector while keeping the dimensional order of the state vector unchanged, and convert all elements of the scene state vector into a floating-point input tensor.

[0211] The input tensor is fed into a random network to distill the target network and perform forward computation to obtain the target feature vector. The input tensor is fed into a random network to distill the prediction network and perform forward computation to obtain the prediction feature vector.

[0212] The difference feature vector is obtained by performing a dimension-wise difference calculation on the target feature vector and the predicted feature vector. The value of each dimension of the difference feature vector is the value of the corresponding dimension of the target feature vector minus the value of the corresponding dimension of the predicted feature vector.

[0213] The scene novelty error value is obtained by performing a squaring operation on the difference feature vector dimension by dimension and summing all dimensions of the squared difference vector.

[0214] The action value corresponding to each vulnerability parameter vector in the action set is calculated based on the conservative action value function, and action value constraints are applied to obtain the constrained action value sequence.

[0215] Select the vulnerability parameter vector corresponding to the maximum value of the constraint action value sequence as the scenario-optimal vulnerability parameter vector;

[0216] Calculate the predicted damage distribution of the scene based on the scene's optimal vulnerability parameter vector;

[0217] The calculation scenario predicts the damage distribution, specifically as follows:

[0218] Read the scene state vector and the scene optimal vulnerability parameter vector, read the number of discrete damage levels and establish a damage level index sequence;

[0219] The optimal vulnerability parameter vector of the scene is structurally expanded to obtain a number of discrete damage levels, a number of level weight vectors, and a number of discrete damage levels, a number of level bias scalars. The dimension of the level weight vector is consistent with the dimension of the scene state vector.

[0220] For each damage level index, a level score is calculated. The level score is the sum of the product of the level weight vector and the scene state vector in each dimension and the sum of the level bias scalar.

[0221] The level score vector is obtained by concatenating all level scores in the order of the damage level index sequence.

[0222] Read the maximum value of the grade score vector and subtract the maximum value from the grade score vector dimension by dimension to obtain the translated grade score vector. Take the exponent of the translated grade score vector dimension by dimension to obtain the exponent vector. Summ up all elements of the exponent vector to obtain the exponent and scalar.

[0223] Divide the exponent and scalar of the exponent vector dimension by dimension to obtain the damage level probability vector, and use the damage level probability vector as the scene prediction damage distribution output.

[0224] Based on the scenario, the damage distribution is predicted, and the analysis results of landslide vulnerability are output for houses built on slopes.

[0225] In this embodiment, a landslide vulnerability analysis system based on reinforcement learning for slope-cutting housing construction includes:

[0226] The data acquisition and preprocessing unit is used to receive landslide disaster samples from slope-cutting construction, on-site monitoring data, engineering survey data, and data of slope-cutting construction scenarios to be analyzed, and output disaster sample datasets, missing label vectors, and scenario data;

[0227] The state vector construction unit is used to extract slope geometric parameters, soil and rock parameters, rainfall triggering parameters, slope cutting construction parameters, and building structure load parameters from disaster sample datasets or scene data, and to fuse missing marker vectors, feature interaction vectors, and observation completeness vectors to generate a set of state vectors or scene state vectors.

[0228] The external reward calculation unit is used to calculate the difference between the predicted damage distribution and the target damage distribution based on the state vector, vulnerability parameter vector and damage level label, and generate state action external reward samples.

[0229] The conservative penalty calculation unit is used to combine the action value network and the target action value network under the conservative Q-learning framework to perform logarithmic exponential aggregation and dispersion adjustment on the data action value and non-data action value, and calculate the conservative penalty amount.

[0230] The novelty reward calculation unit is used to calculate the novelty error value by using random network distillation of the target network and random network distillation of the prediction network, and convert the novelty error value into the intrinsic reward value.

[0231] The state transition building unit is used to fuse external reward values ​​with internal reward values ​​and combine adjacent state vectors and action vectors to construct state transition samples.

[0232] The action value update unit is used to calculate the temporal difference target value based on the state transition sample, update the action value network in conjunction with the conservative penalty amount and synchronously update the target action value network to obtain the conservative action value function.

[0233] The vulnerability analysis output unit is used to calculate the predicted damage distribution of the slope-cutting building scenario based on the conservative action value function and the random network distillation network, and output the vulnerability analysis conclusion of the slope-cutting building landslide.

[0234] Example 1:

[0235] To verify the feasibility of this invention in practice, it was applied to a landslide vulnerability analysis task in a set of slope-cutting housing construction scenarios. The scenario data consisted of disaster sample records, on-site monitoring data, and engineering survey data. The goal was to output the predicted damage distribution of the analyzed scenario and form a landslide vulnerability analysis conclusion for slope-cutting housing construction, under the conditions of limited disaster sample quantity and missing data from multiple sources. In this application scenario, existing methods mostly use fixed-form vulnerability functions or scoring models based on index weighting. The modeling process relies on relatively complete variable observations and sufficient disaster samples. When faced with inputs such as intermittent monitoring, inconsistent survey calibers, and a high proportion of missing data, the predicted damage distribution is prone to deviation, and the model is difficult to update iteratively with new samples, leading to engineering... The stability requirements of the output conclusions are difficult to meet; at the same time, the field data is often dominated by landslide disaster samples, and the damage level labels are unevenly distributed among categories such as minor damage to collapse, which can easily lead to the model biasing towards high-frequency categories and false alarms in non-landslide scenarios during training. In order to address the above problems, this invention deeply integrates conservative Q-learning with random network distillation. It uses missing label vectors and observation completeness vectors to characterize the degree of data incompleteness, uses conservative penalty amount to suppress overestimation of action value, generates intrinsic reward value to supplement the training signal through novelty error value, and introduces negative samples of non-landslide disasters at the data level to improve the sample distribution. Thus, under the condition of scarce samples, it can achieve stable updates of the conservative action value function and output reliable vulnerability analysis conclusions.

[0236] During implementation, when constructing the disaster sample dataset, 186 disaster sample records of landslides were selected as positive samples. Simultaneously, 214 non-landslide negative samples from slope-cutting construction scenarios were added as control samples. Negative samples met the following criteria: valid records of rainfall triggering parameters existed within the monitoring period, the slope showed no signs of slippage, the houses showed no structural damage, and the damage description fields were consistently encoded as "no damage." The fields of both positive and negative samples covered slope geometric parameters, soil and rock parameters, rainfall triggering parameters, slope-cutting construction parameters, and house structural load parameters. The missing marker vector length was 48, and the sample-level missing rate was between 0 and 1. The damage level labels range from 0.08 to 0.43; the damage level labels are divided into 5 levels and the label coding is kept consistent. The 5 levels correspond to no damage, mild damage, moderate damage, severe damage, and collapse, respectively. The damage level labels of negative samples are uniformly mapped to no damage. In order to alleviate the impact of class imbalance on training, samples are extracted stratified according to damage level labels during the training batch construction stage and written into batches in a cyclical alternation, so that the difference in the number of samples of each damage level in each batch does not exceed 1. The negative samples and high damage level samples are kept at the same frequency of entering training, so that the update of the action value network covers both no damage and high risk states at the same time.

[0237] The data acquisition and preprocessing unit unifies the naming, measurement units, and data types of multi-source inputs, and maps the slope units after unifying the geographic coordinates to the same coordinate system. The numerical validity check adopts the constraint that the slope geometric parameter angle value is greater than 0 degrees and less than 90 degrees, and the physical quantity of the soil and rock parameters takes positive values. The cross-field consistency check adopts the constraint that the building structure load parameter matches the building structure type code, forming a consistency marker vector. In order to avoid abnormal records interfering with subsequent state representation, the random network distillation target network and random network distillation prediction network are used to calculate the network output difference by constructing a reliable record subset of records. After sorting the network output difference from smallest to largest, a percentile threshold is selected in the reliable record subset to generate an anomaly marker vector. The corresponding position of the anomaly marker vector is written into the missing marker vector as 0, thus obtaining the disaster sample dataset, missing marker vector, and damage level label for subsequent state vector set construction. In this process, negative samples and positive samples adopt the same field alignment, check, and missing marker vector generation mechanism to keep the input distribution of the undamaged label samples comparable to the landslide samples and avoid introducing additional bias due to the difference in collection scope.

[0238] The state vector construction unit calculates the minimum and maximum values ​​for each numerical field in the disaster sample dataset, performs linear normalization using the minimum and maximum values, and outputs a normalized value of 0 for fields where the minimum value equals the maximum value. The building structure type code is encoded using one-hot encoding and then written into the normalized parameter sub-vector group to form a structure enhancement sub-vector group. To strengthen the coupled expression of triggering factors and disaster-bearing body conditions, the feature interaction vector includes a geometric combination quantity generated by multiplying the normalized slope height and slope angle values, a triggering combination quantity generated by multiplying the normalized cumulative rainfall and rainfall duration values, and a triggering combination quantity generated by multiplying the normalized building area and unit area values. The load combination quantity is generated by multiplying the load normalization values; the observation completeness vector is obtained by the ratio of the number of 1s in the missing label vector to the length of the missing label vector, and the completeness scalar is copied to obtain the observation completeness vector; the structural enhancement sub-vector group, feature interaction vector and observation completeness vector are connected in a fixed splicing order to generate a set of state vectors with a dimension of 96. The negative sample state vector and the positive sample state vector use the same splicing order and the same normalization parameter to ensure that the action value network can learn the separable boundaries of the undamaged state and the damaged state in the state space during the training phase;

[0239] The external reward calculation unit constructs an action set with a size of 64. Each action corresponds to a set of vulnerability parameter vectors. The vulnerability parameter vector structure is expanded into a discrete number of damage level weight vectors and a discrete number of damage level bias scalars. The dimension of the level weight vector is consistent with the dimension of the state vector. For each state vector and action parameter set, a level score is calculated, and the level score vector is Softmax normalized to obtain the predicted damage distribution. The target damage distribution is modulated by a completeness scalar. The dimension corresponding to the damage level label is assigned the completeness scalar value, and the remaining dimensions are assigned the residual probability mass in an even distribution manner. The difference between the predicted damage distribution and the target damage distribution is obtained by taking the natural logarithm of the probability of each dimension, multiplying them, summing them, and taking the opposite number. The external reward value is the opposite number of the difference, forming the state-action external reward sample. On negative samples, the damage level label corresponds to the non-damage dimension. The target damage distribution is concentrated in the non-damage dimension and maintains an even distribution of residual probability mass as the completeness scalar decreases. This ensures that the external reward does not impose an excessive penalty on severely missing non-damage samples, thereby reducing the possibility of non-damage states being attracted by high-risk action errors.

[0240] During the conservative Q-learning training process, the conservative penalty calculation unit inputs the state vector and corresponding vulnerability parameter vector from each sample record into the action value network and the target action value network, respectively, outputting the data action value and the target data action value. It also samples 10 sets of non-data actions different from the corresponding vulnerability parameter vectors from the action set to form a non-data action parameter sequence, which is then input into the action value network and the target action value network to obtain the non-data action value sequence and the target non-data action value sequence. The non-data action value sequence and the target non-data action value sequence are then subtracted term by term to obtain the differencing action value sequence. The difference between the data action value and the target data action value is calculated. The differential action value is obtained by summing the missing rate scalar with 1. The missing rate scalar is the difference between the completeness scalar and 1. The differential action value sequence is scaled by temperature and then subjected to logarithmic exponential aggregation to obtain the aggregated differential action value. The base penalty is obtained by subtracting the differential action value from the aggregated differential action value. The dispersion scalar is the square root of the variance of the differential action value sequence. The scaling coefficient is obtained by summing the dispersion scalar with 1. The conservative penalty is obtained by multiplying the base penalty and the scaling coefficient. This mechanism applies equal suppression to the overestimation of non-data actions on negative samples, so that the action value distribution corresponding to the undamaged samples remains convergent and reduces false alarms.

[0241] The novelty reward calculation unit reads the state vector set in the order of sample identifiers and constructs a mini-batch state sequence of 64 state vectors. The mini-batch state sequence is then input line by line into a random network to distill the target network's output target feature vector sequence, and then input line by line into a random network to distill the prediction network's output prediction feature vector sequence. The novelty error value sequence is obtained by subtracting the values ​​one dimension at a time and summing the squares. The mean and standard deviation scalars of the novelty error value sequence are calculated to obtain a normalized novelty error value sequence. The normalized novelty error value is multiplied by the completeness scalar to obtain a completeness-weighted novelty error value sequence. This completeness-weighted novelty error value sequence is then non-negatively processed, added to 1, and the natural logarithm is taken to obtain the intrinsic reward value sequence. The state transition construction unit, according to sample... This identifier sequentially reads two adjacent state vectors within the state vector set as the current state vector and the successor state vector, reads the corresponding external reward value and internal reward value of the current state vector, takes the difference between the completeness scalar and 1 and limits it to the interval between 0 and 1 for the fusion coefficient, and obtains the fusion reward value by summing the external reward value and the internal reward value according to complementary weights. The current state vector, action vector, fusion reward value and successor state vector are combined in a fixed order to obtain a single state transition record and form a state transition sample. After the negative sample participates in the construction of the state transition sample, the estimation of the temporal difference target value simultaneously covers the non-damaging state and the damaged state, enabling the conservative action value function to have the ability to distinguish between non-damaging and high-damage risk in the action selection stage.

[0242] The action value update unit uses a batch size of 256 to extract training batches from the state transition samples and iterates 8000 times. For each state transition record, 10 sets of vulnerability parameter vectors are sampled from the action set under the successor state vector to form a candidate action vector sequence. The successor state-action pair sequence is input into the target action value network to obtain the target successor action value sequence, and the maximum value is selected as the successor target action value scalar. The discount factor is set to 0.99 and multiplied by the successor completeness scalar. The temporal difference target value is obtained by multiplying the fusion reward value and the discount factor by the successor target action value scalar and summing them. The current state vector and action vector are input into the action value network to obtain the current action value. The temporal difference error is obtained by subtracting the current action value from the temporal difference target value and squaring it to obtain the temporal difference loss value. The joint loss value is obtained by subtracting the current action value from the temporal difference target value and squaring it. The order difference loss value is obtained by summing the conservative penalty amount; the action value network updates the network parameters using an adaptive moment estimation optimization method with a learning rate of 0.001, a first-order moment estimation decay coefficient of 0.9, a second-order moment estimation decay coefficient of 0.999, and a numerical stability term of 0.00000001; the target action value network updates synchronously using an exponential moving average with an update coefficient of 0.005; training monitoring results show that the joint loss value decreases from the initial 1.93 and stabilizes at around 0.29 after the 6100th update; the mean cross-entropy of the predicted damage distribution corresponding to the external reward samples of state actions decreases from 0.97 and stabilizes at 0.40; the intra-batch standard deviation of the scene novelty error value decreases from 0.81 and stabilizes at 0.27; and the intra-batch range of the action value output converges significantly and remains stable.

[0243] In the vulnerability analysis output stage, 30 data points from the slope-cutting house construction scenario to be analyzed are selected as inference samples. Field alignment is performed according to preprocessing rules to generate a scenario missing marker vector, and then a scenario state vector is obtained according to the state vector generation rules. The scenario state vector is input into the random network distillation target network and the random network distillation prediction network to obtain the scenario novelty error value. Based on the conservative action value function, the action value corresponding to each vulnerability parameter vector within the action set is calculated. The action value constraint is generated by the normalized scenario novelty error value and the scenario completeness scalar and applied to the action value to obtain a constrained action value sequence. The vulnerability parameter vector corresponding to the maximum value of the constrained action value sequence is used as the scenario optimal vulnerability parameter vector. The scenario optimal vulnerability parameter vector and the scenario state vector are used for grade score calculation and Softmax normalization to obtain the scenario prediction. The method outputs damage distribution and landslide vulnerability analysis conclusions for slope-cutting housing construction. In the comparative validation, the control methods were the action value learning method with conservative penalty removed and the novelty reward calculation method with intrinsic reward removed. The mean square error of damage distribution, damage level accuracy, and probability calibration error were used as evaluation indicators. On the validation set, the method of this invention had a mean square error of 0.011 for damage distribution, an accuracy of 0.87 for damage level, and a probability calibration error of 0.029. On the negative sample subset, the accuracy of damage level identification without damage was 0.90, and the mean false alarm rate for collapse probability was 0.015. The control method had an accuracy of 0.78 for damage level identification without damage and a mean false alarm rate for collapse probability of 0.041. The results indicate that the introduction of negative samples and the conservative penalty mechanism together reduced false alarms in non-landslide scenarios and improved the calibrability of the damage distribution output.

[0244] Table 1 Comparison of Predicted and Measured Damage Distributions in Slope-Cutting Housing Construction Scenarios

[0245] Sample number Actual measurement of non-damage probability No-damage probability prediction Actual measurement of the probability of minor injury Prediction of the probability of mild injury Measured probability of moderate injury Prediction of the probability of moderate injury Actual measurement of the probability of severe injury Severe injury probability prediction Actual measurement of collapse probability Collapse probability prediction Comprehensive vulnerability index measured Comprehensive Vulnerability Index Prediction A01 0.00 0.01 0.32 0.30 0.28 0.29 0.24 0.25 0.16 0.15 0.56 0.55 A02 0.00 0.01 0.45 0.43 0.25 0.27 0.18 0.17 0.12 0.12 0.49 0.50 A03 0.00 0.01 0.22 0.24 0.26 0.25 0.30 0.29 0.22 0.21 0.63 0.62 A04 0.00 0.02 0.50 0.48 0.24 0.25 0.16 0.17 0.10 0.08 0.45 0.44 A05 0.78 0.75 0.12 0.14 0.06 0.07 0.03 0.03 0.01 0.01 0.09 0.10

[0246] As shown in Table 1, A01 to A04 are damaged samples, and A05 is a non-landslide negative sample. The predicted and measured values ​​for the five samples across the five damage probability indices (no damage, minor damage, moderate damage, severe damage, and collapse) are consistent. In the damaged samples, the mean absolute error (MAO) for the probability of minor damage is 0.02, for moderate damage it is 0.01, for severe damage it is 0.01, for collapse it is 0.01, and for the comprehensive vulnerability index it is 0.012. This indicates that the conservative action value function can output a stable and calibrable probability distribution across different damage levels. In the negative sample A05, the measured probability of no damage is 0.78, and the predicted probability is... The measured collapse probability is 0.75, the predicted collapse probability is 0.01, and the measured comprehensive vulnerability index is 0.09, which is predicted to be 0.10. This indicates that the constraint action value sequence selection mechanism can direct action selection to the low-risk vulnerability parameter vector and suppress false alarms with high damage probability under the condition of no damage. Combining the training convergence process with the table comparison results, this embodiment verifies that the present invention can still construct a set of state vectors, generate state transition samples, and stably update the conservative action value function in scenarios with incomplete data, even under the condition of introducing non-landslide negative samples to improve the sample distribution. It also verifies the stability and consistency between the output scenario predicted damage distribution and the landslide vulnerability analysis conclusions of the slope-cutting house construction scenario to be analyzed.

[0247] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for analyzing landslide vulnerability in slope-cutting housing construction based on reinforcement learning, characterized in that, include: We acquired landslide disaster samples, field monitoring data, and engineering survey data by cutting slopes for housing construction, and preprocessed them to obtain disaster sample datasets, missing label vectors, and damage level labels. Relevant data were extracted from the disaster sample dataset, and a set of state vectors was constructed by fusing missing label vectors. Construct an action set, and select vulnerability parameter vectors for each state vector set. Calculate the difference between the predicted damage distribution generated by the vulnerability parameter vectors and the damage level label as the external reward value to obtain the state-action external reward sample. Initialize the conservative Q-learning action value network and the target action value network, sample multiple sets of non-data actions for the external reward samples of state actions and calculate the action value to obtain the conservative penalty amount; Initialize the random network distillation target network and the random network distillation prediction network, input the set of state vectors and calculate the network output difference, obtain the novelty error value and convert it into the intrinsic reward value; The external reward value and the internal reward value are combined according to the fusion coefficient to generate a fused reward value, and then combined with the adjacent states to construct state transition samples; The temporal difference target value is calculated based on the state transition sample. The action value network is updated by jointly updating the conservative penalty amount and the target action value network simultaneously, thus obtaining the conservative action value function. For the scenario of building houses on slopes to be analyzed, a scenario state vector is constructed, the novelty error value corresponding to the scenario state vector is calculated and applied to the action value constraint, the vulnerability parameter vector is selected according to the conservative action value function and the vulnerability result is output, and the landslide vulnerability analysis conclusion of building houses on slopes is obtained.

2. The landslide vulnerability analysis method based on reinforcement learning for slope-cutting housing construction according to claim 1, characterized in that, The preprocessing specifically includes: Receive samples of landslide disasters caused by building houses on slopes, on-site monitoring data and engineering survey data, unify field naming, units of measurement and data types, and assign sample identifiers to each record to obtain a multi-source raw record table; Spatial benchmark unification is performed on the multi-source original record table. After transforming the geographic coordinates to a unified coordinate system, the slope unit mapping is completed, and the mapping results are written into the corresponding field of the sample identifier to obtain the spatial registration record table. Perform numerical validity checks and cross-field consistency checks on the spatial registration record table to obtain a consistency marker vector; Consistency is filtered based on consistency label vectors. A subset of reliable records is formed by recordings. The network output difference is calculated by inputting the subset of reliable records into the target network and the prediction network of the random network distillation. After sorting the network output difference from smallest to largest, a preset percentile threshold is selected to generate an anomaly label vector. For each field of the spatial registration record table, a missing marker vector is generated to obtain the missing marker vector; Damage description fields were extracted from landslide disaster samples of houses built on slopes and a unified coding system was established. Based on the damage level mapping table, the unified coding was mapped to discrete damage levels to obtain damage level labels. The spatial registration record table, missing marker vector, and damage level label are aligned and merged according to the sample identifier, and unreasonable records are removed to obtain the disaster sample dataset, missing marker vector, and damage level label.

3. The landslide vulnerability analysis method based on reinforcement learning for slope-cutting housing construction according to claim 1, characterized in that, The construction of the state vector set is specifically as follows: Locate a single record in the disaster sample dataset by sample identifier, and read the slope geometry parameters, soil and rock parameters, rainfall triggering parameters, slope cutting construction parameters, building structure load parameters and missing marker vector to obtain the sample parameter tuple; Perform field expansion on the sample parameter tuple to obtain parameter subvectors; In the disaster sample dataset, each numerical field is normalized to obtain a normalized parameter sub-vector group; Read the building structure type code from the parameter sub-vector group, perform one-hot encoding to generate a structure type vector, and write it into the normalized parameter sub-vector group to obtain the structure enhancement sub-vector group; Calculate feature interaction vectors based on structurally enhanced sub-vector groups; A single state vector is generated by concatenating the structure enhancement sub-vector group, feature interaction vector, and observation completeness vector in a fixed splicing order, and a set of state vectors is generated based on all sample identifiers in the disaster sample dataset.

4. The landslide vulnerability analysis method based on reinforcement learning for slope-cutting housing construction according to claim 1, characterized in that, The external report sample for the state action is specifically as follows: Read each state vector in the state vector set and read the damage level label, and count the number of discrete damage levels corresponding to the damage level label; The structure of each vulnerable parameter vector within the action set is expanded to obtain the action parameter set; For each state vector and action parameter group in the state vector set, a grade score is calculated to form a discrete damage grade number grade score, which is then connected in order of damage grade to obtain a grade score vector. Perform Softmax normalization on the grade score vector to form the predicted damage distribution; Read the observation completeness vector from each state vector and take any dimension value as the completeness scalar. Construct the target damage distribution based on the completeness scalar to obtain the target damage distribution. Calculate the difference between the predicted damage distribution and the target damage distribution to obtain the external reward value; Each state vector, vulnerability parameter vector, and external reward value are associated with a sample identifier and written into the sample record to form a state-action external reward sample.

5. The landslide vulnerability analysis method based on reinforcement learning for slope-cutting housing construction according to claim 1, characterized in that, The conservative penalty amount is specifically as follows: During the conservative Q-learning training process, each sample record in the external reward sample of state-action is read. The state vector and the corresponding vulnerability parameter vector in the sample record are input into the action value network and the target action value network, respectively, and the data action value and the target data action value are output. For each sample record, 10 sets of non-data action corresponding vulnerability parameter vectors, which are different from the corresponding vulnerability parameter vectors, are sampled from the action set to obtain the non-data action parameter sequence; Input the state vector and the non-data action parameter sequence into the action value network and the target action value network, respectively, and output the non-data action value sequence and the target non-data action value sequence, respectively. The difference action value sequence is obtained by performing item-by-item difference calculation between the non-data action value sequence and the target non-data action value sequence, and the difference data action value is obtained by performing difference calculation between the data action value and the target data action value. Read the observation completeness vector from the state vector and take the first dimension value as the completeness scalar, and perform temperature scaling on the differential action value sequence; Perform logarithmic exponential aggregation on the temperature-scaled differential action value sequence to obtain the aggregated differential action value, and calculate the base penalty amount; The conservative penalty is calculated based on the differential action value sequence.

6. The landslide vulnerability analysis method based on reinforcement learning for slope-cutting housing construction according to claim 1, characterized in that, The process of obtaining the novelty error value and converting it into an intrinsic reward value is as follows: Read the set of state vectors in the order of sample identifiers and form a mini-batch state sequence with 64 state vectors to obtain the mini-batch state sequence. Input the mini-batch state sequence one by one into the random network to distill the target network and output the target feature vector sequence; input the mini-batch state sequence one by one into the random network to distill the prediction network and output the prediction feature vector sequence. Perform dimension-wise difference between the target feature vector sequence and the predicted feature vector sequence, and sum the result after squaring the difference result dimension-wise to form a novelty error value sequence. The mean scalar and standard deviation scalar are calculated on the novelty error value sequence to obtain the normalized novelty error value sequence; Read the observation completeness vector from each state vector and take the first dimension value as the completeness scalar to obtain the completeness weighted novelty error value sequence; Perform nonnegation on the completeness-weighted novelty error value sequence to obtain the intrinsic reward value sequence; The sample identifier, the novelty error value sequence, and the intrinsic reward value sequence are written into the reward record according to the one-to-one correspondence of the sample identifier, so as to obtain the novelty error value and the intrinsic reward value.

7. The landslide vulnerability analysis method based on reinforcement learning for slope-cutting housing construction according to claim 1, characterized in that, The construction of the state transition sample specifically includes: Read two adjacent state vectors in the state vector set in the order of sample identifiers, mark them as the current state vector and the successor state vector respectively, and read the external reward value and internal reward value corresponding to the current state vector. Read the observation completeness vector from the current state vector and take the first dimension value as the completeness scalar. Calculate the fusion coefficient as the difference between the completeness scalar and 1, and limit the fusion coefficient to the interval between 0 and 1. The external return value and the internal return value are weighted and fused according to the fusion coefficient; Read the vulnerability parameter vector corresponding to the current state vector and use it as the action vector. Combine it with the current state vector, the fusion reward value and the subsequent state vector in a fixed order to obtain a single state transition record. For all state vectors corresponding to consecutive sample identifiers in the state vector set, repeat the calculation of all single state transition records to form state transition samples.

8. The landslide vulnerability analysis method based on reinforcement learning for slope-cutting housing construction according to claim 1, characterized in that, The joint conservative penalty amount updates the action value network and simultaneously updates the target action value network, specifically as follows: Read each state transition record in the state transition sample to obtain the current state vector, action vector, fusion reward value, and successor state vector; For the successor state vector, 10 sets of vulnerable parameter vectors are sampled from the action set to form a candidate action vector sequence, and the successor state vector and the candidate action vector sequence are combined one by one to form a successor state-action pair sequence; The successor state-action pair sequence is input into the target action value network one by one, and the target successor action value sequence is output. The target successor action value with the highest value is selected from the target successor action value sequence as the target action value scalar. Read the observation completeness vector from the successor state vector and take the first dimension value as the successor completeness scalar to calculate the time series difference target value; Input the current state vector and action vector into the action value network to output the current action value, calculate the temporal difference error, and obtain the temporal difference loss value; The conservative penalty amount is read based on the sample identifier, and the joint loss value is obtained by summing the temporal difference loss value and the conservative penalty amount. Backpropagation is performed on the action value network based on the joint loss value to update the network parameters. The updated action value network and the target action value network are output as the conservative action value function.

9. The landslide vulnerability analysis method based on reinforcement learning for slope-cutting housing construction according to claim 1, characterized in that, The specific conclusions of the landslide vulnerability analysis for houses built on slopes are as follows: Receive data on the slope-cutting building scenario to be analyzed and generate a scenario state vector; The novelty error value of the scenario is calculated based on the random network distillation target network and the random network distillation prediction network. The action value corresponding to each vulnerability parameter vector in the action set is calculated based on the conservative action value function, and action value constraints are applied to obtain the constrained action value sequence. Select the vulnerability parameter vector corresponding to the maximum value of the constraint action value sequence as the scenario-optimal vulnerability parameter vector; Calculate the predicted damage distribution of the scene based on the scene's optimal vulnerability parameter vector; Based on the scenario, the damage distribution is predicted, and the analysis results of landslide vulnerability are output for houses built on slopes.

10. A reinforcement learning-based landslide vulnerability analysis system for slope-cutting housing construction, comprising the reinforcement learning-based landslide vulnerability analysis method for slope-cutting housing construction as described in any one of claims 1 to 9, characterized in that, include: The data acquisition and preprocessing unit is used to receive landslide disaster samples from slope-cutting construction, on-site monitoring data, engineering survey data, and data of slope-cutting construction scenarios to be analyzed, and output disaster sample datasets, missing label vectors, and scenario data; The state vector construction unit is used to extract slope geometric parameters, soil and rock parameters, rainfall triggering parameters, slope cutting construction parameters, and building structure load parameters from disaster sample datasets or scene data, and to fuse missing marker vectors, feature interaction vectors, and observation completeness vectors to generate a set of state vectors or scene state vectors. The external reward calculation unit is used to calculate the difference between the predicted damage distribution and the target damage distribution based on the state vector, vulnerability parameter vector and damage level label, and generate state action external reward samples. The conservative penalty calculation unit is used to combine the action value network and the target action value network under the conservative Q-learning framework to perform logarithmic exponential aggregation and dispersion adjustment on the data action value and non-data action value, and calculate the conservative penalty amount. The novelty reward calculation unit is used to calculate the novelty error value by using random network distillation of the target network and random network distillation of the prediction network, and convert the novelty error value into the intrinsic reward value. The state transition building unit is used to fuse external reward values ​​with internal reward values ​​and combine adjacent state vectors and action vectors to construct state transition samples. The action value update unit is used to calculate the temporal difference target value based on the state transition sample, update the action value network in conjunction with the conservative penalty amount and synchronously update the target action value network to obtain the conservative action value function. The vulnerability analysis output unit is used to calculate the predicted damage distribution of the slope-cutting building scenario based on the conservative action value function and the random network distillation network, and output the vulnerability analysis conclusion of the slope-cutting building landslide.

Images