Landslide risk prediction method based on multi-source information fusion and related device

Abstract

The invention provides a landslide risk prediction method based on multi-source information fusion and a related device. The method comprises the following steps: acquiring original data of k monitoring points in a risk landslide to be predicted in an acquisition time period to obtain an original data set containing the k monitoring points; performing standardization processing on k original data in the original data set to obtain a standardized data set; performing space-time diagram construction by adopting the standardized data set and the position information of the corresponding k monitoring points to obtain a target space-time diagram network; performing spatial feature extraction on the target space-time diagram network to obtain spatial features of n acquisition moments in an acquisition time period; performing time feature extraction on the spatial features of the n acquisition moments in the acquisition time period to obtain a spatial-temporal feature vector; and performing prediction decoding processing on the spatial-temporal feature vector to obtain a landslide displacement prediction result. According to the invention, the accuracy and timeliness of landslide prediction are improved, and the emergency response and decision support capabilities of related departments are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of landslide prediction, and in particular to a landslide risk prediction method based on multi-source information fusion and a related device. Background Art

[0002] Landslide is a common geological disaster that seriously affects the safety of people's lives, property and infrastructure. Traditional landslide prediction methods have problems with insufficient prediction accuracy and timeliness, and it is difficult to effectively predict the risk of landslides. Moreover, they mostly rely on a single data source, making it difficult to fully consider the many factors affecting landslide deformation, and the method of integrating heterogeneous information such as groundwater and rainfall, making it difficult to fully reflect the dynamic characteristics of landslide risks. The complex spatiotemporal dependency between landslide monitoring points has not yet been considered, and it cannot adapt to complex geological environments. Therefore, it is particularly important to use multi-source data fusion methods to predict landslide displacement. Summary of the invention

[0003] In view of the defects in the prior art, the present invention provides a landslide risk prediction method and related devices based on multi-source information fusion to improve the accuracy and timeliness of landslide prediction.

[0004] A landslide risk prediction method based on multi-source information fusion, including:

[0005] Collect the original data of k monitoring points in the landslide with risk to be predicted within the collection time period to obtain the original data set containing k monitoring points;

[0006] Standardize k original data in the original data set to obtain a standardized data set;

[0007] The standardized data set and the corresponding k monitoring point location information are used to construct the spatiotemporal graph to obtain the target spatiotemporal graph network;

[0008] Extract spatial features from the target space-time graph network to obtain the spatial features of n acquisition moments within the acquisition time period;

[0009] Extract time features from the spatial features of n acquisition moments within the acquisition time period to obtain a spatiotemporal feature vector;

[0010] The time-space feature vector is predicted and decoded to obtain the landslide displacement prediction result.

[0011] Furthermore, the original data includes monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data.

[0012] Furthermore, k original data in the original data set are standardized to obtain a standardized data set, including:

[0013] The linear interpolation method is used to calculate the displacement data of the monitoring point, the groundwater level data of the monitoring point and the rainfall data of the monitoring point at the corresponding time point to be interpolated according to the displacement data of the monitoring point, the groundwater level data of the monitoring point and the rainfall data of the monitoring point at n acquisition moments within the acquisition time period;

[0014] The Kalman filter method is used to filter the displacement data of the monitoring point, the groundwater level data of the monitoring point and the rainfall data of the monitoring point after interpolation processing at n acquisition moments to obtain filtered monitoring data;

[0015] The filtered monitoring data is normalized to obtain a standardized data set.

[0016] Furthermore, it also includes:

[0017] The spatial features of the target space-time graph network are extracted through the graph neural network to obtain the spatial features of n acquisition moments within the acquisition time period.

[0018] Furthermore, it also includes extracting time features from the spatial features of n acquisition moments within the acquisition time period through Transformer to obtain a spatiotemporal feature vector.

[0019] A landslide risk prediction device based on multi-source information fusion includes an acquisition module, a processing module, a graph construction module, a spatial feature extraction module, a temporal feature extraction module and a decoding module, wherein:

[0020] A collection module is used to collect the original data of k monitoring points in the landslide with risk to be predicted within a collection time period to obtain an original data set containing k monitoring points;

[0021] A processing module, used for performing standardization processing on k original data in the original data set to obtain a standardized data set;

[0022] A graph construction module is used to construct a spatiotemporal graph using a standardized data set and the corresponding k monitoring point location information to obtain a target spatiotemporal graph network;

[0023] The spatial feature extraction module is used to extract spatial features from the target spatiotemporal graph network and obtain the spatial features of n acquisition moments within the acquisition time period;

[0024] The time feature extraction module is used to extract the time features of the space features of n acquisition moments within the acquisition time period to obtain the time-space feature vector;

[0025] The decoding module is used to perform prediction decoding processing on the spatiotemporal feature vector to obtain the landslide displacement prediction result.

[0026] A device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned landslide risk prediction methods based on multi-source information fusion are implemented.

[0027] A storage medium stores a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned landslide risk prediction methods based on multi-source information fusion.

[0028] It can be seen from the above technical scheme that the present invention provides a landslide risk prediction method and related devices based on multi-source information fusion, which improves the accuracy and timeliness of landslide prediction and enhances the emergency response and decision-making support capabilities of relevant departments by fusing heterogeneous information of monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data, and using graph neural network to process the complex spatiotemporal dependency relationship between the monitoring points of the landslide risk to be predicted. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the specific embodiments of the present invention, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below.

[0030] Figure 1 A flow chart of a landslide risk prediction method based on multi-source information fusion provided by an embodiment of the present invention;

[0031] Figure 2 A flowchart of performing standardization processing on the original data in the original data set provided by an embodiment of the present invention;

[0032] Figure 3 A GNN-Transformer model structure provided by an embodiment of the present invention;

[0033] Figure 4 A schematic diagram showing the comparison between the predicted and actual displacements of monitoring points in different directions provided by an embodiment of the present invention;

[0034] Figure 5 A schematic structural diagram of a landslide risk prediction device based on multi-source information fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0036] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0037] In one embodiment, if Figure 1 As shown in the figure, a landslide risk prediction method based on multi-source information fusion is provided, including:

[0038] 1. Collect the original data of k monitoring points in the landslide with risk to be predicted within the collection time period to obtain the original data set containing k monitoring points;

[0039] Preferably, the original data set includes monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data.

[0040] Specifically, the displacement data of the monitoring point includes the displacement data of the monitoring point in the east-west direction, the north-south direction and the elevation direction, and the displacement data of the monitoring point is collected at a sampling interval of 2h (hours).

[0041] The groundwater level data at the monitoring points are collected at a sampling interval of 2 to 4 hours. By monitoring the changes in the groundwater level depth at different times, the groundwater seepage conditions in the landslide risk area to be predicted can be reflected.

[0042] The rainfall data at the monitoring points are collected at sampling intervals of 1 to 24 hours.

[0043] 2. In one embodiment, if Figure 2 As shown, k original data in the original data set are standardized to obtain a standardized data set;

[0044] Preferably, k original data in the original data set are standardized to solve the problems of different sampling frequencies, data missing and noise of the displacement data, groundwater level data and rainfall data of the monitoring points in the original data set, and obtain a standardized data set, including:

[0045] (1) using a linear interpolation method to calculate the corresponding monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data at the time to be interpolated according to the monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data at multiple known acquisition times within the acquisition time period, and obtaining an interpolated data set containing the monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data at n acquisition times after interpolation processing;

[0046] Specifically, the linear interpolation method is first used to complete the low-frequency data, so that the displacement data, groundwater level data and rainfall data of the monitoring points are unified to a sampling interval of 2 hours, so as to achieve their alignment at the sampling time.

[0047] Specifically, for the monitoring point displacement data, monitoring point groundwater level data or monitoring point rainfall data whose sampling interval is not equal to 2h, the monitoring point displacement data, monitoring point groundwater level data or monitoring point rainfall data corresponding to the observation values ​​at the previous known collection time and the next known collection time of two adjacent known collection times are used to perform linear interpolation on the monitoring point displacement data, monitoring point groundwater level data or monitoring point rainfall data at the interpolation time, respectively, to obtain the monitoring point displacement data and monitoring point groundwater level data at n collection times.

[0048] The following formula is used to align the sampling intervals of the displacement data of the monitoring points at the sampling time and fill in the missing data:

[0049]

[0050] Wherein, t is the time to be interpolated, x(t) is the displacement data of the monitoring point at the time to be interpolated, t1 and t2 are the previous known acquisition time and the next known acquisition time of the two adjacent known acquisition times to be interpolated, x(t1) and x(t2) are the displacement data of the monitoring point corresponding to the previous known acquisition time and the next known acquisition time, and t is the time point between t1 and t2.

[0051] The groundwater level data or rainfall data of the monitoring point at the time to be interpolated can be obtained in the same way.

[0052] (2) Using the Kalman filter method to filter the interpolation data in the interpolation data set to obtain the filtered monitoring data set. Using the Kalman filter method to filter the monitoring point displacement data, the monitoring point groundwater level data and the monitoring point rainfall data at n acquisition moments after the interpolation in the interpolation data set to eliminate abnormal data and obtain the filtered monitoring data;

[0053] (3) The filtered monitoring data in the filtered monitoring data set are normalized to obtain a standardized data set.

[0054] The maximum and minimum normalization processing is used for the filtered monitoring data, and each filtered monitoring data value is mapped to the interval [0,1]. This can effectively solve the problems of three heterogeneous data in terms of time, scale, missing values ​​and noise, namely, the displacement data of the monitoring point, the groundwater level data of the monitoring point and the rainfall data of the monitoring point, and obtain a standardized data set containing k monitoring points.

[0055] 3. Use the standardized data set and the corresponding k monitoring point location information to construct the spatiotemporal graph and obtain the target spatiotemporal graph network;

[0056] Assume that there are k monitoring points (including displacement monitoring points, rainfall monitoring points, groundwater level monitoring points, etc.) in the landslide risk area to be predicted.

[0057] Preferably, a spatiotemporal graph can be constructed using a graph neural network based on k standardized data sets and the location information of the corresponding k monitoring points to obtain a target spatiotemporal graph network G = (V, E).

[0058] Where V is a node set, each node represents a monitoring point, and the initial feature vector of each node can be expressed as one of the following information:

[0059] Monitoring point displacement data: the displacement of the landslide at the monitoring point.

[0060] Monitoring point groundwater level data: the height of the groundwater level near the monitoring point.

[0061] Monitoring point rainfall data: rainfall near the monitoring point.

[0062] E is the set of edges between nodes, and the edges between nodes represent the spatial relationship between monitoring points (such as adjacent relationship, physical distance or other association relationship).

[0063] The N-dimensional adjacency matrix A is defined using the spatial coordinates of the location information of the monitoring points N×N , used to represent the association between N nodes.

[0064] Based on the physical distance d between the monitoring points ij The adjacency matrix A is constructed by using the spatial coordinates of the location information of the monitoring points. This matrix will be used to represent the association or connectivity between the nodes (i.e., monitoring points) of the target spatiotemporal graph network. An adjacency matrix is ​​a matrix used to represent a graph, where both rows and columns correspond to nodes in the graph.

[0065] The adjacency matrix elements are defined as:

[0066]

[0067] In the formula, A ij is the adjacency matrix element between node i and node j, indicating the spatial correlation between nodes, d ij is the physical distance between monitoring points i and j, and s is a hyperparameter of the spatial scale. ij When A is smaller, ij The larger the value, the stronger the spatial correlation between the two monitoring points.

[0068] d ij It can be calculated as follows:

[0069]

[0070] In the formula, x i ,y i and h iRepresents the initial spatial coordinates of monitoring point i, and the symbol definition of monitoring point j is similar.

[0071] The adjacency matrix is ​​normalized to ensure the stability of graph neural networks.

[0072] The normalization method usually uses a symmetric normalized adjacency matrix:

[0073]

[0074] Where D is the degree matrix of the adjacency matrix A, D ij =∑ j A ij ,The normalized adjacency matrix A ensures that the influence of each node on its neighbor nodes is more balanced during the information transmission of the target spatiotemporal graph network.

[0075] 4. Extract spatial features from the target space-time graph network to obtain the spatial features of n acquisition moments within the acquisition time period;

[0076] The graph neural network is then forward propagated to update the feature representation of the nodes through information transfer between the nodes of the target spatiotemporal graph network, as follows:

[0077] In each layer of graph neural network GNN, the feature vector of the node is updated by fusing information with that of neighboring nodes.

[0078] Assume that the initial feature vector of the node of the graph neural network is (including monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data, etc.), then the node feature vector of the first layer of the graph neural network is The update formula is:

[0079]

[0080] Where N(i) is the set of neighbor nodes of node i, that is, the nodes directly connected to node i, and W (0) and b (0) is the learnable weight matrix and bias term of the first layer of the graph neural network, σ is the activation function, and ReLU activation is used.

[0081] In order to ensure balanced propagation, a normalized matrix is ​​usually used to update nodes. The normalized graph neural network update formula is:

[0082]

[0083] In the formula, is the normalized adjacency matrix.

[0084] For a multi-layer graph neural network with L layers, the update process is

[0085]

[0086] in, is the feature vector of node i in layer l, σ is the ReLU activation function, N(i) is the set of neighbor nodes of node i, is the normalized adjacency matrix, W (l) and b (l) is the weight matrix and bias vector of the lth layer.

[0087] The node feature vector finally outputted represents the spatial features at the acquisition moment.

[0088] 5. Further, the Transformer network is used to extract the temporal features of the spatial features of the n acquisition moments within the acquisition time period to obtain the spatiotemporal feature vector.

[0089] Specifically, we first use the node feature vector features output by the graph neural network (GNN) as the input sequence of the Transformer, and set it to E = [H 1 ,H 2 ,H 3 ...,H T ], where H T is the node feature vector at the Tth moment.

[0090] Transformer extracts temporal features from the input sequence and generates a spatiotemporal feature vector Z = [Z 1 ,Z 2 ,Z 3 ...,Z T ].

[0091] The encoder part of the Transformer consists of multiple self-attention layers and a feed-forward neural network.

[0092] The calculation formula of the self-attention mechanism of each layer is:

[0093]

[0094] Among them, Q, K1, V1 are query, key and value matrices respectively, d k is the dimension of the key, usually the same as the dimension of the query, and T represents the transpose operation.

[0095] The output of each self-attention layer can be expressed as:

[0096] Z l =Attention(Z l-1 )

[0097] Among them, Z lis the output of the lth layer. In this way, Transformer can adaptively focus on the spatial features of different acquisition moments based on the time information of n acquisition moments, thereby obtaining the spatiotemporal feature vector Z = [Z 1 ,Z 2 ,Z 3 ...,Z T ], extracting the temporal correlation of multi-source monitoring data.

[0098] 6. Perform prediction and decoding processing on the spatiotemporal feature vector to obtain the landslide displacement prediction result.

[0099] After encoding the spatial features at each moment, the output Z of the Transformer will be used as the input of the Transformer decoding layer to predict the landslide displacement value in the next M steps. The output sequence of the Transformer decoder is set to Y = [Y1, Y2, Y3, ... Y M ]:

[0100] Specifically, the decoding layer in the decoder consists of multiple self-attention layers and a feedforward neural network. The input is the output of the decoding layer in the previous step or the output of the encoder. The update process can be expressed as:

[0101]

[0102] In the formula, is the output of the decoding layer in the previous step, and Z is the output of the encoder, combined with time information.

[0103] In the above formula, for each prediction step, a mandatory strategy can be set to use the true value (i.e. historical data or the true prediction value of the previous step) as the input of the current decoding layer instead of the prediction output of the previous step. This can accelerate the convergence of the model and improve the accuracy. The specific update is:

[0104]

[0105] In the formula, It is the true value, which replaces the predicted output of the previous step.

[0106] In one embodiment, if Figure 3 As shown, a GNN-Transformer model structure is also provided, the input of the GNN-Transformer model structure is a standardized data set, and the output is a landslide displacement prediction result.

[0107] Preferably, the GNN-Transformer model is further trained based on heterogeneous data such as displacement data of monitoring points, groundwater level data of monitoring points, and rainfall data of monitoring points obtained from landslide monitoring. In the training stage of the GNN-Transformer model, the mean square error (MSE) is used as the loss function to calculate the predicted value Y (the predicted result of landslide displacement) and the true value The difference between (the final output of the decoding layer):

[0108]

[0109] Preferably, the Adam optimizer is used to update the parameters of the GNN-Transformer model and adjust the learning rate to accelerate convergence. The optimization process is: θ = θ - η▽ θ Loss, where η is the learning rate and θ is the model parameter.

[0110] Preferably, when evaluating the GNN-Transformer model, multiple indicators are used to verify the performance, including root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R 2 ) to fully reflect the predictive power of the GNN-Transformer model:

[0111]

[0112] In the formula, For the i a The true value of a point, is the average of the true values, is the predicted value of the i-th point.

[0113] In terms of model evaluation, as shown in Table 1, the performance comparison results of the GNN-Transformer model, the LSTM-GNN model, the GRU-GNN model, the LSTM model, the GRU model, and the Transformer model are shown.

[0114] Table 1

[0115]

[0116]

[0117] In the prediction results part of the model, such as Figure 4 As shown, the comparison between the landslide displacement prediction results of the GNN-Transformer model on the test set and the actual landslide displacement results is shown.

[0118] The present invention improves the accuracy and timeliness of landslide prediction and enhances the emergency response and decision-making support capabilities of relevant departments by fusing heterogeneous information of monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data, and using graph neural network to process the complex spatiotemporal dependencies between monitoring points of risk-to-be-predicted landslides.

[0119] In one embodiment, Figure 5 As shown, a landslide risk prediction device based on multi-source information fusion is provided, including an acquisition module, a processing module, a graph construction module, a space extraction module, a time feature extraction module and a decoding module, wherein:

[0120] A collection module is used to collect the original data of k monitoring points in the landslide with risk to be predicted within a collection time period to obtain an original data set containing k monitoring points;

[0121] A processing module, used for performing standardization processing on k original data in the original data set to obtain a standardized data set;

[0122] A graph construction module is used to construct a spatiotemporal graph using a standardized data set and the corresponding k monitoring point location information to obtain a target spatiotemporal graph network;

[0123] The spatial feature extraction module is used to extract spatial features from the target spatiotemporal graph network and obtain the spatial features of n acquisition moments within the acquisition time period;

[0124] The time feature extraction module is used to extract the time features of the space features of n acquisition moments within the acquisition time period to obtain the time-space feature vector;

[0125] The decoding module is used to perform prediction decoding processing on the spatiotemporal feature vector to obtain the landslide displacement prediction result.

[0126] In one embodiment, a device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any one of the above-mentioned landslide risk prediction methods based on multi-source information fusion are implemented.

[0127] In one embodiment, a storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in the above embodiment is implemented.

[0128] In the description of the present invention, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0129] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0130] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A landslide risk prediction method based on multi-source information fusion, characterized in that: include: Collect the original data of k monitoring points in the landslide with risk to be predicted within the collection time period to obtain the original data set containing k monitoring points; Standardizing k original data in the original data set to obtain a standardized data set; The standardized data set and the corresponding k monitoring point location information are used to construct the spatiotemporal graph to obtain the target spatiotemporal graph network; Extracting spatial features from the target spatiotemporal graph network to obtain spatial features of n acquisition moments within the acquisition time period; Extracting time features from the spatial features of n acquisition moments within the acquisition time period to obtain a spatiotemporal feature vector; The time-space characteristic vector is subjected to prediction decoding processing to obtain a landslide displacement prediction result.

2. A landslide risk prediction method based on multi-source information fusion as claimed in claim 1, characterized in that: The original data includes monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data.

3. The landslide risk prediction method based on multi-source information fusion according to claim 1 is characterized in that: The k original data in the original data set are standardized to obtain a standardized data set, including: The linear interpolation method is used to calculate the corresponding monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data at the time to be interpolated according to the monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data at multiple known collection times within the collection time period, so as to obtain an interpolated data set containing the monitoring point displacement data, monitoring point groundwater level data and monitoring point rainfall data at n collection times after interpolation processing; Using a Kalman filter method to filter the interpolation processed data in the interpolation processed data set to obtain a filtered monitoring data set; The filtered monitoring data in the filtered monitoring data set is normalized to obtain a standardized data set.

4. A landslide risk prediction method based on multi-source information fusion as claimed in any one of claims 1 to 3, characterized in that: Also includes: The spatial features of the target space-time graph network are extracted through a graph neural network to obtain the spatial features of n acquisition moments within the acquisition time period.

5. The landslide risk prediction method based on multi-source information fusion according to claim 4 is characterized in that: The method also includes extracting time features from the spatial features of n acquisition moments within the acquisition time period through a Transformer to obtain a spatiotemporal feature vector.

6. A landslide risk prediction device based on multi-source information fusion, characterized in that: It includes an acquisition module, a processing module, a graph construction module, a spatial feature extraction module, a temporal feature extraction module and a decoding module, wherein: The acquisition module is used to acquire the original data of k monitoring points in the landslide with risk to be predicted within the acquisition time period to obtain an original data set containing the k monitoring points; The processing module is used to perform standardization processing on k original data in the original data set to obtain a standardized data set; The graph construction module is used to construct a spatiotemporal graph using a standardized data set and corresponding k monitoring point location information to obtain a target spatiotemporal graph network; The spatial feature extraction module is used to extract spatial features from the target spatiotemporal graph network to obtain spatial features of n acquisition moments within the acquisition time period; The time feature extraction module is used to extract time features from the spatial features of n acquisition moments within the acquisition time period to obtain a spatiotemporal feature vector; The decoding module is used to perform prediction decoding processing on the spatiotemporal feature vector to obtain a landslide displacement prediction result.

7. A device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 5 when executing the computer program.

8. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images