Hydraulic concrete structure state monitoring method and system based on acoustic emission and deep learning

Abstract

The invention relates to a hydraulic concrete structure state monitoring method and system based on acoustic emission and deep learning. The method comprises the following steps: collecting acoustic emission signal data generated by a hydraulic concrete structure; performing signal preprocessing on the acoustic emission signal data, and extracting key feature vectors; clustering operation is carried out on the key feature vectors, labels of various types of acoustic emission signal data are marked according to clustering results, the labels comprise data types and corresponding relations between feature parameters and structure states, and the number of clusters is determined through a voting and scoring mechanism; constructing a classification model, and training the classification model by using the data marked with the labels; and monitoring the state grade of the hydraulic concrete structure at the current moment or period in real time by using the trained classification model. According to the method, the physical process of the acoustic emission signal is deeply excavated by combining multiple features and matching with the clustering algorithm, and the hydraulic concrete structure state is accurately captured from early microcrack damage to real-time dynamic tracking evaluation by using the deep learning model.

Description

Technical Field

[0001] The present invention relates to the technical field of hydraulic concrete structure monitoring, and in particular to a hydraulic concrete structure performance monitoring method and system based on acoustic emission and deep learning. Background Art

[0002] In large-scale water conservancy projects such as dams, gates, spillways and other facilities, hydraulic concrete is the main construction material, and its structural properties play a decisive role in the safety and service life of the project. However, during the service process, hydraulic concrete structures will be affected by various loads, environmental erosion, material aging, construction and maintenance defects, and are prone to potential problems such as microcracks, voids, and leakage channels. If these problems are not discovered in time and their development trends are not accurately assessed, they will pose serious risks to the safety and durability of the overall structure.

[0003] With the rapid improvement of information technology and computing power, deep learning algorithms have shown great application potential in many fields. Combining acoustic emission technology with deep learning has opened up a new path for hydraulic concrete structure monitoring. However, there are still obvious shortcomings in the current practical application and research in this field. On the one hand, in the deep analysis of multi-dimensional acoustic emission data, due to the complex characteristics of acoustic emission signals such as multi-parameters, time-varying and non-stationary, the existing analysis methods are difficult to comprehensively and deeply analyze the internal connections and interactions between these data. On the other hand, the research on the effective identification and graded evaluation of different damage evolution stages of structures from micro-damage to severe cracking failure is not perfect. In the micro-damage stage, the acoustic emission signal is often weak and the characteristics are not obvious. The existing deep learning model is difficult to accurately capture these subtle changes and make accurate judgments; in the process of structural damage gradually developing to severe cracking failure, the damage mode is complex and diverse. Different hydraulic concrete structures have different damage evolution laws under different environmental and load conditions. The existing model is difficult to achieve universal accurate identification and graded evaluation.

[0004] In summary, there is an urgent need to develop a hydraulic concrete structure performance monitoring method based on acoustic emission and deep learning, so as to effectively solve the defects of existing methods and provide reliable support for safety warning and maintenance decision-making of large-scale water conservancy projects. Summary of the invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the technical defects existing in the prior art, and propose a hydraulic concrete structure state monitoring method and system based on acoustic emission and deep learning. It combines multiple features with a clustering algorithm to deeply explore the physical process behind the acoustic emission signal, and uses a deep learning model to accurately capture the state of the hydraulic concrete structure from early microcrack damage to real-time dynamic tracking and evaluation, and can issue early warnings in time, take measures before macro cracks appear in the structure, effectively reduce the risk of sudden accidents, and provide strong protection for engineering safety.

[0006] In order to solve the above technical problems, the present invention provides a hydraulic concrete structure performance monitoring method based on acoustic emission and deep learning, comprising the following steps: S10: arranging an acoustic emission sensor on the hydraulic concrete structure to be monitored, and collecting acoustic emission signal data generated by the hydraulic concrete structure during service by means of the acoustic emission sensor; S20: performing signal preprocessing on the collected acoustic emission signal data, and extracting key feature vectors of the acoustic emission signal data after the signal preprocessing from multiple dimensions; S30: performing a clustering operation on the extracted key feature vectors, and labeling various types of acoustic emission signal data according to the clustering results, wherein the labels include data categories and the corresponding relationship between feature parameters and structural states, wherein the number of clusters is determined by a voting scoring mechanism; S40: constructing a classification model, and training the classification model using the labeled data; S50: Using the trained classification model to monitor in real time the state level of the hydraulic concrete structure at the current moment or in a period.

[0007] In one embodiment of the present invention, in S10, at least three acoustic emission sensors are arranged in an array on the hydraulic concrete structure to be monitored.

[0008] In one embodiment of the present invention, in S20, the signal preprocessing of the collected acoustic emission signal data includes denoising and filtering, data synchronization and calibration, and redundancy removal and anomaly detection.

[0009] In one embodiment of the present invention, in S30, the method of performing a clustering operation on the extracted key feature vectors includes: S31: normalize the key feature vectors and remove outliers; S32: Calculate the distance between the key feature vectors after normalization and outlier removal to obtain a distance matrix; S33: Based on the distance matrix, a clustering algorithm is used to perform a clustering operation on the key feature vectors, wherein the number of clusters is determined by a voting scoring mechanism, including determining that the score of the optimal classification number of a single indicator is P, the score of the classification number with the second best effect is P-1, and so on, determining the score of each cluster number, and adding the scores of each classification number of all indicators to obtain the cluster number with the highest score, and selecting the cluster number with the highest score as the classification number of the final cluster; S34: Obtaining data categories of key feature vectors according to the clustering results, and constructing a correspondence between feature parameters and structural states, and marking the data categories and the correspondence between their feature parameters and structural states as labels.

[0010] In one embodiment of the present invention, in S40, the method of training the classification model using labeled data includes: S41: inputting the labeled data into the long short-term memory network classification model, and the long short-term memory network classification model outputs a category prediction value; S42: Selecting hyperparameters of the long short-term memory network classification model based on the input and output mode of the long short-term memory network classification model; S43: Use the gray wolf optimization algorithm to optimize the hyperparameters of the long short-term memory network classification model. By simulating the behavior of gray wolf groups, iteratively search for the optimal solution in the multidimensional search space to obtain the optimal hyperparameter combination of the long short-term memory network classification model. S44: After determining the optimal hyperparameter combination, the long short-term memory network classification model is trained using labeled data to obtain a trained long short-term memory network classification model.

[0011] In one embodiment of the present invention, in S43, the method for optimizing the hyperparameters of the long short-term memory network classification model using the gray wolf optimization algorithm includes: S431: Divide the labeled data into training set and validation set, set the population size and maximum number of iterations , randomly generate the initial position of each gray wolf , and ensure that it is within the given search range; S432: For each , train the LSTM network classification model in the training set, and calculate the validation index as the fitness value in the validation set; S433: Sort and assign values ​​according to fitness values. Wolf, calculate the new hunting strategy according to the gray wolf optimization algorithm , and then perform fitness evaluation after updating the initial position; S434: Repeat steps S432 and S433 until the maximum number of iterations is met or the fitness value tends to be stable, and take the optimal solution corresponding to As the optimal hyperparameter combination for the long short-term memory network classification model.

[0012] In one embodiment of the present invention, in S50, the method of using the trained classification model to monitor the state level of the hydraulic concrete structure in real time at the current moment or period includes: S51: inputting the real-time collected acoustic emission signal data into the trained classification model, and outputting the predicted value of the structural state; S52: Compare and judge the predicted value of the structural state with the preset structural state level, and generate a corresponding safety warning according to the judgment result.

[0013] Based on the same inventive concept, the present invention also provides a hydraulic concrete structure performance monitoring system based on acoustic emission and deep learning, comprising: A data acquisition module, which is used to arrange an acoustic emission sensor on the hydraulic concrete structure to be monitored, and to collect acoustic emission signal data generated by the hydraulic concrete structure during service by means of the acoustic emission sensor; A feature extraction module is used to perform signal preprocessing on the collected acoustic emission signal data and extract key feature vectors of the acoustic emission signal data after signal preprocessing from multiple dimensions; A clustering module is used to perform clustering operations on the extracted key feature vectors, and label various types of acoustic emission signal data according to the clustering results, wherein the labels include data categories and the corresponding relationship between characteristic parameters and structural states, wherein the number of clusters is determined by a voting scoring mechanism; A model training module, which is used to build a classification model and train the classification model using labeled data; The state monitoring module is used to use the trained classification model to monitor the state level of the hydraulic concrete structure in real time at the current moment or period.

[0014] Based on the same inventive concept, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method when executing the program.

[0015] Based on the same inventive concept, the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above-mentioned method are implemented.

[0016] The above technical solution of the present invention has the following advantages compared with the prior art: 1. The present invention provides a method and system for monitoring the properties of hydraulic concrete structures based on acoustic emission and deep learning. The method and system deeply mine the physical process behind the acoustic emission signal by combining multiple features with a clustering algorithm, and utilizes a deep learning model to accurately capture the state of hydraulic concrete structures from early microcrack damage to real-time dynamic tracking and evaluation. The method can also issue early warnings and take measures before macro cracks appear in the structure, effectively reducing the risk of sudden accidents and providing strong protection for engineering safety.

[0017] 2. The present invention provides a method and system for monitoring the properties of hydraulic concrete structures based on acoustic emission and deep learning. The technical architecture of the system can be integrated with the existing structural health monitoring platform, and the system has good scalability for different scales, environments and concrete mix ratios. At the same time, the optimization algorithm can be used to adaptively adjust the model parameters under different working conditions. This high adaptability makes the invention not only suitable for a variety of engineering scenarios, but also can detect potential hidden dangers at an early stage through online monitoring and quantitative evaluation of large-scale hydraulic concrete structures, scientifically guide operation and maintenance arrangements, and significantly improve engineering safety and economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to make the contents of the present invention more clearly understood, the present invention is further described in detail below based on specific embodiments of the present invention in conjunction with the accompanying drawings.

[0019] Figure 1 It is a flow chart of a hydraulic concrete structure performance monitoring method based on acoustic emission and deep learning proposed in the present invention.

[0020] Figure 2 It is a schematic flow chart of step S30 in the method of the present invention.

[0021] Figure 3 It is a schematic flow chart of step S40 in the method of the present invention.

[0022] Figure 4 It is a schematic flow chart of step S50 in the method of the present invention.

[0023] Figure 5 It is a schematic diagram of the arrangement of acoustic emission sensors for hydraulic concrete specimens in the case of the present invention.

[0024] Figure 6 It is a flow chart of the scoring algorithm for determining the optimal number of clusters in the case of the present invention.

[0025] Figure 7 This is a flowchart of using the GWO algorithm to optimize the parameters of the LSTM classification model in the case of the present invention.

[0026] Figure 8These are the index values ​​and scores of different cluster numbers of fully graded specimens in the case of the present invention.

[0027] Fig. 9 It is the clustering result of acoustic emission energy vs center frequency in the case of the present invention.

[0028] Fig.10 It is the clustering result of acoustic emission duration vs amplitude in the case of the present invention.

[0029] Fig.11 It is the clustering result of count, center frequency and amplitude in the case of the present invention.

[0030] Fig.12 It is the average prediction accuracy of the GWO-LSTM classification model in the case of the present invention. DETAILED DESCRIPTION

[0031] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0032] Reference Figure 1 As shown, an embodiment of the present invention provides a hydraulic concrete structure performance monitoring method based on acoustic emission and deep learning, comprising the following steps: S10: Arranging an acoustic emission sensor on the hydraulic concrete structure to be monitored, and collecting acoustic emission signal data generated by the hydraulic concrete structure during service by means of the acoustic emission sensor; S20: performing signal preprocessing on the collected acoustic emission signal data, and extracting key feature vectors of the acoustic emission signal data after the signal preprocessing from multiple dimensions; S30: performing a clustering operation on the extracted key feature vectors, and labeling various types of acoustic emission signal data according to the clustering results, wherein the labels include data categories and the corresponding relationship between feature parameters and structural states, wherein the number of clusters is determined by a voting scoring mechanism; S40: Build a classification model and use the labeled data to train the classification model; S50: Using the trained classification model to monitor in real time the state level of the hydraulic concrete structure at the current moment or in a period.

[0033] The present invention provides a hydraulic concrete structure state monitoring method based on acoustic emission and deep learning. It deeply mines the physical process behind the acoustic emission signal by combining multiple features with a clustering algorithm, and uses a deep learning model to accurately capture the state of the hydraulic concrete structure from early microcrack damage to real-time dynamic tracking and evaluation. It can also issue early warnings and take measures before macro cracks appear in the structure, effectively reducing the risk of sudden accidents and providing strong protection for engineering safety.

[0034] Among them, in step S10, multi-channel acoustic emission sensors are reasonably arranged on the surface and inside of the hydraulic concrete structure to be monitored, such as gravity dams, arch dams, gate piers, etc., or laboratory specimens, according to the structural size, stress distribution and monitoring accuracy requirements. During the service or loading process of the structure, the acoustic emission signal generated by it is collected, and the basic parameters such as amplitude, energy, number of impacts, rise time, duration, peak frequency, etc. of each acoustic emission event are recorded. If the positioning function is enabled, the event occurrence time and locatable coordinate information are saved at the same time.

[0035] As a preferred method, a high-sensitivity broadband acoustic emission sensor suitable for concrete materials can be selected, usually with a frequency range of tens to hundreds of kHz. According to the size, material properties and damage sensitivity of the monitored object (such as concrete dams, test beams, gate piers, etc.), a sensor with a suitable center frequency (such as 100kHz, 150kHz, 300kHz, etc.) can be selected to cover the typical acoustic emission signal frequencies at different stages such as crack initiation and expansion. The sensor and the concrete surface need to be coupled using a special coupling agent (such as vacuum grease, coupling glue, etc.) to enhance the efficiency of sound wave transmission and reduce interface loss. In outdoor or harsh environments, industrial-grade sensors with waterproof and dustproof capabilities and resistance to temperature and humidity changes should be selected to ensure the reliability of long-term monitoring.

[0036] The above-mentioned acoustic emission sensor layout principles need to consider key area coverage, sensor spacing and resolution, installation and fixation, etc. For large hydraulic structures such as dams, spillways, and gate piers, they should be laid out in stress concentration areas or areas prone to cracking (such as dam heels, dam shoulders, surface concrete-rock foundation interfaces, and near reserved construction joints). If it is a laboratory specimen (such as a three-point bending beam, a tensile specimen, etc.), the location where cracks are prone to initiation (notch tip, near the tension zone) is preferred. The layout spacing should comprehensively consider factors such as sound wave attenuation, structural size, signal positioning accuracy, and hardware cost. For large-scale hydraulic concrete structures, preliminary layout can be carried out at a spacing of 5 to 15 meters; for small and medium-scale experimental specimens, the spacing can be reduced to 10 to 50 cm. If three-dimensional positioning is required, no less than 3 to 4 sensors can be arranged in an array; to improve positioning accuracy, the sound wave propagation speed and attenuation characteristics in the concrete can be measured before the sensor is laid, and regular calibration can be carried out during the monitoring process. The sensor can be fixed with a magnetic clamp or strong glue, and a protective cover can be added in outdoor or underwater environments. When pre-buried sensors after construction or drilling, the sealing and durability of the wiring must be ensured to avoid signal distortion or sensor damage due to leakage or vibration.

[0037] Furthermore, a multi-channel acoustic emission data acquisition instrument (acoustic emission monitoring host) can be selected, and the corresponding number of channels (4, 8, 16 or more) can be selected according to the scale of the project and the number of sensors. The data acquisition card should have sufficient sampling rate (such as above 1MS / s) and resolution (12-16 bit ADC) to capture high-speed transient impact signals and obtain clear waveform characteristics. It has a settable preamplifier gain and filtering function, with a common gain of 20-40dB, and the filter is adjustable in the range of several kHz to hundreds of kHz to adapt to acoustic emission signals of different frequency bands. Establish a reasonable threshold setting scheme, and determine the initial trigger threshold by pre-calibrating the noise level; if event positioning is to be achieved, the arrival time (Hit Time) of each channel needs to be recorded and the arrival time difference is automatically calculated by the system; if the demand for high-frequency sensor signals is high, the pre-trigger function (Pre-trigger) can also be enabled to retain the early waveform information of the event. Use triggered or continuous recording mode. Triggered recording can reduce the amount of invalid data, but may miss weak events; continuous recording requires high storage and bandwidth, but can provide complete waveforms and background information. In actual projects, the mode can be switched according to the monitoring cycle or key information needs, and the data can be regularly transmitted to the back-end server or cloud for centralized processing, archiving and analysis.

[0038] In order to distinguish between real structural damage events and interference signals, several accelerometers can be arranged on the surface of the structure or surface vibration sensors can be used for synchronous monitoring. Once a large-scale mechanical vibration is detected, the acoustic emission signal in the corresponding time period will be marked as a possible "noise event". In areas where power equipment and frequency converters are concentrated, shielded cables and metal threading pipes are selected, and good grounding is ensured to reduce power frequency interference. Water flow noise is common in hydraulic engineering. The characteristics of the water flow background signal can be recorded before the test, a "noise template" can be established, and pattern matching can be performed during real-time monitoring to eliminate it, or the characteristic frequency band signal of the water flow can be filtered out through spectrum analysis. In addition, several artificial knocks or special pulse generators are used in the concrete specimen or dam body to measure the propagation time of the sound wave in different paths, so as to obtain the average propagation speed and attenuation coefficient of the sound wave in the concrete material for subsequent positioning and cluster analysis correction. Loading or simulation tests are carried out on complete specimens and areas without obvious defects or cracks in the laboratory or on site to obtain their acoustic emission characteristic data as a benchmark; loading control tests are carried out in areas with artificial cracks or damage to obtain the acoustic emission characteristics corresponding to typical damage modes, thereby providing reference labels for subsequent analysis.

[0039] For large structures, a distributed acquisition module can be built to upload data to a monitoring center or cloud platform through a wireless or wired network for online monitoring and storage. Regularly check the sensor coupling status, cable connection, and system power supply; compare the consistency of responses between sensors. If abnormal jumps occur, troubleshoot the fault in a timely manner and make corresponding corrections. In major engineering applications, the collected acoustic emission data can be encrypted and backed up for disaster recovery to avoid data loss caused by unexpected power outages or network failures, and to ensure the continuity and reliability of monitoring work.

[0040] Among them, in step S20, signal preprocessing includes denoising and filtering, data synchronization and calibration, redundancy removal and abnormality detection processing. In terms of denoising and filtering, for high-noise ratio or invalid acoustic emission events caused by mechanical vibration, water flow sound, electromagnetic interference and other environments, high-pass / low-pass / band-pass filters can be used to eliminate frequency bands that are not related to structural damage, and adaptive algorithms (such as LMS and RLS filtering) can be combined to dynamically identify and track noise characteristics and suppress them, avoiding the problem of poor filtering effect of fixed cutoff frequency filters under different working conditions. In addition, the acoustic emission signal when it is unloaded or not damaged can be collected in advance to obtain the noise power spectrum or characteristic peak distribution, and the event trigger threshold can be set accordingly to exclude false events with too low energy or intensity. Data synchronization and calibration is to ensure that the sampling time base of each channel is consistent when a multi-channel acquisition instrument is used; time alignment can be performed through hardware triggering or synchronization pulses to ensure the accuracy of subsequent positioning and feature analysis. In addition, the acoustic emission device that supports the "pre-trigger" function can retain the original data in a period of time before the event is triggered, so as to fully record the waveform at the start of the event. When multiple channels capture similar waveforms almost simultaneously within a short period of time, they can be regarded as the same acoustic emission event. They can be merged into a single event by judging the arrival time difference to reduce data redundancy.

[0041] In step S20, feature extraction includes time domain features, frequency domain features, time-frequency domain features, statistics and composite features. Acoustic emission signals present multi-scale and multi-dimensional characteristics during crack initiation and expansion. In order to fully capture damage evolution information, key features can be extracted from the time domain, frequency domain, time-frequency domain and other perspectives. Time domain features mainly include: amplitude, energy, number of impacts / counts, rise time, duration, impact rate; frequency domain features mainly include: main frequency, center frequency, bandwidth, spectrum center of gravity and standard deviation; time-frequency domain features include: wavelet packet decomposition coefficients, instantaneous energy, short-time Fourier transform spectrum; statistics and composite features include: b value, Ib value, impact distribution characteristics, RA-AF spectrum. After obtaining a large number of time domain, frequency domain, and time-frequency domain features, feature selection or dimensionality reduction processing is performed to improve the efficiency and accuracy of subsequent deep learning analysis. For example, the Pearson correlation coefficient or mutual information method is used to screen out features that are highly correlated with the crack failure mode; or high-dimensional features are projected into low-dimensional space for visualization, clustering or classification.

[0042] Furthermore, the features of each acoustic emission signal (such as amplitude, energy, rise time, duration, main frequency, etc.) are stored as a fixed-length vector to facilitate the subsequent reading of various algorithms. In addition, in the laboratory or with control tests, known tensile cracks, shear failures and other events can be annotated to provide a reference for supervised training of deep learning models. When the monitoring scale is large and the number of events is huge, the data can be partitioned or stored in time periods, and the distributed computing framework can be combined to accelerate feature extraction and analysis.

[0043] Among them, in step S30, after completing the preprocessing and multi-dimensional feature extraction of the acoustic emission signal, a series of high-dimensional feature vector sets can be obtained, such as [, ](amplitude, energy, rise time, duration, main frequency, etc.). In order to preliminarily identify different types of acoustic emission events (such as tensile cracks, shear cracks, friction or collision interference, etc.), these feature vectors are clustered and analyzed to provide reference labels or data grouping for subsequent deep learning models.

[0044] like Figure 2 As shown, the specific steps of the above clustering process are as follows: S31: normalize the key feature vectors and remove outliers; S32: Calculate the distance between the key feature vectors after normalization and outlier removal to obtain a distance matrix; S33: Based on the distance matrix, a clustering algorithm is used to perform a clustering operation on the key feature vectors, wherein the number of clusters is determined by a voting scoring mechanism, including determining that the score of the optimal classification number of a single indicator is P, the score of the classification number with the second best effect is P-1, and so on, to determine the score of each cluster number, and add the scores of each classification number of all indicators to obtain the cluster number with the highest score, and select the cluster number with the highest score as the classification number of the final cluster; S34: Obtaining data categories of key feature vectors according to the clustering results, and constructing a correspondence between feature parameters and structural states, and marking the data categories and the correspondence between their feature parameters and structural states as labels.

[0045] In step S31, since the dimensions and numerical ranges of different key features may vary greatly (for example, "duration" may be in the range of tens of microseconds to milliseconds, while "energy" may be in the range of hundreds to tens of thousands of amplitude units), it is necessary to standardize or normalize each feature to avoid some features from having an unreasonable excessive impact on the distance metric during the clustering process. Common methods include Z-score standardization, Min-max normalization, etc.

[0046] In step S33, a bottom-up agglomerative hierarchical clustering can be used, and its main process includes: (1) Initialization: each acoustic emission event is regarded as a separate cluster. N clusters; (2) Iterative merging: find the two closest clusters at each step, merge them into a new cluster, and update the distance information related to this new cluster in the distance matrix; (3) Stop condition: stop when all data are merged into one cluster, or stop when the specified number of clusters is merged; (4) Linking method: different linking methods will affect the calculation method of inter-cluster distance. In acoustic emission analysis, in order to balance cohesion and robustness, minimizing the sum of squares of intra-class deviations or the average distance between clusters can be used for better results. The results of hierarchical clustering are usually displayed in a dendrogram, where each branch represents a cluster merging process, and the vertical axis can represent the "distance" or "dissimilarity" during the merging. As the height of the dendrogram increases step by step, it means that a larger distance threshold is required for merging.

[0047] Furthermore, the number of clusters can be evaluated by the Davies-Bouldin (DB) indicator, the Dunn indicator, the Rousseeuw's silhouette (Silhouette) indicator, and the Calinski-Harabaz (CH) indicator. In general, the clustering effect and the optimal number of clusters are determined by the elbow method, but the number of clusters determined by only one indicator is not necessarily referenceable. Therefore, this embodiment preferably determines the optimal number of clusters by a voting scoring mechanism. The voting scoring mechanism is specifically as follows: for a single indicator, the score of the optimal number of classifications determined by the indicator is P, then the number of classifications with the second best effect is scored P-1, and so on. The score of each number of clusters is determined, and then the score of each number of classifications of the above four indicators is added together to obtain the number of clusters with the highest score, and the number of clusters is selected as the number of classifications for the final clustering. This scoring mechanism effectively combines the advantages of different clustering effectiveness indicators and weakens the limitations of a single indicator.

[0048] In step S34, after obtaining the final clusters, the characteristic mean values ​​(such as amplitude, energy, main frequency, duration, etc.) of the center of each cluster or representative sample can be checked and compared with the actual situation or damage mechanism on site to determine the potential acoustic emission type corresponding to the cluster: Cluster A: high energy, short duration, may be brittle tensile cracks; Cluster B: medium energy, long duration, may be friction collision or shear dislocation; Cluster C: low energy, low frequency, may be environmental noise or atypical damage events. Use scatter plots or radar plots to visualize the distribution of various clusters in the multidimensional feature space. If it is found that the differences between some clusters are not obvious, they can also be merged; if a cluster still contains significantly different subclasses, it can be further subdivided. The advantage of hierarchical clustering is that the clustering level can be flexibly selected to form a multi-scale analysis. Further, the laboratory specimens can be combined with the specific crack types (tensile, shear, interface debonding, etc.) observed during the loading process or the later failure surface to verify the clustering results and adjust the weights of the model for feature allocation. At the construction site, if there are visible signs of leakage, crack direction or ultrasonic detection results, the acoustic emission clusters obtained by hierarchical clustering can be compared with the actual damage location and mechanism to verify the clustering accuracy or correct the threshold setting. After verification, labels such as "microcrack stretching", "friction collision" and "shear dislocation" can be given to various clusters, and high-quality training samples can be provided for subsequent supervised learning models or expert systems.

[0049] Among them, in step S40, in order to further realize the accurate evaluation of the structural properties of hydraulic concrete, it is necessary to further use supervised learning methods to accurately classify the structural health status of hydraulic concrete. Long Short-Term Memory (LSTM) is a robust classifier widely used in small sample and high-dimensional scenarios, but its performance is very sensitive to hyperparameters (such as the number of hidden layer units, learning rate, etc.). In order to improve the robustness and classification accuracy of the model, the present invention introduces the Grey Wolf Optimizer (GWO) algorithm to automatically tune the hyperparameters of LSTM. Figure 3 As shown, the method of training a classification model using labeled data specifically includes the following steps: S41: inputting the labeled data into the long short-term memory network classification model, and the long short-term memory network classification model outputs a category prediction value; S42: Selecting the kernel function and hyperparameters of the long short-term memory network classification model based on the input and output mode of the long short-term memory network classification model; S43: Use the gray wolf optimization algorithm to optimize the hyperparameters of the long short-term memory network classification model. By simulating the behavior of gray wolf groups, iteratively search for the optimal solution in the multidimensional search space to obtain the optimal hyperparameter combination of the long short-term memory network classification model. S44: After determining the optimal hyperparameter combination, the long short-term memory network classification model is trained using labeled data to obtain a trained long short-term memory network classification model.

[0050] In the above step S41, the input of the classification model is the data after preliminary clustering and labeling, and each feature vector corresponds to a structural state label, for example: 0 is healthy; 1 is mild or sub-healthy (microcrack initiation); 2 is moderate damage (crack extension); 3 is severe damage (through crack or serious failure). The output is the category prediction value given by the LSTM classification model. The model accuracy can be evaluated by comparing the category prediction value with the true label. If it is a multi-classification scenario, the LSTM combination model can be constructed in a "one-to-one" or "one-to-many" manner to distinguish multiple health levels respectively.

[0051] In the above step S43, the main steps of the gray wolf optimization (GWO) algorithm include: (1) Population initialization: generating a number of gray wolves (solution vectors), each of which has a gene consisting of a combination of LSTM hyperparameters to be optimized, such as [] , can be randomly initialized within a reasonable range (such as , etc.) to cover a sufficiently large search space; (2) Social hierarchy: The individuals with the best, second best, and third best fitness in the population are called , and , and the remaining individuals are called In this problem, fitness corresponds to the validation set or cross-validation accuracy (which can also be measured by indicators such as F1-score and AUC). The higher the value, the better the hyperparameter combination. (3) Encirclement and update strategy: In each round of iteration, other gray wolves ( ) will be based on The position distance of the three wolves is updated, and the update formula usually contains a random factor A , C , simulating the encirclement and pursuit process during hunting; (4) Convergence judgment: When the maximum number of iterations is reached or the fitness converges to a certain threshold, GWO stops and outputs the global optimal parameter combination. Therefore, the method of optimizing the hyperparameters of the long short-term memory network classification model using the gray wolf optimization algorithm specifically includes the following steps: S431: Divide the labeled data into training set and validation set, set the population size and maximum number of iterations , randomly generate the initial position of each gray wolf , and ensure that it is within the given search range; S432: For each , train the LSTM network classification model in the training set, and calculate the validation index as the fitness value in the validation set; S433: Sort and assign values ​​according to fitness values. Wolf, calculate the new hunting strategy according to the gray wolf optimization algorithm , and then perform fitness evaluation after updating the initial position; S434: Repeat steps S432 and S433 until the maximum number of iterations is met or the fitness value tends to be stable, and take the optimal solution corresponding to As the optimal hyperparameter combination for the long short-term memory network classification model.

[0052] In the above step S44, after finding the optimal parameter combination After that, the training set can be combined with the validation set, and the parameters can be used to train the LSTM classification model again to obtain the trained GWO-LSTM classification model. Use an independent test set to evaluate the model accuracy, precision, recall, F1-score, confusion matrix and other indicators; if the test effect is still not ideal, you can expand the features, adjust the kernel function or perform data cleaning, run the GWO-LSTM classification model multiple times under different random initialization populations and data partitioning methods, observe the discrete degree of its final convergence results, and evaluate the sensitivity of the model to different initial conditions. In actual monitoring, the newly collected acoustic emission feature vector is input into the trained GWO-LSTM classification model, and the structural state category (healthy / microcracks / crack extension, etc.) can be output in real time; when more high-risk events are detected, the system can trigger an early warning and suggest on-site personnel to check in time. In long-term monitoring, as the material ages or environmental conditions change, the original model parameters may gradually become invalid. The latest labeled data can be collected periodically or event-driven, and the GWO-LSTM classification model can be re-optimized to maintain the adaptability of the model to new working conditions. If the environmental noise or sensor layout changes significantly, data preprocessing and label calibration will also need to be redone.

[0053] Among them, in step S50, if Figure 4 As shown, the method of using the trained classification model to monitor the state level of the hydraulic concrete structure in real time at the current moment or in a period includes: S51: inputting the real-time collected acoustic emission signal data into the trained classification model, and outputting the predicted value of the structural state; S52: Compare and judge the predicted value of the structural state with the preset structural state level, and generate a corresponding safety warning according to the judgment result.

[0054] In the above step S51, for the multi-classification model, the probability or confidence of each possible state (such as healthy state, slight damage, significant damage, critical state) is output, and the maximum value is selected as the structural state at the current moment. For the regression model, the structural degradation index (such as damage degree, crack width prediction, etc.) is output, and then the value is segmented and thresholded (such as 0-0.3 for health, 0.3-0.6 for moderate damage, >0.6 for severe damage) is judged to obtain the state level. In actual operation, the model prediction results of each cycle or moment are recorded in the database to form a time series for subsequent statistical analysis and evolution trend tracking. If a batch of high-risk state outputs appear, it can trigger a safety warning or manual intervention.

[0055] In the above step S52, according to the characteristics of the hydraulic concrete structure and the test calibration, four levels of status can be set, such as healthy state (level 0): low frequency of acoustic emission events, small energy and characteristic parameters, no obvious cracks in the structure as a whole or only minor defects. Mild damage (level 1): Occasionally, obvious crack acoustic emission signals are detected, but the cracks are still relatively stable and will not immediately affect safety. Moderate or significant damage (level 2): ​​There is an active crack expansion trend in the structure, and the frequency and energy of acoustic emission events are significantly increased. Critical state (level 3): Continuous or high-intensity acoustic emission events indicate that the component may be penetrated or unstable at any time. The above-mentioned level division can be configured with corresponding thresholds or judgment rules in the system. For the model output (category label or damage factor), the alarm threshold is pre-defined: when the prediction result exceeds the set threshold, it is considered to be a state deterioration; if it continues to exceed the limit and shows an upward trend, an alarm is issued. In order to avoid false alarms or jitters, hysteresis processing (such as setting the number of consecutive limit violations and the continuous duration) can be used to determine whether to issue a warning or early warning to improve the stability and reliability of the system.

[0056] Furthermore, when suspected serious damage (such as "significant damage" or "critical condition") is identified, the system automatically triggers an alarm: (1) SMS, email or App notification: promptly inform operation and maintenance personnel and decision makers; (2) On-site sound and light alarm (optional): If in a laboratory or small-scale test scenario, an sound and light reminder can be used to alert nearby staff.

[0057] Furthermore, it can be connected to a remote monitoring platform or SCADA system to coordinate with other sensor data (such as strain gauges, accelerometers) for comprehensive analysis and judgment. If necessary, it can issue water level reduction or dispatch instructions to prevent the expansion of safety hazards.

[0058] Furthermore, the sensor installation coordinates can be combined to draw a three-dimensional or two-dimensional structural model, and the intensity, frequency, status level, etc. of the acoustic emission events at each monitoring point (or monitoring area) can be presented in color or symbol size, so that the operating personnel can intuitively understand the degree of damage in each area. As a preferred embodiment, on the schematic diagram of the dam or pier, a heat map (Heatmap) or bubble map can be used to identify the concentrated damage area, and an enlarged view can be provided when necessary. The key indicators (such as the cumulative energy of the acoustic emission event, the damage level predicted by the model, the impact rate Hits / s, etc.) are displayed in a time series curve to assist engineers in judging whether the health of the structure has deteriorated sharply in the short term. A rolling window (such as 24 hours or 7 days) can be set to dynamically update the chart to reflect the recent evolution; retain historical data for retrospective comparison.

[0059] Furthermore, if the system has the function of sound source positioning or crack imaging, a three-dimensional crack expansion process animation can be generated on the visualization platform to demonstrate the evolution path from microcrack initiation to macro crack penetration, providing an intuitive basis for subsequent repair or maintenance.

[0060] Furthermore, the system can automatically generate a structural health report based on the changes in status levels, warning records and cumulative damage indicators over a period of time. The report contains statistical charts, damage distribution diagrams and targeted suggestions (such as areas that need to be reinforced, monitoring frequency adjustment, etc.).

[0061] Furthermore, if the status is predicted to be mild damage or above, an automated inspection process can be triggered: on-site personnel can be notified to use auxiliary means such as ultrasonic detection and radar detection for precise measurement; or drones and robots can be dispatched to conduct secondary inspections such as close-range photography and infrared temperature measurement of suspicious parts.

[0062] The following is an example of monitoring the structural properties of laboratory hydraulic concrete specimens to illustrate the content of the hydraulic concrete structural properties monitoring method based on acoustic emission and deep learning proposed in the present invention.

[0063] Case: Step 1: Specimen preparation, including making standard three-point bending full-grade hydraulic concrete beam specimens (size 1700×450×1350mm) and prefabricating opening cracks, refer to the attached Figure 5 , 4 acoustic emission sensors are evenly arranged on the surface of the specimen, vaseline is used as the coupling agent, and the sensors are fixed with rubber bands. The monitoring range covers the areas where cracks may occur or expand.

[0064] The second step: data acquisition, specifically including the three-point bending test, which is loaded by the MTS322 electro-hydraulic tester with a range of 500kN. The tester directly loads the specimen through a circular loading head. During the test loading process, the loading rate is controlled by the crack opening displacement, and the loading rate is 0.001mm / s. The acoustic emission sensor is used to collect the acoustic emission signal generated during the stretching process in real time, and the basic parameters such as amplitude, energy, duration, frequency, and load-displacement records of each event are saved.

[0065] The third step: data processing and analysis, including denoising and filtering to extract the multidimensional feature vector of each event; using hierarchical clustering to cluster the signal, using Figure 6 The scoring algorithm shown in the figure determines the number of hierarchical clusters. The index values ​​and scores of each cluster number are as follows: Figure 8 As shown in the figure, cluster analysis was performed on six acoustic emission parameters (including rise time, count, energy, duration, amplitude and center frequency) to obtain Figure 9~Figure 12 result.

[0066] Step 4: Use the labeled data to train the LSTM classification model. During the training process, the Gray Wolf Optimization Algorithm is used to optimize the hyperparameters of the LSTM classification model. The specific optimization diagram is as follows: Figure 7 shown.

[0067] Step 5: Deep learning evaluation model, which specifically includes identifying, predicting or detecting the degree of damage or deterioration of the structure through the trained GWO-LSTM classification model, and judging and grading the health status of the hydraulic concrete structure based on the output results of the deep learning model, and generating corresponding visualization or alarm information.

[0068] The present invention provides a method for monitoring the state of hydraulic concrete structures based on acoustic emission and deep learning. The method arranges acoustic emission sensors on the surface or inside of hydraulic concrete structures (such as dams, gate piers or laboratory specimens) to collect acoustic emission signals released in the process of material damage or crack expansion in real time; then the original signals are filtered, denoised and multi-dimensional feature extracted, and hierarchical clustering analysis is combined to distinguish different types of damage or noise events. On this basis, the present invention applies the optimized deep learning model (LSTM model optimized by GWO) to identify and quantify the structural state of materials at stages such as microcrack initiation and macrocrack expansion. By defining damage classification or evaluation indicators, real-time identification and early warning of the safety state of hydraulic concrete structures are achieved. Compared with traditional monitoring methods, the present invention can more keenly capture early structural damage signals, avoid maintenance only after cracks are formed, and thus significantly reduce major safety hazards and economic losses. The method and system have the advantages of strong real-time performance, wide application range, good scalability, etc., and are suitable for long-term service safety monitoring and management of concrete structures such as large-scale water conservancy projects, tunnels, and bridges.

[0069] The present invention provides a method for monitoring the properties of hydraulic concrete structures based on acoustic emission and deep learning. The technical architecture of the method can be integrated with the existing structural health monitoring platform, and the method has good scalability for different scales, environments and concrete mix ratios. At the same time, the optimization algorithm can be used to adaptively adjust the model parameters under different working conditions. This high adaptability makes the invention not only suitable for a variety of engineering scenarios, but also can detect potential hidden dangers at an early stage through online monitoring and quantitative evaluation of large-scale hydraulic concrete structures, scientifically guide operation and maintenance arrangements, and significantly improve engineering safety and economic benefits.

[0070] Corresponding to the above method embodiment, the embodiment of the present invention further provides a hydraulic concrete structure performance monitoring system based on acoustic emission and deep learning, including: A data acquisition module, which is used to arrange an acoustic emission sensor on the hydraulic concrete structure to be monitored, and to collect acoustic emission signal data generated by the hydraulic concrete structure during service by means of the acoustic emission sensor; A feature extraction module is used to perform signal preprocessing on the collected acoustic emission signal data and extract key feature vectors of the acoustic emission signal data after signal preprocessing from multiple dimensions; A clustering module is used to perform clustering operations on the extracted key feature vectors, and label various types of acoustic emission signal data according to the clustering results, wherein the labels include data categories and the corresponding relationship between characteristic parameters and structural states, wherein the number of clusters is determined by a voting scoring mechanism; A model training module, which is used to build a classification model and train the classification model using labeled data; The state monitoring module is used to use the trained classification model to monitor the state level of the hydraulic concrete structure in real time at the current moment or period.

[0071] The present invention provides a hydraulic concrete structure property monitoring system based on acoustic emission and deep learning. It combines multiple features with a clustering algorithm to deeply explore the physical process behind the acoustic emission signal, and uses a deep learning model to accurately capture the hydraulic concrete structure state from early microcrack damage to real-time dynamic tracking and evaluation. It can also issue early warnings and take measures before macro cracks appear in the structure, effectively reducing the risk of sudden accidents and providing strong protection for engineering safety.

[0072] For the convenience of description, the above system is described by dividing the functions into various system modules. Of course, when implementing the present disclosure, the functions of each system module can be implemented in the same or multiple software and / or hardware.

[0073] The system of the above embodiment is used to implement the hydraulic concrete structure performance monitoring method based on acoustic emission and deep learning in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0074] Corresponding to the above method embodiment, an embodiment of the present invention further provides a computer device, including: A memory for storing computer programs; A processor is used to implement the steps of the hydraulic concrete structure performance monitoring method based on acoustic emission and deep learning when executing a computer program.

[0075] In the embodiment of the present invention, the processor may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices.

[0076] The processor can call the program stored in the memory. Specifically, the processor can execute the operations in the embodiment of the hydraulic concrete structure performance monitoring method based on acoustic emission and deep learning.

[0077] The memory is used to store one or more programs, which may include program codes, and the program codes include computer operating instructions.

[0078] In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0079] Corresponding to the above method embodiments, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for monitoring the properties of hydraulic concrete structures based on acoustic emission and deep learning are implemented.

[0080] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program codes.

[0081] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0082] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0083] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0084] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A method for monitoring the properties of hydraulic concrete structures based on acoustic emission and deep learning, characterized in that: The following steps are involved: S10: arranging an acoustic emission sensor on the hydraulic concrete structure to be monitored, and collecting acoustic emission signal data generated by the hydraulic concrete structure during service by means of the acoustic emission sensor; S20: performing signal preprocessing on the collected acoustic emission signal data, and extracting key feature vectors of the acoustic emission signal data after the signal preprocessing from multiple dimensions; S30: performing a clustering operation on the extracted key feature vectors, and labeling various types of acoustic emission signal data according to the clustering results, wherein the labels include data categories and the corresponding relationship between feature parameters and structural states, wherein the number of clusters is determined by a voting scoring mechanism; S40: constructing a classification model, and training the classification model using the labeled data; S50: Using the trained classification model to monitor in real time the state level of the hydraulic concrete structure at the current moment or in a period.

2. According to claim 1, a method for monitoring the performance of hydraulic concrete structures based on acoustic emission and deep learning is characterized in that: In S10, at least three acoustic emission sensors are arranged in an array on the hydraulic concrete structure to be monitored.

3. A method for monitoring the performance of hydraulic concrete structures based on acoustic emission and deep learning according to claim 1 or 2, characterized in that: In S20, the collected acoustic emission signal data is subjected to signal preprocessing including denoising and filtering, data synchronization and calibration, and redundancy removal and anomaly detection.

4. The method for monitoring the performance of hydraulic concrete structures based on acoustic emission and deep learning according to claim 3 is characterized in that: In S30, the method of performing a clustering operation on the extracted key feature vectors includes: S31: normalize the key feature vectors and remove outliers; S32: Calculate the distance between the key feature vectors after normalization and outlier removal to obtain a distance matrix; S33: Based on the distance matrix, a clustering algorithm is used to perform a clustering operation on the key feature vectors, wherein the number of clusters is determined by a voting scoring mechanism, including determining that the score of the optimal classification number of a single indicator is P, the score of the classification number with the second best effect is P-1, and so on, determining the score of each cluster number, and adding the scores of each classification number of all indicators to obtain the cluster number with the highest score, and selecting the cluster number with the highest score as the classification number of the final cluster; S34: Obtaining data categories of key feature vectors according to the clustering results, and constructing a correspondence between feature parameters and structural states, and marking the data categories and the correspondence between their feature parameters and structural states as labels.

5. The method for monitoring the performance of hydraulic concrete structures based on acoustic emission and deep learning according to claim 3 is characterized in that: In S40, the method of training the classification model using the labeled data includes: S41: inputting the labeled data into the long short-term memory network classification model, and the long short-term memory network classification model outputs a category prediction value; S42: Selecting hyperparameters of the long short-term memory network classification model based on the input and output mode of the long short-term memory network classification model; S43: Use the gray wolf optimization algorithm to optimize the hyperparameters of the long short-term memory network classification model. By simulating the behavior of gray wolf groups, iteratively search for the optimal solution in the multidimensional search space to obtain the optimal hyperparameter combination of the long short-term memory network classification model. S44: After determining the optimal hyperparameter combination, the long short-term memory network classification model is trained using labeled data to obtain a trained long short-term memory network classification model.

6. A method for monitoring hydraulic concrete structure properties based on acoustic emission and deep learning according to claim 5, characterized in that: In S43, a method for optimizing hyperparameters of a long short-term memory network classification model using a gray wolf optimization algorithm includes: S431: Divide the labeled data into training set and validation set, set the population size and maximum number of iterations , randomly generate the initial position of each gray wolf , and ensure that it is within the given search range; S432: For each , train the LSTM network classification model in the training set, and calculate the validation index as the fitness value in the validation set; S433: Sort and assign values ​​according to fitness values. Wolf, calculate the new hunting strategy according to the gray wolf optimization algorithm , and then perform fitness evaluation after updating the initial position; S434: Repeat steps S432 and S433 until the maximum number of iterations is met or the fitness value tends to be stable, and take the optimal solution corresponding to As the optimal hyperparameter combination for the long short-term memory network classification model.

7. The method for monitoring the performance of hydraulic concrete structures based on acoustic emission and deep learning according to claim 5 is characterized in that: In S50, the method of using the trained classification model to monitor the state level of the hydraulic concrete structure in real time at the current moment or in a period includes: S51: inputting the real-time collected acoustic emission signal data into the trained classification model, and outputting the predicted value of the structural state; S52: Compare and judge the predicted value of the structural state with the preset structural state level, and generate a corresponding safety warning according to the judgment result.

8. A hydraulic concrete structure performance monitoring system based on acoustic emission and deep learning, characterized by: include: A data acquisition module, which is used to arrange an acoustic emission sensor on the hydraulic concrete structure to be monitored, and to collect acoustic emission signal data generated by the hydraulic concrete structure during service by means of the acoustic emission sensor; A feature extraction module is used to perform signal preprocessing on the collected acoustic emission signal data and extract key feature vectors of the acoustic emission signal data after signal preprocessing from multiple dimensions; A clustering module is used to perform clustering operations on the extracted key feature vectors, and label various types of acoustic emission signal data according to the clustering results, wherein the labels include data categories and the corresponding relationship between characteristic parameters and structural states, wherein the number of clusters is determined by a voting scoring mechanism; A model training module, which is used to build a classification model and train the classification model using labeled data; The state monitoring module is used to use the trained classification model to monitor the state level of the hydraulic concrete structure in real time at the current moment or period.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images