Cross-modal intelligent perception fusion early warning method and system

Abstract

The invention relates to the technical field of water area public safety, in particular to a cross-modal intelligent perception fusion early warning method and system, and the method comprises the steps: respectively constructing a dam leakage recognition model, a dam deformation recognition model and a dam water pressure recognition model through collecting the sound wave data, image data and water pressure data of a dam in real time; and fast Fourier transform is carried out on the sound wave data through the dam leakage identification model, frequency domain features are extracted, and dam leakage abnormal coefficients are obtained. Recognizing the image data according to the dam deformation recognition model, obtaining dam image contour features and dam image region features, and obtaining a dam deformation abnormal coefficient. And recognizing the water pressure data according to the dam water pressure recognition model to obtain a dam water pressure abnormal coefficient. And obtaining a comprehensive dam abnormal coefficient according to the three abnormal coefficients, comparing the comprehensive dam abnormal coefficient with a preset dam abnormal threshold value, and performing real-time early warning. Through cross-modal data fusion analysis, the intelligent sensing and real-time management and control capabilities of dam public safety are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of water public safety, and in particular to a cross-modal intelligent perception fusion early warning method and system. Background Art

[0002] Dams are key facilities in water conservancy projects. They are mainly built along rivers or lakes to prevent floods, store water, regulate water levels and protect the surrounding ecological environment. Dams can not only effectively prevent the threat of floods to downstream areas and protect people's lives and property, but also regulate the distribution of water resources and support agricultural irrigation and urban water supply. In addition, the construction of dams helps maintain ecological balance and promote the protection of biodiversity in water bodies. Therefore, dams are of great significance in the field of water conservancy.

[0003] As the service life of dams increases, the public safety of dams faces more and more challenges and the safety risks increase significantly. Therefore, it is of great significance to conduct intelligent perception and real-time control of the public safety of dams.

[0004] Traditional dam safety monitoring methods mainly rely on single-modal data, which cannot comprehensively analyze the potential risks of dam safety, resulting in insufficient dam safety monitoring capabilities. Therefore, a cross-modal intelligent perception method is needed to improve the intelligent perception and real-time control capabilities of dam public safety through multi-dimensional perception of dam status, thereby effectively preventing accidents and ensuring water safety.

[0005] To this end, a cross-modal intelligent perception fusion warning method and system are proposed. Summary of the invention

[0006] The purpose of the present invention is to provide a cross-modal intelligent perception fusion early warning method and system. The present invention relates to the field of water public safety technology, specifically a cross-modal intelligent perception fusion early warning method and system. The present invention collects the acoustic wave data, image data and water pressure data of the dam in real time, and respectively constructs a dam leakage recognition model, a dam deformation recognition model and a dam water pressure recognition model. The dam leakage recognition model performs a fast Fourier transform on the acoustic wave data, extracts the frequency domain features, and obtains the dam leakage anomaly coefficient. According to the dam deformation recognition model, the image data is identified to obtain the dam image contour features and the dam image area features, and the dam deformation anomaly coefficient is obtained. According to the dam water pressure recognition model, the water pressure data is identified to obtain the dam water pressure anomaly coefficient. According to the three anomaly coefficients, the comprehensive dam anomaly coefficient is obtained, and it is compared with the preset dam anomaly threshold and a real-time warning is issued. The present invention improves the intelligent perception and real-time control capabilities of dam public safety through cross-modal data fusion analysis.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A cross-modal intelligent perception fusion warning method, characterized by comprising:

[0009] S10. Determine a dam data acquisition strategy, and acquire first acoustic wave data, second image data, and third water pressure data of the dam according to the acquisition strategy;

[0010] S20. Construct a dam leakage identification model, wherein the dam leakage identification model includes a first preprocessing layer, a first identification layer and a first output layer; the first preprocessing layer preprocesses the first acoustic wave data to obtain first acoustic wave identification data, and the first identification layer identifies the first acoustic wave identification data to obtain an abnormal coefficient of dam leakage;

[0011] S30. Construct a dam deformation recognition model, wherein the dam deformation recognition model includes a second preprocessing layer, a second recognition layer, and a second output layer; the second preprocessing layer preprocesses the second image data to obtain second image recognition data, and the second recognition layer analyzes the second image recognition data to obtain a dam deformation anomaly coefficient;

[0012] S40. Construct a dam water pressure identification model, the dam water pressure identification model includes a third preprocessing layer, a third identification layer and a third output layer; the third preprocessing layer preprocesses the third water pressure data to obtain third water pressure identification data, and analyzes the third water pressure identification data according to the third identification layer to obtain a dam water pressure anomaly coefficient;

[0013] S50. Obtain a comprehensive dam anomaly coefficient based on the dam leakage anomaly coefficient, the dam deformation anomaly coefficient and the dam water pressure anomaly coefficient, and compare the comprehensive dam anomaly coefficient with a preset dam anomaly threshold to issue a real-time warning.

[0014] Preferably, the first sound wave data includes sound wave signal data inside the dam obtained by an acoustic wave sensor; the second image data includes an image of the dam surface taken by a high-resolution camera; and the third water pressure data includes the water pressure at the dam outlet and the water pressure at the dam inlet obtained by a water pressure sensor.

[0015] Preferably, the first preprocessing layer denoises the first sound wave data through wavelet transform to obtain the first sound wave identification data; the first identification layer first converts the first sound wave identification data into frequency domain signal data through fast Fourier transform, extracts the first dam leakage characteristics, and analyzes the first dam leakage characteristics to obtain the dam leakage anomaly coefficient, wherein the first dam leakage characteristics include centroid frequency, root mean square frequency, mean square frequency and frequency standard deviation.

[0016] Preferably, the second preprocessing layer includes image denoising and image enhancement, and the second recognition layer performs feature analysis on the second image recognition data through a convolutional neural network to obtain a second dam deformation feature, the second dam deformation feature includes a dam image contour feature and a dam image area feature, the image contour feature includes a contour curvature and a contour edge; the image area feature includes an area area, an area shape index and an area texture, and the dam deformation anomaly coefficient is obtained based on the second dam deformation feature.

[0017] Preferably, the third preprocessing layer denoises the third water pressure data through Kalman filtering to obtain third water pressure identification data; the third identification layer performs feature analysis on the third water pressure identification data through a long short-term memory network to obtain third water pressure anomaly characteristics, and the third water pressure anomaly characteristics include the maximum water pressure at the dam outlet, the maximum water pressure at the dam inlet, the minimum water pressure at the dam outlet, the minimum water pressure at the dam inlet, the average water pressure at the dam outlet and the average water pressure at the dam inlet, and the dam water pressure anomaly coefficient is obtained based on the third water pressure anomaly characteristics.

[0018] Preferably, the comprehensive dam anomaly coefficient is:

[0019] CDAC=ED1*ACDL+ED2*ACDD+ED3*ACDW+exp(ACDL+ACDD+ACDW);

[0020] Among them, CDAC represents the comprehensive dam anomaly coefficient; ED1 represents the first dam anomaly impact weight; ACDL represents the dam leakage anomaly coefficient; ED2 represents the second dam anomaly impact weight; ACDD represents the dam deformation anomaly coefficient; ED3 represents the third dam anomaly impact weight; ACDW represents the dam water pressure anomaly coefficient; exp represents an exponential function with e as the base.

[0021] A cross-modal intelligent perception fusion warning system, comprising:

[0022] A data acquisition module, used for determining a data acquisition strategy for the dam, and acquiring first acoustic wave data, second image data, and third water pressure data of the dam according to the acquisition strategy;

[0023] A dam leakage identification module is used to construct a dam leakage identification model, wherein the dam leakage identification model includes a first preprocessing layer, a first identification layer and a first output layer; the first preprocessing layer preprocesses the first acoustic wave data to obtain first acoustic wave identification data, and the first identification layer identifies the first acoustic wave identification data to obtain an abnormal coefficient of dam leakage;

[0024] A dam deformation recognition module is used to construct a dam deformation recognition model, wherein the dam deformation recognition model includes a second preprocessing layer, a second recognition layer, and a second output layer; the second preprocessing layer preprocesses the second image data to obtain second image recognition data, and the second recognition layer analyzes the second image recognition data to obtain a dam deformation anomaly coefficient;

[0025] A dam water pressure identification module is used to construct a dam water pressure identification model, wherein the dam water pressure identification model includes a third preprocessing layer, a third identification layer and a third output layer; the third preprocessing layer preprocesses the third water pressure data to obtain third water pressure identification data, and analyzes the third water pressure identification data according to the third identification layer to obtain a dam water pressure anomaly coefficient;

[0026] The comprehensive early warning module is used to obtain a comprehensive dam abnormality coefficient based on the dam leakage abnormality coefficient, the dam deformation abnormality coefficient and the dam water pressure abnormality coefficient, and compare the comprehensive dam abnormality coefficient with a preset dam abnormality threshold value to provide real-time early warning.

[0027] Preferably, the first preprocessing layer denoises the first sound wave data through wavelet transform to obtain the first sound wave identification data; the first identification layer first converts the first sound wave identification data into frequency domain signal data through fast Fourier transform, extracts the first dam leakage characteristics, and analyzes the first dam leakage characteristics to obtain the dam leakage anomaly coefficient, wherein the first dam leakage characteristics include centroid frequency, root mean square frequency, mean square frequency and frequency standard deviation.

[0028] Preferably, the second preprocessing layer includes image denoising and image enhancement, and the second recognition layer performs feature analysis on the second image recognition data through a convolutional neural network to obtain a second dam deformation feature, the second dam deformation feature includes a dam image contour feature and a dam image area feature, the image contour feature includes a contour curvature and a contour edge; the image area feature includes an area area, an area shape index and an area texture, and the dam deformation anomaly coefficient is obtained based on the second dam deformation feature.

[0029] Preferably, the third preprocessing layer denoises the third water pressure data through Kalman filtering to obtain third water pressure identification data; the third identification layer performs feature analysis on the third water pressure identification data through a long short-term memory network to obtain third water pressure anomaly characteristics, and the third water pressure anomaly characteristics include the maximum water pressure at the dam outlet, the maximum water pressure at the dam inlet, the minimum water pressure at the dam outlet, the minimum water pressure at the dam inlet, the average water pressure at the dam outlet and the average water pressure at the dam inlet, and the dam water pressure anomaly coefficient is obtained based on the third water pressure anomaly characteristics.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. The present invention collects the first acoustic wave data of the dam and constructs a dam leakage identification model. First, the first acoustic wave data is denoised by wavelet transform; the first identification layer of the dam leakage identification model converts the first acoustic wave identification data from a time domain signal to a frequency domain signal by fast Fourier transform, extracts the first dam leakage characteristics (center of gravity frequency, root mean square frequency, mean square frequency and frequency standard deviation), and analyzes the first dam leakage characteristics to obtain the dam leakage anomaly coefficient, which lays a good data foundation for realizing cross-modal intelligent perception fusion of dams and further enhances the intelligent perception and real-time control capabilities of dam public safety.

[0032] 2. The present invention constructs a dam deformation recognition model by acquiring dam surface image data. The second recognition layer of the dam recognition model processes and analyzes the dam surface image data through a convolutional neural network to obtain dam image contour features and dam image area features. The dam deformation anomaly coefficient is further obtained based on the dam image contour features and dam image area features, which lays a good data foundation for realizing cross-modal intelligent perception fusion of dams and further enhances the intelligent perception and real-time control capabilities of dam public safety.

[0033] 3. The present invention obtains water pressure data of the dam outlet and inlet, constructs a dam water pressure identification model to identify the water pressure data, and the third identification layer of the dam water pressure identification model performs feature analysis on the water pressure identification data through a long short-term memory network to obtain a third water pressure abnormality feature, wherein the third water pressure abnormality feature includes the maximum water pressure at the dam outlet, the maximum water pressure at the dam inlet, the minimum water pressure at the dam outlet, the minimum water pressure at the dam inlet, the average water pressure at the dam outlet and the average water pressure at the dam inlet, and the dam water pressure abnormality coefficient is obtained according to the third water pressure abnormality feature, which lays a good data foundation for realizing cross-modal intelligent perception fusion of dams, and further enhances the intelligent perception and real-time control capabilities of dam public safety.

[0034] 4. The present invention conducts cross-modal intelligent perception analysis on the public safety of the dam by integrating the multi-modal data of the dam, including sound wave data, image data and water pressure data, to obtain the comprehensive abnormality coefficient of the dam, and conducts a comparative analysis based on the comprehensive abnormality coefficient and the preset threshold value to carry out real-time early warning, thereby realizing the cross-modal intelligent perception fusion of the dam and improving the intelligent perception and real-time control capabilities of the public safety of the dam. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of a cross-modal intelligent perception fusion warning method flow chart proposed in an embodiment of the present invention;

[0036] Figure 2A schematic diagram of the structure of a cross-modal intelligent perception fusion warning system proposed in an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of the structure of a dam leakage identification model proposed in an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of the structure of a dam deformation identification model proposed in an embodiment of the present invention;

[0039] Figure 5 A schematic diagram of the structure of a dam water pressure identification model proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Embodiment 1

[0042] In order to improve the intelligent perception and real-time control capabilities of public safety of Dam A, a cross-modal intelligent perception fusion early warning method was applied.

[0043] Reference Figure 1 , which is a flow chart of a cross-modal intelligent perception fusion warning method, including:

[0044] S10. Determine a dam data acquisition strategy, and acquire first acoustic wave data, second image data, and third water pressure data of the dam according to the acquisition strategy;

[0045] The acquisition strategy includes a time window and an acquisition frequency. The specific acquisition strategy for dam data is shown in Table 1;

[0046] Table 1. Dam data collection strategy

[0047] Data Types Time Window Collection frequency First Sound Wave Data 2 seconds 1 time / 1 minute Second image data 15 seconds 1 time / 5 minutes The third water pressure data 10 seconds 1 time / 1 minute

[0048] Furthermore, the first acoustic wave data includes acoustic wave signal data inside the dam obtained by an acoustic wave sensor; the second image data includes images of the dam surface taken by a high-resolution camera; within each time window, a high-resolution camera is used to photograph various areas of the dam surface to ensure that all parts of the dam surface are covered within each time window; the third water pressure data includes the water pressure at the dam outlet and the water pressure at the dam inlet obtained by a water pressure sensor.

[0049] S20. Construct a dam leakage identification model, the dam leakage identification model includes a first preprocessing layer, a first identification layer and a first output layer; the first preprocessing layer preprocesses the first acoustic wave data to obtain first acoustic wave identification data, the first identification layer identifies the first acoustic wave identification data to obtain the dam leakage anomaly coefficient; the schematic diagram of the dam leakage identification model is as follows Figure 3 As shown;

[0050] Furthermore, the first preprocessing layer denoises the first sound wave data by wavelet transform to obtain the first sound wave identification data; the first identification layer first converts the first sound wave identification data into frequency domain signal data by fast Fourier transform to obtain the spectrum of the signal, and extracts the first dam leakage feature, wherein the first dam leakage feature includes centroid frequency, root mean square frequency, mean square frequency and frequency standard deviation;

[0051] The centroid frequency represents the weighted average frequency of the spectrum, reflecting the concentrated position of the signal energy on the frequency axis; the root mean square frequency reflects the distribution of the signal energy on the frequency axis; the mean square frequency is used to represent the characteristics of the frequency distribution; the frequency standard deviation represents the discrete degree of the frequency distribution; according to the above steps, the first dam leakage characteristics of each time window can be obtained, and then the characteristic change rate between multiple time windows is calculated according to each characteristic of the first dam leakage characteristics in each time window, and the average is taken to obtain the characteristic change rate average value of each characteristic in the first dam leakage characteristics, and these characteristic change rate average values ​​are summed to obtain the dam leakage anomaly coefficient;

[0052] The dam leakage anomaly coefficient is:

[0053]

[0054] Where ACDL represents the anomaly coefficient of dam leakage; L represents the characteristic number of the first dam leakage characteristic; η t represents the influence weight of the leakage characteristic of the tth first dam; represents the average rate of change of leakage characteristics of each first dam;

[0055] This embodiment collects the first acoustic wave data of the dam and constructs a dam leakage identification model. The first acoustic wave data is first denoised by wavelet transform. The first identification layer of the dam leakage identification model converts the first acoustic wave identification data from a time domain signal to a frequency domain signal by fast Fourier transform, extracts the first dam leakage characteristics (center of gravity frequency, root mean square frequency, mean square frequency and frequency standard deviation), and analyzes the first dam leakage characteristics to obtain the dam leakage anomaly coefficient, which lays a good data foundation for realizing cross-modal intelligent perception fusion of dams and further enhances the intelligent perception and real-time control capabilities of dam public safety.

[0056] S30. Construct a dam deformation recognition model, the dam deformation recognition model includes a second preprocessing layer, a second recognition layer and a second output layer; the second preprocessing layer preprocesses the second image data to obtain second image recognition data, and the second recognition layer analyzes the second image recognition data to obtain the dam deformation anomaly coefficient; the schematic diagram of the dam deformation recognition model structure is as shown in Figure 4 As shown;

[0057] Furthermore, the second preprocessing layer includes image denoising and image enhancement, and the second recognition layer performs feature analysis on each regional image corresponding to each time window through a convolutional neural network to obtain a second dam deformation feature, the second dam deformation feature includes a dam image contour feature and a dam image regional feature, the image contour feature includes a contour curvature and a contour edge; the image regional feature includes a regional area, a regional shape index and a regional texture, and the dam deformation anomaly coefficient is obtained according to the second dam deformation feature; the regional shape index represents the ratio of the regional perimeter to the area; the regional texture is obtained through a gray level co-occurrence matrix GLCM; the average change rate of the second dam deformation feature of each region in multiple time windows is analyzed to obtain the dam deformation anomaly coefficient;

[0058] The contour curvature reflects the curvature change of the edge of the dam surface; the contour edge is extracted by Canny edge detection;

[0059] The abnormal coefficient of dam deformation is:

[0060]

[0061] Where ACDD represents the abnormal coefficient of dam deformation; C represents the number of regions; ACDD h represents the abnormal coefficient of dam deformation in region h; D represents the characteristic number of the second dam deformation feature; w i,h represents the influence weight of the deformation characteristics of the second dam in region h; represents the average rate of change of deformation characteristics of the i-th second levee in region h;

[0062] This embodiment constructs a dam deformation recognition model by acquiring dam surface image data. The second recognition layer of the dam recognition model processes and analyzes the dam surface image data through a convolutional neural network to obtain dam image contour features and dam image area features. The dam deformation anomaly coefficient is further obtained based on the dam image contour features and dam image area features, which lays a good data foundation for realizing cross-modal intelligent perception fusion of dams and further enhances the intelligent perception and real-time control capabilities of dam public safety.

[0063] S40. Construct a dam water pressure identification model, the dam water pressure identification model includes a third preprocessing layer, a third identification layer and a third output layer; the third preprocessing layer preprocesses the third water pressure data to obtain third water pressure identification data, and analyzes the third water pressure identification data according to the third identification layer to obtain the dam water pressure anomaly coefficient; the structural schematic diagram of the dam water pressure identification model is as follows Figure 5 As shown;

[0064] Furthermore, the third preprocessing layer denoises the third water pressure data through Kalman filtering to obtain third water pressure identification data; the third identification layer performs feature analysis on the third water pressure identification data through a long short-term memory network to obtain third water pressure anomaly features, and the third water pressure anomaly features include the maximum water pressure at the dam outlet, the maximum water pressure at the dam inlet, the minimum water pressure at the dam outlet, the minimum water pressure at the dam inlet, the average water pressure at the dam outlet and the average water pressure at the dam inlet, and calculates the average change rate of the above-mentioned third water pressure anomaly features in multiple time windows to obtain the dam water pressure anomaly coefficient.

[0065] The water pressure anomaly coefficient of the dam is:

[0066]

[0067] Where ACDW represents the dam water pressure anomaly coefficient; W represents the characteristic number of the third water pressure anomaly feature; γ k represents the influence weight of the kth third water pressure anomaly feature; represents the average rate of change of the kth third water pressure anomaly characteristic;

[0068] This embodiment obtains water pressure data at the dam outlet and inlet, constructs a dam water pressure identification model to identify the water pressure data, and the third identification layer of the dam water pressure identification model performs feature analysis on the water pressure identification data through a long short-term memory network to obtain a third water pressure abnormality feature, wherein the third water pressure abnormality feature includes the maximum water pressure at the dam outlet, the maximum water pressure at the dam inlet, the minimum water pressure at the dam outlet, the minimum water pressure at the dam inlet, the average water pressure at the dam outlet and the average water pressure at the dam inlet, and the dam water pressure abnormality coefficient is obtained based on the third water pressure abnormality feature, which lays a good data foundation for realizing cross-modal intelligent perception fusion of dams and further enhances the intelligent perception and real-time control capabilities of dam public safety.

[0069] S50. Obtain a comprehensive dam anomaly coefficient based on the dam leakage anomaly coefficient, the dam deformation anomaly coefficient and the dam water pressure anomaly coefficient, and compare the comprehensive dam anomaly coefficient with a preset dam anomaly threshold to issue a real-time warning.

[0070] Furthermore, the comprehensive dam anomaly coefficient is:

[0071] CDAC=ED1*ACDL+ED2*ACDD+ED3*ACDW+exp(ACDL+ACDD+ACDW);

[0072] Among them, CDAC represents the comprehensive dam anomaly coefficient; ED1 represents the first dam anomaly impact weight; ACDL represents the dam leakage anomaly coefficient; ED2 represents the second dam anomaly impact weight; ACDD represents the dam deformation anomaly coefficient; ED3 represents the third dam anomaly impact weight; ACDW represents the dam water pressure anomaly coefficient; exp represents an exponential function with e as the base.

[0073] This embodiment conducts cross-modal intelligent perception analysis on the public safety of the dam by integrating the multi-modal data of the dam, including sound wave data, image data and water pressure data, to obtain the comprehensive abnormality coefficient of the dam, and conducts a comparative analysis based on the comprehensive abnormality coefficient and the preset threshold to issue a real-time early warning, thereby realizing the cross-modal intelligent perception fusion of the dam and improving the intelligent perception and real-time control capabilities of the public safety of the dam.

[0074] This embodiment collects the acoustic wave data, image data and water pressure data of the dam in real time, and respectively constructs a dam leakage identification model, a dam deformation identification model and a dam water pressure identification model. The acoustic wave data is subjected to a fast Fourier transform through the dam leakage identification model, and the frequency domain features are extracted to obtain the dam leakage anomaly coefficient. The image data is identified according to the dam deformation identification model to obtain the dam image contour features and the dam image area features, and the dam deformation anomaly coefficient is obtained. The water pressure data is identified according to the dam water pressure identification model to obtain the dam water pressure anomaly coefficient. A comprehensive dam anomaly coefficient is obtained based on these three anomaly coefficients, which is compared with the preset dam anomaly threshold and a real-time warning is issued. This embodiment improves the intelligent perception and real-time control capabilities of dam public safety through cross-modal data fusion analysis.

[0075] Embodiment 2

[0076] In order to improve the intelligent perception and real-time control capabilities of public safety of Dam B, a cross-modal intelligent perception fusion early warning system was applied.

[0077] Reference Figure 2 , which is a schematic diagram of the structure of a cross-modal intelligent perception fusion warning system, including:

[0078] A data acquisition module, used for determining a data acquisition strategy for the dam, and acquiring first acoustic wave data, second image data, and third water pressure data of the dam according to the acquisition strategy;

[0079] The acquisition strategy includes a time window and an acquisition frequency. The specific acquisition strategy for dam data is shown in Table 2;

[0080] Table 2 Dam data collection strategy

[0081] Data Types Time Window Collection frequency First Sound Wave Data 3 seconds 1 time / 1 minute Second image data 20 seconds 1 time / 5 minutes The third water pressure data 15 seconds 1 time / 1 minute

[0082] Furthermore, the first acoustic wave data includes acoustic wave signal data inside the dam obtained by an acoustic wave sensor; the second image data includes images of the dam surface taken by a high-resolution camera; within each time window, a high-resolution camera is used to photograph various areas of the dam surface to ensure that all parts of the dam surface are covered within each time window; the third water pressure data includes the water pressure at the dam outlet and the water pressure at the dam inlet obtained by a water pressure sensor.

[0083] The dam leakage identification module is used to construct a dam leakage identification model, and the dam leakage identification model includes a first preprocessing layer, a first identification layer and a first output layer; the first preprocessing layer preprocesses the first acoustic wave data to obtain first acoustic wave identification data, and the first identification layer identifies the first acoustic wave identification data to obtain the dam leakage anomaly coefficient; the schematic diagram of the dam leakage identification model is as follows Figure 3 As shown;

[0084] Furthermore, the first preprocessing layer denoises the first sound wave data by wavelet transform to obtain the first sound wave identification data; the first identification layer first converts the first sound wave identification data into frequency domain signal data by fast Fourier transform to obtain the spectrum of the signal, and extracts the first dam leakage feature, wherein the first dam leakage feature includes centroid frequency, root mean square frequency, mean square frequency and frequency standard deviation;

[0085] The centroid frequency represents the weighted average frequency of the spectrum, reflecting the concentrated position of the signal energy on the frequency axis; the root mean square frequency reflects the distribution of the signal energy on the frequency axis; the mean square frequency is used to represent the characteristics of the frequency distribution; the frequency standard deviation represents the discrete degree of the frequency distribution; according to the above steps, the first dam leakage characteristics of each time window can be obtained, and then the characteristic change rate between multiple time windows is calculated according to each characteristic of the first dam leakage characteristics in each time window, and the average is taken to obtain the characteristic change rate average value of each characteristic in the first dam leakage characteristics, and these characteristic change rate average values ​​are summed to obtain the dam leakage anomaly coefficient;

[0086] The dam leakage anomaly coefficient is:

[0087]

[0088] Where ACDL represents the anomaly coefficient of dam leakage; L represents the characteristic number of the first dam leakage characteristic; η t represents the influence weight of the leakage characteristic of the tth first dam; represents the average rate of change of leakage characteristics of each first dam;

[0089] A dam deformation recognition module is used to construct a dam deformation recognition model, wherein the dam deformation recognition model includes a second preprocessing layer, a second recognition layer, and a second output layer; the second preprocessing layer preprocesses the second image data to obtain second image recognition data, and the second recognition layer analyzes the second image recognition data to obtain a dam deformation anomaly coefficient;

[0090] The structural schematic diagram of the dam deformation identification model is as follows: Figure 4 As shown;

[0091] Furthermore, the second preprocessing layer includes image denoising and image enhancement, and the second recognition layer performs feature analysis on each regional image corresponding to each time window through a convolutional neural network to obtain a second dam deformation feature, the second dam deformation feature includes a dam image contour feature and a dam image regional feature, the image contour feature includes a contour curvature and a contour edge; the image regional feature includes a regional area, a regional shape index and a regional texture, and the dam deformation anomaly coefficient is obtained according to the second dam deformation feature; the regional shape index represents the ratio of the regional perimeter to the area; the regional texture is obtained through a gray level co-occurrence matrix GLCM; the average change rate of the second dam deformation feature of each region in multiple time windows is analyzed to obtain the dam deformation anomaly coefficient;

[0092] The contour curvature reflects the curvature change of the edge of the dam surface; the contour edge is extracted by Canny edge detection;

[0093] The abnormal coefficient of dam deformation is:

[0094]

[0095] Where ACDD represents the abnormal coefficient of dam deformation; C represents the number of regions; ACDD h represents the abnormal coefficient of dam deformation in region h; D represents the characteristic number of the second dam deformation feature; w i,h represents the influence weight of the deformation characteristics of the second dam in region h; represents the average rate of change of deformation characteristics of the i-th second levee in region h;

[0096] The dam water pressure identification module is used to construct a dam water pressure identification model, and the dam water pressure identification model includes a third preprocessing layer, a third identification layer and a third output layer; the third preprocessing layer preprocesses the third water pressure data to obtain third water pressure identification data, and analyzes the third water pressure identification data according to the third identification layer to obtain the dam water pressure anomaly coefficient; the structural schematic diagram of the dam water pressure identification model is shown in Figure 5 As shown;

[0097] Furthermore, the third preprocessing layer denoises the third water pressure data through Kalman filtering to obtain third water pressure identification data; the third identification layer performs feature analysis on the third water pressure identification data through a long short-term memory network to obtain third water pressure anomaly features, and the third water pressure anomaly features include the maximum water pressure at the dam outlet, the maximum water pressure at the dam inlet, the minimum water pressure at the dam outlet, the minimum water pressure at the dam inlet, the average water pressure at the dam outlet and the average water pressure at the dam inlet, and calculates the average change rate of the above-mentioned third water pressure anomaly features in multiple time windows to obtain the dam water pressure anomaly coefficient.

[0098] The water pressure anomaly coefficient of the dam is:

[0099]

[0100] Where ACDW represents the dam water pressure anomaly coefficient; W represents the characteristic number of the third water pressure anomaly feature; γ k represents the influence weight of the kth third water pressure anomaly feature; represents the average rate of change of the kth third water pressure anomaly characteristic;

[0101] The comprehensive early warning module is used to obtain a comprehensive dam abnormality coefficient based on the dam leakage abnormality coefficient, the dam deformation abnormality coefficient and the dam water pressure abnormality coefficient, and compare the comprehensive dam abnormality coefficient with a preset dam abnormality threshold value to provide real-time early warning.

[0102] Furthermore, the comprehensive dam anomaly coefficient is:

[0103] CDAC=ED1*ACDL+ED2*ACDD+ED3*ACDW+exp(ACDL+ACDD+ACDW);

[0104] Among them, CDAC represents the comprehensive dam anomaly coefficient; ED1 represents the first dam anomaly impact weight; ACDL represents the dam leakage anomaly coefficient; ED2 represents the second dam anomaly impact weight; ACDD represents the dam deformation anomaly coefficient; ED3 represents the third dam anomaly impact weight; ACDW represents the dam water pressure anomaly coefficient; exp represents an exponential function with e as the base.

[0105] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A cross-modal intelligent perception fusion warning method, characterized in that: include: S10. Determine a dam data acquisition strategy, and acquire first acoustic wave data, second image data, and third water pressure data of the dam according to the acquisition strategy; S20. Construct a dam leakage identification model, wherein the dam leakage identification model includes a first preprocessing layer, a first identification layer and a first output layer; the first preprocessing layer preprocesses the first acoustic wave data to obtain first acoustic wave identification data, and the first identification layer identifies the first acoustic wave identification data to obtain an abnormal coefficient of dam leakage; S30. Construct a dam deformation recognition model, wherein the dam deformation recognition model includes a second preprocessing layer, a second recognition layer, and a second output layer; the second preprocessing layer preprocesses the second image data to obtain second image recognition data, and the second recognition layer analyzes the second image recognition data to obtain a dam deformation anomaly coefficient; S40. Construct a dam water pressure identification model, the dam water pressure identification model includes a third preprocessing layer, a third identification layer and a third output layer; the third preprocessing layer preprocesses the third water pressure data to obtain third water pressure identification data, and analyzes the third water pressure identification data according to the third identification layer to obtain a dam water pressure anomaly coefficient; S50. Obtain a comprehensive dam anomaly coefficient based on the dam leakage anomaly coefficient, the dam deformation anomaly coefficient and the dam water pressure anomaly coefficient, and compare the comprehensive dam anomaly coefficient with a preset dam anomaly threshold to issue a real-time warning.

2. According to claim 1, a cross-modal intelligent perception fusion warning method is characterized by: The first acoustic wave data includes acoustic wave signal data inside the dam obtained by an acoustic wave sensor; the second image data includes an image of the dam surface taken by a high-resolution camera; and the third water pressure data includes the water pressure at the dam outlet and the water pressure at the dam inlet obtained by a water pressure sensor.

3. The cross-modal intelligent perception fusion warning method according to claim 1 is characterized by: The first preprocessing layer denoises the first sound wave data through wavelet transform to obtain the first sound wave identification data; the first identification layer first converts the first sound wave identification data into frequency domain signal data through fast Fourier transform, extracts the first dam leakage characteristics, and analyzes the first dam leakage characteristics to obtain the dam leakage anomaly coefficient, wherein the first dam leakage characteristics include centroid frequency, root mean square frequency, mean square frequency and frequency standard deviation.

4. The cross-modal intelligent perception fusion warning method according to claim 1 is characterized by: The second preprocessing layer includes image denoising and image enhancement. The second recognition layer performs feature analysis on the second image recognition data through a convolutional neural network to obtain a second dam deformation feature. The second dam deformation feature includes a dam image contour feature and a dam image area feature. The image contour feature includes a contour curvature and a contour edge. The image area feature includes an area, a regional shape index and a regional texture. The dam deformation anomaly coefficient is obtained based on the second dam deformation feature.

5. The cross-modal intelligent perception fusion warning method according to claim 1 is characterized by: The third preprocessing layer denoises the third water pressure data through Kalman filtering to obtain third water pressure identification data; the third identification layer performs feature analysis on the third water pressure identification data through a long short-term memory network to obtain third water pressure abnormality features, and the third water pressure abnormality features include the maximum water pressure at the dam outlet, the maximum water pressure at the dam inlet, the minimum water pressure at the dam outlet, the minimum water pressure at the dam inlet, the average water pressure at the dam outlet and the average water pressure at the dam inlet, and the dam water pressure abnormality coefficient is obtained according to the third water pressure abnormality features.

6. The cross-modal intelligent perception fusion warning method according to claim 1 is characterized by: The comprehensive dam anomaly coefficient is: CDAC=ED1*ACDL+ED2*ACDD+ED3*ACDW+exp(ACDL+ACDD+ACDW); Among them, CDAC represents the comprehensive dam anomaly coefficient; ED1 represents the first dam anomaly impact weight; ACDL represents the dam leakage anomaly coefficient; ED2 represents the second dam anomaly impact weight; ACDD represents the dam deformation anomaly coefficient; ED3 represents the third dam anomaly impact weight; ACDW represents the dam water pressure anomaly coefficient; exp represents an exponential function with e as the base.

7. A cross-modal intelligent perception fusion warning system, characterized in that: include: A data acquisition module, used for determining a data acquisition strategy for the dam, and acquiring first acoustic wave data, second image data, and third water pressure data of the dam according to the acquisition strategy; A dam leakage identification module is used to construct a dam leakage identification model, wherein the dam leakage identification model includes a first preprocessing layer, a first identification layer and a first output layer; the first preprocessing layer preprocesses the first acoustic wave data to obtain first acoustic wave identification data, and the first identification layer identifies the first acoustic wave identification data to obtain an abnormal coefficient of dam leakage; A dam deformation recognition module is used to construct a dam deformation recognition model, wherein the dam deformation recognition model includes a second preprocessing layer, a second recognition layer, and a second output layer; the second preprocessing layer preprocesses the second image data to obtain second image recognition data, and the second recognition layer analyzes the second image recognition data to obtain a dam deformation anomaly coefficient; A dam water pressure identification module is used to construct a dam water pressure identification model, wherein the dam water pressure identification model includes a third preprocessing layer, a third identification layer and a third output layer; the third preprocessing layer preprocesses the third water pressure data to obtain third water pressure identification data, and analyzes the third water pressure identification data according to the third identification layer to obtain a dam water pressure anomaly coefficient; The comprehensive early warning module is used to obtain a comprehensive dam abnormality coefficient based on the dam leakage abnormality coefficient, the dam deformation abnormality coefficient and the dam water pressure abnormality coefficient, and compare the comprehensive dam abnormality coefficient with a preset dam abnormality threshold value to provide real-time early warning.

8. The cross-modal intelligent perception fusion warning system according to claim 7, characterized in that: The first preprocessing layer denoises the first sound wave data through wavelet transform to obtain the first sound wave identification data; the first identification layer first converts the first sound wave identification data into frequency domain signal data through fast Fourier transform, extracts the first dam leakage characteristics, and analyzes the first dam leakage characteristics to obtain the dam leakage anomaly coefficient, wherein the first dam leakage characteristics include centroid frequency, root mean square frequency, mean square frequency and frequency standard deviation.

9. The cross-modal intelligent perception fusion warning system according to claim 7, characterized in that: The second preprocessing layer includes image denoising and image enhancement. The second recognition layer performs feature analysis on the second image recognition data through a convolutional neural network to obtain a second dam deformation feature. The second dam deformation feature includes a dam image contour feature and a dam image area feature. The image contour feature includes a contour curvature and a contour edge. The image area feature includes an area, a regional shape index and a regional texture. The dam deformation anomaly coefficient is obtained based on the second dam deformation feature.

10. The cross-modal intelligent perception fusion warning system according to claim 7, characterized in that: The third preprocessing layer denoises the third water pressure data through Kalman filtering to obtain third water pressure identification data; the third identification layer performs feature analysis on the third water pressure identification data through a long short-term memory network to obtain third water pressure abnormality features, and the third water pressure abnormality features include the maximum water pressure at the dam outlet, the maximum water pressure at the dam inlet, the minimum water pressure at the dam outlet, the minimum water pressure at the dam inlet, the average water pressure at the dam outlet and the average water pressure at the dam inlet, and the dam water pressure abnormality coefficient is obtained according to the third water pressure abnormality features.

Images