Abnormal monitoring data identification method based on machine vision

Abstract

The invention relates to the technical field of engineering facility health monitoring, and discloses a machine vision-based abnormal monitoring data identification method, which comprises the following steps of signal simulation, signal time-frequency feature visualization, image enhancement, image annotation, detection model training, detection model transfer learning and anomaly detection. According to the method, firstly, the abnormal signal is generated and clearly marked through a simulation technology, accurate prior information is provided for the neural network model, and the accuracy of bridge structure health monitoring is improved; secondly, due to automatic exception type superposition and label labeling, time and labor cost are greatly saved, and subjective deviation is reduced at the same time; furthermore, the innovative 2D pseudo-color visualization strategy improves the feature identification degree and enhances the identification precision of the model; and finally, the transfer learning strategy effectively utilizes simulation data, accelerates the training of a new task, improves the model performance, especially has excellent performance on a small-scale data set, and reduces the dependence on a large amount of labeled data.

Description

Technical Field

[0001] The present invention relates to the technical field of engineering facility health monitoring, and in particular to a method for identifying abnormal monitoring data based on machine vision. Background Art

[0002] Bridge structural health monitoring technology is widely used in the management and maintenance of bridge infrastructure. However, the raw signals obtained by these monitoring systems are often affected by various abnormal data patterns, including noise, outliers, missing values, trends, and drift. These anomalies, primarily due to hardware, software, and environmental factors, can severely impact data analysis. To mitigate the impact of abnormal data on the analysis process, identifying abnormal data is a critical first step in the data cleaning process. Only by accurately identifying abnormal data can appropriate measures be taken to address it, such as correction, deletion, or re-collection.

[0003] To this end, the present invention proposes a method for identifying abnormal monitoring data based on machine vision to accurately identify abnormal data, reduce the interference of abnormal data on the analysis process, and provide a reliable data basis for subsequent data analysis and bridge structure health status assessment. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a method for identifying abnormal monitoring data based on machine vision.

[0005] The present invention provides a method for identifying abnormal monitoring data based on machine vision, which is mainly used to solve the problem of data anomalies in bridge structural health monitoring (SHM). The method includes the following steps:

[0006] S1. Signal simulation, used to simulate signal collection in actual operating environment:

[0007] S1.1. Use Midas / Civil to build a simulation model for time-course analysis and simulate the health monitoring system signal acquisition without abnormal signals;

[0008] S1.2. Generate various abnormal time series data using Python programming language;

[0009] S1.3. Randomly mix the signal without abnormality and the abnormal data to obtain the signal with abnormality

[0010] S2. Visualization of signal time-frequency characteristics:

[0011] S2.1. Signal segmentation: Use a shorter sliding window to segment the simulation signal. A shorter sliding window usually covers fewer abnormal situations and helps to avoid the mixing of features of different abnormalities.

[0012] S2.2. Feature extraction: Extract the time domain, frequency domain and energy features of each segment through short-time Fourier transform (STFT);

[0013] S2.3. Visualization features: Use two-dimensional pseudo-color images to represent features from the time domain, frequency domain, and energy. In the two-dimensional pseudo-color image, the Y-axis represents frequency, the X-axis represents time change, and the depth of the color represents the energy contribution of the frequency component;

[0014] S3, Image Enhancement:

[0015] S3.1. The Reds color spectrum is used for numerical mapping instead of the traditional Viridis color spectrum. The Reds color spectrum is a gradient color band dominated by red. The data is mainly mapped on the red channel, which facilitates machine learning to extract frequency and energy time-varying features.

[0016] S3.2. Use the green and blue channels to highlight trends and missing anomalies, mapping the frequency of zero to green and the energy of zero to blue.

[0017] S4. Image annotation:

[0018] S4.1. When the simulation generates abnormal signals, use Python scripts to automatically generate labels and assign corresponding labels to samples based on the abnormal type and location information;

[0019] S4.2. Use the sliding window technique to segment the signal. If the window overlaps with the abnormal signal, the sample is marked as the corresponding abnormal type; if there is no overlap, it is marked as "normal";

[0020] S4.3. Use single-label classification for automatic labeling, using the most important feature as the sample label;

[0021] S5. Training detection model:

[0022] S5.1. Initialize the model: Initialize all layers of the VGG-16 model and set the number of neurons in the last fully connected layer to 5. Also set the number of neurons in the Softmax classification layer and the output layer to 5. This is done to adapt to the five-class acceleration anomaly recognition task.

[0023] S5.2. Training and Evaluation: Use the VGG-16 deep neural network architecture to extract signal features from the enhanced 2D pseudo-color image and train and evaluate it on the acceleration anomaly dataset;

[0024] S6. Detection model transfer learning:

[0025] By performing transfer learning between source domain data and target domain data, and using the pre-trained model trained with simulated data, the training process for new tasks can be accelerated and model performance can be improved.

[0026] S7. Detect anomalies: Use the trained model to perform anomaly detection on the actual bridge engineering health monitoring data to detect whether there are anomalies in the data. If anomalies exist, the type and location of the anomaly are output.

[0027] Furthermore, the source domain data is the mid-span acceleration data of the main beam of the simulation model, and the target domain data is the health monitoring cable acceleration data of the actual bridge project.

[0028] Compared with related technologies, the abnormal monitoring data identification method based on machine vision provided by the present invention has the following beneficial effects:

[0029] 1. Abnormal signals of the same type exhibit consistent characteristics in the time and frequency domains. By generating abnormal signals through simulation technology, the type, occurrence time, and location of the abnormality can be clearly identified, providing the necessary prior information for data labeling, thereby better training the neural network model for anomaly detection in bridge structure health monitoring data.

[0030] 2. Traditional supervised learning requires a lot of time and manpower to manually label large data sets, and is somewhat subjective. By using Python to automatically superimpose anomaly types and automatically label the generated samples, it saves time and manpower and reduces subjective bias.

[0031] 3. The time-domain acceleration signal is converted into a time-frequency data sequence using a short-time Fourier transform (STFT), and this is visualized using 2D pseudo-color to create a dataset. 2D pseudo-color does not use the traditional Viridis color spectrum. Instead, it uses a Reds color spectrum mapping to concentrate features in the red channel, facilitating the machine learning model's extraction of time-varying frequency and energy characteristics. Trend and missing features are relocated to the green and blue channels, effectively avoiding the problem of reduced feature resolution. This strategy can help detect and identify anomalies in the data, improving overall recognition accuracy.

[0032] 4. By using pre-trained models trained on simulation data, useful features between the source and target domains can be extracted, accelerating the training process for new tasks and improving model performance. Traditional methods typically require training a new model from scratch, which consumes a large amount of data and computing resources. Transfer learning, on the other hand, transforms features between source domain data (Midas / Civil simulation analysis) and target domain data (bridge health monitoring data from actual bridge projects), enabling model transfer and training. This achieves better results on small datasets while reducing the need for large amounts of labeled data. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1A flow chart of a method for identifying abnormal monitoring data based on machine vision provided by the present invention;

[0034] Figure 2 Viridis chromatogram before image enhancement and Reds chromatogram after image enhancement;

[0035] Figure 3 It is a trend anomaly feature map;

[0036] Figure 4 is the missing abnormal feature map;

[0037] Figure 5 Automatically annotate images with schematics;

[0038] Figure 6 Schematic diagram of transfer learning. DETAILED DESCRIPTION

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0040] The present invention proposes a method for identifying abnormal monitoring data based on machine vision, which is mainly used to solve the problem of data anomalies in bridge structural health monitoring (SHM). The method includes the following steps (such as Figure 1 (as shown): signal simulation, visualization of signal time-frequency features, image enhancement, image annotation, training detection models, detection model transfer learning, and anomaly detection.

[0041] S1. Signal simulation, used to simulate signal collection in actual operating environment:

[0042] S1.1. Use Midas / Civil to build a simulation model for time-course analysis and simulate the health monitoring system signal acquisition without abnormal signals;

[0043] S1.2. Generate various abnormal time series data using Python programming language;

[0044] S1.3. Randomly mix the signal without abnormality and the abnormal data to obtain the signal with abnormality

[0045] S2. Visualization of signal time-frequency characteristics:

[0046] S2.1. Signal segmentation: Use a shorter sliding window to segment the simulation signal. A shorter sliding window usually covers fewer abnormal situations and helps to avoid the mixing of features of different abnormalities.

[0047] S2.2. Feature extraction: Extract the time domain, frequency domain and energy features of each segment through short-time Fourier transform (STFT);

[0048] S2.3. Visualization features: Use two-dimensional pseudo-color images to represent features from the time domain, frequency domain, and energy. In the two-dimensional pseudo-color image, the Y-axis represents frequency, the X-axis represents time change, and the depth of the color represents the energy contribution of the frequency component;

[0049] S3, Image Enhancement:

[0050] S3.1, select Reds chromatogram for numerical mapping instead of traditional Viridis chromatogram (such as Figure 2 As shown in the figure, the Reds color spectrum is a gradient color band dominated by red, and the data is mainly mapped on the red channel, which is convenient for machine learning to extract the time-varying characteristics of frequency and energy;

[0051] S3.2, using green and blue channels to highlight trends and missing anomaly features, mapping the position with frequency 0 to green (e.g. Figure 3 As shown), the position with energy 0 is mapped to blue (as Figure 4 shown);

[0052] S4, image annotation (e.g. Figure 5 shown):

[0053] S4.1. When the simulation generates abnormal signals, use Python scripts to automatically generate labels and assign corresponding labels to samples based on the abnormal type and location information;

[0054] S4.2. Use the sliding window technique to segment the signal. If the window overlaps with the abnormal signal, the sample is marked as the corresponding abnormal type; if there is no overlap, it is marked as "normal";

[0055] S4.3. Use single-label classification for automatic labeling, using the most important feature as the sample label;

[0056] S5. Training detection model:

[0057] S5.1. Initialize the model: Initialize all layers of the VGG-16 model and set the number of neurons in the last fully connected layer to 5. Also set the number of neurons in the Softmax classification layer and the output layer to 5. This is done to adapt to the five-class acceleration anomaly recognition task.

[0058] S5.2. Training and Evaluation: Use the VGG-16 deep neural network architecture to extract signal features from the enhanced 2D pseudo-color image and train and evaluate it on the acceleration anomaly dataset;

[0059] S6. Detection model transfer learning:

[0060] By transferring the source domain data (simulation model main beam mid-span acceleration data) and the target domain data (actual bridge engineering health monitoring cable acceleration data), the pre-trained model trained with simulation data is used to accelerate the training process of new tasks and improve model performance (such as Figure 6 shown);

[0061] S7. Detect anomalies: Use the trained model to perform anomaly detection on the actual bridge engineering health monitoring data to detect whether there are anomalies in the data. If anomalies exist, the type and location of the anomaly are output.

[0062] Compared with related technologies, the abnormal monitoring data identification method based on machine vision provided by the present invention has the following beneficial effects:

[0063] The machine vision-based abnormal monitoring data identification method provided by the present invention first generates and clearly labels abnormal signals through simulation technology, providing accurate prior information for the neural network model and improving the accuracy of bridge structure health monitoring;

[0064] Secondly, automated anomaly type superposition and labeling significantly saves time and labor costs while reducing subjective bias.

[0065] Furthermore, the innovative 2D pseudo-color visualization strategy utilizes the Reds color spectrum to improve feature recognition, thereby enhancing the recognition accuracy of the model;

[0066] Finally, the transfer learning strategy effectively utilizes simulation data, accelerates the training of new tasks and improves model performance, especially on small-scale datasets, reducing dependence on large amounts of labeled data.

[0067] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for identifying abnormal monitoring data based on machine vision, used to solve the problem of data anomalies in bridge structure health monitoring, characterized by: The method comprises the following steps: S1, signal simulation, used to simulate the actual operating environment signal collection; S2. Signal time-frequency feature visualization, including signal segmentation, feature extraction, and feature visualization; S3, image enhancement; S4, image annotation; S5, training detection model; S6. Detection model transfer learning: By transferring source domain data to target domain data, and using pre-trained models trained on simulated data, we can accelerate the training process for new tasks and improve model performance. S7. Detect anomalies: Use the trained model to perform anomaly detection on the actual bridge engineering health monitoring data to detect whether there are anomalies in the data. If anomalies exist, the type and location of the anomaly are output.

2. The method for identifying abnormal monitoring data based on machine vision according to claim 1, characterized in that: The signal simulation includes the following sub-steps: S1.

1. Use Midas / Civil to build a simulation model for time-course analysis and simulate the health monitoring system signal acquisition without abnormal signals; S1.

2. Generate various abnormal time series data using Python programming language; S1.

3. Randomly mix the signal without abnormality and the abnormal data to obtain the signal containing abnormality.

3. The abnormal monitoring data identification method based on machine vision according to claim 1, characterized in that: The signal time-frequency feature visualization includes the following sub-steps: S2.

1. Signal segmentation: Use a shorter sliding window to segment the simulation signal. A shorter sliding window usually covers fewer abnormal situations and helps to avoid the mixing of features of different abnormalities. S2.

2. Feature extraction: Extract the time domain, frequency domain and energy features of each segment through short-time Fourier transform (STFT); S2.

3. Visualization features: Use two-dimensional pseudo-color images to represent features from the time domain, frequency domain, and energy. In the two-dimensional pseudo-color image, the Y-axis represents frequency, the X-axis represents time changes, and the depth of the color represents the energy contribution of the frequency component.

4. The method for identifying abnormal monitoring data based on machine vision according to claim 1, wherein: The image enhancement comprises the following sub-steps: S3.

1. The Reds color spectrum is used for numerical mapping instead of the traditional Viridis color spectrum. The Reds color spectrum is a gradient color band dominated by red. The data is mainly mapped on the red channel, which facilitates machine learning to extract frequency and energy time-varying features. S3.

2. Use the green and blue channels to highlight trends and missing anomaly features, mapping the position with zero frequency to green and the position with zero energy to blue.

5. The method for identifying abnormal monitoring data based on machine vision according to claim 1, wherein: The image annotation comprises the following sub-steps: S4.

1. When the simulation generates abnormal signals, use Python scripts to automatically generate labels and assign corresponding labels to samples based on the abnormal type and location information; S4.

2. Use the sliding window technique to segment the signal. If the window overlaps with the abnormal signal, the sample is marked as the corresponding abnormal type; if there is no overlap, it is marked as "normal"; S4.

3. Use single-label classification method for automatic labeling, and follow the most important feature as the label of the sample.

6. The method for identifying abnormal monitoring data based on machine vision according to claim 1, wherein: The training detection model includes the following sub-steps: S5.

1. Initialize the model: Initialize all layers of VGG-16 and set the number of neurons in the last fully connected layer to 5. Also set the number of neurons in the Softmax classification layer and the output layer to 5. S5.

2. Training and evaluation: The VGG-16 deep neural network architecture is used to extract signal features from the enhanced 2D pseudo-color image and is trained and evaluated on the acceleration anomaly dataset.

7. The method for identifying abnormal monitoring data based on machine vision according to claim 1, wherein: The source domain data is the mid-span acceleration data of the main beam of the simulation model, and the target domain data is the health monitoring cable acceleration data of the actual bridge project.

Images