A low-strain waveform anomaly detection method based on neural network and sawtooth matching

Abstract

The application relates to a low-strain waveform chart anomaly detection method based on a deep neural network and a sawtooth point matching, which utilizes an OCR pre-training model, straight line detection and a DB-scan clustering algorithm to obtain a pretreated image and mapping of a pixel coordinate system and a real length coordinate system; waveform sequence data is obtained through sawtooth point matching, acceleration filtering and one-dimensional linear interpolation; then, the waveform data is sent into a deep learning model to generate an anomaly prediction; and finally, data post-processing is carried out to correct and format-convert the data results. The application has the advantages that: prior knowledge can be utilized to the maximum extent to help improve the accuracy of the method; the waveform chart range and scale axis digital information are determined through image preprocessing, and when the image information is compressed, the effective prior in the image is retained to the maximum extent; the serialization is carried out by using the method based on the sawtooth point matching, so that the influence of the coordinate axis, the text not erased cleanly and the vertical dashed line can be basically eliminated, and therefore the method has strong robustness.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for detecting low-strain waveform anomalies based on neural networks and sawtooth matching. Background Technology

[0002] In recent years, deep learning technology has developed rapidly and has been applied to various practical scenarios. However, there is currently no anomaly detection method for low-strain waveform images on the market. The main difficulties are: (1) the curve position in low-strain waveform images is not fixed; (2) there are dashed lines, coordinate axis scales and text interference in the image; (3) the proportion of pixels containing effective information in low-strain waveform images is not high; (4) there is no fixed and intuitive pattern for waveform errors. Due to difficulty (4), deep learning is needed to fit the distribution of abnormal data from the data, rather than using traditional expert systems for direct detection. And due to difficulties (2) and (3), directly using the image as model input results in an excessively high signal-to-noise ratio. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of the prior art by providing a low-strain waveform anomaly detection method based on neural networks and sawtooth matching. By combining a neural network model with sawtooth matching, anomalies in images exported from low-strain machines can be predicted.

[0004] The objective of this invention is achieved through the following technical solutions:

[0005] A method for anomaly detection in low-strain waveform images based on deep neural networks and sawtooth point matching is used to predict anomalies in images exported from low-strain machines. The method is characterized by the following steps:

[0006] Step 1: Image preprocessing is performed, with the following steps: All text locations are obtained using a pre-trained text detection and recognition model. The DB-scan clustering algorithm is used, with the y-axis coordinate as the clustering index, to obtain the approximate coordinate axis position. Horizontal line detection is used to detect all lines with a length greater than 50% of the image width, and the line closest to the cluster center on the y-axis is selected as the coordinate axis, thus obtaining the upper and lower boundaries of the waveform. The left and right ranges of the waveform are then determined based on the first and last vertical dashed lines in the image. Simultaneously, the mapping relationship between the pixel coordinate system and the true length coordinate system is obtained based on the clustering results. Then, the text in the image is erased based on the text detection results, and cropped according to the waveform boundaries. Finally, binarization is performed.

[0007] Step 2: Perform waveform serialization on the preprocessed image. The specific steps are as follows: Find all jagged points in the image (including upward and downward points) through jagged point matching, and filter out some outliers through acceleration thresholding. Then, obtain the serialized data through linear interpolation. Finally, normalize the y-axis coordinates of the sequence according to the upper and lower boundaries, and attach the corresponding x-axis coordinates in the real physical coordinate system of the corresponding position.

[0008] Step 3: The sequenced image is fed into the low-strain waveform anomaly detection model that has completed the final training. The low-strain waveform anomaly detection model generates a confidence sequence for the entire sequence based on the sequence and selects the type with the highest probability value as the predicted classification for this point in the sequence.

[0009] The final training includes the following steps: a preliminary data construction step, where manual annotation is used to label the sequences for errors, and the data is divided into training and testing sets; an anomaly detection model training step, where a pre-defined bidirectional LSTM network is trained in a supervised manner based on the training dataset and training labels; a termination condition determination step, where the detection model training step continues until a pre-defined termination condition is met; and an output step, where the BiLSTM network that has completed the network training is used as the low-strain waveform anomaly detection model that has completed the final training. The termination condition is that the prediction results output by the detection model training step are correctly labeled and all confidence scores are 1, or the detection model adjustment step is run 50 times.

[0010] Step 4: Post-processing of predicted data. The specific steps are as follows: Post-process the predicted classification, remove errors below the standard axis and classify them as non-anomaly cases, then merge consecutive error sequences with a length greater than 5 pixels, and take the midpoint as the true physical coordinate position corresponding to the error location for output. This invention provides a low-strain waveform anomaly detection method based on deep neural networks and sawtooth point matching, which also has the following technical features: the preprocessing module uses the DB-scan clustering algorithm to cluster all numbers and obtain the approximate position of the scale axis accordingly.

[0011] The serialization module uses jagged points (a 2*2 matrix in the form of [[255,0],[0,255]] or [[0,255],[255,0]]) as the serialization matching basis to avoid the influence of coordinate axes, dashed lines in the graph, and text that has not been completely erased on the serialization.

[0012] The serialization module uses an acceleration threshold as a filter. It is assumed that velocity is the difference between the y-axis position of the previous sawtooth point and the y-axis position of the current sawtooth point, and acceleration is the difference between the previous velocity and the current velocity. The acceleration threshold sets a fixed threshold on the acceleration to prevent sharp peaks from being accidentally erased.

[0013] The focal loss used in the calculation of the loss is...

[0014] The formula for calculating the focal loss used in the loss calculation is as follows:

[0015]

[0016] Where p t for

[0017] The advantages of this invention are:

[0018] 1) Adopting a serialization-then-prediction approach can maximize the use of prior knowledge, which helps improve the accuracy of the method.

[0019] 2) By determining the waveform range and scale axis digital information through image preprocessing, the effective prior information in the image is preserved to the greatest extent when compressing the image information.

[0020] 3) Using a method based on jagged point matching for serialization can largely eliminate the influence of coordinate axes, text that has not been completely erased, and vertical dashed lines, thus exhibiting strong robustness. Attached Figure Description

[0021] Figure 1 A flowchart of a low-strain waveform anomaly detection method based on deep neural network and sawtooth point matching is provided for one embodiment of the present invention;

[0022] Figure 2 This is a flowchart of image preprocessing in one embodiment of the present invention;

[0023] Figure 3 This is a waveform serialization flowchart in one embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram of an upward-facing sawtooth point in one embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of the downward sawtooth point in one embodiment of the present invention;

[0026] Figure 6 This is a schematic diagram of the post-processing of predicted data in one embodiment of the present invention. Detailed Implementation

[0027] The following examples further illustrate the features and other related characteristics of the present invention in detail, to facilitate understanding by those skilled in the art:

[0028] Example: Figure 1As shown, the low-strain waveform anomaly detection method based on deep neural networks and sawtooth point matching is applied in this embodiment as follows:

[0029] like Figure 1 As shown, in this embodiment, the input image is first preprocessed to obtain the preprocessed image and the mapping relationship between the pixel coordinate system and the real length coordinate system. Then, the waveform sequence is obtained through the waveform serialization module. The waveform sequence is then input into the deep learning model for prediction. Finally, the prediction result is sent to the prediction data post-processing module to obtain the final output result.

[0030] like Figure 2 As shown, step 1 first obtains the bounding boxes B = {B1, B2, ..., B} of all texts through a pre-trained text detection and text recognition module. k}, Confidence level C = {C1, C2, ..., C k The text recognition result T = {T1, T2, ..., T} k}. Obtain the bounding boxes corresponding to the numerical results in T based on the text recognition results. Confidence Text recognition results Clustering was performed using the DB-scan algorithm based on B... Num The y-value of the center point Y center As a clustering criterion, the y-value of the cluster center with the largest number of samples within a cluster is selected as the approximate position of the scale axis. At the same time, the bounding boxes of the two sample points with the highest confidence in that cluster are obtained. and text recognition results This is used to determine α and β in the two-axis mapping function f(x) = αx + β. Then, the length of the line L = {L1, L2, ..., L...} is determined through line detection. r} and the y-values ​​of the horizontal line Y = {Y1, Y2, ..., Y} r}, and according to Y i With Y center The absolute value of the difference is closest to the line length L i Image width greater than 60% (1 mg) width The nearest straight line is used as the scale axis and as the lower boundary, while the Y value is greater than Y. i The two nearest horizontal lines are used as the standard axis and the upper boundary, respectively.

[0031] The image Img is obtained by cropping the image based on its top and bottom boundaries. c1 Then, by detecting Img c1 The vertical dashed line DL in the middle is DL = {DL1,DL2,…,DL p} then according to DL iThe x-values ​​are sorted, and the first and last values ​​are selected as the left and right boundaries of the image. Then, the image is processed according to these left and right boundaries. c1 Img was obtained by cutting. c2 Then, based on B for Img c2 After text erasure, binarization is performed to obtain the final preprocessed image (Img). after_preprocess .

[0032] like Figure 3 As shown, the input for step 2 is the preprocessed image Img. after_preprocess And the coordinate system mapping function f(x) = αx + β, first for Img after_preprocess Perform jagged point matching, and the matched jagged point pattern is as follows: Figure 3 (Upward jagged point pattern) and Figure 4 As shown in the (downward jagged point pattern), find all the jagged points and sort them from left to right along the x-axis. If there are points with the same x-value, take the point with the lower y-value. Then you get S = {S1, S2, ..., S...} l}

[0033] Then, acceleration filtering is performed. The process is as follows: for the sawtooth point set S, the velocity V i =S i -S i-1 acceleration A i =V i -V i-1 Accelerometer filtering, on the other hand, represents any A. i It must not exceed the threshold. A Otherwise, it will be discarded. The sequence is then restored to its original state using linear interpolation. The length, and according to The y-values ​​of the sequence are normalized, and then the x-values ​​are mapped from the pixel coordinate system to the true length coordinate system according to the coordinate system mapping function f(x) = αx + β, thus obtaining the final waveform sequence W = {W1, W2, ..., W...}. r}

[0034] Step 3: Input the final waveform sequence into the deep learning model for prediction. The model structure is a bidirectional LSTM followed by a linear layer, then passed through the SoftMax function to output the result. Among them, for r1, r2, r3, and r4 represent the probability of no error, the probability of an error at the pile head, the probability of an error in the pile body, and the probability of an error at the pile tail, respectively.

[0035] Step 4: Perform post-processing on the obtained model prediction results. First, determine the y-values ​​based on the original sequence data. If the y-value is less than 0, set the corresponding position to [1,0,0,0], indicating that there is no error at that position. Then... r2, r3, and r4 are combined by addition to obtain r'2, and then... The prediction type at the corresponding position is obtained through the argmax function. Then, consecutive error sequences with a length greater than 5 are merged to output the x-value of their midpoint.

[0036] like Figure 6 As shown, one of the squares represents a prediction. Blue indicates a higher probability that the model predicts the point as an error-free point, while yellow indicates a higher probability that the model predicts the point as an outlier. The first continuous error sequence... It was discarded because its length was less than 5, while the second consecutive erroneous sequence... If the length is greater than 5, merge into The corresponding x value.

[0037] This embodiment provides a low-strain waveform anomaly detection method based on deep neural networks and sawtooth point matching. Because this method employs a serialization-before-prediction approach, it maximizes the use of prior knowledge, thus improving accuracy. Since the method determines the waveform range and scale axis information through image preprocessing, it preserves the effective prior information in the image to the greatest extent possible when compressing the image information. Because the method uses a sawtooth point matching-based serialization method, it can largely eliminate the influence of coordinate axes, incompletely erased text, and vertical dashed lines, thus exhibiting strong robustness.

[0038] In this embodiment, cropping based on the waveform boundaries effectively improves the signal-to-noise ratio. This results in higher waveform serialization accuracy, and normalization on the y-axis and the use of a true-length coordinate system on the x-axis enhances the generalization ability of this method across different image formats. In this embodiment, acceleration filtering is used to eliminate a small number of noise points, reducing the distance between the serialized sequence and the original sequence.

[0039] In this embodiment, data post-processing is used to correct the results by combining the sequence y-values, effectively utilizing prior knowledge to help the model achieve better results and have better interpretability.

[0040] Although the above embodiments have described the concept and embodiments of the present invention in detail, those skilled in the art will recognize that various improvements and modifications can still be made to the present invention without departing from the scope of the claims, and therefore will not be elaborated here.

Claims

1. A low-strain waveform anomaly detection method based on neural networks and sawtooth matching, used to predict anomalies in images exported from low-strain machines, characterized in that: Includes the following steps: S1. Perform image preprocessing on the image; S2. The preprocessed image is waveform serialized. Waveform serialization means finding all jagged points in the image by jagged point matching, filtering out abnormal points by acceleration threshold, obtaining serialized data by linear interpolation, and finally normalizing the y-axis coordinate of the sequence according to the upper and lower boundaries and attaching the corresponding x-axis coordinate of the real physical coordinate system of the corresponding position. S3. The sequence of the image after serialization is fed into the low strain waveform anomaly detection model that has completed the final training. The low strain waveform anomaly detection model generates a confidence sequence on the entire sequence based on the sequence and selects the type with the highest probability value as the predicted classification of this point on the sequence. S4. Post-process the predicted classification, remove errors below the standard axis and classify them as no anomalies, then merge error sequences with a continuous length of more than 5 pixels, and take the midpoint as the real physical coordinate position corresponding to the error position for output. The image preprocessing involves obtaining the positions of all text in the image through a trained text detection and recognition model, and using the DB-scan clustering algorithm with the y-axis coordinate as the clustering index to obtain the approximate coordinate position. Horizontal line detection is used to detect all lines with a length greater than 50% of the image width, and the line closest to the cluster center on the y-axis is selected as the coordinate axis, thus obtaining the upper and lower boundaries of the waveform. The left and right ranges of the waveform are then determined based on the first and last vertical dashed lines in the image. Simultaneously, the mapping relationship between the pixel coordinate system and the true length coordinate system is obtained based on the clustering results. Then, the text in the image is erased based on the text detection results, and cropped according to the waveform boundaries. Finally, binarization is performed. The final training of the low-strain waveform anomaly detection model includes the following steps: The preliminary data construction steps include manual annotation of sequences to remove errors, and dividing the data into training and test sets. The anomaly detection model training steps involve supervising the training of a pre-defined bidirectional LSTM network based on the training set and training labels. The termination condition judgment step is followed by the detection model training step until the preset termination condition is reached. The output step takes the bidirectional LSTM network that has completed the network training at this time as the low strain waveform anomaly detection model that has completed the final training. The termination condition is that the prediction result output by the detection model training step is the correct label and the confidence level is 1 for all values, or the detection model training step is run 50 times.

Images