A pipeline scene recognition method based on convolutional neural networks

Abstract

The application provides a pipeline scene recognition method based on a convolutional neural network, comprising the following steps: collecting audio data of multiple pipelines in different scenes to form a label set of scene categories; selecting audio data recorded by the multiple pipelines under different labels to form an original audio file set D; constructing a recognition model; training and evaluating the recognition model; collecting audio data of unrecognised pipelines and performing feature extraction; and recognising the unrecognised pipelines by using the model. According to the pipeline scene recognition method provided by the application, pipeline audio data can be collected by an audio device, and the environment of the pipeline can be recognised without further manual measurement.

Description

Technical Field

[0001] This invention relates to the field of pipeline scene recognition technology, and specifically to a pipeline scene recognition method based on convolutional neural networks. Background Technology

[0002] In urban water supply systems, underground pipes of varying sizes form a water supply network, delivering water to households. The quality of these pipes directly impacts the city's carrying capacity. Urban pipeline scenarios are complex, with different scenarios placing vastly different demands on pipeline operation and post-failure maintenance. Especially in hospital settings, many human factors can cause pipeline malfunctions, and subsequent maintenance must consider factors such as the hospital's normal operation. Therefore, identifying the operating environment of underground pipelines is a crucial aspect of urban pipeline operational status analysis. The results can serve as supplementary information to aid in pipeline event detection and reduce the possibility of misjudgments.

[0003] Current methods for identifying the operating environment of pipelines have the following problems:

[0004] 1. Existing methods for monitoring the working environment of pipelines mainly rely on manual measurement. Staff need to go to the site to record the pipeline status, which has a low level of intelligence and high labor costs.

[0005] 2. For pipelines that have been built and operated for a long time, their working environment may change. If the operating environment data of the pipeline cannot be obtained in a timely manner, the judgment of the pipeline's operating status may be biased, which will affect the safe operation of the pipeline.

[0006] In summary, there is an urgent need for a pipeline scene recognition method based on convolutional neural networks to solve the problems existing in the current technology. Summary of the Invention

[0007] The purpose of this invention is to provide a pipeline scene recognition method based on convolutional neural networks, and the specific technical solution is as follows:

[0008] A pipeline scene recognition method based on convolutional neural networks includes the following steps:

[0009] S1: Collect audio data from multiple pipes in different scenes to form a tag set T = (t1, t2, ..., t3) for scene categories. n ), where t n This represents the nth scenario corresponding to the pipeline;

[0010] S2: Select multiple pipes under different labels to record audio data, forming the original audio file set D = (d1, d2, ..., d...). n ), d nThis represents the nth original audio file; feature extraction is performed on each original audio file in the original audio file set D to form a waveform feature set W = (w1, w2, ..., w...). n As the input dataset for the model, w n This represents the waveform feature matrix corresponding to the pipe in the nth scene; for each original audio file in the original audio file set D, it is one-hot encoded into a vector, forming a probability value set Y = (y1, y2, ..., y...). n ), y n This represents the probability value of the waveform corresponding to the nth scene pipeline;

[0011] S3: Constructing the recognition model: Build a one-dimensional convolutional neural network to process the feature set W, and build a fully connected neural network for classification; the one-dimensional convolutional neural network includes an input layer, a one-dimensional convolutional layer, a max pooling layer, a global average pooling layer, a batch normalization layer, and a Leaky ReLU layer; the fully connected neural network includes an input layer, a fully connected layer, a Dropout layer, a batch normalization layer, a Leaky ReLU layer, and a Softmax layer;

[0012] S4: Divide the waveform feature set W and the probability value set Y, using 80% as the training dataset and the remaining 20% ​​as the evaluation dataset; select cross-entropy as the loss function and stochastic gradient descent as the optimization algorithm during the training process of the recognition model, set the hyperparameters of the model and the expected accuracy; train the recognition model using the training dataset and evaluate the recognition model using the evaluation dataset.

[0013] S5: Collect audio data of unidentified pipes and extract features, then use the model to identify the unidentified pipes.

[0014] Preferably, the feature extraction in step S2 or step S4 specifically includes the following steps: extracting the waveform of the original audio file, and using the FeatureExtractor.extract function in the Dcase-Util tool library, taking the original audio file as the function input, and outputting the waveform feature matrix w.

[0015] Preferably, the one-hot encoding in step S2 specifically includes the following steps: constructing a one-dimensional zero vector with a length equal to the number of scenes n in the tag set T; assuming the original audio file corresponds to the scene category in the tag set T, setting the k-th value of the zero vector to 1 and the rest to 0, to obtain the vector y after one-hot encoding.

[0016] Preferably, the construction process of the one-dimensional convolutional neural network in step S3 is as follows: the input layer, the first one-dimensional convolutional layer, the first max pooling layer, the second one-dimensional convolutional layer, the second max pooling layer, the third one-dimensional convolutional layer, the third max pooling layer, the fourth one-dimensional convolutional layer, the fourth max pooling layer, and a global average pooling layer are constructed sequentially; at the output of each one-dimensional convolutional layer, a batch normalization layer is used to normalize the data, and a Leaky ReLU layer is used as the activation function.

[0017] Preferably, the one-dimensional convolutional neural network is a four-layer stacked Conv-BN-ReLU-Pooling structure, and the expression for the forward propagation process during the training phase is as follows:

[0018]

[0019] in, x represents the output of the batch normalized layer during the training phase. train μ represents the input to the batch normalized layer during the training phase. B σ represents the mean of the set of output values ​​B from the previous convolutional layer. B denoted by , where ∈ represents the variance of the set of output values ​​B of a convolutional layer, ∈ represents a small positive number used to avoid division by zero, γ represents the learnable reconstruction coefficient parameter, and β represents the learnable reconstruction constant parameter.

[0020] The expression for the forward propagation process during the testing phase is as follows:

[0021]

[0022] in, x represents the output of the batch normalization layer during the evaluation phase. tesc Let μ represent the input to the batch normalization layer during the evaluation phase. r σ represents the average of the means of all batches. r This represents an unbiased estimate of the variance for each batch.

[0023] Preferably, the construction process of the fully connected neural network in step S3 is as follows: the first Dropout layer, the first fully connected layer, the second Dropout layer, the second fully connected layer, the third Dropout layer, and the third fully connected layer are constructed sequentially; when the first and second fully connected layers output, a Leaky ReLU layer is used as the activation function, and a batch normalization layer is used to normalize the data; when the third fully connected layer outputs, a Softmax layer is used as the activation function to output the probability value of the label for each scene category.

[0024] Preferably, in the Dropout layer, each neuron is retained with probability p.

[0025] Preferably, the expression for the Softmax layer is as follows:

[0026]

[0027] Where Softmax(x) represents the output of the Softmax layer, x i Let represent the value of the i-th output in the output vector x, and e represent the natural constant.

[0028] Preferably, the expression for the cross-entropy loss function is as follows:

[0029]

[0030] Where L represents the cross-entropy loss function, L i The cross-entropy loss value represents the value of each sample, N represents the number of samples in the training dataset, M represents the number of classes, c represents the class number, and y represents the cross-entropy loss value. ic The sign function is p, which takes the value 1 if the true class of sample i in the training dataset is equal to c, and 0 otherwise. ic This represents the predicted probability that sample i in the training dataset belongs to category c.

[0031] Preferably, the expression for the stochastic gradient descent optimization algorithm is as follows:

[0032]

[0033] Where L(θ;x) i y i ) is the loss function for each sample, and φ(θ) is the regularization term. This represents the calculation of stochastic gradient descent optimization.

[0034] The application of the technical solution of the present invention has the following beneficial effects:

[0035] (1) The pipeline scene recognition method provided in this application only requires collecting pipeline audio data through an audio device to identify the environment in which the pipeline is located, without the need for further manual measurement.

[0036] (2) The pipeline scene recognition method provided in this application can collect environmental audio by deploying a sound pickup device on the pipeline, and update the pipeline's operating environment data in real time after model classification (for example, if a pipeline is identified as being under a highway, the waveform features of all highway labels in the database can be combined and transferred to other tasks (such as fault judgment: the sound waveform features of a highway may be similar to the waveform features of a certain fault type. If it is known that the pipeline is laid under a highway, the misjudgment of operating faults can be avoided)). This improves the accuracy of pipeline operating status recognition.

[0037] (3) The recognition model built in this application retains each neuron with a probability p in the Dropout layer, that is, stops working with a probability of 1-p, so that the neurons retained in each forward propagation are different. The Dropout layer deactivates different neurons each time, which can be regarded as the average of multiple magic. At the same time, since two neurons are not necessarily effective at the same time, the dependence between features is reduced, which forces the recognition model to be more general and can be better used for the recognition of audio from underground pipes in different scenarios.

[0038] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0039] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0040] Figure 1 This is a schematic diagram of the overall process of a preferred embodiment of the present invention;

[0041] Figure 2 yes Figure 1 A schematic diagram of the structure of the recognition model; Detailed Implementation

[0042] To facilitate understanding of the present invention, a more comprehensive description is provided below, along with preferred embodiments. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the present invention.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0044] Example 1:

[0045] See Figure 1 A pipeline scene recognition method based on convolutional neural networks includes the following steps:

[0046] S1: Collect audio data from multiple pipes in different scenes to form a tag set T = (t1, t2, ..., t3) for scene categories. n ), where t n This represents the nth scenario corresponding to the pipeline (e.g., label set T = (hospital, shopping mall, school, highway, office building, subway) with a total of n = 6 scenarios);

[0047] S2: Select multiple pipes under different labels to record audio data, forming the original audio file set D = (d1, d2, ..., d...). n For each original audio file in the original audio file set D, feature extraction is performed (specifically, the waveform of the original audio file is extracted, and the FeatureExtractor.extract function in the Dcase-Util tool library is used as input to output the waveform feature matrix w), forming a waveform feature set W = (w1, w2, ..., w...). n As the input dataset for the model, w n Let represent the waveform feature matrix corresponding to the pipe in the nth scene; for each original audio file in the original audio file set D, it is one-hot encoded into a vector (specifically: construct a one-dimensional zero vector, the length of which is equal to the number of scenes n in the tag set T; let the original audio file correspond to the scene category in the tag set T, set the kth value of the zero vector to 1, and the rest to 0, to obtain the vector y after one-hot encoding), forming a probability value set Y = (y1, y2, ..., y n ), y n This represents the probability value of the waveform corresponding to the nth scene pipeline;

[0048] S3: Building the model, see [link] Figure 2 A one-dimensional convolutional neural network is constructed to process the feature set W, and a fully connected neural network is constructed for classification. The one-dimensional convolutional neural network includes an input layer, a one-dimensional convolutional layer, a max pooling layer, a global average pooling layer, a batch normalization layer, and a Leaky ReLU layer. The fully connected neural network includes an input layer, a fully connected layer, a Dropout layer, a batch normalization layer, a Leaky ReLU layer, and a Softmax layer.

[0049] The construction process of the one-dimensional convolutional neural network is as follows: The input layer, the first one-dimensional convolutional layer, the first max-pooling layer, the second one-dimensional convolutional layer, the second max-pooling layer, the third one-dimensional convolutional layer, the third max-pooling layer, the fourth one-dimensional convolutional layer, the fourth max-pooling layer, and one global average pooling layer are constructed sequentially. At the output of each one-dimensional convolutional layer, a batch normalization layer is used to normalize the data, and a Leaky pooling algorithm is used. The ReLU layer is used as the activation function; the parameters of each layer are as follows: the total number of feature maps in the input layer is set to 1; the total number of feature maps in the first 1D convolutional layer is set to 32, and the kernel size is set to 7; the sampling filter size of the first max pooling layer is set to 8; the total number of feature maps in the second 1D convolutional layer is set to 64, and the kernel size is set to 7; the sampling filter size of the second max pooling layer is set to 8; the total number of feature maps in the third 1D convolutional layer is set to 128, and the kernel size is set to 7; the sampling filter size of the third max pooling layer is set to 8; the total number of feature maps in the fourth 1D convolutional layer is set to 256, and the kernel size is set to 7; the sampling filter size of the fourth max pooling layer is set to 8.

[0050] The construction process of the fully connected neural network is as follows: The first Dropout layer, the first fully connected layer, the second Dropout layer, the third Dropout layer, and the third fully connected layer are constructed sequentially. When the first and second fully connected layers output, a Leaky ReLU layer is used as the activation function, and a batch normalization layer is used to normalize the data. When the third fully connected layer outputs, a Softmax layer is used as the activation function to output the probability value of the label for each scene category. The parameters for each layer are as follows: the total number of feature maps in the first fully connected layer is set to 128; the total number of feature maps in the second fully connected layer is set to 64; the total number of feature maps in the third fully connected layer is set to n, where n equals the number of scenes in the label set T. In the Dropout layers (including the first, second, and third Dropout layers), each neuron is retained with probability p.

[0051] S4: Divide the waveform feature set W and the probability value set Y, using 80% as the training dataset and the remaining 20% ​​as the evaluation dataset; select cross-entropy as the loss function, and this embodiment also provides a specific expression for the cross-entropy loss function:

[0052]

[0053] Where L represents the cross-entropy loss function, L iThe cross-entropy loss value represents the value of each sample, N represents the number of samples in the training dataset, M represents the number of classes, c represents the class number, and y represents the cross-entropy loss value. ic p represents the sign function, which takes the value 1 if the true class of sample i is equal to c, and 0 otherwise. ic This represents the predicted probability that sample i in the training dataset belongs to category c.

[0054] Stochastic gradient descent is used as an optimization algorithm in the training process of a recognition model. This embodiment also provides a specific expression for the stochastic gradient descent optimization algorithm:

[0055]

[0056] Wherein, L(θ; x i y i ) is the loss function for each sample, and φ(θ) is the regularization term. This indicates the calculation of stochastic gradient descent optimization.

[0057] Set the model's hyperparameters (e.g., learning rate, number of iterations, number of layers, number of neurons per layer, batch size, weights of each part of the loss function, number of categories, etc.) and the expected accuracy (95%) according to the actual situation; train the recognition model using the training dataset (specifically: input the waveform feature set W into a one-dimensional convolutional neural network, and compare the output of the classification module with the probability value set Y); evaluate the recognition model using the evaluation dataset;

[0058] S5: Collect audio data of unidentified pipes and extract features (same as feature extraction in step S2), and use the model to identify the unidentified pipes.

[0059] In this embodiment, the one-dimensional convolutional neural network is characterized by a four-layer stack of Conv-BN-ReLU-Pooling structures, and the expression for the forward propagation process during the training phase is as follows:

[0060]

[0061] in, x represents the output of the batch normalized layer during the training phase. train μ represents the input to the batch normalized layer during the training phase. B σ represents the mean of the set of output values ​​B from the previous convolutional layer. B denoted by , where ∈ represents the variance of the set of output values ​​B of a convolutional layer, ∈ represents a small positive number used to avoid division by zero, γ represents the learnable reconstruction coefficient parameter, and β represents the learnable reconstruction constant parameter.

[0062] The expression for the forward propagation process during the testing phase is as follows:

[0063]

[0064] in, x represents the output of the batch normalization layer during the evaluation phase. test Let μ represent the input to the batch normalization layer during the evaluation phase. r σ represents the average of the means of all batches. r This represents an unbiased estimate of the variance for each batch.

[0065] In this embodiment, the expression for the Softmax layer is as follows:

[0066]

[0067] Where Softmax(x) represents the output of the Softmax layer, x i Let represent the value of the i-th output in the output vector x, and e represent the natural constant.

[0068] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for pipe scene recognition based on a convolutional neural network, characterized in that, The method comprises the following steps: S1: Collect audio data of multiple pipes in different scenes to form a label set T=(t1, t2,..., t n ), wherein t n represents the nth scene corresponding to the pipe; S2: Select multiple pipeline recording audio data under different labels to form a set of original audio files D=(d1, d2,..., d n ), d n n represents the nth original audio file; feature extraction is performed on each original audio file in the set of original audio files D to form a set of waveform features W=(w1, w2,..., w n ) as the input data set of the model, w n n represents the waveform feature matrix corresponding to the pipeline in the nth scene; For each original audio file in the original audio file set D, it is one-hot encoded into a vector, forming a probability value set Y = (y1, y2, ..., y3). n ), y n This represents the probability value of the waveform corresponding to the nth scene pipeline; S3: constructing an identification model: a one-dimensional convolutional neural network is built for processing the feature set W, and a fully connected neural network is built for classification; the one-dimensional convolutional neural network comprises an input layer, a one-dimensional convolutional layer, a maximum pooling layer, a global average pooling layer, a batch normalization layer and a Leaky ReLU layer; the fully connected neural network comprises an input layer, a fully connected layer, a Dropout layer, a batch normalization layer, a Leaky ReLU layer and a Softmax layer; S4: dividing the waveform feature set W and the probability value set Y, taking 80% of them as a training data set and the remaining 20% as an evaluation data set; selecting cross-entropy as a loss function, using stochastic gradient descent as an optimization algorithm in the training process of the identification model, setting the hyperparameters of the model and the expected accuracy; training the identification model through the training data set and evaluating the identification model through the evaluation data set; S5: collecting audio data of an un-identified pipeline and performing feature extraction, and using the model to identify the un-identified pipeline.

2. The pipe scene recognition method of claim 1, wherein, The feature extraction in the step S2 or the step S4 specifically comprises the following steps: extracting a waveform diagram of the original audio file, using a FeatureExtractor.extract function in a Dcase-Util tool library, inputting the original audio file into the function, and outputting a waveform feature matrix w.

3. The pipe scene recognition method of claim 1, wherein, The one-hot encoding in the step S2 specifically comprises the following steps: constructing a one-dimensional zero vector, the length of the vector being equal to the number n of scene categories in the label set T; setting a scene category corresponding to the original audio file in the label set T, and setting the kth value of the zero vector to be 1 and the remaining values to be 0, to obtain a vector y after one-hot encoding.

4. The pipe scene recognition method of claim 1, wherein, The one-dimensional convolutional neural network in the step S3 is built in the following process: sequentially building an input layer, a first one-dimensional convolutional layer, a first maximum pooling layer, a second one-dimensional convolutional layer, a second maximum pooling layer, a third one-dimensional convolutional layer, a third maximum pooling layer, a fourth one-dimensional convolutional layer, a fourth maximum pooling layer and a global average pooling layer; using a batch normalization layer to normalize the data and using a Leaky ReLU layer as an activation function when each one-dimensional convolutional layer outputs.

5. The pipe scene recognition method of claim 4, wherein, The structure of the one-dimensional convolutional neural network is a four-layer stack of Conv-BN-ReLU-Pooling structures, and the expression of the forward propagation process in the training stage is as follows: wherein, represents the output of the batch normalization layer in the training phase, x train represents the input of the batch normalization layer in the training phase, μ B represents the mean of the output numerical set B of the previous convolutional layer, σ B represents the variance of the output numerical set B of a convolutional layer, ∈ represents a small positive number used to avoid division by zero, γ represents a learnable reconstruction coefficient parameter, and β represents a learnable reconstruction constant parameter; The expression of the forward propagation process in the test stage is as follows: where, represents the output of the batch normalization layer in the evaluation phase, x test represents the input of the batch normalization layer in the evaluation phase, μ r represents the average of the means of all batches, σ r represents the unbiased estimate of the variance of each batch.

6. The pipe scene recognition method of claim 1, wherein, The fully connected neural network in the step S3 is built in the following process: sequentially building a first Dropout layer, a first fully connected layer, a second Dropout layer, a second fully connected layer, a third Dropout layer and a third fully connected layer; using a batch normalization layer to normalize the data and using a Leaky ReLU layer as an activation function when the first fully connected layer and the second fully connected layer output; using a Softmax layer as an activation function when the third fully connected layer outputs, to output the probability value of the label of each scene category.

7. The pipe scene recognition method of claim 6, wherein, In the Dropout layer, each neuron is kept with probability p.

8. The pipe scene recognition method of claim 6, wherein, The expression of the Softmax layer is as follows: where softmax(x) represents the output of the Softmax layer, x i represents the value of the i-th output in the output vector x, and e represents the natural constant.

9. The pipe scene recognition method of claim 1, wherein, The expression of the cross-entropy loss function is as follows: where L denotes the cross-entropy loss function, L i represents the cross-entropy loss value of each sample, N represents the number of training dataset samples, M represents the number of categories, c represents the category number, y ic represents the sign function, which takes 1 if the real category of the training dataset sample i is equal to c, and 0 otherwise, p ic represents the predicted probability of the training dataset sample i belonging to category c.

10. The pipe scene recognition method of claim 1, wherein the expression of the stochastic gradient descent optimization algorithm is as follows: where L(0; x i , y i ) is the loss function for each sample, φ(θ) is the regularizer, denotes stochastic gradient descent optimization computation.

Images