Method for generating audio dataset of expanded steel plate shearing and falling when cutting the head and tail of steel plate

Abstract

The present invention belongs to the field of deep learning, and specifically relates to a data generation method for expanding an audio data set of steel plate shearing and dropping when the steel plate is cut at the head and tail, which is used to identify the head and tail dropping state. First, the acquired few-sample audio data set is trained with dilated convolution using the Wavenet speech generation network, and a predicted audio data set is continuously iterated using a time series model; at the same time, for the generated unpaired type data, the CycleGan network is used to learn its data distribution and calculate the loss function and similarity between the generated data and the predicted data, thereby generating an audio waveform data set that is highly similar to the real audio data. The present invention can effectively solve the problem of few audio samples of steel plate shearing and dropping.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning, and specifically relates to a method for generating an expanded audio data set of steel plate shearing and falling when the steel plate is cut at the head and tail, which is used to identify the head and tail falling state of the steel plate when the steel plate is cut at the head and tail on a plate and strip production line. Background Art

[0002] With the advent of intelligent manufacturing, the steel manufacturing industry is fully embracing digitalization, unmanned operations, and intelligent technology. When cutting the ends of steel plates on steel strip production lines, determining whether the plates have fallen properly during the shearing process relies primarily on the experience of technicians. This is not only time-consuming but also results in high noise levels, low production efficiency, and a high risk factor.

[0003] Neural network training models can provide reliable real-time predictions of the sound of steel plates shearing and falling. However, neural network training requires a large amount of sample data. In practice, collecting large amounts of data samples is often difficult, and obtaining data on abnormal shearing is even more challenging. Consequently, a number of deep learning-based data augmentation algorithms and data generation networks have emerged.

[0004] In the existing technology, the following solutions are available for generating and enhancing image data: the Pixel CNN algorithm continuously generates new pixels in the form of convolution kernels, but has certain limitations in generating pixels for the waveform of speech data, and the generated data is not much different from the predicted data; the VAE algorithm generates data using the maximum expected probability of the grayscale value distribution, but the grayscale value distribution cannot serve as a good feature item for the speech waveform; the Generative Adversarial Network (GAN) is widely used, but in general GAN ​​networks, the generator cannot maximize the realism of the image. Summary of the Invention

[0005] In response to the above technical problems, the present invention provides a method for generating an expanded audio dataset of steel plate shearing and dropping when cutting the head and tail of steel plates. Taking the audio waveform as the object, the Wavenet speech generation network is combined with the cycle consistency generative adversarial network CycleGan to generate audio data that is as similar as possible to the predicted data. This method can solve the problem of insufficient samples of steel plate shearing and dropping audio data and achieve the purpose of expanding the steel plate shearing and dropping audio dataset.

[0006] The present invention is achieved through the following technical solutions:

[0007] A method for generating data of an audio data set of steel plate shearing and dropping when the steel plate is cut at the head and tail, the method comprising:

[0008] A dilated convolution operation is performed on the obtained real data sample set of steel plate shearing and falling to generate a predicted audio dataset; the predicted audio dataset is input into the CycleGan network, and the CycleGan network is trained to generate a generated dataset that is highly similar to the predicted audio data to expand the steel plate shearing and falling audio dataset.

[0009] Furthermore, the method comprises the following steps:

[0010] Step 1: Obtain a real data sample set; the real data sample set includes samples of normal shearing sound of steel plates and samples of abnormal shearing sound of steel plates, and the number of samples of normal shearing sound of steel plates is greater than the number of samples of abnormal shearing sound;

[0011] This step obtains a small set of real-world data samples of steel plate shearing and dropping, approximately 50 audio data points. In practice, the sound data of normal steel plate shearing and dropping is easy to collect, but it is difficult to simulate abnormal steel plate shearing and dropping, so the abnormal dropping data obtained will be relatively small.

[0012] Step 2: Use the Wavenet speech generation network to perform dilated convolution operations on the real data sample set to expand the network's receptive field. Based on the input of the historical moment and the current moment, the waveform of the next moment is predicted. This is repeated over and over again to generate a predicted audio dataset based on the time series.

[0013] Step 3: Input the generated predicted audio data set into the CycleGan generative network to generate a generated data set that is highly similar to the predicted audio data, and calculate the loss function and similarity between the generated data and the predicted audio data;

[0014] Step 4: By continuously modifying the training loss function, the similarity between the generated data and the predicted data is improved, and the similarity between the generated data and the predicted audio data is determined. When the similarity reaches a certain range, the data generation is completed, and an audio waveform graph that is highly similar to the predicted audio data is obtained, expanding the data set of the audio waveform graph of steel plate shearing and falling.

[0015] Furthermore, in step 2, an input layer, an output layer, and two hidden layers are set in the Wavenet speech generation network, and the expansion rate parameters of the dilated convolution are set to 1, 2, and 4, respectively; during the generation process, each generated point is placed at the last point of the input layer, and the prediction value for the next moment is iteratively generated until the time series of the audio data ends, and new sequence data is predicted to obtain a generated predicted audio data set.

[0016] Furthermore, in step three, the predicted audio dataset generated by the Wavenet speech generation network is of unpaired data type, and the predicted audio dataset includes the predicted audio of normal shearing of steel plates and the predicted audio of abnormal shearing of steel plates. The generated predicted audio dataset is input into the CycleGan generation network, and the predicted audio dataset is randomly divided into two groups. The two groups of data are used to train the generator and the discriminator at the same time, and the loss function and similarity between the generated data and the predicted audio data are calculated. The training loss function is continuously corrected to optimize the similarity to ensure that the generated data can be converted into an image that is highly similar to the predicted audio data through the generator.

[0017] Furthermore, the predicted audio data set is randomly divided into two groups, denoted as Group A and Group B, and the generators G1 and G2 are trained at the same time. The data of Group A is generated by the generator G1 to generate data C. B , and then generate C B Input into generator G2 to generate A'; group B data passes through generator G2 to generate data C A , and then generate C A Input into generator G1 to generate B';

[0018] The data generated by the generator for the two groups of predicted data, A and B, can be converted into images similar to the predicted audio data through the generator;

[0019] D1 and D2 are discriminators, D1 discriminates A and C A , whether A and A' meet the similarity range, D2 judges B and C B , B and B' meet the similarity range, and improve A and C by continuously calculating the training loss function A , A and A', B and C B , the similarity between B and B'.

[0020] Furthermore, in step 4, the logarithmic power spectrum feature of the audio is used as the standard for determining the similarity. When the similarity reaches the range of 0.85-0.95, the data generation is completed.

[0021] Beneficial technical effects of the present invention:

[0022] The method described in the present invention uses a Wavenet speech generation network to perform dilated convolution operations on real sample data, predicts the waveform at the next moment based on inputs at historical moments and the current moment, and continuously iterates. Finally, a series of dense layers are used to combine all previous results to generate a predicted audio dataset; the predicted audio dataset is input into a CycleGan network. The CycleGan network can avoid that the generated data is exactly the same as the predicted data, and can make the generated data and the predicted data belong to the same category (normal drop or abnormal drop) but not exactly the same, thereby generating a dataset that is more similar to real data.

[0023] The method proposed in the present invention combines the use of two generative networks. First, the Wavenet network is used to generate an audio dataset predicted based on a time series model. The predicted audio dataset is divided into two groups, and the predicted audio dataset is used to approximately replace the real dataset. Then, the CycleGan network is used to expand and optimize the dataset to solve the problem of insufficient audio samples in steel plate shearing. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of a method for generating audio data of steel plate shearing and falling according to an embodiment of the present invention;

[0025] Figure 2 Schematic diagram of the Wavenet dilated convolution algorithm in an embodiment of the present invention;

[0026] Figure 3 This is a diagram of the CycleGan network structure in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0028] On the contrary, the present invention covers any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention as defined by the claims. Furthermore, to facilitate a better understanding of the present invention, certain specific details are described in detail below in the detailed description of the present invention. Those skilled in the art will be able to fully understand the present invention without these details.

[0029] An embodiment of the present invention provides a data generation method for expanding a data set of audio recordings of steel plates shearing and dropping when cutting the head and tail of steel plates on a plate and strip production line. By combining a Wavenet speech generation network with a CycleGan generation network, the problem of insufficient audio samples of steel plates shearing and dropping is solved.

[0030] Specifically, the obtained few-sample real data is used to perform dilated convolution operations through the Wavenet speech generation network to generate a predicted audio dataset based on the time series; then, the CycleGan network is trained to generate a dataset that is highly similar to the predicted audio data, improving the availability and authenticity of the data and solving the problem of few audio samples of steel plate shearing and falling.

[0031] like Figure 1 As shown, the method includes the following steps:

[0032] Step 1: Obtain a real data sample set; the real data sample set includes samples of normal shearing sound of steel plates and samples of abnormal shearing sound of steel plates, and the number of samples of normal shearing sound of steel plates is greater than the number of samples of abnormal shearing sound;

[0033] Step 2: If Figure 2 As shown. Figure 2 In the algorithm, each convolutional layer convolves the previous layer and selects connected nodes based on the set dilation rate parameter to expand the perceptual range. During the generation process, each generated point is added to the last point in the input layer, and generation continues iteratively. Using the Wavenet speech generation network, dilated convolution operations are performed on the real data sample set to expand the network's receptive field. Based on historical and current inputs, the waveform at the next moment is predicted. This process continues with iteration, generating a predicted audio dataset based on the time series.

[0034] Step 3: Input the generated predicted audio data set into the CycleGan generative network to generate a generated data set that is highly similar to the predicted audio data, and calculate the loss function and similarity between the generated data and the predicted audio data;

[0035] In this embodiment, the similarity determination range is set to 0.85-0.95, and the specific evaluation index item is the logarithmic power spectrum of the audio.

[0036] Step 4: Improve the similarity between generated data and predicted data by continuously modifying the training loss function, and use the logarithmic power spectrum features of the audio as the standard for judging similarity; judge the similarity between the generated data and the predicted data. When it reaches the range of 0.85-0.95, data generation is completed.

[0037] Specifically, in step 1, in actual situations, the sound data of normal shearing and falling of steel plates is easy to collect, but it is impossible to simulate the abnormal shearing and falling of steel plates, so the abnormal falling data obtained will be relatively small.

[0038] Specifically, in step 2, the input layer, output layer, and two hidden layers are set, and the dilation convolution rate parameters are 1, 2, and 4 respectively. During the generation process, each point is added to the last point of the input layer, and the next prediction value is generated iteratively until the time series of the audio data ends. New sequence data is predicted to obtain the generated predicted audio data set.

[0039] Specifically, in step 3, the generated prediction audio data set (including normal and abnormal shearing of steel plates) is randomly divided into two groups, denoted as group A and group B. Generators G1 and G2 are trained at the same time, and the data of group A is generated by generator G1 to generate data C. B , and then generate C BInput into generator G2 to generate A'; group B data passes through generator G2 to generate data C A , and then generate C A Input into the generator G1 to generate B'; so that the data generated by the generator of the two sets of prediction data A and B can also be converted into images similar to the prediction data through the generator. Among them, D1 and D2 are discriminators, D1 discriminates A and C A , whether A and A' meet the similarity range, D2 judges B and C B , B and B' meet the similarity range, and improve A and C by continuously calculating the training loss function A , A and A', B and C B , B and B', when the similarity reaches the range of 0.85-0.95, it means that the generated data belongs to the category of the predicted data (normal drop and abnormal drop) but is not exactly the same as the predicted data, which meets the requirements of expanding the model training data set. Figure 3 shown.

[0040] The method provided in the embodiment of the present invention should acquire as much sample data as possible when acquiring samples. However, in actual situations, the sound data of normal shearing of steel plates is easy to acquire, but it is impossible to simulate the abnormal shearing of steel plates. The data of abnormal shearing of steel plates cannot be acquired too much. Therefore, in the process of generating abnormal shearing data in step three, possible background noise in the steel plant can be added, or noise points and noise types (such as Gaussian noise) can be randomly added to make the generated data more similar to the real data. In the Wavenet speech generation network, two hidden layers are set to avoid the waveform graph from losing too many features; on the basis of the CycleGan network, the judgment of the similarity of A and A', B and B' data is added, and the similarity of A and C is compared. A , B and C B The similarity also constitutes the discrimination criterion, which enhances the accuracy of judging whether the generated data and the predicted data are highly similar.

Claims

1. A data generation method for expanding a steel plate shearing and dropping audio data set when cutting the head and tail of a steel plate, characterized in that: The method comprises: Performing a dilated convolution operation on the acquired real data sample set of steel plate shearing and falling to generate a predicted audio dataset; inputting the predicted audio dataset into the CycleGan network, and generating a generated dataset that is highly similar to the predicted audio data through CycleGan network training to expand the steel plate shearing and falling audio dataset; The method comprises the following steps: Step 1: Obtain a real data sample set: the real data sample set includes samples of normal shearing sound of steel plates and samples of abnormal shearing sound of steel plates, and the number of samples of normal shearing sound of steel plates is greater than the number of samples of abnormal shearing sound; Step 2: Use the Wavenet speech generation network to perform dilated convolution operations on the real data sample set, predict the waveform at the next moment based on the input at the historical moment and the current moment, and iterate continuously to generate a predicted audio dataset based on the time series; Step 3: Input the generated predicted audio data set into the CycleGan generative network to generate a generated data set that is highly similar to the predicted audio data, and calculate the loss function and similarity between the generated data and the predicted audio data; Step 4: By continuously modifying the training loss function, the similarity between the generated data and the predicted data is improved, and the similarity between the generated data and the predicted audio data is determined. When the similarity reaches a certain range, the data generation is completed, and an audio waveform graph that is highly similar to the predicted audio data is obtained. This process is continuously iterated to achieve the purpose of expanding the steel plate shearing and falling audio waveform graph dataset.

2. The method for generating data of an audio data set of steel plate shearing and dropping when the steel plate is cut off at the head and tail as claimed in claim 1, characterized in that: In step 2, an input layer, an output layer, and two hidden layers are set in the Wavenet speech generation network, and the expansion rate parameters of the dilated convolution are set to 1, 2, and 4 respectively; during the generation process, each generated point is placed at the last point of the input layer, and the next moment prediction value is iteratively generated until the time series of the audio data ends, and new sequence data is predicted to obtain a generated predicted audio data set.

3. The method for generating data of an audio data set of steel plate shearing and dropping when the steel plate is cut off at the head and tail as claimed in claim 1, characterized in that: In step three, the predicted audio data set generated by the Wavenet speech generation network is of unpaired data type, and the predicted audio data set includes the predicted audio of normal shearing of steel plates and the predicted audio of abnormal shearing of steel plates. The generated predicted audio data set is input into the CycleGan generation network, and the predicted audio data set is randomly divided into two groups. The two groups of data are used to train the generator and the discriminator at the same time, and the loss function and similarity between the generated data and the predicted audio data are calculated. The training loss function is continuously corrected to optimize the similarity to ensure that the generated data can be converted into an image that is highly similar to the predicted audio data through the generator.

4. The method for generating data of an audio data set of steel plate shearing and dropping when the steel plate is cut off at the head and tail as claimed in claim 3, characterized in that: The predicted audio data set is randomly divided into two groups, denoted as Group A and Group B. Generators G1 and G2 are trained simultaneously. The data of Group A is passed through Generator G1 to generate data CB, which is then input into Generator G2 to generate A'; the data of Group B is passed through Generator G2 to generate data CA, which is then input into Generator G1 to generate B'. The data generated by the generator for the two groups of predicted data, A and B, can be converted into images similar to the predicted audio data through the generator; D1 and D2 are discriminators. D1 determines whether A and CA, A and A' meet the similarity range. D2 determines whether B and CB, B and B' meet the similarity range. The similarity between A and CA, A and A', B and CB, B and B' is improved by continuously calculating the training loss function.

5. The method for generating data of an audio data set of steel plate shearing and dropping when the steel plate is cut off at the head and tail as claimed in claim 1, characterized in that: In step 4, the logarithmic power spectrum features of the audio are used as the standard for judging similarity. When the similarity reaches the range of 0.85-0.95, the data generation is completed.

Images