An automatic data augmentation method and device for indicator diagrams

Abstract

The present invention relates to the field of artificial intelligence technology, and specifically provides a method and device for automatically enhancing dynamometric diagram data, which have the following steps: S1. The automatic enhancement method based on teacher knowledge alternately updates the parameters of the target model and the enhancement model. The enhancement method includes a color augmentation model and a geometric augmentation model. Let the target model be the teacher model, and the teacher model can be any model as long as it is different from the target model; S2. Set a search algorithm and a search space, and use the color augmentation model and the geometric augmentation model to update the structure of the search space; S3. Generate a semantic auxiliary field; S4. Image and semantic multimodal fusion; S5. Dataset preparation. Compared with the prior art, the present invention can improve the sample generation efficiency, increase the working condition diagnosis accuracy rate, and reduce the enterprise cost.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and particularly provides a method and device for automatically enhancing the data of an indicator diagram. Background Art

[0002] Condition diagnosis is an important evaluation method for the operating state of a mechanical equipment system. As the data source for condition diagnosis, the quality of the indicator diagrams will have a significant impact on the diagnosis results. Currently, machine learning methods are mostly used for condition diagnosis. When training a machine learning model, what is actually done is to adjust the parameters so that it can map the input (such as a picture) to the output (such as a label). Our optimization goal is to pursue the optimal point with a lower model loss, which will occur when the parameters are adjusted in the correct way. Obviously, if there are many parameters, a sufficient proportion of samples need to be given to the model. However, in the actual mechanical production process, the probability of a failure is small, and the number of samples of failure types is small. Therefore, the number of samples between different types of indicator diagrams is significantly unbalanced.

[0003] As a method for increasing the number of samples and balancing the differences between classes, data augmentation has received the attention of many researchers. However, traditional data augmentation methods need to be designed manually, and the experimental results are not satisfactory. Summary of the Invention

[0004] In view of the above deficiencies of the prior art, the present invention provides a practical method for automatically enhancing the data of an indicator diagram.

[0005] A further technical task of the present invention is to provide a device for automatically enhancing the data of an indicator diagram with reasonable design, safety and applicability.

[0006] The technical solution adopted by the present invention to solve its technical problems is as follows:

[0007] A method for automatically enhancing the data of an indicator diagram has the following steps:

[0008] S1. Alternately update the parameters of the target model and the enhancement model using an automatic enhancement method based on teacher knowledge. The enhancement method includes a color augmentation model and a geometric augmentation model . Let be the teacher model, and the teacher model can be any model as long as it is different from the target model ;

[0009] S2. Set a search algorithm and a search space, and use the color augmentation model and the geometric augmentation model to update the structure of the search space;

[0010] S3. Generate a semantic auxiliary field;

[0011] S4. Image and semantic multimodal fusion;

[0012] S5. Dataset preparation.

[0013] Furthermore, in step S1, two teacher models are used, one is a pre-trained model and the other is an EMA model;

[0014] In the automatic enhancement method of the teacher knowledge, the enhancement model and the target model are alternately updated by stochastic gradient descent ; ;

[0015] Suppose represents the image sampled from the dataset X, represents the augmentation function with parameter and the corresponding function is ;

[0016] Let be the calculation of the loss, and the function objective is set as follows:

[0017] ;

[0018] Furthermore, in step S2, the main body of the search algorithm is reinforcement learning, which consists of two parts, namely a controller and a training algorithm, and the controller is served by an RNN;

[0019] Next, the generated enhancement policy will be applied to the training of the sub-model. After the sub-model is trained, it will be evaluated on the validation set. The accuracy of the sub-model will be fed back to the RNN controller as a "reward signal". Finally, the RNN controller will be updated through the reward signal and under the optimization algorithm.

[0020] Furthermore, the search space consists of many sub-policies, where one sub-policy is randomly selected for each image in each batch. The sub-policy consists of two operations, each of which has an image processing function. The search algorithm is used to find the best policy so that the neural network can produce the highest validation accuracy on the target dataset.

[0021] Furthermore, in step S3, the text descriptions of different working conditions in actual production are combined as the source for text feature extraction, and accurate professional knowledge is used to semantically assist different types of indicator diagrams to obtain text data for subsequent feature fusion.

[0022] Further, in step S4, an attention-based feature fusion mechanism is adopted to calculate a context vector of a word for each sub-region of the image. When generating the image, the image sub-regions are matched with the most relevant words, and for each part of the input picture, the most relevant word vectors are matched to constrain its generation, thereby increasing the fine-grained details of the image.

[0023] Further, in step S5, the sample data points of the dynamometer card of the actual pumping unit are collected and plotted into a dynamometer card. The data in the dynamometer card is preprocessed using the min-max normalization method to obtain the working condition samples. Then, a data augmentation method is used to increase the number of samples. Subsequently, a dynamometer card data set is made, and the dynamometer card data set is divided into a training set and a test set according to a ratio of 8:2.

[0024] An automatic data augmentation device for dynamometer cards includes: at least one memory and at least one processor;

[0025] The at least one memory is used to store machine-readable programs;

[0026] The at least one processor is used to call the machine-readable program and execute an automatic data augmentation method for dynamometer cards.

[0027] Compared with the prior art, an automatic data augmentation method and device for dynamometer cards of the present invention have the following prominent beneficial effects:

[0028] The present invention uses teacher knowledge for automatic data augmentation and combines semantic-assisted multi-modal data for more accurate data amplification. It can automatically select the optimal data augmentation method according to different types of dynamometer cards and generate more accurate images. A solution is proposed for the problem of unbalanced dynamometer card sample quantities, improving the sample generation efficiency while increasing the working condition diagnosis accuracy rate and reducing the enterprise cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following-described drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0030] Att Figure 1 is a training flow chart of an automatic data augmentation method for dynamometer cards;

[0031] Att Figure 2 is a schematic diagram of the mode of an automatic data augmentation method for dynamometer cards;

[0032] Att Figure 3It is a conceptual diagram of an automatic data augmentation method for indicator diagrams;

[0033] Appendix Figure 4 It is an error curve graph of an automatic data augmentation method for indicator diagrams and other comparison methods in the test stage;

[0034] Appendix Figure 5 It is an accuracy curve graph of an automatic data augmentation method for indicator diagrams and other comparison methods in the test stage. Detailed implementation method

[0035] To enable those skilled in the art to better understand the solution of the present invention, the present invention will be further described in detail below in conjunction with specific implementation methods. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative work belong to the scope of protection of the present invention.

[0036] The following gives a best embodiment:

[0037] As Figures 1-3 shown, an automatic data augmentation method for indicator diagrams in this embodiment has the following steps:

[0038] S1. Alternately update the parameters of the target model and the augmentation model based on the automatic augmentation method of teacher knowledge. The augmentation method includes a color augmentation model and a geometric augmentation model . Let be the teacher model, and the teacher model can be any model as long as it is different from the target model ;

[0039] In the automatic augmentation method of teacher knowledge, the augmentation model and the target model are alternately updated through stochastic gradient descent;

[0040] Suppose represents an image sampled from the dataset X, represents the augmentation function with parameters , and the corresponding function is ;

[0041] Let be the calculation of the loss, and the function objective formula is set as follows:

[0042] .

[0043] For the target model, this objective and the adversarial data augmentation method have the same components. However, this augmentation function not only requires maximizing the loss of the objective function but also minimizing the loss of the teacher model. Using this method can effectively avoid the disappearance of the inherent meaning of the image because when the image becomes unrecognizable, the teacher model will experience gradient explosion. This indicates that the introduced teacher model loss requires the augmentation function to transform the image so that the augmentation result is adversarial to the target model and recognizable to the teacher model.

[0044] S2. Set up the search algorithm and the search space, and use the color augmentation model and the geometric augmentation model to update the structure of the search space;

[0045] The main body of the search algorithm is reinforcement learning, which consists of two parts: the controller and the training algorithm. The controller is served by an RNN (Recurrent Neural Network). Next, the generated augmentation strategy will be applied to the training of the sub-model. After the sub-model is trained, it will be evaluated on the validation set, and the accuracy of the sub-model will be fed back to the RNN controller as a "reward signal". Finally, the controller will update through this signal and under the optimization algorithm.

[0046] In this article, we focus on the method of updating the strategy in an online manner for two reasons: (1) They can directly search for data augmentation strategies on the target network using all training data, and (2) this unifies the search and training processes and simplifies the framework.

[0047] The strategies in the search space consist of many sub-strategies, and one sub-strategy is randomly selected for each image in each batch. A sub-strategy consists of two operations, each of which has an image processing function, such as translation, rotation, or shear, as well as the probability and magnitude of these operations being adopted. The search algorithm is used to find the best strategy so that the neural network can produce the highest validation accuracy on the target dataset.

[0048] For example:

[0049] In the search space, the strategy consists of 5 sub-strategies, and each sub-strategy consists of 2 image operations to be applied in sequence. Additionally, each operation is also associated with two hyperparameters: 1) the probability of applying the operation; 2) the magnitude of the operation.

[0050] S3. Generate semantic auxiliary fields;

[0051] Combine the text descriptions of different working conditions in actual production as the source for text feature extraction, and use the most accurate professional knowledge to semantically assist different types of indicator diagrams, so that several pieces of text data can be obtained for subsequent feature fusion.

[0052] S4. Image and semantic multimodal fusion;

[0053] Adopt an attention-based feature fusion mechanism to calculate a context vector of a word for each sub-region of the image. When generating an image, match the image sub-region with the most relevant word, and for each part of the input picture, match the most relevant word vector to constrain its generation, increasing the fine-grained details of the image.

[0054] S5. Dataset preparation;

[0055] Collect the sample data points of the dynamometer diagrams of actual pumping units from an oilfield and plot them into dynamometer diagrams. Preprocess the data. Considering that the numerical ranges of working condition parameters among different pumping units are different, to ensure that this does not interfere with the working condition recognition, we adopt the min-max normalization method. In this way, 6 types of working condition samples can be obtained, with 700 samples in each type, a total of 4200 dynamometer diagrams. Use the data augmentation method to uniformly increase the number of samples to 1000 in each type, so that a total of 6000 samples are obtained.

[0056] Subsequently, start making the dynamometer diagram dataset. Here, referring to the format of the standard dataset "cifar10", change the image size to 32 × 32 and make it into a standardized data file. In terms of dataset division, we divide each type of dynamometer diagram into a training set and a test set according to a ratio of 8:2. That is to say, the number of samples in the training set and test set of each type is 800 and 200 respectively.

[0057] Regarding the model training problem, this teacher knowledge-based auto-augmentation method alternately updates the parameters of the target model and the augmentation model . The training process of this method is as Figure 1 shown. As Figure 1 shown, this method solves the problem by alternately updating the augmentation function and the target model through stochastic gradient descent. The model first updates the target network for n steps, and then updates the parameters of the augmentation function. The reason why the augmentation function can be updated by the gradient method is that the parameters of the proposed neural network are differentiable with respect to the parameters . This method neither regularizes the augmentation function based on domain knowledge nor restricts the difficulty of transforming the images to ensure that the transformed images are recognizable.

[0058] This paper proposes a data augmentation method using a neural network with parameters . This method mainly includes two models, the color augmentation model and the geometric augmentation model . This model can update the parameters through the gradient method and uses two functions to update the structure of the search space.

[0059] Comparative experiments were conducted to compare this method with traditional data augmentation methods and the original samples without data augmentation. The superiority of this method has been proven, and indicator diagrams were generated for actual production.

[0060] As Figures 4-5 shown, it can be seen from the loss curve that the error loss drops very quickly at the beginning, slows down after twenty iteration cycles, and finally remains stable. The accuracy change curve is also very fast in the initial stage, slows down after reaching a certain value, and finally reaches stability. The following is the numerical analysis. The final loss of the traditional data augmentation method in the working condition diagnosis task is 0.000512, and the accuracy is 93.32%. The final loss of the original data on the test set is 0.00632, and the accuracy is 89.46%. Obviously, the experimental performance of the automatic data augmentation method based on teacher knowledge semantics assistance is better than that of the other two comparative experiments.

[0061] The conceptual diagram of this method is as Figure 1 shown. The adversarial data augmentation method increases the loss value of the target model by transforming the data. The image data enhanced in this way is usually difficult to generate value (such as Figure 1 the black and noisy parts in ). The images enhanced without any constraints are difficult to recognize. For the problem of working condition diagnosis, providing directional guidance is helpful for the convergence speed of the model for indicator diagram classification. The method we proposed performs data transformation, making it adversarial to the target model but recognizable to the teacher model

[0062] . Therefore, the enhanced images will be richer in information than ordinary adversarial data augmentation.

[0063] The at least one memory is used to store machine-readable programs;

[0064] The at least one processor is used to call the machine-readable program and execute an automatic data augmentation method for indicator diagrams.

[0065] The above specific implementation manners are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above specific implementation manners. Any technical solution that conforms to the above specific implementation manners of the present invention and any appropriate changes or substitutions made by those of ordinary skill in the art shall fall within the patent protection scope of the present invention.

[0066] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An automatic data augmentation method for indicator diagrams, characterized in that, It has the following steps: S1. The automatic enhancement method based on teacher knowledge alternately updates the parameters of the target model and the enhancement model . The enhancement method includes a color augmentation model and a geometric augmentation model . Let be the teacher model, and the teacher model can be any model as long as it is different from the target model . Use two teacher models, one is a pre-trained model and the other is an EMA model; In the automatic enhancement method of the teacher's knowledge, the enhancement model and the target model are alternately updated by stochastic gradient descent and the target model ; Hypothesis represents an image sampled from the dataset X, represents an augmentation function with parameters , and the corresponding function is ; Let To calculate the loss, the function objective is set as follows: ； S2. Set the search algorithm and search space, and use the color augmentation model and the geometric augmentation model to update the structure of the search space; The main body of the search algorithm is reinforcement learning, which consists of two parts, namely a controller and a training algorithm, and the controller is served by an RNN; Next, the generated enhanced policy will be applied to the training of the sub-model. After the sub-model is trained, it will be evaluated on the validation set. The accuracy of the sub-model will be fed back to the RNN controller as a "reward signal". Finally, the RNN controller will be updated through the reward signal and under the optimization algorithm; The search space consists of many sub-policies. One of the sub-policies is randomly selected for each image in each batch. The sub-policy consists of two operations, each of which has an image processing function. Use the search algorithm to find the best policy so that the neural network can produce the highest validation accuracy on the target dataset; S3. Generate a semantic auxiliary field; Combine the text descriptions of different working conditions in actual production as the source for text feature extraction, use accurate professional knowledge to semantically assist different types of dynamometer cards, and obtain text data for subsequent feature fusion; S4. Image and semantic multi-modal fusion; Adopt an attention-based feature fusion mechanism to calculate a context vector of a word for each sub-region of the image. When generating the image, match the image sub-region with the most relevant word, and for each part of the input picture, match the most relevant word vector to constrain its generation, increasing the fine-grained details of the image; S5. Dataset preparation; Collect the sample data points of the dynamometer cards of actual pumping units, draw them into dynamometer cards, preprocess the data in the dynamometer cards, adopt the min-max normalization method to obtain the working condition samples, and then use the data augmentation method to increase the number of samples. Subsequently, start making the dynamometer card dataset, and divide the dynamometer card dataset into a training set and a test set according to the ratio of 8:

2.

2. An indicator diagram automatic data enhancement device, characterized in that Including: At least one memory and at least one processor; The at least one memory is used to store machine-readable programs; The at least one processor is used to call the machine-readable program and execute the method described in claim 1.

Images