Data field elimination adaptive switching method based on autonomous learning

Abstract

The invention discloses a data field elimination adaptive switching method based on autonomous learning, relates to the technical field of launcher control, and solves the problems that the existing data field elimination needs to manually tell a system to switch which field elimination method, and for long-period data with continuously changing data characteristics, if the field elimination method needs to be manually switched every time, the field elimination method cannot be manually switched. The method comprises the following steps of manually marking outlier data, carrying out rule identification on the manually marked outlier data, analyzing data characteristics of front and back data and outliers, and trying to customize any one of threshold value outlier elimination, quartile distance threshold value outlier elimination and change rate outlier elimination to carry out data outlier elimination. According to the method, the memory occupied by the deep learning network architecture is greatly reduced, the optimal field elimination method can be effectively and adaptively matched and selected to perform data field elimination in cooperation with different field elimination methods and the deep learning network architecture formed by the stacked drilling data, the convenience and the data accuracy are effectively improved, and manual intervention is reduced.

Description

Technical Field

[0001] The present invention relates to the field of missile control technology, and in particular to a data culling adaptive switching method based on autonomous learning. Background Art

[0002] There are many methods for data elimination, such as custom threshold elimination, interquartile range threshold elimination, 3-σ threshold elimination, elimination based on median absolute deviation (MAD), change rate elimination, etc. For a certain type of data, different elimination methods should be used under different working conditions to achieve the best elimination effect. For example, taking the wheel speed as an example, under a stable working condition with a stable speed, the elimination method based on median absolute deviation (MAD) should be adopted, and under a working condition with large rotation fluctuations, the change rate elimination method should be adopted.

[0003] However, in actual engineering, for a certain type of data, the system does not know which working state it is currently running in and which wild-field elimination method should be switched to. Manual instructions are needed to tell the system which wild-field elimination method to switch to. For long-period data with constantly changing data characteristics, if the wild-field elimination method needs to be switched manually every time, work efficiency cannot be improved. Therefore, it does not meet existing needs. In this regard, we propose a data wild-field elimination adaptive switching method based on autonomous learning. Summary of the invention

[0004] The purpose of the present invention is to provide a data de-wiping adaptive switching method based on autonomous learning, so as to solve the problem that the existing data de-wiping proposed in the above background technology requires manual telling of which de-wiping method to switch to. For long-period data with constantly changing data characteristics, if the de-wiping method needs to be manually switched every time, work efficiency cannot be improved.

[0005] To achieve the above object, the present invention provides the following technical solution: a data culling adaptive switching method based on autonomous learning, comprising the following steps: Step 1: Manually mark outlier data, identify the regularity of the manually marked outlier data, analyze the characteristics of the data before and after and the outlier data, and try to use any of the custom threshold elimination, interquartile range threshold elimination and change rate elimination to eliminate outliers in the data; Step 2: For the data elimination method with the best effect, record it in the system to form a knowledge base, build a data elimination 3D inversion data set, and generate sample models. The sample models are subdivided into multiple subtypes, covering the variation of different working conditions, speed values, and speed stability change factors. Ten thousand model samples are generated and divided into training set, verification set, and test set in a ratio of 5:1:2; Step 3: Perform forward simulation on the designed model to generate corresponding two-dimensional data to remove abnormal data. The generated data is used as the input of the neural network and used together with the corresponding model parameters to train the model; Step 4: Use a 3D dimension converter to convert the two-dimensional features extracted by the encoder into three-dimensional features, perform nonlinear transformation on the three-dimensional features, capture the relationship between the three-dimensional features, and further improve the expressiveness of the features; Step 5: Perform an inversion simulation on the designed model. The data remaining after excluding the optimal data in a single calculation link is the residual data. The parameters corresponding to each residual data directly pass its input information to the subsequent layer to learn the residual of the input information, the change or update amount of the input information, and then keep the input and output consistent, loop, stack multiple residual data, and build a deep learning network architecture; Step 6: When eliminating unknown data, match the features in the deep learning network architecture with the data to be eliminated. Automatically select the optimal elimination method through matching among custom threshold elimination, interquartile range threshold elimination, and change rate elimination to perform data elimination and realize automatic switching of elimination methods.

[0006] Preferably, after the 3D dimension converter converts the two-dimensional features extracted by the encoder into three-dimensional features, the data de-wiping decoder is used to reconstruct the three-dimensional data de-wiping structure. The data de-wiping decoder converts the features extracted by the inversion network encoder into a three-dimensional data de-wiping structure, and the five cascaded deconvolution blocks of the data de-wiping decoder start working.

[0007] Preferably, the deconvolution block includes a feature recombination module and an upsampling module, wherein the feature recombination module utilizes a convolution kernel to reorganize features and performs nonlinear transformation through a Leaky ReLU activation function, and the convolution kernel of the upsampling module upsamples features while reducing the number of deconvolution filters to reduce the number of feature maps.

[0008] Preferably, the custom threshold elimination comprises the following steps: D1: According to the upper and lower limit boundaries, the data of the super-high boundary is eliminated, and the time of retaining the data can be understood as no telemetry at that time; D2: Set upper and lower thresholds for different telemetry and eliminate them.

[0009] Preferably, the upper and lower limits of the thresholds for different telemetry may be set by setting a set of the same upper and lower limit thresholds for different telemetry, or by setting respective threshold value pair information for different telemetry.

[0010] Preferably, the interquartile range threshold elimination comprises the following steps: E1: First, calculate the quartiles corresponding to the box plot, and get the distance between the upper quartile and the lower quartile, called the interquartile range (IQR). Then calculate the upper and lower limits based on the IQR. Values ​​beyond the upper and lower limits are considered outliers. E2: Calculate the upper and lower thresholds based on the deviation coefficient and IQR. The deviation coefficient can be customized to adjust the range of the upper and lower limits to make it more applicable to different scenarios.

[0011] Preferably, the calculation formula of the interquartile range IQR is: IQR=Q3-Q1; Among them, Q3 is the upper quartile and Q1 is the lower quartile; The calculation formula for the upper limit of IQR, IQR max, is: IQR max=Q3+m IQR; The calculation formula for the lower limit of IQR, IQR min, is: IQR min = Q1-m IQR; Where m is the deviation coefficient.

[0012] Preferably, the change rate elimination comprises the following steps: F1: According to the data change rate, the data that exceeds the change rate range is eliminated, and the time corresponding to the data is retained. Change rate elimination first needs to calculate the first-order or multi-order change rate of the data, and the change of the data is divided by the change of time. The first-order change rate is mainly used for elimination; F2: Similar to the interquartile range threshold outlier removal, the rate of change outlier removal can also customize the reasonable range of the rate of change, or calculate the threshold range of the rate of change for the data in the selected time period according to the statistical distribution characteristics and distance dispersion, which is used to process outliers by means of the data's own threshold outlier removal.

[0013] Preferably, the order of the custom change rate is to obtain the data change rate in different time periods, and then convert the time columns in different formats into a standard time format. If there is no available time column, the change rate is calculated according to the data order interval.

[0014] Preferably, the 3D dimension converter performs feature reorganization and nonlinear transformation sequentially, compresses and reorganizes the feature map, and doubles the number of channels to enhance feature expression capabilities.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention uses a 3D dimensional converter to sequentially reorganize features and perform nonlinear transformation, compress and reorganize feature maps, and double the number of channels to enhance feature expression capabilities. The deconvolution block performs nonlinear transformation through the Leaky ReLU activation function and cooperates with the convolution kernel of the upsampling module to upsample the features. At the same time, the number of deconvolution filters is reduced to reduce the number of feature maps, so that the memory occupied by the deep learning network architecture is greatly reduced. The deep learning network architecture composed of different de-wiring methods and stacked training data can effectively and adaptively match and select the optimal de-wiring method to realize automatic switching of de-wiring methods for data de-wiring, which effectively improves convenience, data accuracy and reduces manual intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the data culling adaptive switching method based on autonomous learning of the present invention; Figure 2 This is a system diagram of the interquartile range threshold elimination method of the present invention. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0018] See also Figure 1 to Figure 2 , an embodiment of the present invention provides: a data culling adaptive switching method based on autonomous learning, comprising the following steps: Step 1: Manually mark outlier data, identify the regularity of the manually marked outlier data, analyze the characteristics of the data before and after and the outlier data, and try to use any of the custom threshold elimination, interquartile range threshold elimination and change rate elimination to eliminate outliers in the data; Step 2: For the data elimination method with the best effect, record it in the system to form a knowledge base, build a data elimination 3D inversion data set, and generate sample models. The sample models are subdivided into multiple subtypes, covering the variation of different working conditions, speed values, and speed stability change factors. Ten thousand model samples are generated and divided into training set, verification set, and test set in a ratio of 5:1:2; Step 3: Perform forward simulation on the designed model to generate corresponding two-dimensional data to remove abnormal data. The generated data is used as the input of the neural network and used together with the corresponding model parameters to train the model; Step 4: Use a 3D dimension converter to convert the two-dimensional features extracted by the encoder into three-dimensional features, perform nonlinear transformation on the three-dimensional features, capture the relationship between the three-dimensional features, and further improve the expressiveness of the features. The 3D dimension converter sequentially reorganizes and nonlinearly transforms the features, compresses and reorganizes the feature maps, and doubles the number of channels to enhance the expressiveness of the features. After the 3D dimension converter converts the 2D features extracted by the encoder into 3D features, the data de-wiping decoder is used to reconstruct the 3D data de-wiping structure. The data de-wiping decoder converts the features extracted by the inversion network encoder into a 3D data de-wiping structure. The five cascaded deconvolution blocks of the data de-wiping decoder start working. The deconvolution block includes a feature recombination module and an upsampling module. The feature recombination module uses a convolution kernel to reorganize features and performs nonlinear transformation through a Leaky ReLU activation function. The convolution kernel of the upsampling module upsamples the features and reduces the number of deconvolution filters to reduce the number of feature maps.

[0019] Step 5: Perform an inversion simulation on the designed model. The data remaining after excluding the optimal data in a single calculation link is the residual data. The parameters corresponding to each residual data directly pass its input information to the subsequent layer to learn the residual of the input information, the change or update amount of the input information, and then keep the input and output consistent, loop, stack multiple residual data, and build a deep learning network architecture; Step 6: When eliminating unknown data, match the features in the deep learning network architecture with the data to be eliminated. Automatically select the optimal elimination method through matching among custom threshold elimination, interquartile range threshold elimination, and change rate elimination to perform data elimination and realize automatic switching of elimination methods.

[0020] The above-mentioned custom threshold culling removes data with super-high boundaries according to the upper and lower limits. The time for retaining the data can be understood as no telemetry at that time. Set the upper and lower limits of the thresholds for different telemetry and cull them. Setting the upper and lower limits of the thresholds for different telemetry can set a set of the same upper and lower limit thresholds for different telemetry, or set different threshold value pair information for different telemetry.

[0021] The above-mentioned interquartile range threshold outlier elimination method first calculates the quartiles corresponding to the box plot, obtains the distance between the upper quartile and the lower quartile, called the interquartile range IQR, and calculates the upper and lower limits based on the IQR. Values ​​beyond the upper and lower limits are outliers. The upper and lower limit thresholds are calculated based on the deviation coefficient and IQR. The deviation coefficient can be customized to adjust the range of the upper and lower limits, making it more applicable to more scenarios.

[0022] like Figure 2 As shown, the calculation formula for the interquartile range IQR is: IQR=Q3-Q1; IQR max=Q3+m IQR; IQR min = Q1-m IQR; Where Q3 is the upper quartile, Q1 is the lower quartile, IQR max is the upper limit of IQR calculation, IQR min is the lower limit of IQR calculation, and m is the deviation coefficient, which can be 1.5.

[0023] The above-mentioned rate of change elimination is based on the rate of change of the data, and the data that exceeds the rate of change range is eliminated, and the time corresponding to the data is retained. The rate of change elimination first needs to calculate the first-order or multi-order rate of change of the data. The change of the data is divided by the change of time, and the first-order rate of change is mainly used for elimination. Similar to the interquartile range threshold elimination, the rate of change elimination can also customize the reasonable range of the rate of change, or calculate the threshold range of the rate of change for the data in the selected time period according to the statistical distribution characteristics and distance dispersion, which is used to process the wild values ​​by the data's own threshold elimination method.

[0024] The order of custom change rate is to obtain the data change rate in different time periods, and then convert the time columns in different formats into the standard time format. If there is no available time column, the change rate is calculated according to the data order interval; In summary, the deep learning network architecture composed of different wild-wild removal methods and stacked training data in this application can effectively and adaptively match and select the optimal wild-wild removal method to realize automatic switching of wild-wild removal methods for data wild-wild removal, effectively improving convenience, data accuracy and reducing manual intervention.

[0025] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A data culling adaptive switching method based on autonomous learning, characterized in that: The steps include: Step 1: Manually mark outlier data, identify the regularity of the manually marked outlier data, analyze the characteristics of the data before and after and the outlier data, and try to use any of the custom threshold elimination, interquartile range threshold elimination and change rate elimination to eliminate outliers in the data; Step 2: For the data elimination method with the best effect, record it in the system to form a knowledge base, build a data elimination 3D inversion data set, and generate sample models. The sample models are subdivided into multiple subtypes, covering the variation of different working conditions, speed values, and speed stability change factors. Ten thousand model samples are generated and divided into training set, verification set, and test set in a ratio of 5:1:2; Step 3: Perform forward simulation on the designed model to generate corresponding two-dimensional data to remove abnormal data. The generated data is used as the input of the neural network and used together with the corresponding model parameters to train the model; Step 4: Use a 3D dimension converter to convert the two-dimensional features extracted by the encoder into three-dimensional features, perform nonlinear transformation on the three-dimensional features, capture the relationship between the three-dimensional features, and further improve the expressiveness of the features; Step 5: Perform an inversion simulation on the designed model. The data remaining after excluding the optimal data in a single calculation link is the residual data. The parameters corresponding to each residual data directly pass its input information to the subsequent layer to learn the residual of the input information, the change or update amount of the input information, and then keep the input and output consistent, loop, stack multiple residual data, and build a deep learning network architecture; Step 6: When eliminating unknown data, match the features in the deep learning network architecture with the data to be eliminated. Automatically select the optimal elimination method through matching among custom threshold elimination, interquartile range threshold elimination, and change rate elimination to perform data elimination and realize automatic switching of elimination methods.

2. According to the method of claim 1, the method is characterized by: After the 3D dimension converter converts the two-dimensional features extracted by the encoder into three-dimensional features, the data de-wiping decoder is used to reconstruct the three-dimensional data de-wiping structure. The data de-wiping decoder converts the features extracted by the inversion network encoder into a three-dimensional data de-wiping structure, and the five cascaded deconvolution blocks of the data de-wiping decoder start working.

3. The method for adaptive switching of data culling based on autonomous learning according to claim 2, characterized in that: The deconvolution block includes a feature recombination module and an upsampling module. The feature recombination module uses a convolution kernel to reorganize features and performs nonlinear transformation through a Leaky ReLU activation function. The convolution kernel of the upsampling module upsamples the features and reduces the number of deconvolution filters to reduce the number of feature maps.

4. The method for adaptive switching of data culling based on autonomous learning according to claim 1, characterized in that: The custom threshold culling comprises the following steps: D1: According to the upper and lower limit boundaries, the data of the super-high boundary is eliminated, and the time of retaining the data can be understood as no telemetry at that time; D2: Set upper and lower thresholds for different telemetry and eliminate them.

5. The method for adaptive switching of data culling based on autonomous learning according to claim 4 is characterized in that: To set the upper and lower limits of the thresholds for different telemetry, you can set a set of the same upper and lower limit thresholds for different telemetry, or you can set separate threshold value pair information for different telemetry.

6. The method for adaptive switching of data culling based on autonomous learning according to claim 1, characterized in that: The interquartile range threshold elimination comprises the following steps: E1: First, calculate the quartiles corresponding to the box plot, and get the distance between the upper quartile and the lower quartile, called the interquartile range (IQR). Then calculate the upper and lower limits based on the IQR. Values ​​beyond the upper and lower limits are considered outliers. E2: Calculate the upper and lower thresholds based on the deviation coefficient and IQR. The deviation coefficient can be customized to adjust the range of the upper and lower limits to make it more applicable to different scenarios.

7. The method for adaptive switching of data culling based on autonomous learning according to claim 6, characterized in that: The calculation formula for the interquartile range (IQR) is: IQR=Q3-Q1; Among them, Q3 is the upper quartile and Q1 is the lower quartile; The calculation formula for the upper limit of IQR, IQR max, is: IQR max=Q3+m IQR; The calculation formula for the lower limit of IQR, IQR min, is: IQR min=Q1-m IQR; Where m is the deviation coefficient.

8. The method for adaptive switching of data culling based on autonomous learning according to claim 1, characterized in that: The change rate elimination comprises the following steps: F1: According to the data change rate, the data that exceeds the change rate range is eliminated, and the time corresponding to the data is retained. Change rate elimination first needs to calculate the first-order or multi-order change rate of the data, and the change of the data is divided by the change of time. The first-order change rate is mainly used for elimination; F2: Similar to the interquartile range threshold outlier removal, the change rate outlier removal can also customize the reasonable range of the change rate, or calculate the threshold range of the change rate for the data in the selected time period according to the statistical distribution characteristics and distance dispersion, which is used to process outliers by the data's own threshold outlier removal method.

9. The method for adaptive switching of data culling based on autonomous learning according to claim 8, characterized in that: The order of the custom change rate is to obtain the data change rate in different time periods, and then convert the time columns in different formats into the standard time format. If there is no available time column, the change rate is calculated according to the data order interval.

10. The method for adaptive switching of data culling based on autonomous learning according to claim 1, characterized in that: The 3D dimension converter performs feature reorganization and nonlinear transformation sequentially, compresses and reorganizes the feature map, and doubles the number of channels to enhance the feature expression capability.

Images