Data screening method and system for recording type convolution track distance

Abstract

The invention discloses a data screening method and system for recording a class convolution track distance, and the method comprises the steps: processing the data of preset batches of multiple products, and obtaining a data track matrix; distinguishing a track matrix of same test data of multiple batches of products and a track matrix of single test data of a single product; obtaining the difference quantity between a single product and other products according to the track matrix of the same test data of multiple batches of products; abnormal data products in the multiple batches of products are screened out according to the difference quantity of the single product and other products and a preset product abnormal threshold value; obtaining a performance data exception influence quantity according to the single-product single test data track matrix; screening out an abnormal performance index according to the performance data abnormal influence quantity and a preset performance index abnormal threshold value; and obtaining abnormal products according to the abnormal data products and the abnormal performance indexes in the multiple batches of products, and positioning specific abnormal performance indexes. According to the invention, batch abnormity and index abnormity of multiple batches of data can be effectively identified.

Description

Technical Field

[0001] The present invention belongs to the technical field of product screening, and in particular relates to a data screening method and system for recording convolutional trajectory distances. Background Art

[0002] With the development of existing technologies, the complexity of product performance and testing has gradually increased. For multi-batch, small-batch, and high-complexity products, there is a lack of a specific and operational method for screening various types of large-scale data to identify data anomalies caused by factors such as unstable batch product quality. For data anomaly screening of multi-batch, small-batch, and high-complexity products, it is currently mainly based on product designers. According to design requirements, component conditions, research and development level and other factors, independent abnormal judgment intervals are set according to each indicator, and in actual data analysis, the abnormal state of a single indicator is mainly focused on. When all indicators are not in the abnormal interval, the product is judged to be normal as a whole, ignoring the abnormal deviation of the indicator data of a few products from the corresponding indicator data of most products. When the deviation anomaly of the above-mentioned minority products is superimposed on the average processing of multi-batch product indicators, the abnormal deviation anomaly of the minority products is easily diluted by the mean processing and is difficult to detect. On the other hand, the current multi-batch data analysis method is not fixed, and the batch anomaly screening process and quality are greatly affected by the technical ability and subjective factors of the analysts. The data analysis of intermittent batch products lacks a unified dimension and cannot effectively support the multi-data analysis of the entire batch. Summary of the invention

[0003] The technical problem solved by the present invention is: to overcome the prior art and provide a data screening method and system for recording convolution-like trajectory distances, which can effectively identify batch anomalies and indicator anomalies of multiple batches of data.

[0004] The object of the present invention is achieved through the following technical scheme: a data screening method for recording convolution-like trajectory distances, comprising: linearizing the data of preset batches of multiple products to obtain a data trajectory matrix; distinguishing the trajectory matrix of the same test data of multiple batches of products from the trajectory matrix of single test data of a single product in the data trajectory matrix; obtaining the difference between a single product and other products according to the trajectory matrix of the same test data of multiple batches of products; screening out abnormal data products in multiple batches of products according to the difference between the single product and other products and a preset product abnormality threshold; obtaining the abnormal impact of performance data according to the trajectory matrix of single test data of a single product; screening out abnormal performance indicators according to the abnormal impact of performance data and a preset performance indicator abnormality threshold; obtaining abnormal products and locating specific abnormal performance indicators according to the abnormal data products and abnormal performance indicators in multiple batches of products.

[0005] In the data screening method of recording the convolutional trajectory distance, the data of preset batches of multiple products are linearly processed to obtain a data trajectory matrix, including: collecting the performance data of each indicator of each product, and calibrating the collected performance data of each indicator according to time to form a data table; according to the time series, each performance test data in the data table forms a time-performance data trajectory; the product performance data corresponding to the same test process are placed in the same coordinate system to form a data trajectory matrix.

[0006] In the data screening method of recording the convolutional trajectory distance, a trajectory distance algorithm based on convolution is used according to the trajectory matrix of the same test data of multiple batches of products to obtain the difference between a single product and other products.

[0007] In the above data screening method of recording convolutional trajectory distance, the difference between a single product and other products is obtained by the following formula: ; in, For the The difference between a product and other products, For the The products are in the data table The indicators listed are the same as those of this batch of products except Other products besides products are in List the minimum similarity of trajectories of the same indicator, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, For the first moment, is the total number of products in this batch, For the second moment, is the number of columns of the total performance index of the product, The product serial number.

[0008] In the above data screening method for recording convolutional trajectory distance, It is obtained by the following formula:

[0009] in, To consider the time shift error Performance and The similarity between performance, For the Products in A collection of column indicator values, For this batch of products except Other products besides products are in A collection of column indicator values, For The sequence introduces displacement, is the adjustable displacement threshold, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, is the third moment, m is the Products in The total time of the column metric test.

[0010] In the data screening method of the above-mentioned record-type convolutional trajectory distance, if the difference between a single product and other products is greater than a preset product abnormality threshold, then the product is an abnormal data product.

[0011] In the above-mentioned data screening method of recording convolutional trajectory distance, the abnormal impact of performance data is obtained by the following formula: ; in, is the abnormal impact of performance data, is the starting time of the time period partition, is the end time of the partition according to the time period, is the column position number of the performance indicator, is the total number of columns of performance indicators, Preset time in time zone. for List The indicator value at the moment, for List arrive The average value of the indicator at the time, is the total number of products in this batch, For the fourth moment, For the The product performance indicator matrix at all times, It is the average value matrix of product performance indicators.

[0012] A data screening system for recording convolution-like trajectory distances, comprising: a first module, used for linearizing data of preset batches of multiple products to obtain a data trajectory matrix; a second module, used for distinguishing the trajectory matrix of the same test data of multiple batches of products from the trajectory matrix of single-time test data of a single product in the data trajectory matrix; a third module, used for obtaining the difference between a single product and other products based on the trajectory matrix of the same test data of multiple batches of products; a fourth module, used for screening out abnormal data products in multiple batches of products based on the difference between the single product and other products and a preset product abnormality threshold; a fifth module, used for obtaining the abnormal impact of performance data based on the trajectory matrix of single-time test data of a single product; a sixth module, used for screening out abnormal performance indicators based on the abnormal impact of performance data and a preset performance indicator abnormality threshold; a seventh module, used for obtaining abnormal products and locating specific abnormal performance indicators based on the abnormal data products and abnormal performance indicators in multiple batches of products.

[0013] In the data screening system that records the convolutional trajectory distance, the data of preset batches of multiple products are linearly processed to obtain a data trajectory matrix, including: collecting the performance data of each indicator of each product, and calibrating the collected performance data of each indicator according to time to form a data table; according to the time series, each performance test data in the data table forms a time-performance data trajectory; the product performance data corresponding to the same test process are placed in the same coordinate system to form a data trajectory matrix.

[0014] In the data screening system for recording the convolutional trajectory distance, a trajectory distance algorithm based on convolution is used according to the trajectory matrix of the same test data of multiple batches of products to obtain the difference between a single product and other products.

[0015] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention only performs correlation analysis on the provided data, quickly screening abnormal data in the testing process of multiple batches, small batches, and high-complexity products, and solves the problem that it is difficult to form a unified data analysis standard for different categories of products; (2) The present invention uses the idea of ​​data trajectory to abstract each column of indicators and time in the data into a two-dimensional trajectory, and abstracts multi-indicator, multi-product, and multi-batch data into high-dimensional space trajectories for description, thus solving the problem of unified description of multi-time, multi-individual, and multi-dimensional complex data; (3) The present invention applies the trajectory distance algorithm to perform multi-batch distance calculation on the data abstracted as multi-dimensional trajectories, and quantifies the relationship and trend of the test data using distance, thus solving the problem of unifying the dimensions of different indicators. (4) The present invention supplements the dynamic convolution-like algorithm and implements round-robin comparison in the trajectory distance algorithm process. By finding the minimum value of the trajectory distance within a certain period of time as the final result, the time shift error caused by the misalignment of the time dimension can be offset, and the problem of false errors caused by the time misalignment of test data of different individuals and batches of products can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 It is a flow chart of a data screening method for recording convolutional trajectory distances provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to be able to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0018] Figure 1 : is a flow chart of a data screening method for recording convolutional trajectory distances provided by an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps: Linearize the data of preset batches of multiple products to obtain a data trajectory matrix; Distinguish between the trajectory matrix of the same test data of multiple batches of products and the trajectory matrix of the single test data of a single product in the data trajectory matrix; The difference between a single product and other products is obtained based on the trajectory matrix of the same test data of multiple batches of products; Filter out abnormal data products in multiple batches of products based on the difference between a single product and other products and the preset product abnormality threshold; Obtain the abnormal impact of performance data based on the single test data trajectory matrix of a single product; According to the abnormal impact of performance data and the preset abnormal threshold of performance indicators Screen out abnormal performance indicators; Based on the abnormal data products and abnormal performance indicators in multiple batches of products, abnormal products are obtained and specific abnormal performance indicators are located.

[0019] The data of preset batches of multiple products are linearized to obtain a data trajectory matrix, including: collecting the performance data of each indicator of each product, and calibrating the collected performance data of each indicator according to time to form a data table; according to the time series, each performance test data in the data table forms a time-performance data trajectory; the product performance data corresponding to the same test process are placed in the same coordinate system to form a data trajectory matrix.

[0020] According to the trajectory matrix of the same test data of multiple batches of products, the convolution-based trajectory distance algorithm is used to obtain the difference between a single product and other products.

[0021] The difference between a single product and other products is obtained by the following formula: ; in, For the The difference between a product and other products, For the The products are in the data table The indicators listed are the same as those of this batch of products except Other products besides products are in List the minimum similarity of trajectories of the same indicator, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, For the first moment, is the total number of products in this batch, For the second moment, is the number of columns of the total performance index of the product, The product serial number.

[0022] It is obtained by the following formula:

[0023] in, To consider the time shift error Performance and The similarity between performance, For the Products in A collection of column indicator values, For this batch of products except Other products besides products are in A collection of column indicator values, For The sequence introduces displacement, is the adjustable displacement threshold, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, is the third moment, m is the Products in The total time of the column metric test.

[0024] If the difference between a single product and other products is greater than the preset product anomaly threshold, the product is an abnormal data product.

[0025] The abnormal impact of performance data is obtained by the following formula: ; in, is the abnormal impact of performance data, is the starting time of the time period partition, is the end time of the partition according to the time period, is the column position number of the performance indicator, is the total number of columns of performance indicators, Preset time in time zone. for List The indicator value at the moment, for List arrive The average value of the indicator at the time, is the total number of products in this batch, For the fourth moment, For the The product performance indicator matrix at all times, It is the average value matrix of product performance indicators.

[0026] Specifically, the method comprises the following steps: S1. Input product data.

[0027] S2. Linearize the product data to obtain a complex trajectory matrix describing the performance of a single product.

[0028] S21. In different stages of development such as simulation, debugging, and environmental testing, data of various indicators are usually collected according to time, and the collected data is calibrated according to time to form a data table in .csv format, as shown in the following table: Table 1 Collection format of data to be screened

[0029] S22. According to the time series, each performance test data can form a time-performance data trajectory, and the distance between trajectories can describe the differences between indicator data; S23. Product performance data corresponding to the same test process can be placed in the same coordinate system to form a data trajectory matrix; S3. Distinguish the trajectory matrix of the same test data of multiple batches of products from the trajectory matrix of a single test data of a single product.

[0030] S4. Apply the trajectory distance algorithm to the trajectory matrix of the same test data of multiple batches of products to calculate the difference between a single product and other products.

[0031] S41. For the test data of the same category of different products with a total batch size of p, the data information is saved in a matrix list in csv format. For each product ( ) is stored in the m-th time (row) Performance indicators (columns) in matrix form, The total number of performance indicators (columns) of a single product in this batch, product ( ) and other products' matrix test data differences can be describe; S42, quantitatively describe the difference between each two columns of data. First, the selected two columns of performance indicator data are defined as: ; in, is the value at time 1 (row 1) Column indicator value, For time 2 (row 2) Column indicator value, is the value at time 3 (row 3) Column indicator value, is the value at time m (row m) Column indicator value, is the value at time 1 (row 1) Column indicator value, For time 2 (row 2) Column indicator value, is the value at time 3 (row 3) Column indicator value, is the value at time m (row m) Column indicator value.

[0032] S43, data may also face time shift errors in the performance trajectory curve, which can be eliminated by integrating the convolution calculation process. The sequence is processed to eliminate the potential time shift error caused by time misalignment. Sequence-Introduced Displacement , and set the displacement threshold make , then the displacement is included New sequence It can be defined as:

[0033] S44, therefore, in the case of the above time-shift error, if the calculation is simply based on the trajectory distance, a gray inherent trajectory error will appear, but this error does not mean that the two curves have different trajectory forms. In order to eliminate the time-shift error, the present invention introduces displacement to the data and uses a convolution-like round-robin method to find the minimum trajectory distance. Specifically, the interval is the shift threshold. In the interval, the new sequence In a convolutional form, Calculate the similarity:

[0034] S45, and The minimum similarity between can be defined as:

[0035] S46. In the actual screening of abnormal products, we hope to locate the specific performance indicators with problems. Therefore, we use grouping based on different attributes, sharding data by time series, and calculating the mean square error within the same attribute and shard, and then normalizing it. We find the abnormal device attributes that change dramatically in a short period of time and mark them as abnormal. Specifically, we define a single product ( ) and the overall difference in the data matrix of other products ,but:

[0036] S5. Filter out abnormal data products through product abnormality thresholds.

[0037] S51. Set the product abnormality judgment threshold to E according to specific product characteristics; S52, through the preset product abnormality threshold E, when When the product ( ) is an abnormal product; S6. Calculate the impact of abnormal performance data on a single product test data trajectory matrix.

[0038] S61. Since the product test time is uncontrollable, in order to reduce the complexity of the calculation, the time period is If the test matrix is ​​partitioned by rows, the performance index of a certain q (column) will cause anomalies The influence quantity is defined as ; S62, resulting in a single product ( ) with the overall anomaly of the test data matrix of other products The specific performance indicators are The influence amount can be calculated by the following formula:

[0039] S7. Filter out specific abnormal performance indicators within a certain time period through performance abnormality thresholds.

[0040] S71. Set performance indicator abnormal thresholds based on specific product characteristics ; S72: Therefore, the preset performance indicator abnormal threshold is used ,when When you are in the area The performance index of q (column) is the product ( ) The main cause of the abnormality; S8. Summarize the abnormal product screening information, identify the abnormal products and locate the specific abnormal performance indicators, and filter out the abnormal data.

[0041] The present embodiment also provides a data screening system for recording convolution-like trajectory distances, the system comprising: a first module, used to linearize the data of preset batches of multiple products to obtain a data trajectory matrix; a second module, used to distinguish the trajectory matrix of the same test data of multiple batches of products from the trajectory matrix of single test data of a single product in the data trajectory matrix; a third module, used to obtain the difference between a single product and other products based on the trajectory matrix of the same test data of multiple batches of products; a fourth module, used to screen out abnormal data products in multiple batches of products based on the difference between the single product and other products and a preset product abnormality threshold; a fifth module, used to obtain the abnormal impact of performance data based on the trajectory matrix of single test data of a single product; a sixth module, used to screen out abnormal performance indicators based on the abnormal impact of performance data and a preset performance indicator abnormality threshold; a seventh module, used to obtain abnormal products and locate specific abnormal performance indicators based on the abnormal data products and abnormal performance indicators in multiple batches of products.

[0042] This embodiment only performs correlation analysis on the provided data to quickly screen abnormal data in the testing process of multiple batches, small batches, and high-complexity products, thereby solving the problem that it is difficult to form a unified data analysis standard for products of different categories; this embodiment uses the idea of ​​data trajectory to abstract each column of indicators and time in the data into a two-dimensional trajectory, and abstracts multi-indicator, multi-product, and multi-batch data into high-dimensional space trajectories for description, thereby solving the problem of unified description of multi-time, multi-individual, and multi-dimensional complex data; this embodiment applies a trajectory distance algorithm to perform multi-batch distance calculation on the data abstracted as multi-dimensional trajectories, and quantifies the relationship and trend of the test data using distance, thereby solving the problem of unified dimensioning of different indicators; this embodiment supplements the dynamic convolution-like algorithm, implements round-robin comparison in the process of the trajectory distance algorithm, and finds the minimum value of the trajectory distance within a certain period of time as the final result, thereby offsetting the time shift error caused by the misalignment of the time dimension, thereby solving the problem of false errors caused by the time misalignment of test data of different individuals and batches of products.

[0043] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A data screening method for recording convolutional trajectory distances, characterized in that include: Linearize the data of preset batches of multiple products to obtain a data trajectory matrix; Distinguish between the trajectory matrix of the same test data of multiple batches of products and the trajectory matrix of the single test data of a single product in the data trajectory matrix; The difference between a single product and other products is obtained based on the trajectory matrix of the same test data of multiple batches of products; Filter out abnormal data products in multiple batches of products based on the difference between a single product and other products and the preset product abnormality threshold; Obtain the abnormal impact of performance data based on the single test data trajectory matrix of a single product; Filter out abnormal performance indicators based on the abnormal impact of performance data and the preset abnormal threshold of performance indicators; Based on the abnormal data products and abnormal performance indicators in multiple batches of products, abnormal products are obtained and specific abnormal performance indicators are located.

2. The data screening method for recording convolutional trajectory distance according to claim 1 is characterized in that: The data trajectory matrix obtained by linearizing the preset batch multi-product data includes: Collect the performance data of each indicator of each product, and calibrate the collected performance data of each indicator according to time to form a data table; According to the time series, each performance test data in the data table forms a track of time-performance data; The product performance data corresponding to the same test process are placed in the same coordinate system to form a data trajectory matrix.

3. The data screening method for recording convolutional trajectory distance according to claim 1, characterized in that: According to the trajectory matrix of the same test data of multiple batches of products, the convolution-based trajectory distance algorithm is used to obtain the difference between a single product and other products.

4. The data screening method for recording convolutional trajectory distance according to claim 1 or 3, characterized in that: The difference between a single product and other products is obtained by the following formula: ； in, For the The difference between a product and other products, For the The products are in the data table The indicators listed are the same as those of this batch of products except Other products besides products are in List the minimum similarity of trajectories of the same indicator, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, For the first moment, is the total number of products in this batch, For the second moment, is the number of columns of the total performance index of the product, The product serial number.

5. The data screening method for recording convolutional trajectory distance according to claim 4, characterized in that: It is obtained by the following formula: in, To consider the time shift error Performance and The similarity between performance, For the Products in A collection of column indicator values, For this batch of products except Other products besides products are in A collection of column indicator values, For The sequence introduces displacement, is the adjustable displacement threshold, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, For Column indicator The indicator value at the moment, is the third moment, m is the Products in The total time of the column metric test.

6. The data screening method for recording convolutional trajectory distance according to claim 4, characterized in that: If the difference between a single product and other products is greater than the preset product anomaly threshold, the product is an abnormal data product.

7. The data screening method for recording convolutional trajectory distance according to claim 1, characterized in that: The abnormal impact of performance data is obtained by the following formula: ； in, is the abnormal impact of performance data, is the starting time of the time period partition, is the end time of the partition according to the time period, is the column position number of the performance indicator, is the total number of columns of performance indicators, Preset time in time zone. for List The indicator value at the moment, for List arrive The average value of the indicator at the time, is the total number of products in this batch, For the fourth moment, For the The product performance indicator matrix at all times, It is the average value matrix of product performance indicators.

8. A data screening system for recording convolutional trajectory distances, characterized in that include: The first module is used to perform linear processing on the data of preset batches of multiple products to obtain a data trajectory matrix; The second module is used to distinguish between the trajectory matrix of the same test data of multiple batches of products and the trajectory matrix of the single test data of a single product in the data trajectory matrix; The third module is used to obtain the difference between a single product and other products based on the trajectory matrix of the same test data of multiple batches of products; The fourth module is used to screen out abnormal data products in multiple batches of products based on the difference between a single product and other products and a preset product abnormality threshold; The fifth module is used to obtain the abnormal impact of performance data based on the single test data trajectory matrix of a single product; The sixth module is used to filter out abnormal performance indicators according to the abnormal impact of performance data and the preset abnormal threshold of performance indicators; The seventh module is used to obtain abnormal products and locate specific abnormal performance indicators based on abnormal data products and abnormal performance indicators in multiple batches of products.

9. The data screening system for recording convolutional trajectory distance according to claim 8, characterized in that: The data trajectory matrix obtained by linearizing the preset batch multi-product data includes: Collect the performance data of each indicator of each product, and calibrate the collected performance data of each indicator according to time to form a data table; According to the time series, each performance test data in the data table forms a track of time-performance data; The product performance data corresponding to the same test process are placed in the same coordinate system to form a data trajectory matrix.

10. The data screening system for recording convolutional trajectory distance according to claim 8, characterized in that: According to the trajectory matrix of the same test data of multiple batches of products, the convolution-based trajectory distance algorithm is used to obtain the difference between a single product and other products.

Images