A vehicle experimental data impurity cleaning method and device based on working condition clustering and rule model

Abstract

The application discloses a vehicle experimental data impurity cleaning method and device based on working condition clustering and a rule model, and relates to the field of data cleaning. The application overcomes the two challenges of disconnection between actual working conditions and data cleaning steps and insufficient precision caused by excessively wide data cleaning granularity. The method comprises the following steps: step 1, automatic labeling of working condition labels based on clustering; step 2, application of a classification model based on the labeled working condition label data of the clustering in step 1 to obtain classification rules, application of the classification rules to extract a rule model, and completion of establishment of the rule model; step 3, inspection of to-be-cleaned data by using the rule model; step 4, cleaning processing of data meeting the rule model, storage of the cleaned data, and storage of the cleaned data; and step 5, cleaning processing of data not meeting the rule model, buffering of the cleaned data, and updating of the rule model. The application is also suitable for application of clustering rule information to the field of data cleaning.

Description

Technical Field

[0001] This invention relates to the field of data cleaning technology, specifically to a method and apparatus for cleaning impurities from vehicle experimental data based on working condition clustering and rule models. Background Technology

[0002] In the automotive industry, evaluating vehicle performance is crucial, impacting comfort, fuel economy, and safety. With continuous advancements in automotive technology, vehicle performance testing has gradually shifted from laboratory testing to operational performance experiments. These experiments provide a more comprehensive understanding of a vehicle's performance in real-world conditions, and are thus gaining increasing importance.

[0003] Performance testing is a method that simulates real-world driving environments. By testing vehicles under specific conditions, a more comprehensive understanding of their performance under different driving states can be obtained. For example, in analyzing vehicle driving performance, the driving conditions are considered, vehicle speed is collected, and speed-time curves are plotted to analyze these conditions. Furthermore, driving conditions can be further subdivided into acceleration, deceleration, and constant speed driving conditions. Under different driving conditions, specific data is combined to evaluate performance indicators such as power, stability, and fuel economy. Similarly, in analyzing vehicle steering conditions, parameters such as the steering pump drive mechanism, zero-axis drive mechanism, and steering wheel angle are collected and plotted to analyze steering conditions. These conditions can be further subdivided into left turns, right turns, straight driving, and sharp turns. Under different steering conditions, specific data is combined to evaluate performance indicators such as controllability and stability.

[0004] Most of the data from performance tests are collected by sensors. As the vehicle moves, sensors monitor various parameters during operation, such as local temperature, control component pressure, and speed, converting these into digital signals for storage and generating a large amount of time-series data. This time-series data is then analyzed through performance tests to understand the vehicle's various performance characteristics.

[0005] Before conducting experimental analysis on the data, it is necessary to perform certain preprocessing, i.e., data cleaning. Data cleaning refers to identifying and correcting errors in the dataset to improve data quality. In industrial production, the collected data is mostly sensor data. Except in special cases where sensors malfunction and collect erroneous data, the collected data is generally correct, and it is impossible to declare any data as erroneous. Correcting erroneous data is even more difficult. Simply considering the cleaning of erroneous data is insufficient to effectively improve the model's performance.

[0006] Sensor data contains some data points that, while their values ​​are generally valid, can negatively impact model performance if used directly. We need to clean this type of data. In practical applications, we primarily clean the following types of data impurities: First, with a large number of sensors, some sensors, such as those in temporarily stopped equipment or fault detection sensors in normal operation, maintain stable, unchanging data over extended periods. While this data is correct, it is meaningless for data analysis and can be safely deleted. This type of data impurity needs to be cleaned. Second, when equipment sensors have extremely high, fixed sampling frequencies—often several times or even tens of times higher than the changing frequency—the collected data often exhibits repetition within a certain timeframe. This redundancy wastes storage space and reduces analysis efficiency. While this data is correct, it can be compressed for better subsequent experimental analysis. This type of data impurity also needs to be cleaned.

[0007] Traditional vehicle test data impurity cleaning techniques have the following shortcomings:

[0008] I. There is a disconnect between actual operating conditions and data cleaning steps. Data cleaning aims to improve the efficiency and effectiveness of subsequent operating condition experimental analysis, and has a clear objective. Traditional data cleaning often overlooks this objective, failing to consider actual operating conditions and corresponding requirements during the data cleaning process. Operating condition application requirements indicate the type of data needed for experimental analysis, containing heuristic rules for data cleaning. Ignoring this information means missing a significant opportunity for performance improvement.

[0009] Second, the granularity of data cleaning is too broad and the precision is insufficient. Traditional data cleaning techniques mostly only consider filling in missing values, checking and correcting erroneous values, and deleting duplicate values. They only clean the data from the perspective of data correctness and cannot effectively improve the subsequent experimental analysis. More specific data cleaning techniques need to be developed based on the characteristics of the vehicle experimental data and the needs of the operating condition analysis. At the same time, traditional data cleaning often uses the same cleaning standard for all data. This method is logically clear but the effect is insufficient. In practical applications, different operating condition analyses may require different data and have different cleaning requirements. Using a single cleaning standard will lead to inefficient use of data in subsequent experimental analysis. Summary of the Invention

[0010] To overcome the challenges of the disconnect between actual working conditions and data cleaning steps, and the insufficient accuracy caused by overly broad data cleaning granularity, this invention proposes a method and apparatus for cleaning impurities from vehicle experimental data based on working condition clustering and rule models. The technical solution adopted includes:

[0011] Option 1: This invention proposes a method for cleaning impurities in vehicle experimental data based on working condition clustering and rule models. The method includes the following steps:

[0012] Step 1: Automated labeling of working conditions based on clustering;

[0013] Step 2: Based on the clustered working condition label data labeled in Step 1, apply the classification model to obtain its classification rules, and apply the classification rules to extract the rule model to complete the establishment of the rule model.

[0014] Step 3: Use the rule model described in Step 2 to verify the data to be cleaned;

[0015] Step 4: Clean the data that conforms to the rule model and store the cleaned data;

[0016] Step 5: Clean the data that does not conform to the rule model, and cache the cleaned data for updating the rule model.

[0017] Furthermore, a preferred embodiment is provided, wherein the method for automated labeling of working condition tags based on clustering in step 1 is as follows:

[0018] Step 1.1: Determine the key attribute set for cluster analysis;

[0019] Step 1.2: Using the key attribute set as features, select a clustering model to cluster the data and obtain the data clustering results;

[0020] Step 1.3: Obtain statistical information for each category;

[0021] Step 1.4: Based on the statistical information of each category, assign working condition labels to each category according to actual needs.

[0022] Furthermore, a preferred embodiment is provided, in step 2, a rule model is extracted from the clustered working condition label data labeled in step 1 using a rule set. Extended attribute set Attribute constraint set accomplish.

[0023] Furthermore, a preferred implementation method is provided, using a rule set. Extended attribute set Attribute constraint set The implementation method is as follows:

[0024] The rule set is obtained by combining a classification model and an IF THEN rule extraction model. Get the rule set Then, correlation analysis was performed on its attributes to expand the attribute set. In expanding the attribute set During the acquisition process, the association constraints between recorded attributes constitute the attribute constraint set. This allows for the establishment of a rule model.

[0025] Furthermore, a preferred implementation method is provided, in which the IF TEHN rule extraction model is implemented based on a decision tree.

[0026] Furthermore, a preferred embodiment is provided, wherein the method for cleaning the data conforming to the rule model in step 4 and storing the cleaned data is as follows:

[0027] Step 4.1: Obtain the working condition to which the data belongs based on the judgment result of the rule model;

[0028] Step 4.2: Obtain the extended attribute set for this working condition based on the rule model;

[0029] Step 4.3: Delete and clean up other attributes outside of this extended attribute set;

[0030] Step 4.4: Aggregate and sample the attributes involved in the extended attribute set and store the corresponding aggregated values.

[0031] Furthermore, a preferred embodiment is provided in which the method for cleaning data that does not conform to the rule model in step 5 is implemented by aggregation statistical analysis.

[0032] Option 2: A vehicle test data impurity cleaning device based on working condition clustering and rule model, the device comprising:

[0033] The labeling module is used for automated labeling of work conditions based on clustering.

[0034] The rule model building module is used to apply a classification model to obtain the classification rules based on the clustered working condition label data labeled by the annotation module, and then apply the classification rules to extract the rule model to complete the establishment of the rule model.

[0035] The verification module is used to apply a classification model to the labeled clustered working condition label data to obtain its classification rules, apply the classification rules to extract the rule model, and complete the establishment of the rule model.

[0036] The storage module is used to clean the data that conforms to the rule model and store the cleaned data.

[0037] The update module is used to clean up data that does not conform to the rule model, cache the cleaned data, and then use it to update the rule model.

[0038] Option 3: A computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method described in any one of Options 1.

[0039] Option 4: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in any one of Options 1.

[0040] The advantages of this invention are:

[0041] This invention presents a vehicle experimental data impurity cleaning method based on operating condition clustering and rule models, which cleverly integrates clustering analysis and rule models, and achieves adaptive processing of operating conditions. A comprehensive system is meticulously constructed that tightly integrates operating condition clustering rules with the data cleaning process, ensuring that clustering rule information can be effectively applied to the data cleaning process, guaranteeing that the cleaning operation is well-founded, highly customized, and highly accurate. This significantly improves the detail of data cleaning, thereby greatly enhancing the quality and reliability of the cleaned data.

[0042] This invention is also applicable to the field of applying clustering rule information to the data cleaning process. Attached Figure Description

[0043] Figure 1 This is a flowchart of the vehicle experimental data impurity cleaning method based on working condition clustering and rule model as described in Implementation Method 1.

[0044] Figure 2 This is a flowchart illustrating the application of the rule model in the vehicle experimental data impurity cleaning method based on working condition clustering and rule model as described in Implementation Method Eleven. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0046] Implementation Method 1: This implementation method provides a method for cleaning impurities in vehicle experimental data based on working condition clustering and rule models. The method includes the following steps:

[0047] Step 1: Automated labeling of working conditions based on clustering;

[0048] Step 2: Based on the clustered working condition label data labeled in Step 1, apply the classification model to obtain its classification rules, and apply the classification rules to extract the rule model to complete the establishment of the rule model.

[0049] Step 3: Use the rule model described in Step 2 to verify the data to be cleaned;

[0050] Step 4: Clean the data that conforms to the rule model and store the cleaned data;

[0051] Step 5: Clean the data that does not conform to the rule model, and cache the cleaned data for updating the rule model.

[0052] Implementation Method Two: This implementation method further defines the vehicle test data impurity cleaning method based on working condition clustering and rule models described in Implementation Method One. The method for automated labeling of working condition tags based on clustering in step 1 is as follows:

[0053] Step 1.1: Determine the key attribute set for cluster analysis;

[0054] Step 1.2: Using the key attribute set as features, select a clustering model to cluster the data and obtain the data clustering results;

[0055] Step 1.3: Obtain statistical information for each category;

[0056] Step 1.4: Based on the statistical information of each category, assign working condition labels to each category according to actual needs.

[0057] Implementation Method 3: This implementation method further defines the vehicle test data impurity cleaning method based on working condition clustering and rule models described in Implementation Method 1. In step 2, rule models are extracted from the working condition label data labeled in step 1 using a rule set. Extended attribute set Attribute constraint set accomplish.

[0058] Implementation Method Four: This implementation method further refines the vehicle test data impurity cleaning method based on working condition clustering and rule models described in Implementation Method Two, using rule sets... Extended attribute set Attribute constraint set The implementation method is as follows:

[0059] The rule set is obtained by combining a classification model and an IF THEN rule extraction model. Get the rule set Then, correlation analysis was performed on its attributes to expand the attribute set. In expanding the attribute set During the acquisition process, the association constraints between recorded attributes constitute the attribute constraint set. This allows for the establishment of a rule model.

[0060] Implementation Method 5: This implementation method further defines the vehicle test data impurity cleaning method based on working condition clustering and rule model described in Implementation Method 4. The IF TEHN rule extraction model is implemented based on decision tree.

[0061] Implementation Method Six: This implementation method further defines the vehicle test data impurity cleaning method based on working condition clustering and rule model described in Implementation Method One. In step 4, the data conforming to the rule model is cleaned, and the cleaned data is stored as follows:

[0062] Step 4.1: Obtain the working condition to which the data belongs based on the judgment result of the rule model;

[0063] Step 4.2: Obtain the extended attribute set for this working condition based on the rule model;

[0064] Step 4.3: Delete and clean up other attributes outside of this extended attribute set;

[0065] Step 4.4: Aggregate and sample the attributes involved in the extended attribute set and store the corresponding aggregated values.

[0066] Implementation Method Seven: This implementation method further defines the vehicle test data impurity cleaning method based on working condition clustering and rule model described in Implementation Method One. The method for cleaning data that does not conform to the rule model in step 5 is implemented by cluster statistical analysis.

[0067] Implementation Method 8: This implementation method proposes a vehicle test data impurity cleaning device based on working condition clustering and rule model. The device includes:

[0068] The labeling module is used for automated labeling of work conditions based on clustering.

[0069] The rule model building module is used to obtain classification rules based on the clustered working condition label data labeled by the annotation module, apply the classification model, extract the rule model, and complete the establishment of the rule model.

[0070] The verification module is used to verify the data to be cleaned using the rule model described in the rule model building module.

[0071] The storage module is used to clean the data that conforms to the rule model and store the cleaned data.

[0072] The update module is used to clean up data that does not conform to the rule model, cache the cleaned data, and then use it to update the rule model.

[0073] Implementation Method Nine: This implementation method provides a computer device, including a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the method described in any one of Implementation Methods One to Seven.

[0074] Implementation Method 10: This implementation method provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in any one of Implementation Methods 1 to 7.

[0075] Implementation Method Eleven: This implementation method provides an example, which is used to explain the above-described implementation methods one through eight. The specific example is as follows:

[0076] See Figures 1 to 2 This implementation method proposes an innovative vehicle operation and maintenance data impurity cleaning technology that cleverly integrates cluster analysis and rule models, and achieves adaptive processing based on operating conditions. A comprehensive system is carefully constructed that tightly integrates operating condition clustering rules with the data cleaning process, ensuring that clustering rule information can be effectively applied to the data cleaning process, guaranteeing that the cleaning operation is well-founded, highly customized, and highly accurate. Its core objective is to overcome two major challenges in current data cleaning processes: first, the disconnect between actual operating conditions and data cleaning steps; and second, insufficient accuracy caused by overly broad data cleaning granularity.

[0077] This implementation constructs a comprehensive system that tightly integrates operational condition clustering rules with the data cleaning process. In this system, clustering rule information is effectively applied to the data cleaning process, ensuring that the cleaning operations are not only well-founded but also highly customized and accurate. This design significantly improves the meticulousness of data cleaning, thereby greatly enhancing the quality and reliability of the cleaned data.

[0078] This implementation first trains a rule model, then uses this rule model to test the data to be cleaned, and finally performs corresponding data cleaning based on the different test results of the rule model. The overall process includes five steps: Step 1, automated labeling of work conditions based on clustering; Step 2, applying a classification model to the labeled work condition data to learn classification rules, and then applying a rule extraction model to extract corresponding rules, thus completing the establishment of the rule model; Step 3, using the rule model to test the data to be cleaned; Step 4, cleaning the data that conforms to the model rules and storing the cleaned data for subsequent experimental analysis; Step 5, cleaning the data that does not conform to the model rules and caching the cleaned data to periodically update the rule model.

[0079] Specifically, the following steps are included:

[0080] Step 1 involves automated labeling of operating conditions based on clustering. Using automated labeling technology reduces the manual cost of data labeling and improves the feasibility and effectiveness of the overall solution. The core objective of Step 1 is to obtain the operating condition labeling data. Due to the massive volume of vehicle sensor data and the high cost of labeling, a clustering model is used to automatically obtain clustered labeling data. After selecting the attribute set relevant to operating condition analysis, clustering is performed using these attributes, and each category is assigned a label related to the operating condition, thus achieving automated labeling of operating conditions. Through cluster analysis, labeled data containing rules is obtained.

[0081] Specifically, this can be achieved through the following steps: 1. Determine the key attribute set for operating condition analysis; 2. Using the key attribute set as features, select a clustering model to cluster the data and obtain the data clustering results; 3. Obtain important statistical information for each category; 4. Based on the statistical information of each category and combined with actual needs, assign meaningful operating condition labels to each category.

[0082] The aforementioned key attribute set A is defined as a set of attributes directly related to the operating conditions: For example, in vehicle driving condition analysis, the attribute directly related to the driving state is speed. Through the speed attribute, the driving condition can be divided into acceleration condition, deceleration condition, constant speed condition, etc. Therefore, the key attribute set for this type of driving condition analysis is {speed}. Similarly, in vehicle steering condition analysis, the attributes directly related to the steering state are the steering pump drive mechanism and the zero-axis drive mechanism. Through this attribute, the steering condition can be divided into different categories. Therefore, the key attribute set for this type of steering condition analysis is {steering pump drive mechanism, zero-axis drive mechanism}.

[0083] Based on the key attribute set, the clustering model can be targeted during the clustering process, enabling efficient and accurate segmentation of specific working conditions according to requirements. This allows for the utilization of rule information for different working conditions during subsequent data cleaning, linking data cleaning with working condition rules and improving data cleaning quality. After obtaining the clustering results, simply treating each category as a different working condition has some effect, but it is not intuitive and the use of subsequent labeled data is inconvenient. Additional operations are required: statistical analysis of the data in each category to obtain important statistical values ​​of the category, such as the mean, which can quickly grasp the distribution characteristics of each category, determine the working conditions represented by the category, and realize the labeling of working condition data. For example, after clustering, if data in category 1 has a higher average speed and data in category 2 has a lower average speed, then data belonging to category 1 can be labeled as "high-speed driving condition" and data belonging to category 2 can be labeled as "low-speed driving condition," thus achieving the labeling of data conditions. In addition, the data used to train the clustering model is dynamically updated, and the results of subsequent data cleaning can update the data used to train the model, so that the condition clustering rules can be updated as data arrives, strengthening the connection between data cleaning and condition clustering, and improving the overall adaptability of the model.

[0084] The core objective of step 2 is to extract a rule model from the work condition annotation data. The rule model consists of three parts: the rule set. Extended attribute set Attribute constraint set .

[0085] A rule set is defined as a collection of rules: Among them, a single rule The format is defined as follows: .in, The condition is a judgment on the data distribution space, which is a combination of a series of basic conditions, i.e. Each of these basic conditions It is a set of key attributes A certain attribute in The judgment, in the form of > , indicating the first The fundamental condition is the attributes in the key attribute set. The judgment is made based on whether the attribute is greater than the value. ; The conclusion is a judgment made on the corresponding working condition label under that condition, that is... ;rule Overall, it shows that in satisfying its At that time, the corresponding operating condition label for the data is .

[0086] An extended attribute set is defined as a set of attribute sets: ,in, For a collection of attributes { }, stores the operating conditions Analyze relevant attributes that are helpful. Furthermore, The source of attributes in the middle consists of two parts: those from the key attribute set and those related to... Helpful attributes for operating condition analysis : The remaining attribute set from the original data for this working condition Analyzing helpful attributes : .

[0087] An attribute constraint set is defined as a collection of attribute constraint sets: ,in, For attribute collections A series of constraints Indicates operating conditions. Corresponding extended attribute set In this context, certain attributes must satisfy specific association constraints. For example, constraints... It can be represented as "attribute" With attributes The correlation should be no less than 0.6. In practical applications, depending on the specific requirements, the degree of correlation between attributes can be measured using Pearson correlation coefficient, Spearman rank correlation coefficient, or similar methods.

[0088] The rule set can be obtained by combining classification models and IF THEN rule extraction models. In obtaining the rule set Then, correlation analysis was performed on the relevant attributes to expand the attribute set. In expanding the attribute set During the acquisition process, the association constraints between recorded attributes constitute the attribute constraint set. This allows for the establishment of a rule model.

[0089] After establishing the rule model, subsequent steps are taken using rule sets. It allows for quick identification of data conditions using only a few attributes, and then leverages the relevant information from those conditions for more refined data cleaning; by expanding the attribute set... It can clearly identify which attributes are helpful for the analysis of different operating conditions, ensuring that important information related to the operating condition analysis is not lost in the stored data; through attribute constraint sets It can further ensure that the stored data conforms to specific operating conditions, is helpful for experimental analysis, and has higher reliability.

[0090] The rule set was obtained through the above process. Extended attribute set Attribute constraint set This established a rule model, providing a theoretical basis for subsequent data cleaning: data can be cleaned based on the rule set. Quickly locate the operating conditions of the data, and then use the relevant information of those conditions to perform more refined data cleaning; expand the attribute set. The system expands the attributes to be stored, enabling the location of data conditions to be determined with fewer attributes, while ensuring that important information related to the condition analysis is not lost; it also introduces attribute constraint sets. It can further ensure that the stored data conforms to specific operating conditions, is helpful for experimental analysis, and has higher reliability.

[0091] In step 3, the incoming data is tested using a rule model. The test is based on whether the data belongs to a certain working condition and satisfies the association constraints under that condition. If the data passes the rule model test, it belongs to a known working condition and is helpful for experimental analysis; it should be stored subsequently. If the rule model test fails, the data may belong to an unknown emerging working condition, requiring more careful consideration.

[0092] To use a rule-based model, the data to be cleaned is evaluated. There are two possible outcomes: conforming to the model rules or not conforming. The subsequent data cleaning operations will differ depending on the outcome. Data conforming to the rule-based model is defined as data simultaneously meeting two conditions: Condition 1 is that for a given set of data… , This makes the data satisfy In Condition 2 is for items within the window that belong to the label. All data , If condition 1 is met, it means that the incoming data can be located to the current working condition according to the clustering rules; if condition 2 is met, it means that the data also meets the attribute constraints of its working condition. When the data meets both conditions 1 and 2, the data conforms to the model rules; otherwise, the data does not conform to the model rules.

[0093] Step 4 involves data cleaning of data that conforms to the model rules. This data conforms to the model rules and the specific operating condition, making it helpful for analyzing that condition and safe to store. This can be achieved through the following steps: 1. Obtain the operating condition to which the data belongs based on the rule model's judgment results; 2. Obtain the extended attribute set for that operating condition based on the rule model; 3. Delete and clean other attributes outside the extended attribute set; 4. Perform cluster sampling on the attributes involved in the extended attribute set and store the corresponding cluster values. During cluster sampling, it is necessary to save the number of data repetitions, the mean, and other aggregated statistical values. These statistical values ​​can serve as additional information to assist subsequent operating condition analysis and prevent the loss of operating condition characteristic distribution information due to data impurity cleaning.

[0094] Since the data satisfies the model's rule constraints, steps 1 and 2 are guaranteed to yield results. This also demonstrates that during the data cleaning process: the clustering information of the rule model was utilized, and the working condition clustering rules provided detailed guidance for subsequent data cleaning, resulting in an organically unified overall framework and improved data cleaning quality; step 3 used relevant information from the rule model to retain only data helpful for working condition analysis, cleaning irrelevant attributes and improving data quality; step 4 performed aggregated sampling before data storage, efficiently cleaning redundant data impurities. Redundant data impurities are a frequent phenomenon in time-series data. Due to the extremely high sampling frequency of sensors, which can reach several times or even tens of times the data change frequency, the information density of the original data is low, and the storage efficiency is low, requiring compression. Compressed storage improves data storage efficiency while retaining aggregated values ​​to record data redundancy, without changing the data distribution characteristics or affecting subsequent experimental analysis.

[0095] Step 5 aims to improve the model's adaptability to changes in operating conditions by cleaning data that does not conform to the model's rules. This data contradicts the model's rules and all existing operating conditions; therefore, it cannot be stored and should be carefully considered, as it may indicate a change in the distribution of operating conditions. This can be achieved through the following two steps: 1. Using clustered statistical analysis, the data is screened to remove meaningless data impurities; 2. The remaining meaningful data is cached. When the cached data reaches a certain level, it is used to update the dataset in Step 1 of the overall framework.

[0096] In step 5, when the operating condition classification is stable and satisfactory, each data point should satisfy the rule model and will not proceed to this step. However, when the operating condition rules change, the existing rule model, built on historical data, cannot adapt to the new operating conditions. Therefore, data that does not conform to the current rules and may belong to the new operating conditions requires additional processing and should not be arbitrarily deleted. First, it is necessary to determine whether the data that does not conform to the current rules has any value: using cluster statistical analysis, if a certain attribute remains constant over a long time window, it indicates that this part of the data is ineffective for data analysis and belongs to data impurities, which can be safely deleted. By filtering out data impurities, valid data that may be beneficial to updating the rule model is retained. Ultimately, this valid data will update the clustering training dataset in step 1, thereby achieving an overall model update and realizing operating condition adaptation.

[0097] In Step 1, meaningless data impurities refer to data whose values ​​have not changed within an extremely long time window, such as data collected by certain fault condition detection sensors. This type of data frequently appears in time series data, offers little help for operating condition analysis, and is therefore meaningless data impurity requiring appropriate cleaning. In Step 2, the clustering model training dataset from Step 1 of the overall framework will be updated. Specifically, when there are emerging operating conditions in the data to be cleaned that the existing rule model has not yet learned, this emerging operating condition data will eventually reach this fifth stage through the overall framework. Here, this part of the data is cached and used to update the clustering model training dataset from Step 1 of the overall framework. Using the updated dataset, the rule model is updated, learns the patterns of emerging operating conditions, and adapts to changes in operating conditions. This process demonstrates that data cleaning can update the rule model, improve the operating condition clustering effect, and enhance the model's adaptability to operating conditions. It proves the organic unity of the operating condition clustering rule-data cleaning overall framework and further demonstrates the advantages of this framework.

[0098] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0099] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A vehicle experimental data impurity cleaning method based on working condition clustering and rule model, characterized in that, The method comprises the following steps: Step 1, automatic labeling of the working condition label based on clustering; Step 2, applying a classification model to the labeled working condition label data based on step 1 to obtain the classification rule, applying the classification rule to extract a rule model, and completing the establishment of the rule model; Step 3, using the rule model of step 2 to test the data to be cleaned; Step 4, cleaning the data that meets the rule model, and storing the cleaned data; Step 5, cleaning the data that does not meet the rule model, and buffering the cleaned data for updating the rule model; The method for automatic labeling of the working condition label based on clustering in step 1 is as follows: Step 1.1, determining the key attribute set of the clustering working condition analysis; Step 1.2, selecting a clustering model to cluster the data based on the key attribute set as the feature to obtain the data clustering result; Step 1.3, obtaining the statistical information of each category; Step 1.4, assigning a working condition label to each category according to the statistical information of each category and combining the actual demand; By a rule set , an extended attribute set , an attribute constraint set The method is implemented as follows: The classification model and the IF THEN rule extraction model are combined to obtain a rule set , obtain a rule set The attributes are analyzed and expanded, and an expanded attribute set is obtained , the expanded attribute set The association constraints between the attributes in the obtaining process are recorded to form an attribute constraint set , so that the rule model is established The IFTEHN rule extraction model is implemented based on a decision tree; The method for cleaning the data that meets the rule model in step 4 and storing the cleaned data is as follows: Step 4.1, obtaining the working condition to which the data belongs according to the judgment result of the rule model; Step 4.2, obtaining the extended attribute set under the working condition according to the rule model; Step 4.3, deleting and cleaning other attributes outside the extended attribute set; Step 4.4, aggregating and sampling the attributes involved in the extended attribute set, and storing the corresponding aggregated value; The key attribute set A is defined as a set of attributes directly related to the working condition: In the vehicle driving condition analysis, the attribute directly related to the driving state is speed, and the driving condition is divided into acceleration condition, deceleration condition and constant speed condition through the speed attribute, i.e. the key attribute set of the driving condition analysis is {speed}; in the vehicle steering condition analysis, the attributes directly related to the steering state are the steering pump driving mechanism and the zero shaft driving mechanism, and the steering condition is divided differently through the steering attribute, i.e. the key attribute set of the steering condition analysis is {steering pump driving mechanism, zero shaft driving mechanism}.

2. The vehicle experimental data impurity cleaning method based on working condition clustering and rule model according to claim 1, characterized in that, Extracting rule model in step 2 based on the cluster labeled in step 1 , expanding attribute set , attribute constraint set implementation.

3. The method of claim 1, wherein the method is based on a condition-based clustering and rule model for vehicle experimental data impurity cleaning. The method for cleaning the data that does not meet the rule model in step 5 is implemented by an aggregation statistical analysis method.

4. A vehicle experimental data impurity cleaning device based on working condition clustering and rule model, characterized in that, The device is implemented based on the method of claim 1, and the device comprises: A labeling module for automatic labeling of the working condition label based on clustering; A rule model establishment module for applying a classification model to the labeled working condition label data to obtain the classification rule, applying the classification rule to extract a rule model, and completing the establishment of the rule model; A testing module for using the rule model of the rule model establishment module to test the data to be cleaned; A storage module for cleaning the data that meets the rule model, and storing the cleaned data; An updating module for cleaning the data that does not meet the rule model, buffering the cleaned data, and updating the rule model.

5. Computer device comprising a memory and a processor, characterized in that The memory stores a computer program, and when the processor executes the computer program stored in the memory, the processor executes the method of any one of claims 1 to 3.

6. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 3.

Images