A multi-source heterogeneous battery time series data standardization processing and analysis method
By constructing a standard metadata table, terminology mapping, variational autoencoder, and hidden Markov model, the standardization and quality assessment of time series data of multi-source heterogeneous lithium batteries were solved, enabling efficient utilization and hierarchical management of the data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUZHOU CYCLE INTELLIGENT TECHNOLOGY CO LTD
- Filing Date
- 2026-03-13
- Publication Date
- 2026-06-09
AI Technical Summary
Existing technologies for processing time-series data from multi-source heterogeneous lithium batteries suffer from problems such as complex data sources, chaotic field naming, inconsistent time formats, difficulty in data fusion and analysis, and data quality evaluation methods that rely on subjective settings, have poor generalization ability, are difficult to capture abnormal data, and produce erroneous segmentation under complex operating conditions.
A standardization method for time-series data from multi-source heterogeneous batteries is adopted. By constructing a standard metadata table, terminology mapping, variational autoencoder, and hidden Markov model, data standardization, anomaly detection, and segmentation are achieved, and data quality is evaluated by combining information entropy.
It has achieved standardized processing and quality assessment of multi-source heterogeneous lithium battery data, improved the data utilization effect, built a database of data of different quality, and supported hierarchical utilization.
Smart Images
Figure CN122173770A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of battery data processing, specifically to a method for standardizing and analyzing time-series data of multi-source heterogeneous batteries. Background Technology
[0002] With the rapid development of the new energy industry, the data on the entire life cycle of lithium-ion batteries is growing explosively. However, in the actual application and modeling of this data, we face serious problems of "multi-source heterogeneity" and "inconsistent quality".
[0003] First, the data sources are extremely complex. Real-world vehicle data typically follows the GB / T 32960 standard, energy storage data originates from EMS or SCADA systems, while laboratory data comes from testing equipment from different manufacturers such as Arbin and Maccor. The naming of these data fields is inconsistent (for example, voltage fields may be labeled "Volt," "U_cell," or "single cell voltage"), and the time formats vary, making direct fusion analysis impossible.
[0004] Secondly, existing technologies have significant bottlenecks in in-depth data processing and value assessment. In the data cleaning and operating condition segmentation stages, existing methods heavily rely on fixed physical thresholds set by experts (such as fixed voltage extremes or current magnitude settings). This rule-based state machine not only struggles to capture the hidden dirty data in the early stages of battery micro-short circuits, which are "within normal ranges but with abnormal distribution patterns," but also easily generates a large amount of missegmented and fragmented data under complex and fluctuating operating conditions (such as frequent start-stop cycles in congested urban areas).
[0005] In terms of data quality assessment, existing assessment methods mostly rely on subjectively set fuzzy rule bases or simple linear weighting, which not only have poor generalization ability, but also ignore the objective information entropy of the data distribution itself, making it difficult to provide an objective, scientific and adaptive comprehensive assessment of the quality of different batches of data. Summary of the Invention
[0006] To address the aforementioned shortcomings of existing technologies, this invention provides a method for standardized processing and analysis of time-series data from multi-source heterogeneous batteries.
[0007] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: A method for standardizing and analyzing time-series data of multi-source heterogeneous batteries is provided, which includes the following steps: S1: Input multi-source heterogeneous data related to lithium batteries, construct a standard metadata table for the entire life cycle of lithium batteries as a standard metadata target, extract effective metadata from the multi-source heterogeneous data based on the standard metadata target, and calculate the information acquisition degree coefficient of the data. ; S2: Based on the lithium battery data terminology mapping dictionary, standardize the naming of heterogeneous and non-standard fields in multi-source heterogeneous data, and count the number of fields that have completed the standard naming. Parse various time strings in multi-source heterogeneous data, calculate the acquisition time difference between adjacent data, and calculate the average sampling frequency of the data. ; Based on the number of fields and average sampling frequency Calculate the combined coefficient of field coverage and frequency. It outputs a CSV data stream with standardized fields and time alignment; S3: Construct a variational autoencoder (VAE) to extract multidimensional time-series features of batteries from the CSV data stream, obtain the original input vector, and map it to the low-dimensional distribution of the hidden layer to obtain the reconstructed vector; calculate the weighted mean square error between the reconstructed vector and the original output vector, locate the points of data reconstruction anomalies, record the anomaly type labels, and calculate the data availability coefficient. ; S4: A Hidden Markov Model (HMM) is used to describe the hidden state sequence of the battery's actual operating conditions. Continuous clean time-series data is used as input, and the Viterbi algorithm is employed to find a hidden state path that maximizes the global observation probability. The timestamps of state transitions are identified, and the clean time-series data is segmented to obtain data segments for charging, discharging, and resting states. A charging state database, a discharging state database, and a resting state database are then constructed, and the data quality coefficient is calculated. ; S5: Extract deep state segments from the charging state database, discharging state database, and idle state database, and calculate the actual power throughput using the ampere-hour integration method. Combined with the theoretical charge changes corresponding to the deep state segments Calculate the logical reliability index coefficient of the data. ; S6: Information Acquisition Degree Coefficient Field coverage and frequency combined index coefficient Data availability coefficient Data quality coefficient and logic reliability index coefficient As evaluation features, the information entropy of each evaluation feature is calculated to obtain the objective weight of each evaluation feature. The objective weight is then used to calculate the comprehensive score of data quality to evaluate the quality of the data.
[0008] Further, step S1 includes: S11: Input BMS data of actual vehicles related to lithium batteries, EMS logs of energy storage power stations and laboratory test reports as multi-source heterogeneous data; S12: Construct a standard metadata table for the entire life cycle of lithium batteries as a standard metadata target. The standard metadata target contains basic static information, production traceability information and dynamic configuration information of lithium batteries. S13: Extract effective metadata from multi-source heterogeneous data based on standard metadata targets, and statistically analyze the proportion of required metadata items defined in the standard metadata targets appearing in the multi-source heterogeneous data to obtain the information acquisition degree coefficient of the data. ; ; in, This represents the number of valid metadata items actually extracted from multi-source heterogeneous data. For the required metadata items defined in the standard metadata target, i The index number of each required metadata item in the standard metadata target. For the first i The weight coefficient of each required metadata item.
[0009] Further, step S2 includes: S21: Based on the lithium battery data terminology mapping dictionary, heterogeneous fields in multi-source heterogeneous data are uniformly mapped to standard field names. For non-standard fields not included in the lithium battery data terminology mapping dictionary, the numerical distribution characteristics of 10,000 rows of data before and after the non-standard field are read. The meaning of the non-standard field is inferred based on the data distribution characteristics, and standard field names are generated. The number of fields successfully named as standard fields is counted. ; S22: Parse various time strings in multi-source heterogeneous data, convert all time columns into Unix timestamps, use Unix timestamps as the unique primary key index of the subsequent time series database, eliminate the time zone differences in the collection of multi-source heterogeneous data, and ensure alignment on the same time axis. S23: Calculate the acquisition time difference between adjacent data in multi-source heterogeneous data. , For the first i +1 row of data collection time, For the first i Data collection time for rows; S24: Statistical satisfaction of sampling time difference The proportion of data rows, and the average sampling frequency of the data are calculated. ; ; in, m The data is numbered based on the sampling time difference. MThe number of data points representing the sampling time difference. For the first m Each sampling time difference; S25: Based on the average sampling frequency and number of fields Calculate the combined coefficient of field coverage and frequency of the data. It outputs a CSV data stream with standardized fields and time alignment; ; in, The total number of standard fields required for modeling, For the target ideal frequency, Let be the weight coefficient, and satisfy... .
[0010] Further, step S3 includes: S31: Construct a variational autoencoder (VAE), including an encoder and a decoder. The encoder extracts multidimensional temporal features of the battery from the CSV data stream. x The original input vector is obtained. And the multidimensional time-series characteristics of the battery x Low-dimensional distribution mapped to hidden layers middle, z These are latent variables; S32: Decoder from low-dimensional distribution Mid-sampled latent variables z and hidden variables z Reconstruct the original input This yields the reconstruction vector corresponding to each standard field. ; S33: Calculate the reconstructed vector Compared with the original output vector Weighted mean square error between ; ; in, M The total number of standard fields, For the first j The weighting coefficients of each standard field. For the first j The first standard field corresponding to the first k One time-series feature, For the first j The first standard field corresponding to the first k One reconstructed temporal feature; S34: Weighted mean square error for each time series feature Real-time monitoring is performed, if the weighted mean square error is... Exceeding the set deviation threshold If the data reconstruction at that point in time is abnormal, then the weighted mean square error is used to determine the anomaly. The standard field corresponding to the maximum value is used to locate the specific physical dimension and anomaly point where the reconstruction anomaly occurred. S35: Set the abnormal point to null or insert an abnormal state, and record the abnormal point's abnormal type label in the metadata database; S36: Based on weighted mean square error Calculate the data availability coefficient ; ; in, K The number of features for the multidimensional time-series characteristics of the battery. For indicator functions, when satisfying When the condition is met, the indicator function is 1; otherwise, the indicator function is 0. S37: Perform smoothing repair on the data at outlier points in the CSV data stream to obtain continuous, clean time-series data.
[0011] Further, step S4 includes: S41: Using Hidden Markov Models (HMMs) to describe the hidden state sequence of the battery under actual operating conditions. S And this is manifested through the observed sequence; hidden state sequence , Indicates a sequence of charging status tags. Represents the discharge state tag sequence, The sequence represents the stationary state label sequence; the observation sequence is the current directly measured by external sensors. I and sliding variance ; S42: Set the state transition probability, take the current time series data from the continuous clean time series data as input, and use the Viterbi algorithm to find a hidden state path that maximizes the global observation probability. , n The number of hidden state sequences. For the first n A sequence of hidden states; S43: Based on the state label sequence in the hidden state sequence, identify the timestamp of the state transition, and eliminate invalid segments caused by short-term disturbances that are less than the minimum working condition duration threshold by setting a minimum working condition duration threshold. S44: Based on the timestamp of the state transition, the continuous clean time series data is divided into independent data segments to obtain data segments of the charging state, the discharging state, and the static state; and the effective metadata corresponding to the data segments of the charging state, the discharging state, and the static state is extracted to obtain the effective metadata of the charging state, the discharging state, and the static state, respectively. S45: Valid metadata for charging status, discharging status, and resting status is named and packaged in the format of operating condition type_start time_temperature.csv and stored in the charging status database, discharging status database, and resting status database respectively. S46: Based on each segmented charging state data segment and discharging state data segment, calculate the change in state of charge at the beginning and end of each data segment. ,like If the condition is met, the data segment is determined to be a depth state segment; otherwise, the data segment is determined to be a non-depth state segment. These represent the charge states corresponding to the beginning and end of the data segment, respectively. The threshold for the change in state of charge is set. S47: Based on the sum of the durations of all depth state segments The sum of battery runtime Calculate the gold content coefficient of the data ; ; in, W This represents the number of depth state fragments contained in the data.
[0012] Further, step S5 includes: S51: Extract deep state segments from the charging state database, discharging state database, and idle state database, and then use the corresponding current timing data in the deep state segments. The actual electricity throughput was calculated using the ampere-hour integration method. ; ; in, These are the start and end times of the depth state segment, respectively. S52: Based on the state-of-charge data segments at the beginning and end of the deep state segment reported by the Battery Management System (BMS). Calculate the theoretical change in electricity , This refers to the battery's rated capacity. S53: If the power throughput Changes in theoretical electricity If the deviation exceeds the set threshold, it is determined that there is a physical-logical inconsistency in the depth state segment, and the time period corresponding to the depth state segment is marked as a potential sensor failure point or algorithm failure point. S54: Based on power throughput Changes in theoretical electricity Calculate the logical reliability index coefficient of the data ; .
[0013] Further, step S6 includes: S61: Information Acquisition Degree Coefficient Field coverage and frequency combined index coefficient Data availability coefficient Data quality coefficient and logic reliability index coefficient As evaluation features, these features are encapsulated into a five-dimensional feature vector. ; S62: Perform range standardization on the evaluation features, and then calculate the information entropy of each evaluation feature. , v The evaluation feature is numbered; according to The proportion determines the objective weight of each evaluation feature. Utilizing objective weights Calculate the overall score for data quality. Y ; ; S63: If If the data is deemed to be of high quality, it will be stored in a high-value modeling library for algorithm engineers to directly use in training models. like If the data is deemed to be of medium quality, it will be stored in a regular analysis database for statistical analysis. like If the data is deemed low-quality, it will be stored in the original archive.
[0014] The beneficial effects of this invention are as follows: This invention can standardize the processing of heterogeneous battery data from multiple sources, and perform automated cleaning and smoothing operations on data with poor quality, thereby improving the subsequent utilization of heterogeneous battery data, conducting comprehensive quality assessment of the data, constructing a database of data with different quality, and realizing the hierarchical utilization of data with different quality. Attached Figure Description
[0015] Figure 1 A flowchart for the standardization processing and analysis method of time series data of multi-source heterogeneous batteries. Detailed Implementation
[0016] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.
[0017] like Figure 1 As shown, a method for standardizing and analyzing time-series data of multi-source heterogeneous batteries includes the following steps: S1: Input multi-source heterogeneous data related to lithium batteries, construct a standard metadata table for the entire life cycle of lithium batteries as a standard metadata target, extract effective metadata from the multi-source heterogeneous data based on the standard metadata target, and calculate the information acquisition degree coefficient of the data. Step S1 specifically includes the following steps: S11: Input BMS data of actual vehicles related to lithium batteries, EMS logs of energy storage power stations and laboratory test reports as multi-source heterogeneous data; When accessing multi-source heterogeneous data, the system first builds a compatibility layer and accesses three typical data sources through API interfaces, direct database connections, or file stream parsing: real vehicle BMS data that conforms to the GB / T 32960 national standard, historical operation logs from the energy storage power station EMS system, and raw reports (such as .res and .xls formats) exported from high-precision testing equipment in the laboratory.
[0018] S12: Construct a standard metadata table for the entire life cycle of lithium batteries as a standard metadata target. The standard metadata target contains basic static information, production traceability information and dynamic configuration information of lithium batteries. While accessing heterogeneous data from multiple sources, the system pre-configures a "Lithium Battery Full Lifecycle Standard Metadata Table." It's important to clarify that this table is not raw data, but rather a built-in "verification checklist" or "template" used to define the "identity information" that qualified lithium battery data should possess. This table defines in detail the three-layer information structure that lithium battery data must have: The first layer is basic static information (Level 1), including battery type (such as NCM811 ternary lithium or LFP lithium iron phosphate), rated capacity, rated voltage and pack topology (number of series and parallel connections). The second layer is production traceability information (Level 2), including cell manufacturer, module batch number and manufacturing date; The third layer is dynamic configuration information (Level 3), which covers sampling frequency, BMS hardware version, and cooling method.
[0019] S13: Extract effective metadata from multi-source heterogeneous data based on standard metadata targets, and statistically analyze the proportion of required metadata items defined in the standard metadata targets appearing in the multi-source heterogeneous data to obtain the information acquisition degree coefficient of the data. ; ; in, This represents the number of valid metadata items actually extracted from multi-source heterogeneous data. For the required metadata items defined in the standard metadata target, i The index number of each required metadata item in the standard metadata target. For the first i The weight coefficient of each required metadata item.
[0020] The index number serves as a unique index number for each required metadata item, used to uniquely identify each specific required metadata item defined in the standard metadata target. Weighting coefficient. Based on the practical application value of lithium battery data processing / modeling, an importance weight is set for each necessary metadata item to reflect the differentiated impact of missing / extracted different necessary metadata items, upgrading the information acquisition degree coefficient from "simple quantitative proportion" to "weighted effective proportion", which is more in line with engineering practice.
[0021] To quantify the completeness of accessed data, this invention defines an information acquisition degree coefficient. Indicators; The system will automatically scan the header information or associated description files of multi-source heterogeneous data and calculate the proportion of non-empty fields in the above standard metadata targets.
[0022] S2: Based on the lithium battery data terminology mapping dictionary, standardize the naming of heterogeneous and non-standard fields in multi-source heterogeneous data, and count the number of fields that have completed the standard naming. Parse various time strings in multi-source heterogeneous data, calculate the acquisition time difference between adjacent data, and calculate the average sampling frequency of the data. ; Based on the number of fields and average sampling frequency Calculate the combined coefficient of field coverage and frequency. It outputs a CSV data stream with standardized fields and time alignment.
[0023] Step S2 specifically includes the following steps: S21: Based on the lithium battery data terminology mapping dictionary, heterogeneous fields in multi-source heterogeneous data are uniformly mapped to standard field names. For non-standard fields not included in the lithium battery data terminology mapping dictionary, the numerical distribution characteristics of 10,000 rows of data before and after the non-standard field are read. The meaning of the non-standard field is inferred based on the data distribution characteristics, and standard field names are generated. The number of fields successfully named as standard fields is counted. ; In this embodiment, the lithium battery data terminology mapping dictionary contains lithium battery terms with multiple languages and abbreviations. For example, heterogeneous fields such as “Volt”, “U_cell”, “Voltage_V” or “single cell voltage” appearing in multi-source heterogeneous data are uniformly mapped to the standard field name Cell_Voltage; “Current”, “I” and “current” are uniformly mapped to Current.
[0024] Statistical Feature Inference: For non-standard fields not included in the lithium battery data terminology mapping dictionary, the system initiates secondary inference logic based on statistical features. This step is not merely "inference," but more importantly, it performs a "uniform renaming" operation to generate standardized table headers. The system will read the numerical distribution characteristics of 10,000 rows of data before and after the non-standard field: If a non-standard field value fluctuates between 2.0 and 4.5 for a long period without negative values, the system infers its physical meaning as "cell voltage" and automatically renames it to Cell_Voltage; if the value is between -50 and 80 and changes gradually, it is inferred as ambient temperature and renamed to Temperature.
[0025] S22: Parse various time strings in multi-source heterogeneous data, convert all time columns into Unix timestamps, use Unix timestamps as the unique primary key index of the subsequent time series database, eliminate the time zone differences in the collection of multi-source heterogeneous data, and ensure alignment on the same time axis. To address the inconsistency in time formats (such as the mixing of "yyyy-MM-dd" with relative time in seconds), this invention enforces time cleaning and data format standardization, converting all time strings (primarily Voltaic format, BDF format, MATLAB date vector format, UTC timestamp, compressed BCD code format, ISO8601, Unix timestamp, and human-readable formats) into Unix timestamps (millisecond precision).
[0026] S23: Calculate the acquisition time difference between adjacent data in multi-source heterogeneous data. , For the first i +1 row of data collection time, For the first i Data collection time for rows; S24: Statistical satisfaction of sampling time difference The proportion of data rows, and the average sampling frequency of the data are calculated. ; ; in, m The data is numbered based on the sampling time difference. MThe number of data points representing the sampling time difference. For the first m Each sampling time difference; S25: Based on the average sampling frequency and number of fields Calculate the combined coefficient of field coverage and frequency of the data. It outputs a CSV data stream with standardized fields and time alignment; ; in, The total number of standard fields required for modeling, For the target ideal frequency, Let be the weight coefficient, and satisfy... ; To more accurately reflect data quality, this embodiment combines field coverage and frequency index coefficients. The upgrade to a "combined score of fields and frequency" means that data quality depends not only on the completeness of core standard fields but also on the sampling frequency. The total number of standard fields required for modeling... (Can be freely defined, such as SOC, total current, total voltage, average individual cell voltage, maximum individual cell voltage, minimum individual cell voltage, highest temperature, lowest temperature, insulation resistance, etc.), target ideal frequency. (For example, set to 0.1Hz, i.e., one point every 10 seconds, as the benchmark for a perfect score; if the actual frequency is 1Hz, it would be higher, but the min function limits the coefficient to a maximum of 1); This embodiment sets... α =0.6、 β =0.4, meaning that field integrity accounts for 60% and data density accounts for 40%.
[0027] If standard fields (such as voltage) are missing, the number of fields... Reduction lowers the combined coefficient of field coverage and frequency. If data is sparse (e.g., sampling only once every 60 seconds), the average sampling frequency decreases. Smaller, combined coefficient of field coverage and frequency It will also be significantly reduced, thus being identified as low-precision data in subsequent evaluations.
[0028] S3: Construct a variational autoencoder (VAE) to extract multidimensional time-series features of batteries from the CSV data stream, obtain the original input vector, and map it to the low-dimensional distribution of the hidden layer to obtain the reconstructed vector; calculate the weighted mean square error between the reconstructed vector and the original output vector, locate the points of data reconstruction anomalies, record the anomaly type labels, and calculate the data availability coefficient. .
[0029] Step S3 specifically includes the following steps: S31: Construct a variational autoencoder (VAE), including an encoder and a decoder. The encoder extracts multidimensional temporal features of the battery from the CSV data stream. x The original input vector is obtained. And the multidimensional time-series characteristics of the battery x Low-dimensional distribution mapped to hidden layers middle, z These are latent variables; S32: Decoder from low-dimensional distribution Mid-sampled latent variables z and hidden variables z Reconstruct the original input This yields the reconstruction vector corresponding to each standard field. ; In this embodiment, the battery's multidimensional timing characteristics x The feature vector consists of fields including time-series signals such as current, temperature, and voltage. Since the performance of batteries varies greatly at different rates, the variational autoencoder (VAE) only learns the multidimensional time-series feature distribution of this type of battery under normal operating conditions during the training phase. This makes the variational autoencoder (VAE) have extremely high reconstruction accuracy for "normal data".
[0030] S33: Calculate the reconstructed vector Compared with the original output vector Weighted mean square error between ; ; in, M The total number of standard fields, For the first j The weighting coefficients of each standard field. For the first j The first standard field corresponding to the first k One time-series feature, For the first j The first standard field corresponding to the first k One reconstructed temporal feature; In battery systems, minute perturbations at voltage sampling points often foreshadow severe internal short circuits or sampling failures, thus giving the voltage field a high weight; while temperature, due to its thermal inertia, changes more slowly and has a relatively low weight. By weighted summing of reconstruction errors from multiple standard fields, the overall deviation of the data at that moment can be assessed more scientifically.
[0031] S34: Weighted mean square error for each time series feature Real-time monitoring is performed, if the weighted mean square error is... Exceeding the set deviation threshold If the data reconstruction at that point in time is abnormal, then the weighted mean square error is used to determine the anomaly. The standard field corresponding to the maximum value is used to locate the specific physical dimension and anomaly point where the reconstruction anomaly occurred. By tracing back to the standard field with the highest contribution of weighted mean square error, the specific physical dimension of the anomaly can be located (for example, if the voltage reconstruction error accounts for 80%, it will be automatically located as "voltage sampling anomaly").
[0032] S35: Set the abnormal point to null or insert an abnormal state, and record the abnormal point's abnormal type label in the metadata database; The system does not blindly repair the original input CSV data stream. Instead, it performs "empty" or "insert abnormal status bit" operations on abnormal points and records the abnormal type label of the abnormal point in the metadata database, which makes it convenient for R&D personnel to trace specific failure conditions (such as sensor zero drift, poor contact).
[0033] S36: Based on weighted mean square error Calculate the data availability coefficient ; ; in, K The number of features for the multidimensional time-series characteristics of the battery. For indicator functions, when satisfying When the condition is met, the indicator function is set to 1; otherwise, the indicator function is set to 0.
[0034] This invention uses data availability coefficient The proportion of "low reconstruction error samples" indicates that if the weighted mean square error of a batch of data is generally high, it means that the batch of data has completely deviated from the pre-trained normal battery distribution model, and its reference value in subsequent modeling is extremely low.
[0035] S37: Perform smoothing repair on the data at outlier points in the CSV data stream to obtain continuous, clean time-series data.
[0036] S4: A Hidden Markov Model (HMM) is used to describe the hidden state sequence of the battery's actual operating conditions. Continuous clean time-series data is used as input, and the Viterbi algorithm is employed to find a hidden state path that maximizes the global observation probability. The timestamps of state transitions are identified, and the clean time-series data is segmented to obtain data segments for charging, discharging, and resting states. A charging state database, a discharging state database, and a resting state database are then constructed, and the data quality coefficient is calculated. .
[0037] Step S4 specifically includes the following steps: S41: Using Hidden Markov Models (HMMs) to describe the hidden state sequence of the battery under actual operating conditions. SAnd this is manifested through the observed sequence; hidden state sequence , Indicates a sequence of charging status tags. Represents the discharge state tag sequence, The sequence represents the stationary state label sequence; the observation sequence is the current directly measured by external sensors. I and sliding variance ; Traditional "current thresholding methods" are prone to generating numerous fragmented small files when faced with complex and fluctuating operating conditions in real vehicles (such as frequent start-stop cycles during urban congestion), due to the instantaneous return of current to zero. Hidden Markov Models (HMMs), by introducing a state transition probability matrix, can leverage the physical constraint that "the current state is related to the previous state" to forcibly smooth out instantaneous spike signals, thereby achieving a more robust segmentation effect than regular state machines.
[0038] S42: Set the state transition probability, take the current time series data from the continuous clean time series data as input, and use the Viterbi algorithm to find a hidden state path that maximizes the global observation probability. , n The number of hidden state sequences. For the first n A sequence of hidden states; For example, the probability of jumping directly from "charging state" to "discharging state" is set to an extremely low value, thereby avoiding the jitter of switching between charging and discharging.
[0039] S43: Based on the state label sequence in the hidden state sequence, identify the timestamp of the state transition, and eliminate invalid segments caused by short-term disturbances that are less than the minimum working condition duration threshold by setting a minimum working condition duration threshold. S44: Based on the timestamp of the state transition, the continuous clean time series data is divided into independent data segments to obtain data segments of the charging state, the discharging state, and the static state; and the effective metadata corresponding to the data segments of the charging state, the discharging state, and the static state is extracted to obtain the effective metadata of the charging state, the discharging state, and the static state, respectively. S45: Valid metadata for charging status, discharging status, and resting status is named and packaged in the format of operating condition type_start time_temperature.csv and stored in the charging status database, discharging status database, and resting status database respectively. This approach ensures that subsequent algorithms (such as capacity estimation and internal resistance calculation) can directly retrieve the relevant operating condition data.
[0040] S46: Based on each segmented charging state data segment and discharging state data segment, calculate the change in state of charge at the beginning and end of each data segment. ,like If the condition is met, the data segment is determined to be a depth state segment; otherwise, the data segment is determined to be a non-depth state segment. These represent the charge states corresponding to the beginning and end of the data segment, respectively. The threshold for the change in state of charge is set. S47: Based on the sum of the durations of all depth state segments The sum of battery runtime Calculate the gold content coefficient of the data ; ; in, W This represents the number of depth state fragments contained in the data.
[0041] Data quality coefficient This reflects the "quality" of the data. Even if a piece of data is clean and complete, However, if all data involves only shallow charging, shallow releasing, or long-term stagnation with minimal SOC changes, then the data's value factor is... Extremely low.
[0042] S5: Extract deep state segments from the charging state database, discharging state database, and idle state database, and calculate the actual power throughput using the ampere-hour integration method. Combined with the theoretical charge changes corresponding to the deep state segments Calculate the logical reliability index coefficient of the data. .
[0043] Step S5 specifically includes the following steps: S51: Extract deep state segments from the charging state database, discharging state database, and idle state database, and then use the corresponding current timing data in the deep state segments. The actual electricity throughput was calculated using the ampere-hour integration method. ; ; in, These are the start and end times of the depth state segment, respectively. S52: Based on the state-of-charge data segments at the beginning and end of the deep state segment reported by the Battery Management System (BMS). Calculate the theoretical change in electricity , This refers to the battery's rated capacity. S53: If the power throughput Changes in theoretical electricity If the deviation exceeds the set threshold, it is determined that there is a physical-logical inconsistency in the depth state segment, and the time period corresponding to the depth state segment is marked as a potential sensor failure point or algorithm failure point. S54: Based on power throughput Changes in theoretical electricity Calculate the logical reliability index coefficient of the data ; .
[0044] Logical reliability index coefficient It intuitively reflects the degree of consistency between current data and status labels (SOC / SOH), if the logic reliability index coefficient The smaller value indicates a serious conflict in the underlying physical logic of this batch of data (such as current sensor range drift or large jumps in SOC). Even if it appears "smooth and noise-free", its reference value at the business level is still extremely low.
[0045] S6: Information Acquisition Degree Coefficient Field coverage and frequency combined index coefficient Data availability coefficient Data quality coefficient and logic reliability index coefficient As evaluation features, the information entropy of each evaluation feature is calculated to obtain the objective weight of each evaluation feature. The objective weight is then used to calculate the comprehensive score of data quality to evaluate the quality of the data.
[0046] Step S6 specifically includes the following steps: S61: Information Acquisition Degree Coefficient Field coverage and frequency combined index coefficient Data availability coefficient Data quality coefficient and logic reliability index coefficient As evaluation features, these features are encapsulated into a five-dimensional feature vector. ; S62: Perform range standardization on the evaluation features, and then calculate the information entropy of each evaluation feature. , v The evaluation feature is numbered; according to The proportion determines the objective weight of each evaluation feature. Utilizing objective weights Calculate the overall score for data quality. Y ; ; S63: If If the data is deemed to be of high quality, it will be stored in a high-value modeling library for algorithm engineers to directly use in training models. like If the data is deemed to be of medium quality, it will be stored in a regular analysis database for statistical analysis. like If the data is deemed low-quality, it will be stored in the original archive.
Claims
1. A method for standardizing and analyzing time-series data of multi-source heterogeneous batteries, characterized in that, Includes the following steps: S1: Input multi-source heterogeneous data related to lithium batteries, construct a standard metadata table for the entire life cycle of lithium batteries as a standard metadata target, extract effective metadata from the multi-source heterogeneous data based on the standard metadata target, and calculate the information acquisition degree coefficient of the data. ; S2: Based on the lithium battery data terminology mapping dictionary, standardize the naming of heterogeneous and non-standard fields in multi-source heterogeneous data, and count the number of fields that have completed the standard naming. Parse various time strings in multi-source heterogeneous data, calculate the acquisition time difference between adjacent data, and calculate the average sampling frequency of the data. ; Based on the number of fields and average sampling frequency Calculate the combined coefficient of field coverage and frequency. It outputs a CSV data stream with standardized fields and time alignment; S3: Construct a variational autoencoder (VAE) to extract multidimensional time-series features of batteries from the CSV data stream, obtain the original input vector, and map it to the low-dimensional distribution of the hidden layer to obtain the reconstructed vector; calculate the weighted mean square error between the reconstructed vector and the original output vector, locate the points of data reconstruction anomalies, record the anomaly type labels, and calculate the data availability coefficient. ; S4: A Hidden Markov Model (HMM) is used to describe the hidden state sequence of the battery's actual operating conditions. Continuous clean time-series data is used as input, and the Viterbi algorithm is employed to find a hidden state path that maximizes the global observation probability. The timestamps of state transitions are identified, and the clean time-series data is segmented to obtain data segments for charging, discharging, and resting states. A charging state database, a discharging state database, and a resting state database are then constructed, and the data quality coefficient is calculated. ; S5: Extract deep state segments from the charging state database, discharging state database, and idle state database, and calculate the actual power throughput using the ampere-hour integration method. Combined with the theoretical charge changes corresponding to the deep state segments Calculate the logical reliability index coefficient of the data. ; S6: Information Acquisition Degree Coefficient Field coverage and frequency combined index coefficient Data availability coefficient Data quality coefficient and logic reliability index coefficient As evaluation features, the information entropy of each evaluation feature is calculated to obtain the objective weight of each evaluation feature. The objective weight is then used to calculate the comprehensive score of data quality to evaluate the quality of the data.
2. The method for standardizing and analyzing time-series data of multi-source heterogeneous batteries according to claim 1, characterized in that, Step S1 includes: S11: Input BMS data of actual vehicles related to lithium batteries, EMS logs of energy storage power stations and laboratory test reports as multi-source heterogeneous data; S12: Construct a standard metadata table for the entire life cycle of lithium batteries as a standard metadata target. The standard metadata target contains basic static information, production traceability information and dynamic configuration information of lithium batteries. S13: Extract effective metadata from multi-source heterogeneous data based on standard metadata targets, and statistically analyze the proportion of required metadata items defined in the standard metadata targets appearing in the multi-source heterogeneous data to obtain the information acquisition degree coefficient of the data. ; ； in, This represents the number of valid metadata items actually extracted from multi-source heterogeneous data. For the required metadata items defined in the standard metadata target, i The index number of each required metadata item in the standard metadata target. For the first i The weight coefficient of each required metadata item.
3. The method for standardizing and analyzing time-series data of multi-source heterogeneous batteries according to claim 2, characterized in that, Step S2 includes: S21: Based on the lithium battery data terminology mapping dictionary, heterogeneous fields in multi-source heterogeneous data are uniformly mapped to standard field names. For non-standard fields not included in the lithium battery data terminology mapping dictionary, the numerical distribution characteristics of 10,000 rows of data before and after the non-standard field are read. The meaning of the non-standard field is inferred based on the data distribution characteristics, and standard field names are generated. The number of fields successfully named as standard fields is counted. ; S22: Parse various time strings in multi-source heterogeneous data, convert all time columns into Unix timestamps, use Unix timestamps as the unique primary key index of the subsequent time series database, eliminate the time zone differences in the collection of multi-source heterogeneous data, and ensure alignment on the same time axis. S23: Calculate the acquisition time difference between adjacent data in multi-source heterogeneous data. , For the first i +1 row of data collection time, For the first i Data collection time for rows; S24: Statistical satisfaction of sampling time difference The proportion of data rows, and the average sampling frequency of the data are calculated. ; ； in, m The data is numbered based on the sampling time difference. M The number of data points representing the sampling time difference. For the first m Each sampling time difference; S25: Based on the average sampling frequency and number of fields Calculate the combined coefficient of field coverage and frequency of the data. It outputs a CSV data stream with standardized fields and time alignment; ； in, The total number of standard fields required for modeling, For the target ideal frequency, Let be the weight coefficient, and satisfy... .
4. The method for standardizing and analyzing time-series data of multi-source heterogeneous batteries according to claim 3, characterized in that, Step S3 includes: S31: Construct a variational autoencoder (VAE), including an encoder and a decoder. The encoder extracts multidimensional temporal features of the battery from the CSV data stream. x The original input vector is obtained. And the multidimensional time-series characteristics of the battery x Low-dimensional distribution mapped to hidden layers middle, z These are latent variables; S32: Decoder from low-dimensional distribution Mid-sampled latent variables z and hidden variables z Reconstruct the original input This yields the reconstruction vector corresponding to each standard field. ; S33: Calculate the reconstructed vector Compared with the original output vector Weighted mean square error between ; ； in, M The total number of standard fields, For the first j The weighting coefficients of each standard field. For the first j The first standard field corresponding to the first k One time-series feature, For the first j The first standard field corresponding to the first k One reconstructed temporal feature; S34: Weighted mean square error for each time series feature Real-time monitoring is performed, if the weighted mean square error is... Exceeding the set deviation threshold If the data reconstruction at that point in time is abnormal, then the weighted mean square error is used to determine the anomaly. The standard field corresponding to the maximum value is used to locate the specific physical dimension and anomaly point where the reconstruction anomaly occurred. S35: Set the abnormal point to null or insert an abnormal state, and record the abnormal point's abnormal type label in the metadata database; S36: Based on weighted mean square error Calculate the data availability coefficient ; ； in, K The number of features for the multidimensional time-series characteristics of the battery. For indicator functions, when satisfying When the condition is met, the indicator function is 1; otherwise, the indicator function is 0. S37: Perform smoothing repair on the data at outlier points in the CSV data stream to obtain continuous, clean time-series data.
5. The method for standardizing and analyzing time-series data of multi-source heterogeneous batteries according to claim 4, characterized in that, Step S4 includes: S41: Using Hidden Markov Models (HMMs) to describe the hidden state sequence of the battery under actual operating conditions. S And this is manifested through the observed sequence; hidden state sequence , Indicates a sequence of charging status tags. Represents the discharge state tag sequence, The sequence represents the stationary state label sequence; the observation sequence is the current directly measured by external sensors. I and sliding variance ; S42: Set the state transition probability, take the current time series data from the continuous clean time series data as input, and use the Viterbi algorithm to find a hidden state path that maximizes the global observation probability. , n The number of hidden state sequences. For the first n A sequence of hidden states; S43: Based on the state label sequence in the hidden state sequence, identify the timestamp of the state transition, and eliminate invalid segments caused by short-term disturbances that are less than the minimum working condition duration threshold by setting a minimum working condition duration threshold. S44: Based on the timestamp of the state transition, the continuous clean time series data is divided into independent data segments to obtain data segments of the charging state, the discharging state, and the static state; and the effective metadata corresponding to the data segments of the charging state, the discharging state, and the static state is extracted to obtain the effective metadata of the charging state, the discharging state, and the static state, respectively. S45: Valid metadata for charging status, discharging status, and resting status is named and packaged in the format of operating condition type_start time_temperature.csv and stored in the charging status database, discharging status database, and resting status database respectively. S46: Based on each segmented charging state data segment and discharging state data segment, calculate the change in state of charge at the beginning and end of each data segment. ,like If the condition is met, the data segment is determined to be a depth state segment; otherwise, the data segment is determined to be a non-depth state segment. These represent the charge states corresponding to the beginning and end of the data segment, respectively. The threshold for the change in state of charge is set. S47: Based on the sum of the durations of all depth state segments The sum of battery runtime Calculate the gold content coefficient of the data ; ； in, W This represents the number of depth state fragments contained in the data.
6. The method for standardizing and analyzing time-series data of multi-source heterogeneous batteries according to claim 5, characterized in that, Step S5 includes: S51: Extract deep state segments from the charging state database, discharging state database, and idle state database, and then use the corresponding current timing data in the deep state segments. The actual electricity throughput was calculated using the ampere-hour integration method. ; ； in, These are the start and end times of the depth state segment, respectively. S52: Based on the state-of-charge data segments at the beginning and end of the deep state segment reported by the Battery Management System (BMS). Calculate the theoretical change in electricity , This refers to the battery's rated capacity. S53: If the power throughput Changes in theoretical electricity If the deviation exceeds the set threshold, it is determined that there is a physical-logical inconsistency in the depth state segment, and the time period corresponding to the depth state segment is marked as a potential sensor failure point or algorithm failure point. S54: Based on power throughput Changes in theoretical electricity Calculate the logical reliability index coefficient of the data ; 。 7. The method for standardizing and analyzing time-series data of multi-source heterogeneous batteries according to claim 6, characterized in that, Step S6 includes: S61: Information Acquisition Degree Coefficient Field coverage and frequency combined index coefficient Data availability coefficient Data quality coefficient and logic reliability index coefficient As evaluation features, these features are encapsulated into a five-dimensional feature vector. ; S62: Perform range standardization on the evaluation features, and then calculate the information entropy of each evaluation feature. , v The evaluation feature is numbered; according to The proportion determines the objective weight of each evaluation feature. Utilizing objective weights Calculate the overall score for data quality. Y ; ； S63: If If the data is deemed to be of high quality, it will be stored in a high-value modeling library for algorithm engineers to directly use in training models. like If the data is deemed to be of medium quality, it will be stored in a regular analysis database for statistical analysis. like If the data is deemed low-quality, it will be stored in the original archive.