Spectrum turning time identification method and device, electronic equipment and storage medium
By constructing clustered tree structure data and combining multiple algorithms to identify spectral inflection points, the shortcomings of existing technologies that rely on preset thresholds or human experience are overcome, and efficient and accurate identification of spectral inflection time is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JIHUA LAB
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-21
AI Technical Summary
Existing technologies rely on preset thresholds or human experience to identify spectral inflection points in samples, making it difficult to accurately identify weak or inconspicuous inflection points in complex and variable sample systems. They are also prone to false alarms and lack automated and intelligent identification methods.
We construct clustering tree structure data using agglomerative hierarchical clustering method, Mahalanobis distance, minimum variance method, and time series. Combining sliding window algorithm and Fourier transform algorithm, we identify turning point data that meet the conditions of abrupt change in cluster distance and jump in frequency domain features by merging distances, and verify the turning points using the isolated forest algorithm.
It improves the efficiency of spectral transition time identification, enables objective analysis of the intrinsic structural changes in spectral data, accurately identifies transition time points, reduces noise interference, and improves the accuracy and robustness of identification.
Smart Images

Figure CN121434995B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of spectral analysis, and more specifically, to a method, apparatus, electronic device, and storage medium for identifying spectral transition times. Background Technology
[0002] Real-time monitoring of sample conditions is crucial in many fields, including food production, biomedicine, and environmental monitoring. During production or reaction processes, the internal components or physicochemical properties of samples often change. These changes are not always gradual but occur at specific points in time. For example, during food fermentation, the accumulation of microbial metabolites can cause a sudden change in spectral absorption characteristics at a certain moment; in chemical reactions, the formation or consumption of key intermediates can also cause drastic changes in spectral signals. Accurately identifying these turning points is of great significance for understanding sample change patterns, optimizing production processes, and ensuring product quality.
[0003] However, existing technologies have limitations in identifying spectral inflection points in samples. Traditional spectral monitoring methods typically rely on preset thresholds or human experience, which are highly subjective, difficult to adapt to complex and variable sample systems, and prone to missing weak or inconspicuous inflection points. Furthermore, while some statistical analysis-based methods can identify outliers in spectral data, they often struggle to distinguish between random noise and genuine inflection points, leading to high false alarm rates. Current technologies lack an automated and intelligent method that can effectively capture the inherent structural changes in spectral data and accurately identify key inflection points. Especially when dealing with continuously acquired real-time spectral data, efficiently and accurately extracting representative inflection information from massive amounts of data and eliminating noise interference is a pressing technical challenge.
[0004] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention
[0005] The purpose of this application is to provide a method, device, electronic device, and storage medium for identifying spectral transition times. By employing agglomerative hierarchical clustering, Mahalanobis distance, minimum variance method, and time series analysis, a clustering tree structure data corresponding to real-time spectral data is constructed. Based on the sliding window algorithm and Fourier transform algorithm, combined with merging distance, transition point data satisfying the abrupt change condition of clustering distance and the jump condition of frequency domain features are identified within the clustering tree structure data to confirm the transition time point. This solves the problem that existing spectral transition time identification methods rely on preset thresholds or manual experience judgment, making it difficult to accurately identify spectral transition time points. It can objectively analyze the inherent structural changes of spectral data and improve the efficiency of spectral transition time identification.
[0006] Firstly, this application provides a method for identifying spectral transition times, including:
[0007] Periodically acquire real-time spectral data of the sample to be tested;
[0008] A cohesive hierarchical clustering method is adopted, which combines Mahalanobis distance and minimum variance method, to merge the real-time spectral data into corresponding cluster tree structure data according to the time series.
[0009] Using the sliding window algorithm and Fourier transform algorithm, based on the merging distance in the clustering tree structure data, the turning point data that satisfies the preset clustering distance mutation condition and the preset frequency domain feature jump condition are determined.
[0010] Based on the isolated forest algorithm, the inflection point data is verified to determine the time point corresponding to the inflection point data of non-abnormal data as the inflection time point.
[0011] The spectral transition time identification method provided in this application can identify the transition time points of spectral data. By using agglomerative hierarchical clustering, Mahalanobis distance, minimum variance method, and time series, a clustering tree structure data corresponding to real-time spectral data is constructed. Based on the sliding window algorithm and Fourier transform algorithm, combined with the merging distance, the transition point data in the clustering tree structure data that meets the conditions of abrupt change in clustering distance and jump in frequency domain features are identified to confirm the transition time points. This solves the problem that existing spectral transition time identification methods rely on preset thresholds or human experience to accurately identify spectral transition time points. It can objectively analyze the internal structural changes of spectral data and improve the identification efficiency of spectral transition time.
[0012] Optionally, an agglomerative hierarchical clustering method is employed, combining Mahalanobis distance and minimum variance methods, to merge the real-time spectral data into corresponding clustering tree structure data according to the time series, including:
[0013] According to the time series, each spectral data of the real-time spectral data is divided into a separate cluster to calculate the spectral mean vector of each cluster;
[0014] Using Mahalanobis distance and based on the spectral mean vector, the covariance values between all pairs of clusters are calculated.
[0015] Using the minimum variance method, with the covariance value as the merging distance, the real-time spectral data are sequentially merged into a cluster to obtain the corresponding clustering tree structure data.
[0016] The spectral transition time identification method provided in this application can identify the transition time points of spectral data. By using Mahalanobis distance and the minimum variance method, it can more accurately measure the similarity between spectral data, thereby constructing a clustering tree that can reflect the internal structural changes of spectral data, providing a more reliable basis for subsequent identification of transition points.
[0017] Optionally, using the minimum variance method, with the covariance value as the merging distance, the real-time spectral data are sequentially merged into a cluster to obtain the corresponding clustering tree structure data, including:
[0018] A1, merge the two clusters corresponding to the minimum value of the covariance values into a new cluster, record the new cluster in the same branch of the tree diagram, and determine the corresponding covariance value as the merging distance of the corresponding branch;
[0019] A2, recalculate the new covariance values between all clusters, merge the two clusters corresponding to the minimum value of the new covariance values into a new cluster, record the new cluster in the same branch of the tree diagram, and determine the corresponding covariance value as the merging distance of the corresponding branch.
[0020] A3. Repeat step A2 to merge the clusters corresponding to the real-time spectral data into one cluster in sequence to obtain the corresponding clustering tree structure data.
[0021] Optionally, using a sliding window algorithm and a Fourier transform algorithm, based on the merging distance in the clustering tree structure data, inflection point data satisfying preset clustering distance abrupt change conditions and preset frequency domain feature jump conditions are calculated, including:
[0022] According to the order in which the merge distances were obtained, the merge distances corresponding to each branch in the clustering tree structure data are sorted to construct a distance sequence;
[0023] The sliding window algorithm is used to calculate the mean distance and standard deviation of the distance sequence within each first preset window, which are then used to determine the real-time spectral data corresponding to the merged distance that meets the preset clustering distance mutation conditions as the initial inflection point data.
[0024] The spectral frequency component changes of the initial inflection point data are analyzed using the Fourier transform algorithm to determine the initial inflection point data that meets the preset frequency domain characteristic jump conditions as the inflection point data.
[0025] The spectral transition time identification method provided in this application can identify the transition time points of spectral data. By combining the abrupt change in cluster distance and the change in spectral frequency components, it captures the transition features of spectral data from multiple dimensions, effectively distinguishes between real transitions and noise interference, and improves the accuracy and robustness of transition point identification.
[0026] Optionally, the mean distance and standard deviation of the distance sequence within each first preset window are calculated using a sliding window algorithm. This is used to determine the real-time spectral data corresponding to the merged distance that satisfies the preset clustering distance abrupt change conditions as the initial inflection point data, including:
[0027] Using a sliding window algorithm, a first preset window is set with each of the merged distances as the center, and the mean distance and standard deviation of the distance sequence within each first preset window are calculated.
[0028] Based on the mean distance and the standard deviation of distance, a clustering distance mutation threshold is constructed within each of the first preset windows;
[0029] Sequentially determine whether each of the merged distances in the distance sequence is greater than the clustering distance mutation threshold within the corresponding first preset window; if yes, determine that the merged distance meets the preset clustering distance mutation condition, and determine the real-time spectral data corresponding to the merged distance as the initial turning point data; if no, determine that the merged distance does not meet the preset clustering distance mutation condition.
[0030] Optionally, the spectral frequency component changes of the initial inflection point data are analyzed using a Fourier transform algorithm to determine the initial inflection point data that satisfies the preset frequency domain characteristic jump condition as the inflection point data, including:
[0031] The spectral frequency components of all the real-time spectral data were calculated using the Fourier transform algorithm.
[0032] Based on the spectral frequency component data, the change value of the spectral frequency component between each real-time spectral data and the previous real-time spectral data is calculated according to the time series.
[0033] Using a sliding window algorithm, a second preset window is set with each real-time spectral data as the center, and the mean value and standard deviation of the spectral frequency component change within each second preset window are calculated to construct the frequency domain feature jump threshold within each second preset window.
[0034] Sequentially determine whether the change value of the spectral frequency component corresponding to each real-time spectral data is greater than the frequency domain feature jump threshold within the corresponding second preset window; if yes, determine that the real-time spectral data meets the preset frequency domain feature jump condition, and determine the initial inflection point data that meets the preset frequency domain feature jump condition as the inflection point data; if no, determine that the real-time spectral data does not meet the preset frequency domain feature jump condition.
[0035] Optionally, according to the isolated forest algorithm, the inflection point data is verified to determine the time point corresponding to the inflection point data of non-anomaly data as the inflection time point, including:
[0036] The isolated forest algorithm is used to divide the real-time spectral data into isolated forest partitions corresponding to each real-time spectral data.
[0037] Determine the path length of each real-time spectral data in the corresponding isolated forest partition, and calculate the absolute value of the difference between the path length and the average of all path lengths, so as to determine that the real-time spectral data corresponding to the path length whose absolute value of the difference is less than or equal to a preset abnormal isolation threshold is non-abnormal data.
[0038] The time point corresponding to the turning point data that is determined to be non-abnormal data is the turning point time point.
[0039] Secondly, this application provides a spectral transition time identification device, comprising:
[0040] The acquisition module is used to periodically acquire real-time spectral data of the sample to be tested;
[0041] The merging module is used to merge the real-time spectral data into corresponding clustering tree structure data according to the time series by employing an agglomerative hierarchical clustering method, combined with Mahalanobis distance and the minimum variance method.
[0042] The determination module is used to determine, based on the merging distance in the clustering tree structure data, the turning point data that satisfies the preset clustering distance abrupt change condition and the preset frequency domain feature jump condition, by using the sliding window algorithm and the Fourier transform algorithm.
[0043] The verification module is used to verify the inflection point data according to the isolated forest algorithm, so as to determine the time point corresponding to the inflection point data of non-abnormal data as the inflection time point.
[0044] This spectral transition time identification device constructs a clustering tree structure data corresponding to real-time spectral data through agglomerative hierarchical clustering, Mahalanobis distance, minimum variance method, and time series. Based on the sliding window algorithm and Fourier transform algorithm, combined with the merging distance, it identifies the transition point data in the clustering tree structure data that meets the conditions of abrupt change in clustering distance and jump in frequency domain features, thus confirming the transition time point. This solves the problem that existing spectral transition time identification methods rely on preset thresholds or human experience to accurately identify spectral transition time points. It can objectively analyze the internal structural changes of spectral data and improve the identification efficiency of spectral transition time.
[0045] Thirdly, this application provides an electronic device including a processor and a memory, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, it runs the steps in the spectral transition time identification method described above.
[0046] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the spectral transition time identification method described above.
[0047] Beneficial effects: The spectral transition time identification method, device, electronic device, and storage medium provided in this application construct a clustering tree structure data corresponding to real-time spectral data through agglomerative hierarchical clustering method, Mahalanobis distance, minimum variance method, and time series. Based on the sliding window algorithm and Fourier transform algorithm, combined with the merging distance, the transition point data in the clustering tree structure data that meets the conditions of abrupt change in clustering distance and jump in frequency domain features are identified to confirm the transition time point. This solves the problem that existing spectral transition time identification methods rely on preset thresholds or human experience to accurately identify spectral transition time points. It can objectively analyze the internal structural changes of spectral data and improve the identification efficiency of spectral transition time. Attached Figure Description
[0048] Figure 1 A flowchart of the spectral transition time identification method provided in the embodiments of this application.
[0049] Figure 2 This is a schematic diagram of the spectral transition time identification device provided in an embodiment of this application.
[0050] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0051] Figure 4 This is a detailed structural diagram of the clustered tree data.
[0052] Figure 5 This is a simplified structural diagram of clustered tree data.
[0053] Figure 6 This is a schematic diagram showing the change in the cleanliness of the liquid sample inside the processing tank over time.
[0054] Labeling Explanation: 1. Acquisition Module; 2. Merging Module; 3. Determination Module; 4. Verification Module; 301. Processor; 302. Memory; 303. Communication Bus. Detailed Implementation
[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0056] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0057] Please refer to Figure 1 , Figure 1 This application provides a method for identifying spectral transition time points in some embodiments, used to identify transition time points in spectral data, including the following steps:
[0058] Step S101: Periodically acquire real-time spectral data of the sample to be tested;
[0059] Step S102: Using an agglomerative hierarchical clustering method, combined with Mahalanobis distance and minimum variance method, real-time spectral data are merged into corresponding clustering tree structure data according to the time series.
[0060] Step S103: Using the sliding window algorithm and Fourier transform algorithm, based on the merging distance in the clustering tree structure data, determine the turning point data that satisfies the preset clustering distance abrupt change condition and the preset frequency domain feature jump condition.
[0061] Step S104: According to the isolated forest algorithm, verify the inflection point data to determine the time point corresponding to the inflection point data of non-abnormal data as the inflection time point.
[0062] This spectral transition time identification method constructs a clustering tree structure data corresponding to real-time spectral data through agglomerative hierarchical clustering, Mahalanobis distance, minimum variance method, and time series. Based on the sliding window algorithm and Fourier transform algorithm, combined with the merging distance, it identifies the transition point data in the clustering tree structure data that meets the conditions of abrupt change in clustering distance and jump in frequency domain features, thus confirming the transition time point. This solves the problem that existing spectral transition time identification methods rely on preset thresholds or human experience to accurately identify spectral transition time points. It can objectively analyze the internal structural changes of spectral data and improve the identification efficiency of spectral transition time.
[0063] Specifically, in step S101, the spectrometer automatically collects spectral data of the sample at preset time points (such as per second, per minute, or longer). The sample is a substance whose chemical or physical properties can be detected by the spectrometer and change over time during the production, processing, or treatment process. This substance includes, but is not limited to, liquids (such as cleaning liquid in beverage processing tanks, reaction liquid in fermentation tanks, etc.), solids (such as food in the baking process, pharmaceutical powder in drying, tea in fermentation, etc.), semi-solids (such as cosmetic pastes, food slurries, etc.), and gases (fermentation exhaust gas, chemical process gases, etc.).
[0064] Specifically, in step S102, an agglomerative hierarchical clustering method is used, combining Mahalanobis distance and the minimum variance method, to merge real-time spectral data into corresponding clustering tree structure data according to the time series, including:
[0065] According to the time series, each spectral data point of the real-time spectral data is divided into a separate cluster to calculate the spectral mean vector of each cluster;
[0066] The covariance values between all pairs of clusters are calculated using Mahalanobis distance and based on the spectral mean vector.
[0067] Using the minimum variance method, with the covariance value as the merging distance, real-time spectral data are sequentially merged into a cluster to obtain the corresponding clustering tree structure data.
[0068] In step S102, at the initial stage of the clustering process, each independent spectral data in the real-time spectral data is sorted according to the time series, and each independent spectral data is regarded as an independent clustering unit, that is, an independent cluster (which is the initial cluster). For each independent cluster, the spectral mean vector of the cluster is calculated based on its corresponding spectral data. The spectral mean vector represents the average characteristics of the spectral data within the cluster, providing a basis for subsequent distance calculation.
[0069] Based on Mahalanobis distance and using the spectral mean vector as the basis, the covariance value between clusters is calculated. This covariance value reflects the similarity between different clusters (spectral data); a larger value indicates greater difference, and a smaller value indicates higher similarity. The specific formula for calculating the covariance value is as follows:
[0070] ;
[0071] in, For clusters and cluster The covariance value; For clusters The amount of spectral data in the data; For clusters The amount of spectral data in the data; For clusters spectral mean vector; For clusters spectral mean vector; This is the norm symbol.
[0072] Specifically, in step S102, the minimum variance method is used, with the covariance value as the merging distance, to sequentially merge the real-time spectral data into a single cluster, resulting in the corresponding clustering tree structure data, including:
[0073] A1, merge the two clusters corresponding to the minimum covariance values into a new cluster, record the new cluster in the same branch of the tree diagram, and determine the corresponding covariance value as the merging distance of the corresponding branch;
[0074] A2, recalculate the new covariance values between all clusters, merge the two clusters corresponding to the minimum value of the new covariance values into a new cluster, record the new cluster in the same branch of the tree diagram, and determine the corresponding covariance value as the merging distance of the corresponding branch.
[0075] A3. Repeat step A2 to merge the clusters corresponding to the real-time spectral data into one cluster in sequence to obtain the corresponding clustering tree structure data.
[0076] In step S102, among all possible cluster pairs, two clusters with the smallest covariance value are selected for merging. After merging these two clusters, a new cluster is formed, and a branch is formed in the clustering tree structure data. The merging distance of this branch is the covariance value when the two clusters are merged.
[0077] The merged new cluster is treated as a whole, and the new covariance value is recalculated with the remaining clusters (including the original unmerged clusters and the new clusters formed by previous merges). The two clusters with the smallest new covariance value are selected from all cluster pairs for merging, and their merging distance and branch information are recorded.
[0078] Therefore, the above steps are repeated to continuously merge clusters, selecting the two closest clusters (with the smallest covariance values) among all current clusters at each merge, until all real-time spectral data are finally merged into a single cluster, constructing a complete clustering tree structure. Each merge operation corresponds to a branch in the tree, and the merge distance represents the similarity of that branch during the merge.
[0079] Specifically, in step S103, using the sliding window algorithm and Fourier transform algorithm, based on the merging distance in the clustering tree structure data, the inflection point data that satisfies the preset clustering distance abrupt change condition and the preset frequency domain feature jump condition are determined, including:
[0080] The merge distances corresponding to each branch in the clustering tree structure data are sorted according to the order in which the merge distances are obtained, so as to construct a distance sequence.
[0081] The sliding window algorithm is used to calculate the mean distance and standard deviation of the distance sequence within each first preset window, which are used to determine the real-time spectral data corresponding to the merged distance that meets the preset clustering distance mutation conditions as the initial inflection point data.
[0082] The Fourier transform algorithm is used to analyze the changes in the spectral frequency components of the initial inflection point data, in order to determine the initial inflection point data that meets the preset frequency domain characteristic jump conditions as the inflection point data.
[0083] In step S103, by arranging the merged distances according to the merged order, a distance sequence reflecting the evolution of the clustering process can be formed, which provides basic data for subsequent inflection point detection.
[0084] Specifically, in step S103, the mean distance and standard deviation of the distance sequence within each first preset window are calculated using a sliding window algorithm. These are used to determine the real-time spectral data corresponding to the merged distance that meets the preset clustering distance abrupt change conditions as the initial inflection point data, including:
[0085] Using the sliding window algorithm, a first preset window is set with each merged distance as the center, and the mean distance and standard deviation of the distance sequence within each first preset window are calculated.
[0086] Based on the distance mean and distance standard deviation, a clustering distance mutation threshold is constructed within each first preset window;
[0087] The system sequentially determines whether each merged distance in the distance sequence is greater than the corresponding clustering distance mutation threshold within the first preset window. If so, it determines that the merged distance meets the preset clustering distance mutation condition and determines the real-time spectral data corresponding to the merged distance as the initial inflection point data. If not, it determines that the merged distance does not meet the preset clustering distance mutation condition.
[0088] In step S103, the mean distance and standard deviation of distance are calculated within a first preset window using a sliding window algorithm, which quantifies the central tendency and dispersion of the merged distances within the first preset window. The length of the first preset window is set according to actual needs.
[0089] Based on the distance mean and distance standard deviation, a clustering distance mutation threshold is constructed within each first preset window. For example, the 3σ criterion can be used to set the threshold, setting it as the sum of the distance mean and three times the distance standard deviation. , The mean distance within the m-th (m≤n) preset window (i.e., the preset window corresponding to the m-th merged distance). denoted as the standard deviation of the distance within the m-th preset window.
[0090] The algorithm sequentially checks whether each merge distance exceeds the corresponding cluster distance abrupt change threshold within a first preset window to identify merge distances that significantly deviate from their local average level. These deviations typically indicate significant changes in the spectral data structure, thus pointing to potential inflection points. When a merge distance exceeds the corresponding cluster distance abrupt change threshold within the first preset window, it is determined that the merge distance meets the preset cluster distance abrupt change condition. The real-time spectral data of the cluster corresponding to the merge distance (i.e., the two clusters corresponding to the merge distance at the time of acquisition) is determined as the initial inflection point data, indicating that the sample's spectral data may have undergone a significant change in cluster structure at the corresponding time point, which is a key characteristic of inflection points. When a merge distance does not exceed the corresponding cluster distance abrupt change threshold within the first preset window, it indicates that the merge distance does not meet the preset cluster distance abrupt change condition, and it is determined that the sample's spectral data has not changed at the corresponding time point.
[0091] Specifically, in step S103, the spectral frequency component changes of the initial inflection point data are analyzed using a Fourier transform algorithm to determine the initial inflection point data that meets the preset frequency domain characteristic jump conditions as the inflection point data, including:
[0092] The spectral frequency components of all real-time spectral data were calculated using the Fourier transform algorithm.
[0093] Based on the spectral frequency component data, the change in spectral frequency components between each real-time spectral data and the previous real-time spectral data is calculated according to the time series.
[0094] Using a sliding window algorithm, a second preset window is set with each real-time spectral data as the center. The mean value and standard deviation of the spectral frequency component changes within each second preset window are calculated to construct the frequency domain feature jump threshold within each second preset window.
[0095] Sequentially determine whether the change value of the spectral frequency component corresponding to each real-time spectral data is greater than the frequency domain feature jump threshold within the corresponding second preset window; if yes, determine that the real-time spectral data meets the preset frequency domain feature jump condition, and determine the initial inflection point data that meets the preset frequency domain feature jump condition as the inflection point data; if no, determine that the real-time spectral data does not meet the preset frequency domain feature jump condition.
[0096] In step S103, by performing a Fourier transform on the real-time spectral data, its energy distribution at different frequencies can be obtained, i.e., spectral frequency component data. This frequency component data can reflect the intrinsic structure and variation patterns of the spectral signal, such as the vibrational frequencies of specific chemical bonds or the characteristic frequencies of molecular structures.
[0097] Based on spectral frequency component data, the change in spectral frequency components between each real-time spectral data and the previous real-time spectral data is calculated according to the time series, so as to quantify the dynamic evolution of spectral frequency characteristics over time.
[0098] Using a sliding window algorithm, a second preset window (with the same length as the first preset window, adjustable as needed) is set centered on each real-time spectral data point. The mean and standard deviation of spectral frequency component changes within each second preset window are calculated to construct a frequency domain feature jump threshold for each window. This threshold can be adjusted based on the position of each preset window, thereby more accurately identifying abnormal and significant frequency jumps. The frequency domain feature jump threshold can also be set using the aforementioned 3σ criterion, or adjusted as needed.
[0099] The system sequentially checks whether the change in spectral frequency components corresponding to each real-time spectral data exceeds the corresponding frequency domain feature jump threshold within a second preset window. This accurately identifies spectral data that undergo significant jumps in the frequency domain, thus ultimately determining the initial inflection point data that meets the conditions as inflection point data. This effectively eliminates the interference of random noise and improves the accuracy and robustness of inflection point identification. When the change in spectral frequency components exceeds the corresponding frequency domain feature jump threshold within the second preset window, the corresponding real-time spectral data is determined to meet the preset frequency domain feature jump condition. The initial inflection point data that meets the preset frequency domain feature jump condition is then identified as inflection point data. This confirms that the initial inflection point data in the sample that meets the preset frequency domain feature jump condition has undergone a significant change in cluster structure at the corresponding time point, effectively capturing the true inflection point of sample state change. When the change in spectral frequency components does not exceed the corresponding frequency domain feature jump threshold within the second preset window, it indicates that the corresponding real-time spectral data does not meet the preset frequency domain feature jump condition, and the spectral data of the sample has not changed at the corresponding time point.
[0100] Specifically, in step S104, the inflection point data is verified according to the isolated forest algorithm to determine the time point corresponding to the inflection point data of non-abnormal data as the inflection time point, including:
[0101] The isolated forest algorithm is used to divide the real-time spectral data into isolated forest partitions corresponding to each real-time spectral data.
[0102] Determine the path length of each real-time spectral data in the corresponding isolated forest partition, and calculate the absolute value of the difference between the path length and the average of all path lengths. The real-time spectral data corresponding to the path length whose absolute value of the difference is less than or equal to the preset abnormal isolation threshold is considered as non-abnormal data.
[0103] The time point corresponding to the turning point data that is determined to be non-abnormal data is the turning point time point.
[0104] In step S104, the real-time spectral data is processed using an isolation forest algorithm (a pre-trained isolation forest algorithm that, after training, can identify anomalous data by randomly selecting features and split points) to obtain the isolation forest partition to which each spectral data belongs. Specifically, this can be achieved by constructing multiple isolation trees. Each isolation tree recursively partitions the data by randomly selecting features and split points until each spectral data is isolated or a preset tree depth is reached. Due to their sparsity, anomalous points are usually isolated within a few splitting steps, thus forming specific isolation forest partitions.
[0105] The path length from the root node of the isolation tree to the isolated leaf node of each real-time spectral data point is calculated. The absolute value of the difference between this path length and the average of all path lengths is then calculated. This absolute value is compared to a preset outlier threshold (which can be adjusted as needed) to determine if the corresponding spectral data is abnormal. If the absolute value of the difference is less than or equal to the preset outlier threshold, the corresponding real-time spectral data is considered normal. If the absolute value of the difference is greater than the preset outlier threshold, the corresponding real-time spectral data is considered abnormal.
[0106] After verification using the Isolation Forest algorithm, the timestamps corresponding to the inflection point data that were determined to be non-anomaly data were ultimately confirmed as the inflection points of the spectrum. The inflection point data that were determined to be anomaly data were thus confirmed as such because their anomalies caused changes in the clustering structure.
[0107] For example, after processing a certain beverage, the processing tank needs to be cleaned for 90 minutes. The cleanliness level of the tank after cleaning can be identified by detecting the Raman spectra of the liquid samples in the tank during the cleaning process. Obviously, the longer the cleaning time, the cleaner the processing tank will be. However, the changes in the liquid samples in the processing tank are often not smooth and gradual, but rather abrupt changes occur near a few turning points. Sample spectra are collected every 3 minutes to obtain clustering tree structure data, such as... Figure 4 and Figure 5 As shown (where, Figure 4 A detailed structural diagram is provided. Figure 5 (To simplify the structural diagram), where T represents time (in minutes), from Figure 4 and Figure 5 It can be seen that four turning points were identified through clustering tree structure data: 15 minutes, 30 minutes, 66 minutes, and 69 minutes. Figure 4 The spectral data for the 66-minute period was also identified as an anomalous sample. Therefore, a schematic diagram of the change in the cleanliness of the liquid sample within the processing tank over time can be plotted, such as... Figure 6 As shown.
[0108] As can be seen from the above, this spectral transition time identification method periodically acquires the real-time spectral data of the sample to be tested, adopts an agglomerative hierarchical clustering method, and combines Mahalanobis distance and minimum variance method to merge the real-time spectral data into corresponding cluster tree structure data according to the time series. Through the sliding window algorithm and Fourier transform algorithm, based on the merging distance in the cluster tree structure data, the transition point data that meets the preset clustering distance abrupt change condition and the preset frequency domain feature jump condition are determined. The transition point data is verified according to the isolated forest algorithm to determine the time point corresponding to the transition point data of non-abnormal data as the transition time point. Therefore, by using agglomerative hierarchical clustering, Mahalanobis distance, minimum variance method, and time series, a clustering tree structure data corresponding to real-time spectral data is constructed. Based on the sliding window algorithm and Fourier transform algorithm, combined with the merging distance, the inflection point data in the clustering tree structure data that meet the conditions of abrupt change in clustering distance and jump in frequency domain features are identified to confirm the inflection time point. This solves the problem that existing spectral inflection time identification methods rely on preset thresholds or human experience judgment, making it difficult to accurately identify spectral inflection time points. It can objectively analyze the internal structural changes of spectral data and improve the identification efficiency of spectral inflection time.
[0109] refer to Figure 2 This application provides a spectral transition time identification device for identifying transition time points in spectral data, including:
[0110] Acquisition module 1 is used to periodically acquire real-time spectral data of the sample to be tested;
[0111] Merging module 2 is used to merge real-time spectral data into corresponding clustering tree structure data by using an agglomerative hierarchical clustering method, combined with Mahalanobis distance and minimum variance method, according to the time series.
[0112] Module 3 is used to determine the turning point data that satisfies the preset clustering distance abrupt change condition and the preset frequency domain feature jump condition based on the merging distance in the clustering tree structure data by using the sliding window algorithm and the Fourier transform algorithm.
[0113] Verification module 4 is used to verify the inflection point data according to the isolated forest algorithm, so as to determine the time point corresponding to the inflection point data of non-abnormal data as the inflection time point.
[0114] This spectral transition time identification device constructs a clustering tree structure data corresponding to real-time spectral data through agglomerative hierarchical clustering, Mahalanobis distance, minimum variance method, and time series. Based on the sliding window algorithm and Fourier transform algorithm, combined with the merging distance, it identifies the transition point data in the clustering tree structure data that meets the conditions of abrupt change in clustering distance and jump in frequency domain features, thus confirming the transition time point. This solves the problem that existing spectral transition time identification methods rely on preset thresholds or human experience to accurately identify spectral transition time points. It can objectively analyze the internal structural changes of spectral data and improve the identification efficiency of spectral transition time.
[0115] Specifically, when module 1 is executed, it automatically collects spectral data of the sample using a spectrometer at preset time points (such as per second, per minute, or longer). The sample is a substance whose chemical or physical properties can be detected by the spectrometer and change over time during production, processing, or treatment. This substance includes, but is not limited to, liquids (such as cleaning liquids in beverage processing tanks, reaction liquids in fermentation tanks, etc.), solids (such as food in the baking process, pharmaceutical powders in drying, tea leaves in fermentation, etc.), semi-solids (such as cosmetic pastes, food slurries, etc.), and gases (fermentation exhaust gases, chemical process gases, etc.).
[0116] Specifically, when merging module 2 uses an agglomerative hierarchical clustering method, combined with Mahalanobis distance and minimum variance method, to merge real-time spectral data into corresponding cluster tree structure data according to the time series, it executes the following:
[0117] According to the time series, each spectral data point of the real-time spectral data is divided into a separate cluster to calculate the spectral mean vector of each cluster;
[0118] The covariance values between all pairs of clusters are calculated using Mahalanobis distance and based on the spectral mean vector.
[0119] Using the minimum variance method, with the covariance value as the merging distance, real-time spectral data are sequentially merged into a cluster to obtain the corresponding clustering tree structure data.
[0120] When the merging module 2 is executed, in the initial stage of the clustering process, it sorts each independent spectral data in the real-time spectral data according to the time series, and regards each independent spectral data as an independent clustering unit, that is, an independent cluster (this cluster is the initial cluster). For each independent cluster, the spectral mean vector of the cluster is calculated based on its corresponding spectral data. This spectral mean vector represents the average characteristics of the spectral data within the cluster, providing a basis for subsequent distance calculations.
[0121] Based on Mahalanobis distance and using the spectral mean vector as the basis, the covariance value between clusters is calculated. This covariance value reflects the similarity between different clusters (spectral data); a larger value indicates greater difference, and a smaller value indicates higher similarity. The specific formula for calculating the covariance value is as follows:
[0122] ;
[0123] in, For clusters and cluster The covariance value; For clusters The amount of spectral data in the data; For clusters The amount of spectral data in the data; For clusters spectral mean vector; For clusters spectral mean vector; This is the norm symbol.
[0124] Specifically, when merging module 2 uses the minimum variance method, with the covariance value as the merging distance, to sequentially merge real-time spectral data into a single cluster to obtain the corresponding clustering tree structure data, it executes the following:
[0125] A1, merge the two clusters corresponding to the minimum covariance values into a new cluster, record the new cluster in the same branch of the tree diagram, and determine the corresponding covariance value as the merging distance of the corresponding branch;
[0126] A2, recalculate the new covariance values between all clusters, merge the two clusters corresponding to the minimum value of the new covariance values into a new cluster, record the new cluster in the same branch of the tree diagram, and determine the corresponding covariance value as the merging distance of the corresponding branch.
[0127] A3. Repeat step A2 to merge the clusters corresponding to the real-time spectral data into one cluster in sequence to obtain the corresponding clustering tree structure data.
[0128] When the merging module 2 is executed, it selects two clusters with the smallest covariance value from all possible cluster pairs and merges them. The two clusters are merged to form a new cluster and form a branch in the clustering tree structure data. The merging distance of this branch is the covariance value of the two clusters when they are merged.
[0129] The merged new cluster is treated as a whole, and the new covariance value is recalculated with the remaining clusters (including the original unmerged clusters and the new clusters formed by previous merges). The two clusters with the smallest new covariance value are selected from all cluster pairs for merging, and their merging distance and branch information are recorded.
[0130] Therefore, the above steps are repeated to continuously merge clusters, selecting the two closest clusters (with the smallest covariance values) among all current clusters at each merge, until all real-time spectral data are finally merged into a single cluster, constructing a complete clustering tree structure. Each merge operation corresponds to a branch in the tree, and the merge distance represents the similarity of that branch during the merge.
[0131] Specifically, when module 3 determines the inflection point data that satisfies the preset clustering distance abrupt change condition and the preset frequency domain feature jump condition based on the merging distance in the clustering tree structure data through the sliding window algorithm and Fourier transform algorithm, it executes:
[0132] The merge distances corresponding to each branch in the clustering tree structure data are sorted according to the order in which the merge distances are obtained, so as to construct a distance sequence.
[0133] The sliding window algorithm is used to calculate the mean distance and standard deviation of the distance sequence within each first preset window, which are used to determine the real-time spectral data corresponding to the merged distance that meets the preset clustering distance mutation conditions as the initial inflection point data.
[0134] The Fourier transform algorithm is used to analyze the changes in the spectral frequency components of the initial inflection point data, in order to determine the initial inflection point data that meets the preset frequency domain characteristic jump conditions as the inflection point data.
[0135] When module 3 is executed, it can form a distance sequence that reflects the evolution of the clustering process by arranging the merged distances in the order of merging. This distance sequence provides basic data for subsequent inflection point detection.
[0136] Specifically, when module 3 calculates the mean distance and standard deviation of the distance sequence within each first preset window using the sliding window algorithm, and uses this to determine the real-time spectral data corresponding to the merged distance that meets the preset clustering distance abrupt change conditions as the initial inflection point data, it executes the following:
[0137] Using the sliding window algorithm, a first preset window is set with each merged distance as the center, and the mean distance and standard deviation of the distance sequence within each first preset window are calculated.
[0138] Based on the distance mean and distance standard deviation, a clustering distance mutation threshold is constructed within each first preset window;
[0139] The system sequentially determines whether each merged distance in the distance sequence is greater than the corresponding clustering distance mutation threshold within the first preset window. If so, it determines that the merged distance meets the preset clustering distance mutation condition and determines the real-time spectral data corresponding to the merged distance as the initial inflection point data. If not, it determines that the merged distance does not meet the preset clustering distance mutation condition.
[0140] When module 3 is executed, it uses a sliding window algorithm to calculate the mean and standard deviation of distances within a first preset window, which quantifies the central tendency and dispersion of the merged distances within that first preset window. The length of the first preset window is set according to actual needs.
[0141] Based on the distance mean and distance standard deviation, a clustering distance mutation threshold is constructed within each first preset window. For example, the 3σ criterion can be used to set the threshold, setting it as the sum of the distance mean and three times the distance standard deviation. , The mean distance within the m-th (m≤n) preset window (i.e., the preset window corresponding to the m-th merged distance). denoted as the standard deviation of the distance within the m-th preset window.
[0142] The algorithm sequentially checks whether each merge distance exceeds the corresponding cluster distance abrupt change threshold within a first preset window to identify merge distances that significantly deviate from their local average level. These deviations typically indicate significant changes in the spectral data structure, thus pointing to potential inflection points. When a merge distance exceeds the corresponding cluster distance abrupt change threshold within the first preset window, it is determined that the merge distance meets the preset cluster distance abrupt change condition. The real-time spectral data of the cluster corresponding to the merge distance (i.e., the two clusters corresponding to the merge distance at the time of acquisition) is determined as the initial inflection point data, indicating that the sample's spectral data may have undergone a significant change in cluster structure at the corresponding time point, which is a key characteristic of inflection points. When a merge distance does not exceed the corresponding cluster distance abrupt change threshold within the first preset window, it indicates that the merge distance does not meet the preset cluster distance abrupt change condition, and it is determined that the sample's spectral data has not changed at the corresponding time point.
[0143] Specifically, when module 3 analyzes the spectral frequency component changes of the initial inflection point data using the Fourier transform algorithm to determine the initial inflection point data that meets the preset frequency domain characteristic jump conditions, it executes the following:
[0144] The spectral frequency components of all real-time spectral data were calculated using the Fourier transform algorithm.
[0145] Based on the spectral frequency component data, the change in spectral frequency components between each real-time spectral data and the previous real-time spectral data is calculated according to the time series.
[0146] Using a sliding window algorithm, a second preset window is set with each real-time spectral data as the center. The mean value and standard deviation of the spectral frequency component changes within each second preset window are calculated to construct the frequency domain feature jump threshold within each second preset window.
[0147] Sequentially determine whether the change value of the spectral frequency component corresponding to each real-time spectral data is greater than the frequency domain feature jump threshold within the corresponding second preset window; if yes, determine that the real-time spectral data meets the preset frequency domain feature jump condition, and determine the initial inflection point data that meets the preset frequency domain feature jump condition as the inflection point data; if no, determine that the real-time spectral data does not meet the preset frequency domain feature jump condition.
[0148] When module 3 is executed, it performs a Fourier transform on the real-time spectral data to obtain its energy distribution at different frequencies, i.e., spectral frequency component data. This frequency component data can reflect the intrinsic structure and variation patterns of the spectral signal, such as the vibrational frequencies of specific chemical bonds or the characteristic frequencies of molecular structures.
[0149] Based on spectral frequency component data, the change in spectral frequency components between each real-time spectral data and the previous real-time spectral data is calculated according to the time series, so as to quantify the dynamic evolution of spectral frequency characteristics over time.
[0150] Using a sliding window algorithm, a second preset window (with the same length as the first preset window, adjustable as needed) is set centered on each real-time spectral data point. The mean and standard deviation of spectral frequency component changes within each second preset window are calculated to construct a frequency domain feature jump threshold for each window. This threshold can be adjusted based on the position of each preset window, thereby more accurately identifying abnormal and significant frequency jumps. The frequency domain feature jump threshold can also be set using the aforementioned 3σ criterion, or adjusted as needed.
[0151] The system sequentially checks whether the change in spectral frequency components corresponding to each real-time spectral data exceeds the corresponding frequency domain feature jump threshold within a second preset window. This accurately identifies spectral data that undergo significant jumps in the frequency domain, thus ultimately determining the initial inflection point data that meets the conditions as inflection point data. This effectively eliminates the interference of random noise and improves the accuracy and robustness of inflection point identification. When the change in spectral frequency components exceeds the corresponding frequency domain feature jump threshold within the second preset window, the corresponding real-time spectral data is determined to meet the preset frequency domain feature jump condition. The initial inflection point data that meets the preset frequency domain feature jump condition is then identified as inflection point data. This confirms that the initial inflection point data in the sample that meets the preset frequency domain feature jump condition has undergone a significant change in cluster structure at the corresponding time point, effectively capturing the true inflection point of sample state change. When the change in spectral frequency components does not exceed the corresponding frequency domain feature jump threshold within the second preset window, it indicates that the corresponding real-time spectral data does not meet the preset frequency domain feature jump condition, and the spectral data of the sample has not changed at the corresponding time point.
[0152] Specifically, when verification module 4 verifies the inflection point data according to the isolated forest algorithm to determine the inflection point time corresponding to the inflection point data of non-abnormal data, it performs the following:
[0153] The isolated forest algorithm is used to divide the real-time spectral data into isolated forest partitions corresponding to each real-time spectral data.
[0154] Determine the path length of each real-time spectral data in the corresponding isolated forest partition, and calculate the absolute value of the difference between the path length and the average of all path lengths. The real-time spectral data corresponding to the path length whose absolute value of the difference is less than or equal to the preset abnormal isolation threshold is considered as non-abnormal data.
[0155] The time point corresponding to the turning point data that is determined to be non-abnormal data is the turning point time point.
[0156] During execution, verification module 4 processes the real-time spectral data using the Isolation Forest algorithm (a pre-trained algorithm that, after training, can identify outliers by randomly selecting features and split points) to obtain the isolation forest partition to which each spectral data belongs. Specifically, this can be achieved by constructing multiple isolation trees. Each isolation tree recursively partitions the data by randomly selecting features and split points until each spectral data is isolated or a preset tree depth is reached. Due to their sparsity, outliers are typically isolated within a few splitting steps, thus forming specific isolation forest partitions.
[0157] The path length from the root node of the isolation tree to the isolated leaf node of each real-time spectral data point is calculated. The absolute value of the difference between this path length and the average of all path lengths is then calculated. This absolute value is compared to a preset outlier threshold (which can be adjusted as needed) to determine if the corresponding spectral data is abnormal. If the absolute value of the difference is less than or equal to the preset outlier threshold, the corresponding real-time spectral data is considered normal. If the absolute value of the difference is greater than the preset outlier threshold, the corresponding real-time spectral data is considered abnormal.
[0158] After verification using the Isolation Forest algorithm, the timestamps corresponding to the inflection point data that were determined to be non-anomaly data were ultimately confirmed as the inflection points of the spectrum. The inflection point data that were determined to be anomaly data were thus confirmed as such because their anomalies caused changes in the clustering structure.
[0159] For example, after processing a certain beverage, the processing tank needs to be cleaned for 90 minutes. The cleanliness level of the tank after cleaning can be identified by detecting the Raman spectra of the liquid samples in the tank during the cleaning process. Obviously, the longer the cleaning time, the cleaner the processing tank will be. However, the changes in the liquid samples in the processing tank are often not smooth and gradual, but rather abrupt changes occur near a few turning points. Sample spectra are collected every 3 minutes to obtain clustering tree structure data, such as... Figure 4 and Figure 5 As shown (where, Figure 4 A detailed structural diagram is provided. Figure 5 (To simplify the structural diagram), where T represents time (in minutes), from Figure 4 and Figure 5 It can be seen that four turning points were identified through clustering tree structure data: 15 minutes, 30 minutes, 66 minutes, and 69 minutes. Figure 4 The spectral data for the 66-minute period was also identified as an anomalous sample. Therefore, a curve showing the change in the cleanliness of the liquid sample within the processing tank over time can be plotted, approximately as follows: Figure 6 As shown.
[0160] As can be seen from the above, this spectral transition time identification device periodically acquires real-time spectral data of the sample to be tested, adopts an agglomerative hierarchical clustering method, and combines Mahalanobis distance and minimum variance method to merge the real-time spectral data into corresponding cluster tree structure data according to the time series. Through the sliding window algorithm and Fourier transform algorithm, based on the merging distance in the cluster tree structure data, the transition point data that meet the preset clustering distance abrupt change conditions and the preset frequency domain feature jump conditions are determined. According to the isolated forest algorithm, the transition point data is verified to determine the time point corresponding to the transition point data of non-abnormal data as the transition time point. Therefore, by using agglomerative hierarchical clustering, Mahalanobis distance, minimum variance method, and time series, a clustering tree structure data corresponding to real-time spectral data is constructed. Based on the sliding window algorithm and Fourier transform algorithm, combined with the merging distance, the inflection point data in the clustering tree structure data that meet the conditions of abrupt change in clustering distance and jump in frequency domain features are identified to confirm the inflection time point. This solves the problem that existing spectral inflection time identification methods rely on preset thresholds or human experience judgment, making it difficult to accurately identify spectral inflection time points. It can objectively analyze the internal structural changes of spectral data and improve the identification efficiency of spectral inflection time.
[0161] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device includes a processor 301 and a memory 302. The processor 301 and the memory 302 are interconnected and communicate with each other via a communication bus 303 and / or other connection mechanisms (not shown). The memory 302 stores a computer program executable by the processor 301. When the electronic device is running, the processor 301 executes the computer program to perform the spectral transition time identification method in any optional implementation of the above embodiments, to achieve the following functions: periodically acquiring real-time spectral data of the sample to be tested; using an agglomerative hierarchical clustering method, combined with Mahalanobis distance and minimum variance method, merging the real-time spectral data into corresponding clustering tree structure data according to the time series; using a sliding window algorithm and a Fourier transform algorithm, determining the transition point data that satisfies the preset clustering distance abrupt change condition and the preset frequency domain feature jump condition based on the merging distance in the clustering tree structure data; verifying the transition point data according to the isolated forest algorithm to determine the time point corresponding to the transition point data of non-abnormal data as the transition time point.
[0162] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it executes the spectral transition time identification method in any optional implementation of the above embodiments to achieve the following functions: periodically acquiring real-time spectral data of the sample to be tested; using an agglomerative hierarchical clustering method, combined with Mahalanobis distance and minimum variance method, merging the real-time spectral data into corresponding clustering tree structure data according to the time series; using a sliding window algorithm and a Fourier transform algorithm, determining the transition point data that meets the preset clustering distance abrupt change condition and the preset frequency domain feature jump condition based on the merging distance in the clustering tree structure data; verifying the transition point data according to the isolated forest algorithm, and determining the time point corresponding to the transition point data of non-abnormal data as the transition time point. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0163] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.
[0164] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.
[0165] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0166] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.
[0167] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A method for identifying spectral transition times, used to identify transition time points in spectral data, characterized in that, Including the following steps: Periodically acquire real-time spectral data of the sample to be tested; A cohesive hierarchical clustering method is adopted, which combines Mahalanobis distance and minimum variance method, to merge the real-time spectral data into corresponding cluster tree structure data according to the time series. Using the sliding window algorithm and Fourier transform algorithm, based on the merging distance in the clustering tree structure data, the turning point data that satisfies the preset clustering distance abrupt change condition and the preset frequency domain feature jump condition are determined. Based on the isolated forest algorithm, the inflection point data is verified to determine the time point corresponding to the inflection point data of non-abnormal data as the inflection time point. A cohesive hierarchical clustering method, combining Mahalanobis distance and minimum variance, is used to merge the real-time spectral data into corresponding cluster tree structure data according to the time series, including: According to the time series, each spectral data of the real-time spectral data is divided into a separate cluster to calculate the spectral mean vector of each cluster; Using Mahalanobis distance and based on the spectral mean vector, the covariance values between all pairs of clusters are calculated. Using the minimum variance method, with the covariance value as the merging distance, the real-time spectral data are sequentially merged into a cluster to obtain the corresponding clustering tree structure data. Using the sliding window algorithm and Fourier transform algorithm, based on the merging distance in the clustering tree structure data, inflection point data that satisfy preset clustering distance abrupt change conditions and preset frequency domain feature jump conditions are determined, including: According to the order in which the merge distances were obtained, the merge distances corresponding to each branch in the clustering tree structure data are sorted to construct a distance sequence; The sliding window algorithm is used to calculate the mean distance and standard deviation of the distance sequence within each first preset window, which are then used to determine the real-time spectral data corresponding to the merged distance that meets the preset clustering distance mutation conditions as the initial inflection point data. The spectral frequency component changes of the initial inflection point data are analyzed using the Fourier transform algorithm to determine the initial inflection point data that meets the preset frequency domain characteristic jump conditions as the inflection point data. Based on the isolated forest algorithm, the inflection point data is verified to determine the time points corresponding to the inflection point data of non-anomaly data as inflection time points, including: The isolated forest algorithm is used to divide the real-time spectral data into isolated forest partitions corresponding to each real-time spectral data. Determine the path length of each real-time spectral data in the corresponding isolated forest partition, and calculate the absolute value of the difference between the path length and the average of all path lengths, so as to determine that the real-time spectral data corresponding to the path length whose absolute value of the difference is less than or equal to a preset abnormal isolation threshold is non-abnormal data. The time point corresponding to the turning point data that is determined to be non-abnormal data is the turning point time point.
2. The spectral transition time identification method according to claim 1, characterized in that, Using the minimum variance method, with the covariance value as the merging distance, the real-time spectral data are sequentially merged into a single cluster to obtain the corresponding clustering tree structure data, including: A1, merge the two clusters corresponding to the minimum value of the covariance values into a new cluster, record the new cluster in the same branch of the tree diagram, and determine the corresponding covariance value as the merging distance of the corresponding branch; A2, recalculate the new covariance values between all clusters, merge the two clusters corresponding to the minimum value of the new covariance values into a new cluster, record the new cluster in the same branch of the tree diagram, and determine the corresponding covariance value as the merging distance of the corresponding branch. A3. Repeat step A2 to merge the clusters corresponding to the real-time spectral data into one cluster in sequence to obtain the corresponding clustering tree structure data.
3. The spectral transition time identification method according to claim 1, characterized in that, Using a sliding window algorithm, the mean distance and standard deviation of the distance sequence within each first preset window are calculated. These are used to determine the real-time spectral data corresponding to the merged distance that meets the preset clustering distance abrupt change conditions as the initial inflection point data, including: Using a sliding window algorithm, a first preset window is set with each of the merged distances as the center, and the mean distance and standard deviation of the distance sequence within each first preset window are calculated. Based on the mean distance and the standard deviation of distance, a clustering distance mutation threshold is constructed within each of the first preset windows; Sequentially determine whether each of the merged distances in the distance sequence is greater than the clustering distance mutation threshold within the corresponding first preset window; if yes, determine that the merged distance meets the preset clustering distance mutation condition, and determine the real-time spectral data corresponding to the merged distance as the initial turning point data; if no, determine that the merged distance does not meet the preset clustering distance mutation condition.
4. The spectral transition time identification method according to claim 1, characterized in that, The spectral frequency component changes of the initial inflection point data are analyzed using a Fourier transform algorithm to determine the initial inflection point data that meets the preset frequency domain characteristic jump conditions as inflection point data, including: The spectral frequency components of all the real-time spectral data were calculated using the Fourier transform algorithm. Based on the spectral frequency component data, the change value of the spectral frequency component between each real-time spectral data and the previous real-time spectral data is calculated according to the time series. Using a sliding window algorithm, a second preset window is set with each real-time spectral data as the center, and the mean value and standard deviation of the spectral frequency component change within each second preset window are calculated to construct the frequency domain feature jump threshold within each second preset window. Sequentially determine whether the change value of the spectral frequency component corresponding to each real-time spectral data is greater than the frequency domain feature jump threshold within the corresponding second preset window; if yes, determine that the real-time spectral data meets the preset frequency domain feature jump condition, and determine the initial inflection point data that meets the preset frequency domain feature jump condition as the inflection point data; if no, determine that the real-time spectral data does not meet the preset frequency domain feature jump condition.
5. A spectral transition time identification device, used to identify the transition time points of spectral data, characterized in that, include: The acquisition module is used to periodically acquire real-time spectral data of the sample to be tested; The merging module is used to merge the real-time spectral data into corresponding clustering tree structure data according to the time series by employing an agglomerative hierarchical clustering method, combined with Mahalanobis distance and the minimum variance method. The determination module is used to determine, based on the merging distance in the clustering tree structure data, the turning point data that satisfies the preset clustering distance abrupt change condition and the preset frequency domain feature jump condition, by using the sliding window algorithm and the Fourier transform algorithm. The verification module is used to verify the inflection point data according to the isolated forest algorithm, so as to determine the time point corresponding to the inflection point data of non-abnormal data as the inflection time point; The merging module is used to merge the real-time spectral data into corresponding clustering tree structure data according to the time series using an agglomerative hierarchical clustering method, combined with Mahalanobis distance and minimum variance method, including: According to the time series, each spectral data of the real-time spectral data is divided into a separate cluster to calculate the spectral mean vector of each cluster; Using Mahalanobis distance and based on the spectral mean vector, the covariance values between all pairs of clusters are calculated. Using the minimum variance method, with the covariance value as the merging distance, the real-time spectral data are sequentially merged into a cluster to obtain the corresponding clustering tree structure data. The determining module is used to determine, based on the merging distance in the clustering tree structure data and using a sliding window algorithm and a Fourier transform algorithm, inflection point data that satisfies preset clustering distance abrupt change conditions and preset frequency domain feature jump conditions, including: According to the order in which the merge distances were obtained, the merge distances corresponding to each branch in the clustering tree structure data are sorted to construct a distance sequence; The sliding window algorithm is used to calculate the mean distance and standard deviation of the distance sequence within each first preset window, which are then used to determine the real-time spectral data corresponding to the merged distance that meets the preset clustering distance mutation conditions as the initial inflection point data. The spectral frequency component changes of the initial inflection point data are analyzed using the Fourier transform algorithm to determine the initial inflection point data that meets the preset frequency domain characteristic jump conditions as the inflection point data. The verification module is used to verify the inflection point data according to the isolated forest algorithm, so as to determine the time point corresponding to the inflection point data of non-abnormal data as the inflection time point, including: The isolated forest algorithm is used to divide the real-time spectral data into isolated forest partitions corresponding to each real-time spectral data. Determine the path length of each real-time spectral data in the corresponding isolated forest partition, and calculate the absolute value of the difference between the path length and the average of all path lengths, so as to determine that the real-time spectral data corresponding to the path length whose absolute value of the difference is less than or equal to a preset abnormal isolation threshold is non-abnormal data. The time point corresponding to the turning point data that is determined to be non-abnormal data is the turning point time point.
6. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program executable by the processor, which, when executing the computer program, performs the steps of the spectral transition time identification method as described in any one of claims 1-4.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the steps in the spectral transition time identification method as described in any one of claims 1-4.
Citation Information
Patent Citations
Multi-stage spatial-temporal clustering method and system based on fused mahalanobis distance
CN121051489A