Digital factory equipment anomaly detection method and system based on voiceprint recognition

Abstract

The application provides a kind of digital factory equipment anomaly detection method and system based on voiceprint recognition, time-frequency domain joint analysis processing is carried out by obtaining the voiceprint signal stream of digital factory equipment, time-frequency hybrid feature set is generated, feature mapping processing is carried out, voiceprint embedding vector is generated, device working condition information corresponding to voiceprint signal stream is called, voiceprint embedding vector is associated with device working condition information Mapping, constructing feature expansion matrix, using hierarchical attention mechanism to execute key abnormal feature extraction processing, screening to obtain first key abnormal feature subset and second key abnormal feature subset, dynamic weighted fusion processing is carried out to the two, generate fusion feature vector, compare and analyze according to the energy distribution of abnormal feature and preset abnormal threshold, determine whether the equipment exists abnormal running state.The application can avoid the information break or redundancy problem in large-scale voiceprint data processing process, guarantee the stability and continuity of detection.

Description

Technical Field

[0001] This invention relates to the fields of voiceprint recognition and anomaly detection, and more specifically, to a method and system for anomaly detection in digital factory equipment based on voiceprint recognition. Background Technology

[0002] With the development of digital factories, capturing acoustic signals generated during equipment operation and analyzing the equipment operating status information contained in the signals can enable timely detection and judgment of equipment anomalies, ensuring the stable operation of factory equipment. Currently, the common practice is to acquire the acoustic signals of the equipment, perform simple time-frequency domain analysis to extract features, and then use preset thresholds or basic models for anomaly judgment. However, this processing method often fails to accurately capture subtle changes in the acoustic signals related to equipment anomalies, does not fully explore the intrinsic correlation between acoustic features and equipment operating conditions, and is easily interfered with by irrelevant signals or redundant information, resulting in insufficient accuracy and reliability of anomaly identification results, and failing to meet the high precision and adaptability requirements of digital factories for equipment anomaly detection. Summary of the Invention

[0003] This invention provides a method and system for detecting abnormalities in digital factory equipment based on voiceprint recognition.

[0004] In a first aspect, embodiments of the present invention provide a method for detecting anomalies in digital factory equipment based on voiceprint recognition. The method includes: acquiring a voiceprint signal stream generated by digital factory equipment during operation, the voiceprint signal stream carrying time-frequency domain fluctuation information and a timestamp marker corresponding to the equipment's operating state; performing time-frequency domain joint analysis processing on the voiceprint signal stream to generate a time-frequency hybrid feature set containing time-dimensional fluctuation features and frequency-dimensional spectral features; inputting the time-frequency hybrid feature set into a preset self-supervised pre-trained model for feature mapping processing to generate a voiceprint embedding vector; retrieving equipment operating condition information corresponding to the voiceprint signal stream; associating and mapping the voiceprint embedding vector with the equipment operating condition information to construct a feature expansion matrix with operating condition labels; performing key anomaly feature extraction processing based on the feature expansion matrix using a hierarchical attention mechanism, obtaining a first key anomaly feature subset and a second key anomaly feature subset through multi-level feature filtering; performing dynamic weighted fusion processing on the first key anomaly feature subset and the second key anomaly feature subset to generate a fused feature vector; and comparing and analyzing the energy distribution of the anomaly features in the fused feature vector with a preset anomaly threshold to determine whether the equipment has an abnormal operating state.

[0005] Secondly, embodiments of the present invention provide a computer system, including: a memory storing a computer program; and a processor for loading the computer program to implement the digital factory equipment anomaly detection method based on voiceprint recognition as described above.

[0006] This invention acquires the acoustic signature signal stream during equipment operation and performs joint time-frequency domain analysis to generate a time-frequency hybrid feature set containing time-dimensional fluctuation features and frequency-dimensional spectral features. A self-supervised pre-trained model is used to map the time-frequency hybrid feature set to obtain an acoustic signature embedding vector. Combined with equipment operating condition information, a feature expansion matrix with operating condition labels is constructed. A hierarchical attention mechanism is used to extract key abnormal feature subsets and dynamically weighted and fused. The equipment operating status is determined by comparing the energy distribution of the fused feature vector with a preset abnormality threshold. This method can accurately capture subtle changes in the acoustic signature signal during equipment operation and the inherent correlation between acoustic signature features and operating condition information, reducing interference from irrelevant information and effectively improving the accuracy of equipment abnormality identification. Furthermore, through the synergistic effect of time-frequency domain joint analysis and the hierarchical attention mechanism, it can specifically strengthen the identification of features related to equipment abnormalities. By refining the feature representation of key components and mitigating the influence of redundant features, the extraction of abnormal features becomes more targeted, thereby improving the reliability of anomaly detection results. Furthermore, the method of constructing a feature extension matrix by combining equipment operating condition information can adapt to the feature change patterns under different operating scenarios, enhancing the adaptability of the method in complex operating environments. In addition, the dynamic weighted fusion processing method can adjust the fusion weights according to the actual contribution of different key abnormal feature subsets, making the fused feature vector more reflective of the actual operating status of the equipment, further ensuring the accuracy of anomaly identification results. At the same time, the entire processing flow does not rely on complex manual intervention and can automatically complete the entire process from voiceprint signal acquisition to anomaly status determination, improving the efficiency of equipment anomaly detection, avoiding information breakage or redundancy problems in large-scale voiceprint data processing, and ensuring the stability and continuity of the detection process. Attached Figure Description

[0007] Figure 1 This is a flowchart of a digital factory equipment anomaly detection method based on voiceprint recognition provided in an embodiment of the present invention.

[0008] Figure 2 This is a schematic diagram of the composition of a computer system provided in an embodiment of the present invention. Detailed Implementation

[0009] Please see Figure 1 This is a flowchart of a digital factory equipment anomaly detection method based on voiceprint recognition provided by an embodiment of the present invention. The method can be executed by a computer system and may include the following steps: Step S100: Obtain the acoustic signature signal stream generated by the digital factory equipment during operation. The acoustic signature signal stream carries time-frequency domain fluctuation information and timestamp markers corresponding to the equipment's operating status.

[0010] Time-frequency domain fluctuation information reflects the changes in sound signals in the time and frequency dimensions during equipment operation. Different equipment operating states will produce different time-frequency domain fluctuation patterns. Timestamps assign precise time information to each data point in the acoustic signature signal stream. To acquire the acoustic signature signal stream, multiple high-sensitivity microphones can be strategically placed around the equipment in the digital factory. These microphones can capture the sound generated during equipment operation in real time and convert it into electrical signals. Subsequently, the electrical signals are processed by an analog-to-digital converter and converted into a digitized acoustic signature signal stream. During the acquisition process, the system automatically adds timestamps to each acquired signal data point. These timestamps are accurate to the millisecond level, thus ensuring the accuracy of the time information in the acoustic signature signal stream.

[0011] Step S200: Perform joint time-frequency domain analysis on the acoustic signature signal stream to generate a time-frequency hybrid feature set containing time-dimensional fluctuation features and frequency-dimensional spectral features.

[0012] Joint time-frequency domain analysis is an analytical method that comprehensively considers the characteristics of acoustic signature signals in both time and frequency dimensions. The time-dimensional fluctuation characteristics mainly reflect the changes in the acoustic signature signal over time, such as fluctuations in signal amplitude and phase changes; the frequency-dimensional spectral characteristics reflect the distribution and energy of different frequency components in the acoustic signature signal. The time-frequency hybrid feature set is the result of integrating the time-dimensional fluctuation characteristics and the frequency-dimensional spectral characteristics, providing a more comprehensive and accurate reflection of the device's operating status.

[0013] In one implementation, step S200 may specifically include the following steps S210 to S260: Step S210: According to the timestamp of the voiceprint signal stream, the voiceprint signal stream is segmented into segments with continuous and non-overlapping fixed time windows to obtain multiple orderly arranged segmented voiceprint signal units. Each segmented voiceprint signal unit is bound to the corresponding timestamp start point and end point information.

[0014] A fixed time window is a pre-defined time length used to segment the voiceprint signal stream. By segmenting according to timestamps, a continuous voiceprint signal stream can be divided into multiple independent segmented voiceprint signal units. Each segmented voiceprint signal unit contains voiceprint signal information within that time period and is bound to corresponding timestamp start and end points. In practice, a suitable fixed time window length is first determined based on the characteristics of the voiceprint signal stream and the needs of subsequent analysis. Then, based on the timestamps in the voiceprint signal stream, segmentation is performed sequentially according to the fixed time window length, starting from the beginning of the signal stream. For example, for a voiceprint signal stream that lasts for a relatively long time, a shorter fixed time window is set, and a segmented voiceprint signal unit is created every other fixed time window, starting from the first timestamp. During the segmentation process, the start and end timestamps of each segmented voiceprint signal unit are recorded to ensure that the time range of each unit is clearly defined.

[0015] Step S220: Perform time-dimensional point-by-point analysis on each segmented voiceprint signal unit, record the changes in signal amplitude at each sampling moment and the evolution trajectory of phase over time, obtain the original time-dimensional change sequence, and simultaneously associate the boundary change data of adjacent segmented voiceprint signal units.

[0016] Time-dimensional point-by-point analysis is a process of detailed analysis of each sampling point in the segmented acoustic signature signal unit. The change in signal amplitude at each sampling moment reflects the intensity change of the acoustic signature signal over time, while the phase evolution trajectory over time reflects the dynamic change of the signal phase in the time dimension. The original time-dimensional change sequence is a sequence obtained by arranging these amplitude changes and phase evolution trajectories in chronological order. Correlating the boundary change data of adjacent segmented acoustic signature signal units can help analyze the transition characteristics of the signal between adjacent time periods, and better understand the continuity and variability of the device's operating state.

[0017] In one implementation, step S220 may specifically include the following steps S221 to S226: Step S221: Extract the amplitude and phase data of each segmented acoustic signal unit at all sampling times, arrange them sequentially according to the sampling time order to obtain the original amplitude sequence and the original phase sequence, so that the data order is consistent with the timestamp order.

[0018] Amplitude data represents the intensity of the acoustic signature signal at each sampling moment, while phase data represents the phase angle of the signal at that moment. By extracting and arranging this data in chronological order of sampling time, original amplitude and phase sequences can be obtained, ensuring that the data order is consistent with the timestamp order. Signal processing algorithms can be used to process the segmented acoustic signature signal units during the extraction of amplitude and phase data. For example, the amplitude and phase information of each sampling point can be extracted from the segmented acoustic signature signal units. Then, according to the chronological order of sampling time, these amplitude and phase data are arranged sequentially to form the original amplitude and phase sequences. Based on this, each data point in the sequence corresponds one-to-one with the timestamp in the acoustic signature signal stream, ensuring the accuracy of the data's temporal order.

[0019] Step S222: Perform point-by-point difference calculation on the original amplitude sequence to obtain the amplitude change at adjacent sampling times, and perform point-by-point difference calculation on the original phase sequence to obtain the phase change at adjacent sampling times, thus obtaining the amplitude change sequence and the phase change sequence.

[0020] For the original amplitude sequence, the amplitude change at adjacent sampling times can be obtained by calculating the amplitude difference between two adjacent sampling times. Arranging these changes in order forms the amplitude change sequence. Similarly, the same point-by-point difference calculation is performed on the original phase sequence to obtain the phase change sequence. These change sequences more clearly reflect the rate and trend of change of the acoustic signature signal over time. During the point-by-point difference calculation, starting from the first data point of both the original amplitude and phase sequences, the difference between adjacent data points is calculated sequentially. For example, for the original amplitude sequence, the amplitude of the previous data point is subtracted from the amplitude of the subsequent data point to obtain the amplitude change. Arranging all the calculated amplitude changes in the order of the original sequence yields the amplitude change sequence. The same operation is performed on the original phase sequence to obtain the phase change sequence.

[0021] Step S223: Divide the amplitude change sequence into sub-intervals, calculate the dispersion of the change in each sub-interval, mark the sub-intervals whose dispersion meets the preset standard as amplitude stable intervals, and mark the sub-intervals whose dispersion does not meet the preset standard as amplitude fluctuation intervals, and record the starting sampling point and ending sampling point of each interval.

[0022] Dividing the signal into sub-intervals allows for localized analysis of amplitude variation sequences, providing a better understanding of signal changes across different time periods. The degree of dispersion can be measured using pre-defined methods, such as calculating a statistical characteristic of the changes within a sub-interval. The pre-defined standard is a threshold set based on extensive experimental data and experience, used to determine whether the dispersion within a sub-interval meets the requirements. A stable amplitude interval indicates relatively smooth amplitude changes within that time period, while a fluctuating amplitude interval indicates more drastic amplitude changes. Recording the start and end sampling points of each interval clarifies its time range, facilitating targeted analysis of different intervals later. When dividing the signal into sub-intervals, the appropriate sub-interval length and division method are first determined. Based on the characteristics of the amplitude variation sequence and analytical needs, the sequence can be divided into multiple continuous sub-intervals. Then, for each sub-interval, the degree of dispersion of its internal changes is calculated. For example, a statistical method can be used to process the amplitude changes within the sub-interval, yielding a value representing the degree of dispersion. This value is compared to the pre-defined standard; if it meets the standard, the sub-interval is marked as a stable amplitude interval; otherwise, it is marked as a fluctuating amplitude interval. At the same time, the starting and ending sampling points of each interval are recorded for subsequent analysis.

[0023] Step S224: Perform the same sub-interval division and dispersion calculation on the phase change sequence. Mark the sub-intervals that meet the preset standard as phase stable intervals and the sub-intervals that do not meet the preset standard as phase fluctuating intervals. Determine the sampling point range of each interval and compare the time overlap range of the amplitude and phase stable intervals and fluctuating intervals.

[0024] Similar to the processing of amplitude change sequences, the phase change sequences are divided into sub-intervals and their dispersion is calculated to determine the stable phase intervals and phase fluctuation intervals. By comparing the time overlap range of the stable and fluctuation intervals of amplitude and phase, the correlation characteristics of equipment operating status in both time and phase dimensions can be discovered, further revealing information about the equipment operating status. The same methods and standards are used for sub-interval division and dispersion calculation of the phase change sequences as for amplitude change sequences. After determining the stable and fluctuation intervals of phase, the sampling point range of each interval is recorded. Then, the stable and fluctuation intervals of amplitude are compared with those of phase over time to identify their overlap range.

[0025] Step S225: Retrieve the last sampling point data of the previous segment voiceprint signal unit and the starting sampling point data of the next segment voiceprint signal unit, and calculate the amplitude difference and phase difference between the current segment and the previous and next segments at the boundary sampling points.

[0026] Boundary sampling point data reflects the transition information between adjacent segmented acoustic signal units. Calculating the amplitude difference and phase difference between the current segment and the segments before and after it at the boundary sampling points can analyze the abrupt changes in acoustic signals between adjacent time periods, which helps to detect sudden changes in the operating status of the equipment.

[0027] In practice, based on the timestamp information of the current segmented voiceprint signal unit, the last sampling point of the preceding segment and the first sampling point of the following segment are located. Then, the amplitude and phase data of these sampling points are extracted, and the amplitude and phase differences between the current segment and the preceding and following segments at the boundary sampling points are calculated. For example, the amplitude difference is obtained by subtracting the amplitude of the last sampling point of the preceding segment from the amplitude of the current segment's first sampling point; the phase difference is obtained by subtracting the phase of the last sampling point of the preceding segment from the phase of the current segment's first sampling point. The same calculation is performed for the end sampling point of the current segment and the beginning sampling point of the following segment.

[0028] Step S226: Integrate the original amplitude sequence, the original phase sequence, the amplitude change sequence, the phase change sequence, and the boundary difference data to obtain the original change sequence in the time dimension. Each sequence is bound with the corresponding interval label and boundary association information.

[0029] During the integration process, the original amplitude sequence, original phase sequence, amplitude change sequence, phase change sequence, and calculated boundary difference data are combined according to certain rules. Simultaneously, corresponding amplitude stability intervals, amplitude fluctuation intervals, phase stability intervals, and phase fluctuation interval markers, as well as boundary correlation information, are added to each sequence. For example, this information can be stored in the form of a certain data structure, so that each sequence can clearly reflect its corresponding interval characteristics and boundary correlation.

[0030] Step S230: Decompose each segmented voiceprint signal unit into frequency dimension, analyze the distribution range of various frequency components contained in the signal and their respective energy transfer, obtain the original frequency dimension distribution sequence, and simultaneously record the occurrence order and duration of different frequency components.

[0031] When performing frequency-dimensional decomposition, signal processing methods such as Fourier transform can be used to convert the segmented acoustic signature signal units from the time domain to the frequency domain. By analyzing the frequency domain signal, the distribution range of various frequency components within the signal can be determined, i.e., the frequency range of each frequency component can be identified. Simultaneously, the energy transfer of each frequency component is calculated, such as analyzing its magnitude and variation trend. After obtaining the original frequency-dimensional distribution sequence, the occurrence order and duration of different frequency components are recorded synchronously. This information can help in further analyzing the dynamic changes in the equipment's operating status.

[0032] Step S240: Align the original time dimension change sequence of each segmented voiceprint signal unit with the original frequency dimension distribution sequence by timestamp, establish the real-time correspondence between the two, and generate a preliminary time-frequency feature association pair.

[0033] During timestamp alignment, the timestamp information in the original time-dimension variation sequence and the original frequency-dimension distribution sequence is matched temporally. For example, the timestamp corresponding to each data point in the original time-dimension variation sequence is found, and then the frequency component information corresponding to the same timestamp is found in the original frequency-dimension distribution sequence, and they are associated. Based on this, preliminary time-frequency feature association pairs are generated for each segmented acoustic signature unit.

[0034] Step S250: Perform cross-segment feature fusion on the preliminary time-frequency feature association pairs of all segmented voiceprint signal units, extract the time-frequency feature evolution rules between adjacent segments, supplement boundary transition feature information, and obtain complete time-frequency feature association pairs.

[0035] When performing cross-segment feature fusion, the initial comparison and analysis of the time-frequency feature correlation pairs between adjacent segments are conducted. For example, the energy changes of the same frequency components in adjacent segments, and the trends of amplitude and phase changes over time are observed. This analysis extracts the evolution patterns of time-frequency features between adjacent segments. Then, based on the boundary sampling point data and the previously calculated boundary difference information, boundary transition feature information is supplemented. For example, the abrupt changes in amplitude and phase at the boundaries of adjacent segments are considered, and this information is incorporated into the cross-segment feature fusion. Finally, a complete time-frequency feature correlation pair is obtained, which can more comprehensively and accurately reflect the changes in the time-frequency features of the voiceprint signal over different time periods.

[0036] Step S260: Integrate and arrange all time-frequency features in complete association according to timestamp order to obtain a time-frequency hybrid feature set containing time-dimensional fluctuation features and frequency-dimensional spectral features. Each feature component retains the corresponding segment identifier and timestamp information.

[0037] Integrating all time-frequency feature pairs in complete association according to timestamp order forms an ordered time-frequency hybrid feature set. This set contains comprehensive feature information of the voiceprint signal in both time and frequency dimensions. Each feature component retains its corresponding segment identifier and timestamp information, facilitating subsequent querying and analysis of features from different time periods and segments. During the integration process, the time-frequency feature pairs are sorted chronologically based on their timestamp information. Then, the sorted time-frequency feature pairs are combined to form the time-frequency hybrid feature set. Simultaneously, corresponding segment identifiers and timestamp information are added to each feature component, ensuring that each feature clearly belongs to its corresponding segment and time range.

[0038] Step S300: Input the time-frequency hybrid feature set into the preset self-supervised pre-trained model for feature mapping processing to generate a voiceprint embedding vector, retrieve the equipment operating condition information corresponding to the voiceprint signal stream, associate and map the voiceprint embedding vector with the equipment operating condition information, and construct a feature expansion matrix with operating condition labels.

[0039] A self-supervised pre-trained model is a pre-trained model that performs feature mapping on the input time-frequency mixed feature set, converting it into a low-dimensional, dense voiceprint embedding vector. The voiceprint embedding vector is a compact representation of the time-frequency mixed feature set, preserving the key feature information of the voiceprint signal. Equipment operating condition information reflects various working states and environmental conditions of the equipment during operation. Associating the voiceprint embedding vector with the equipment operating condition information allows for the construction of a feature extension matrix with operating condition labels. This matrix can more comprehensively reflect the operating status of the equipment.

[0040] In one implementation, step S300 may specifically include the following steps S310 to S360: Step S310: Perform dimensional verification on each time-frequency feature pair in the time-frequency hybrid feature set, check the dimensional consistency between the time dimension fluctuation feature and the frequency dimension spectrum feature, perform dimensional alignment processing on features with inconsistent dimensions, and generate a standardized time-frequency hybrid feature set.

[0041] Dimension verification ensures that the time-dimensional fluctuation features and the frequency-dimensional spectral features in the time-frequency mixed feature set are consistent in dimension. Inconsistent dimensions negatively impact the processing performance of subsequent models. Dimension alignment adjusts features with inconsistent dimensions to achieve consistency. The standardized time-frequency mixed feature set, after dimension verification and alignment, possesses a unified dimensional structure, facilitating its input into a self-supervised pre-trained model. During dimensional verification, each complete association pair of time-frequency features in the mixed feature set is checked, comparing the number and structure of dimensions between the time-dimensional fluctuation features and the frequency-dimensional spectral features. If inconsistencies are found, dimension alignment is performed. For example, features with fewer dimensions can be expanded, or features with more dimensions can be compressed to achieve consistency. This ultimately generates a standardized time-frequency mixed feature set, providing a unified input for subsequent feature mapping processing.

[0042] Step S320: Input the standardized time-frequency hybrid feature set into the input layer of the preset self-supervised pre-trained model, perform hierarchical mapping of the features through feature transformation within the model, gradually compress the feature dimension while retaining the core information, and generate a low-dimensional dense voiceprint embedding vector.

[0043] The pre-trained self-supervised model is a pre-trained neural network model whose input layer receives a standardized time-frequency mixture feature set. The feature transformation process within the model comprises multiple layers, each performing pre-defined processing and transformation on the input features. During the layer-by-layer mapping process, the model progressively filters and integrates the features, removing redundant information while retaining the parts that represent the key features of the voiceprint signal. Through continuous feature transformation and mapping, the model gradually compresses the feature dimensionality, transforming the originally high-dimensional standardized time-frequency mixture feature set into a low-dimensional, dense voiceprint embedding vector.

[0044] Taking a self-supervised pre-trained model structure as an example, the input layer receives a standardized time-frequency mixed feature set and passes it to the hidden layer. The hidden layer consists of multiple neurons, each of which performs a weighted summation of the input features and a non-linear transformation through an activation function. Different hidden layers abstract and refine the features to varying degrees. For example, shallower hidden layers extract some local, simple features, while deeper hidden layers further combine and abstract these local features to form higher-level feature representations. As the layers deepen, the dimensionality of the features gradually decreases, while their expressive power gradually increases. Finally, after processing by multiple hidden layers, the output layer outputs a low-dimensional, dense voiceprint embedding vector.

[0045] Step S330: Based on the timestamp range of the voiceprint signal stream, retrieve equipment operating information within the same time range from the digital factory equipment operating condition database. The equipment operating condition information includes equipment operating load, cumulative working time, ambient temperature and humidity, and the sequence of executed operation instructions.

[0046] The digital factory equipment condition database is a database that stores various operating condition information of equipment. The data is indexed and managed according to timestamps. Based on the timestamp range of the voiceprint signal stream, equipment condition information within the same time range can be accurately filtered from the database. Equipment operating load reflects the workload of the equipment during operation; different operating loads will affect the equipment's voiceprint signal. Cumulative working time reflects the equipment's usage; prolonged operation may lead to changes in equipment performance, thus affecting voiceprint characteristics. Ambient temperature and humidity affect the physical properties of the equipment and sound propagation characteristics, thereby influencing the voiceprint signal. The sequence of executed operation commands records the various operations performed by the equipment during operation; different operation commands will cause different operating states of the equipment, which will then be reflected in the voiceprint signal.

[0047] In practice, the start and end timestamps are first extracted from the voiceprint signal stream. These timestamps are then used as query criteria to retrieve information from the digital factory equipment condition database. The database filters the equipment condition information that meets the criteria based on the timestamp range and returns it. For example, for a voiceprint signal stream that starts at one time and ends at another, the database will retrieve the equipment's operating load data, cumulative working time, ambient temperature and humidity data, and the sequence of executed operation instructions during that time period.

[0048] Step S340: Perform structured transformation on the retrieved equipment operating condition information, convert non-numerical operation instruction sequences into coded data in a unified format, and standardize the range of numerical operating load, temperature, humidity, and other data to generate standardized equipment operating condition data.

[0049] Structured transformation involves organizing and standardizing the retrieved equipment operating information to give it a unified format and structure, facilitating subsequent processing and analysis. Non-numerical operation command sequences are usually in text form and need to be converted into coded data in a unified format so that they can be integrated and processed with other numerical data. Range normalization of numerical data such as operating load, temperature, and humidity aims to eliminate differences in dimensions and numerical ranges between different data points, placing them within a relatively uniform numerical interval.

[0050] For the conversion of operation command sequences, an encoding mapping method can be used. For example, an operation command encoding table can be predefined, mapping different operation commands to different encoding values. After obtaining the operation command sequence, each operation command is converted into its corresponding encoding value according to the encoding table, forming coded data. For numerical data such as operating load, temperature, and humidity, range normalization can be achieved through normalization or standardization. Normalization maps data to a set interval, such as [0, 1], ensuring that the data values ​​fall within this interval. Standardization, on the other hand, converts the data into a distribution with a mean of 0 and a standard deviation of 1.

[0051] Step S350: Correlate the voiceprint embedding vector corresponding to each segmented voiceprint signal unit with the standardized equipment operating condition data of the same period in a dimension-by-dimensional manner, explore the inherent correspondence between voiceprint features and operating condition data, and generate a voiceprint-operating condition correlation feature vector.

[0052] In one implementation, step S350 may specifically include the following steps S351 to S356: Step S351: Divide the various feature dimensions of the voiceprint embedding vector into amplitude-related dimensions, phase-related dimensions, and time-frequency-related dimensions according to functional attributes, and divide the standardized equipment operating condition data into operating status dimensions, environmental impact dimensions, and operation command dimensions.

[0053] Voiceprint embedding vectors contain multiple feature dimensions, which can be categorized into different types based on their functional attributes. The amplitude correlation dimension primarily reflects the amplitude characteristics of the voiceprint signal, related to factors such as the equipment's operating intensity and energy consumption. The phase correlation dimension reflects the phase information of the voiceprint signal, potentially related to the equipment's vibration and sound propagation characteristics. The time-frequency correlation dimension focuses on the joint characteristics of the voiceprint signal in the time and frequency dimensions, reflecting the dynamic changes in the equipment's operating status. Standardized equipment operating condition data can also be divided into different dimensions based on its properties. The operating status dimension includes information such as the equipment's operating load, reflecting the equipment's workload and operating status. The environmental impact dimension includes data such as ambient temperature and humidity, reflecting the impact of the environmental conditions on the equipment's operation. The operation command dimension records the sequence of operation commands executed by the equipment, reflecting the equipment's operational status.

[0054] Step S352: Calculate the correspondence between amplitude correlation dimension and operating status dimension, analyze the variation law of amplitude characteristics under different operating loads, generate amplitude-operation correlation data, and calculate the correspondence between phase correlation dimension and environmental impact dimension, generate phase-environment correlation data.

[0055] Calculating the correspondence between the amplitude correlation dimension and the operating status dimension aims to identify the variation patterns of the acoustic signature signal amplitude characteristics under different operating loads. Different operating loads cause varying vibrations and energy consumption in the equipment, resulting in changes in the acoustic signature signal amplitude. By analyzing this correspondence, a correlation model between amplitude characteristics and operating load can be established, providing a basis for monitoring and diagnosing equipment operating status. Calculating the correspondence between the phase correlation dimension and the environmental influence dimension considers the impact of environmental factors such as temperature and humidity on the phase of the acoustic signature signal. Changes in environmental conditions affect the speed and path of sound propagation, thus influencing the phase characteristics of the acoustic signature signal.

[0056] In one implementation, step S352 may include the following steps S3521 to S3526: Step S3521: Extract feature data of all amplitude-related dimensions in the voiceprint embedding vector, arrange them in chronological order to obtain the amplitude feature sequence, and extract the operating load data of the operating status dimension in the standardized equipment operating condition data to obtain the operating load sequence, so that the timestamps of the two are aligned.

[0057] The feature data of the amplitude-related dimension contains key information related to amplitude in the acoustic signature signal. Arranging this data in chronological order forms an amplitude feature sequence. Operating load data is an important component of the operating status dimension in standardized equipment condition data. Extracting and arranging it in chronological order yields the operating load sequence. Aligning the timestamps of both ensures a temporal correspondence between the amplitude feature sequence and the operating load sequence, facilitating subsequent synchronous analysis. In practice, all feature data of the amplitude-related dimension are first selected from the acoustic signature embedding vector, and then these data are sorted according to their timestamps to form the amplitude feature sequence. Similarly, operating load data of the operating status dimension is extracted from the standardized equipment condition data and sorted according to timestamps to obtain the operating load sequence.

[0058] Step S3522: Perform sliding window analysis on the amplitude characteristic sequence and the operating load sequence simultaneously. Each window covers the same time range. Calculate the statistical data of the amplitude characteristics and the statistical data of the operating load within each window to obtain the window-level amplitude statistics and window-level load statistics.

[0059] Statistical analysis is performed on the data within a fixed-size window by sliding it across the sequence. When performing simultaneous sliding window analysis on amplitude characteristic sequences and operating load sequences, each window covers the same time range, ensuring comparative analysis of amplitude characteristics and operating load within the same time period. Calculating the statistical data of amplitude characteristics and operating load within each window yields window-level amplitude statistics and window-level load statistics, which reflect the overall situation of amplitude characteristics and operating load within that time period. In practice, the size of the sliding window, i.e., the time range it covers, is first determined. Then, starting from the starting point of the amplitude characteristic sequence and operating load sequence, the window is slid sequentially to each position in the sequence. Within each window, statistical calculations are performed on the amplitude characteristic data and operating load data, such as calculating the mean and variance, to obtain window-level amplitude statistics and window-level load statistics. As the window slides, a series of window-level statistical values ​​are obtained, reflecting the changes in amplitude characteristics and operating load over different time periods.

[0060] Step S3523: Analyze the changes in window-level amplitude statistics with window-level load statistics within each window, record the relationship of synchronous increase, synchronous decrease, or reverse change between the two, and mark the window range and change magnitude corresponding to each relationship.

[0061] Analyzing the changes in window-level amplitude statistics with window-level load statistics can reveal the dynamic relationship between amplitude characteristics and operating load. A synchronous increase indicates that as operating load increases, the amplitude characteristic also increases; a synchronous decrease indicates that as operating load decreases, the amplitude characteristic also decreases; and an inverse change indicates that the trends of operating load and amplitude characteristics are opposite. Recording the window range and magnitude of change corresponding to each relationship provides a more detailed understanding of the temporal distribution and degree of change of this relationship.

[0062] For example, step S3523 may include the following steps S35231 to S35236: Step S35231: Calculate the rate of change of the amplitude of the window-level amplitude statistics within each window. The rate of change of amplitude is the difference between the amplitude statistics of the current window and the previous window divided by the time interval. Also, the rate of change of the load statistics of the window-level load is obtained using the same calculation method.

[0063] The amplitude change rate and load change rate reflect the rate of change of window-level amplitude statistics and window-level load statistics between adjacent windows, respectively. By calculating the change rate, the dynamic changes in amplitude characteristics and operating load can be analyzed more accurately. In actual calculations, for each window, the amplitude change rate is obtained by subtracting the window-level amplitude statistics of the previous window from the current window's window-level amplitude statistics, and then dividing by the time interval between the two windows. Similarly, the same calculation is performed on the window-level load statistics to obtain the load change rate.

[0064] Step S35232: Compare the positive and negative signs of the amplitude change rate and load change rate of each window. If the signs are the same and the values ​​are both greater than zero, it is determined to be a synchronous upward relationship; if the signs are the same and the values ​​are both less than zero, it is determined to be a synchronous downward relationship; if the signs are opposite, it is determined to be an inverse change relationship.

[0065] By comparing the signs of the amplitude change rate and the load change rate, the relationship between the two can be quickly and accurately determined. When both signs are the same and positive, it indicates that both the amplitude characteristic and the operating load are increasing, i.e., rising synchronously; when both signs are the same and negative, it indicates that both the amplitude characteristic and the operating load are decreasing, i.e., falling synchronously; when the signs are opposite, it indicates that the amplitude characteristic and the operating load are changing in opposite directions, i.e., changing in opposite directions. In the actual comparison process, the signs of the amplitude change rate and the load change rate for each window are checked. Appropriate judgments are made based on the signs, and the judgment results are recorded.

[0066] Step S35233: Calculate the ratio of amplitude change rate to load change rate within each window. This ratio is used to quantify the degree of matching between the two change amplitudes. The closer the ratio is to the preset benchmark value, the more matched the change amplitudes are. Record the deviation between the ratio and the benchmark value.

[0067] The ratio of the amplitude change rate to the load change rate quantifies the degree of matching between their change magnitudes. A preset benchmark value is a standard value pre-set based on extensive experimental data and experience, used to measure the consistency of their change magnitudes. The deviation of the ratio from the benchmark value reflects the degree of difference in their change magnitudes; the smaller the deviation, the more matched their change magnitudes.

[0068] In actual calculations, for each window, the amplitude change rate is divided by the load change rate to obtain a ratio. Then, the absolute value of the difference between this ratio and the preset benchmark value is calculated as the deviation. The ratio and deviation for each window are recorded for subsequent analysis.

[0069] Step S35234: Mark the window number, window start timestamp and end timestamp corresponding to each change relationship, clarify the position of the window in the overall sequence, and record the maximum and minimum values ​​of amplitude statistics and load statistics within the window.

[0070] Marking the window number, start timestamp, and end timestamp corresponding to each change relationship clarifies the specific location of that change relationship in the time series. Recording the maximum and minimum values ​​of amplitude statistics and load statistics within the window provides a more detailed understanding of the amplitude characteristics and the range of operating load changes within that window. In practice, for each window determined to be a synchronous increase, synchronous decrease, or reverse change relationship, its window number, start timestamp, and end timestamp are recorded. Simultaneously, the maximum and minimum values ​​of amplitude statistics and load statistics within that window are identified and recorded.

[0071] Step S35235: For the window of synchronous rise and synchronous fall relationship, analyze whether the change rhythm of amplitude change rate and load change rate is consistent, that is, whether the peak occurrence time is synchronous, and record the cases of consistent and inconsistent rhythms and the corresponding window information.

[0072] Analyzing whether the rate of change of amplitude and the rate of change of load change are consistent can further reveal the dynamic relationship between amplitude characteristics and operating load. If the peak values ​​occur synchronously, it indicates that their change rates are consistent; if the peak values ​​occur asynchronously, it indicates that their change rates are inconsistent. Recording the cases of consistent and inconsistent rates, along with the corresponding window information, can provide more detailed information for subsequent correlation model building and equipment status diagnosis. In actual analysis, for windows with synchronous rising and synchronous falling relationships, the peak occurrence times of the rate of change of amplitude and the rate of change of load are identified respectively. These two times are compared; if they are the same, the rate of change is considered consistent; if they are different, the rate of change is considered inconsistent. The window number, start timestamp, and end timestamp, etc., corresponding to each case are recorded.

[0073] Step S35236: Integrate and record the change relationship type, window range, change rate ratio, deviation, extreme value data and rhythm synchronization of each window to obtain a complete record of window change relationships.

[0074] Integrating the relevant information for each window creates a complete record of window change relationships, containing detailed information on the relationship between amplitude characteristics and operating load within each window. Analyzing and summarizing these records provides a more comprehensive understanding of the correspondence between amplitude characteristics and operating load. In the actual integration process, information such as the change relationship type (synchronous increase, synchronous decrease, or reverse change), window range (start and end timestamps), change rate ratio, deviation, maximum and minimum values ​​of amplitude and load statistics, and rhythm synchronization status for each window are organized and recorded to obtain a complete record of window change relationships.

[0075] Step S3524: Classify and statistically analyze the change relationships of all windows, identify change relationships that meet the preset core screening conditions, and record change relationships that do not match the preset typical change relationship set to obtain a change relationship classification set. Each classification includes the corresponding number of windows and time distribution.

[0076] The purpose of categorizing and statistically analyzing the changing relationships across all windows is to systematically identify various variation patterns between amplitude characteristics and operating load. Pre-set core screening criteria are standards pre-defined based on extensive experimental data and experience, used to select changing relationships that are crucial for analyzing equipment operating status. A pre-set set of typical changing relationships contains common and representative changing relationship patterns. By identifying changing relationships that meet the pre-set core screening criteria, key changing patterns can be highlighted, providing an important basis for subsequently constructing corresponding relationship models. Recording changing relationships that do not match the pre-set set of typical changing relationships can reveal some special changes that may indicate potential equipment anomalies.

[0077] Specifically, the changes in all windows (synchronous increase, synchronous decrease, reverse change, etc.) are first categorized and organized. For each change relationship, the number of windows that appear is counted, and the distribution of these windows over time is recorded, i.e., in which time periods the change relationship occurred. Then, these changes are filtered according to preset core filtering criteria to identify those that meet the criteria. Simultaneously, each change relationship is checked to see if it matches a pattern in a preset set of typical change relationships; mismatched changes are recorded separately. The final result is a set of change relationship categories, where each category explicitly includes the corresponding number of windows and time distribution information.

[0078] Step S3525: Based on the change relationship classification set, construct a correspondence model between amplitude characteristics and operating load, which clarifies the amplitude characteristic range and change trend corresponding to different operating load intervals, and marks the applicable time range of the model.

[0079] The change relationship classification set contains information on the change relationship between amplitude characteristics and operating load within different windows. Based on this information, a correspondence model between amplitude characteristics and operating load can be constructed. This model can clearly define the amplitude characteristic range and change trend corresponding to different operating load intervals, providing a reference for monitoring and diagnosing equipment operating status. Labeling the applicable time range of the model ensures its effectiveness within a specific time period.

[0080] When constructing the model, the classification set of changing relationships is first analyzed and summarized to identify the changing patterns of amplitude characteristics under different operating load ranges. For example, the range and trend of amplitude characteristics are determined in low, medium, and high operating load ranges. Then, corresponding relationship models are constructed based on these patterns. Simultaneously, the applicable time range of the model is marked according to the data collection time and analysis results, such as from one moment to another.

[0081] Step S3526: Integrate the change relationship classification set, correspondence model and related window statistics to generate amplitude-run correlation data, which includes the correspondence between features and load, change patterns and applicable conditions information.

[0082] Integrating the change relationship classification set, correspondence model, and related window statistics can generate amplitude-operation correlation data, which contains comprehensive correspondence information between amplitude characteristics and operating load. The correspondence between characteristics and load clarifies the values ​​and changes of amplitude characteristics under different operating loads; the change pattern describes the changing patterns and dynamic relationships between amplitude characteristics and operating load; and the applicable condition information indicates the conditions under which this correlation data is applicable. In the actual integration process, the change relationship classification set, correspondence model, and window statistics (such as various information in the window change relationship record) are organized and combined to form amplitude-operation correlation data. It is ensured that this data includes key information such as the correspondence between characteristics and load, change patterns, and applicable conditions.

[0083] Step S353: Calculate the correspondence between the time-frequency correlation dimension and the operation command dimension, analyze the response of the voiceprint time-frequency characteristics when different operation commands are executed, generate time-frequency-operation correlation data, and record the corresponding strength and stability of various correlation data.

[0084] Calculating the correspondence between the time-frequency correlation dimension and the operation command dimension aims to identify the impact of different operation commands on the time-frequency characteristics of the voiceprint. Different operation commands cause the device to operate in different states, thus leading to changes in the time-frequency characteristics of the voiceprint. By analyzing this correspondence, time-frequency-operation correlation data can be generated, which records the correlation between different operation commands and the time-frequency characteristics of the voiceprint. Recording the correspondence strength and stability of various correlation data can assess the reliability and effectiveness of this correlation.

[0085] In practical calculations, feature data in the time-frequency correlation dimension is first extracted from the voiceprint embedding vector, and operation command sequences in the operation command dimension are extracted from standardized equipment operating condition data. Then, the changes in the feature data in the time-frequency correlation dimension are analyzed when different operation commands are executed. For example, changes in the frequency component distribution and energy transfer in the voiceprint time-frequency features are observed when a certain operation command is executed. Based on the analysis results, time-frequency-operation correlation data is generated, and the corresponding strength and stability of different correlation relationships are recorded. Correspondence strength can be measured by calculating the magnitude or correlation of feature data changes, while stability can be evaluated by observing the consistency of feature data changes when the same operation command is executed multiple times.

[0086] Step S354: Perform cross-dimensional fusion of amplitude-operation correlation data, phase-environment correlation data, and time-frequency-operation correlation data, extract common change trends between different correlation dimensions, supplement cross-correlation features, and obtain comprehensive correlation data.

[0087] Cross-dimensional fusion is a process of comprehensively processing amplitude-operational correlation data, phase-environmental correlation data, and time-frequency-operational correlation data. Through cross-dimensional fusion, common trends in change across different correlation dimensions can be extracted, that is, identifying patterns of change present in various correlation relationships. Supplementing with cross-correlation features further enriches the information in the correlation data, considering the interactions and influences between different dimensions. The resulting comprehensive correlation data more fully and accurately reflects the complex relationship between acoustic signature features and equipment operating condition information.

[0088] In the actual fusion process, the amplitude-operation correlation data, phase-environment correlation data, and time-frequency-operation correlation data are first analyzed and compared. Common trends among them are identified; for example, in some cases, amplitude, phase, and time-frequency characteristics may simultaneously change similarly with variations in equipment operating status. Then, based on these common trends and the interrelationships between different dimensions, cross-correlation features are added. For example, the combined effects of operating load and ambient temperature and humidity on the acoustic signature's time-frequency characteristics are considered. Finally, this information is integrated to obtain comprehensive correlation data.

[0089] Step S355: Based on the feature distribution of the comprehensive association data, filter the feature dimensions of the voiceprint embedding vector and the standardized equipment operating condition data, retain the feature dimensions that match the corresponding association type in the comprehensive association data according to the preset conditions, and remove the feature dimensions that do not match any of the association types in the comprehensive association data according to the preset conditions.

[0090] Feature dimension filtering based on the feature distribution of the comprehensive correlation data can remove some feature dimensions that are not important for equipment condition monitoring and diagnosis, reducing data redundancy and improving the efficiency and accuracy of subsequent analysis. The preset conditions are a standard pre-set based on a large amount of experimental data and experience, used to judge the degree of matching between feature dimensions and corresponding correlation types in the comprehensive correlation data. Retaining feature dimensions that meet the matching conditions ensures that the filtered data can better reflect the correlation between voiceprint features and equipment operating condition information. In the actual filtering process, the feature distribution of the comprehensive correlation data is first analyzed to determine the key features and variation patterns of different correlation types. Then, for each feature dimension of the voiceprint embedding vector and standardized equipment operating condition data, its matching degree with the corresponding correlation type in the comprehensive correlation data is evaluated. If the matching degree meets the preset conditions, the feature dimension is retained; if the matching degree with all correlation types does not meet the preset conditions, the feature dimension is removed.

[0091] Step S356: Arrange the filtered voiceprint embedding vector features and standardized equipment operating condition data features in order of correlation strength, and integrate them to obtain a voiceprint-operating condition correlation feature vector, wherein each feature component is labeled with the corresponding correlation type and correlation strength information.

[0092] Arranging the selected voiceprint embedding vector features and standardized equipment condition data features in order of correlation strength makes the voiceprint-condition correlation feature vector more structured and hierarchical. Correlation strength reflects the closeness of the association between features and equipment condition information; arranging them in order of correlation strength highlights important feature information. Labeling each feature component with its corresponding correlation type and correlation strength information makes the feature vector clearer and easier to understand, facilitating subsequent analysis and application. In the actual arrangement and integration process, the correlation strength of the selected voiceprint embedding vector features and standardized equipment condition data features is first evaluated. Based on factors such as the correlation and consistency of changes between the features and the corresponding correlation types in the integrated correlation data, the correlation strength of each feature is determined. Then, the features are arranged in order of correlation strength from high to low. Finally, the arranged features are integrated to form the voiceprint-condition correlation feature vector, and each feature component is labeled with its corresponding correlation type and correlation strength information.

[0093] Step S360: Arrange the voiceprint-operating condition association feature vectors corresponding to all segmented voiceprint signal units in the order of timestamps to obtain a multi-row, multi-column matrix structure. Each row corresponds to a segmented association feature, and each column corresponds to a feature dimension, generating a feature expansion matrix with operating condition labels.

[0094] Arranging the voiceprint-operating condition association feature vectors corresponding to all segmented voiceprint signal units in timestamp order forms a multi-row, multi-column matrix structure. This matrix structure clearly displays the association between voiceprint features and equipment operating condition information within different time periods. Each row corresponds to the association feature of a segment, reflecting the comprehensive association between voiceprint features and equipment operating condition information within that time period; each column corresponds to a feature dimension, reflecting the value of different segments on that feature dimension. Generating a feature extension matrix with operating condition labels can provide richer and more structured data input for subsequent equipment anomaly detection. In the actual arrangement process, based on the timestamp information of each segmented voiceprint signal unit, the corresponding voiceprint-operating condition association feature vectors are arranged in chronological order. The arranged feature vectors are combined into a matrix structure, ensuring that each row corresponds to a segment and each column corresponds to a feature dimension. Simultaneously, operating condition labels are added to the matrix to clarify the correspondence between each feature and equipment operating condition information.

[0095] Step S400: Based on the feature expansion matrix, a hierarchical attention mechanism is used to perform key anomaly feature extraction processing, and the first key anomaly feature subset and the second key anomaly feature subset are obtained through multi-level feature filtering.

[0096] The hierarchical attention mechanism is a mechanism that automatically assigns attention weights, weighting different features in the feature expansion matrix according to their importance. Key anomaly feature extraction involves filtering key features from the feature expansion matrix that reflect the abnormal state of the equipment. Through multi-level feature filtering, the feature range can be gradually narrowed, improving the accuracy and reliability of key anomaly features. The first and second subsets of key anomaly features contain different types of key anomaly features, which are of great significance for the detection and diagnosis of abnormal equipment states.

[0097] In one implementation, step S400 may specifically include the following steps S410 to S460: Step S410: Input the feature expansion matrix into the input interface of the hierarchical attention mechanism, and divide it into time-related feature layer, working condition-related feature layer and cross-related feature layer according to feature attributes.

[0098] The input interface of the hierarchical attention mechanism is the entry point for receiving the feature expansion matrix. Dividing the feature expansion matrix into time-related feature layers, operating condition-related feature layers, and cross-correlation feature layers according to feature attributes allows for the separation and classification of different types of features, facilitating subsequent targeted processing and analysis of these different feature types. The time-related feature layer contains features related to the time dimension, such as the amplitude and phase changes of the acoustic signature signal over different time periods. The operating condition-related feature layer contains features related to equipment operating condition information, such as operating load, ambient temperature and humidity, etc. The cross-correlation feature layer contains correlation features between different types of features, such as amplitude-operation correlation data, time-frequency-operation correlation data, etc. Specifically, based on the meaning and properties of each feature in the feature expansion matrix, it is classified into the corresponding feature layer. For example, features related to the time dimension of the acoustic signature signal are classified into the time-related feature layer; features related to equipment operating load are classified into the operating condition-related feature layer; and features reflecting the correlation between different types of features are classified into the cross-correlation feature layer.

[0099] Step S420: Assign attention weights to the time-related feature layer, analyze the correlation between the feature vectors corresponding to different timestamps and the evolution of the device's operating state, assign corresponding attention weights to each timestamp feature, and generate a time attention weight matrix.

[0100] In one implementation, step S420 may specifically include the following steps S421 to S426: Step S421: Extract the feature vector corresponding to each timestamp in the time-related feature layer, calculate the degree of difference between each feature vector and the standard time feature vector when the device is running normally, and obtain the feature difference degree of each timestamp. The difference degree reflects the degree to which the feature deviates from the normal state.

[0101] In the time-related feature layer, the feature vector corresponding to each timestamp contains the feature information of the voiceprint signal in the time dimension at that time point. The standard time feature vector during normal device operation is a reference vector predetermined based on a large amount of normal operation data, representing the time dimension features of the device under normal conditions. Calculating the degree of difference between each feature vector and the standard time feature vector can assess the degree to which the features at that time point deviate from the normal state, yielding the feature difference degree. Specifically, for the feature vector corresponding to each timestamp in the time-related feature layer, an appropriate method is used to calculate its difference from the standard time feature vector. For example, indices such as Euclidean distance and cosine similarity between the two vectors can be calculated, and these indices can be used as measures of feature difference degree. The greater the difference degree, the more the features at that time point deviate from the normal state.

[0102] Step S422: Analyze the characteristic difference data of all timestamps, identify the intervals where the difference evolves in a continuous and consistent direction and the time points where the difference deviates from the adjacent numerical range, record the range and direction of the difference change before and after the deviation time point, and mark the evolution stability information of the intervals where the difference evolves in a continuous and consistent direction.

[0103] Analyzing the characteristic difference data across all timestamps reveals patterns and anomalies in characteristic difference. Identifying intervals where difference evolves in a consistent direction identifies periods of continuous increase or decrease, potentially reflecting gradual changes in equipment operating status. Identifying points where difference deviates from adjacent value ranges reveals sudden changes in characteristic difference, possibly corresponding to abrupt shifts in equipment operating status. Recording the range and direction of difference changes before and after these deviations provides a more detailed understanding of the anomaly's occurrence. Marking the stability of consecutively evolving intervals allows for an assessment of the stability of characteristic difference changes within those intervals.

[0104] Specifically, the characteristic difference data of all timestamps are sorted and analyzed to identify intervals where the difference continuously increases or decreases, and the start and end timestamps of these intervals are recorded. Simultaneously, it is checked whether the characteristic difference of each timestamp deviates from the range of adjacent values. If it does, the time point is recorded, and the range and direction of change in difference before and after it are analyzed. For intervals that evolve in a continuous and consistent direction, the stability of their evolution is assessed, for example, by calculating the fluctuation of the rate of change of difference within the interval, and the corresponding information is labeled.

[0105] Step S423: Retrieve the standard time feature evolution template corresponding to normal equipment operation. The feature evolution template records the standard evolution range and evolution rate of feature differences within the entire operating cycle of the equipment, and clarifies the reasonable range of differences corresponding to different operating stages. Match and verify each feature difference with the reasonable range of the corresponding stage in the standard time feature evolution template according to the timestamp order, mark the difference data that exceeds the preset matching range, and record the exceeded range and the corresponding timestamp information.

[0106] As one implementation method, in step S423, each feature difference degree is matched and verified with the reasonable interval of the corresponding stage in the standard time feature evolution template according to the timestamp order. The difference degree data that exceeds the preset matching range is marked, and the range that exceeds the range and the corresponding timestamp information are recorded. Specifically, this may include the following steps S4231~S4236: Step S4231: Retrieve the parameter information of the standard time feature evolution template. The parameter information includes the standard range of difference degree, standard evolution rate and stage boundary timestamp threshold corresponding to each operating stage of the equipment, and determine the allowable fluctuation range and evolution direction constraints of the difference degree in different stages.

[0107] The standard range of variability for each operational stage of the equipment defines the normal range of characteristic variability values ​​at different operational stages; the standard evolution rate describes the normal rate of change of characteristic variability at different stages; and the stage boundary timestamp threshold is used to divide different operational stages. Determining the allowable fluctuation range and evolution direction constraints of variability at different stages can provide clear standards for subsequent matching verification.

[0108] In the actual retrieval process, parameter information is obtained from the database or file storing the standard time feature evolution template. Based on this parameter information, the allowable fluctuation range of feature variability in each operational stage is determined, i.e., the upper and lower limits of variability. Simultaneously, the constraints on the evolution direction of feature variability in different stages are clarified; for example, variability should increase in some stages and decrease in others.

[0109] Step S4232: According to the stage boundary timestamp threshold of the standard time feature evolution template, divide all current timestamps into corresponding running stages. Each timestamp belongs to a unique running stage, so that the stage division is completely aligned with the template and there are no cross-stage timestamp omissions.

[0110] Dividing all timestamps into corresponding operational stages based on stage boundary timestamp thresholds ensures that each timestamp accurately corresponds to a specific operational stage of the device. Aligning the stage division with the template guarantees that subsequent matching and verification are performed within the correct stage range, avoiding incorrect cross-stage matching. Ensuring no cross-stage timestamp omissions ensures that all timestamps are properly processed and analyzed. Specifically, all current timestamps are sorted and divided according to the stage boundary timestamp thresholds of the standard time feature evolution template. Each timestamp is assigned to a unique operational stage, ensuring consistency between the stage division and the template. It is then checked whether any timestamps have been incorrectly assigned to any stage; if so, adjustments are made to ensure no cross-stage timestamp omissions.

[0111] Step S4233: Extract the feature difference data of each timestamp one by one, compare the values ​​with the difference standard range of the corresponding operating stage, determine whether the difference data is within the standard range, and verify whether its evolution rate meets the standard evolution rate requirements of the stage.

[0112] Extracting the feature difference data for each timestamp and comparing it with the standard range of difference for the corresponding operational stage can determine whether the feature difference at that time point is normal. Verifying whether its evolution rate meets the standard evolution rate requirements for that stage can further assess whether the change in feature difference is reasonable. If the feature difference exceeds the standard range or the evolution rate does not meet the requirements, it indicates that the features at that time point may be abnormal.

[0113] In the actual comparison and verification process, for each timestamp, its characteristic difference data is extracted and compared with the difference standard range of its corresponding operational stage. If the difference data is within the standard range, the evolution rate is further verified to ensure it meets the standard evolution rate requirements for that stage. The evolution rate can be obtained by calculating the difference in characteristic differences between adjacent timestamps and dividing it by the time interval, and then comparing it with the standard evolution rate.

[0114] Step S4234: Mark the feature difference data whose values ​​exceed the standard range or whose evolution rate does not meet the requirements, record the specific value of the difference data, the timestamp to which it belongs, the operating stage to which it belongs, and the specific dimension that exceeds the standard, and obtain the abnormal difference mark list.

[0115] Marking feature variance data whose values ​​exceed standard ranges or whose evolution rates do not meet requirements can highlight anomalies. Recording the specific numerical value of the variance data, its corresponding timestamp, the relevant operational stage, and the specific dimension exceeding the standard provides detailed information about the anomaly, forming an anomaly variance marker list. During the actual marking and recording process, for feature variance data that does not meet the requirements, a marker is added, for example, by setting a special flag. Simultaneously, the specific numerical value of the variance data, its corresponding timestamp, the relevant operational stage, and the specific dimension exceeding the standard are recorded, such as the variance of a certain feature component exceeding the standard range. This information is then compiled into an anomaly variance marker list.

[0116] Step S4235: Combine the equipment operating condition information and operation instruction sequence retrieved at the same time, analyze the background of each data point in the abnormality difference label list, distinguish between abnormalities caused by data acquisition interference and abnormalities caused by deviations from the actual operating status of the equipment, and label the interference type or deviation attribute respectively.

[0117] By combining the equipment operating condition information and operation command sequences retrieved concurrently, the background of each data point in the anomaly difference labeling list can be analyzed to identify the cause of the anomaly. Distinguishing between anomalies caused by data acquisition interference and those caused by deviations from the actual equipment operating status allows for accurate determination of the anomaly's nature. Labeling the interference type or deviation attribute provides accurate information for subsequent processing and decision-making. In the actual analysis process, for each data point in the anomaly difference labeling list, the concurrent equipment operating condition information and operation command sequences are examined. If the anomaly is found to be caused by interference factors during data acquisition, such as sensor failure or environmental noise, the interference type is labeled; if it is caused by actual changes in the equipment operating status, such as equipment failure or sudden changes in operating load, the deviation attribute is labeled.

[0118] Step S4236: For abnormal difference data caused by data acquisition interference, correct the values ​​by trend interpolation based on the standard evolution rate of the relevant operating stage and the reasonable difference data of adjacent timestamps; for abnormal difference data caused by actual deviation of equipment operating status, retain the original values ​​and add key attention markers, and complete the matching verification and classification processing of all feature difference data.

[0119] For anomalous difference data caused by data acquisition interference, a reasonable feature difference value is estimated using trend interpolation based on the standard evolution rate of the relevant operational phase and the reasonable difference data of adjacent time stamps. This value then replaces the original anomalous value. For anomalous difference data caused by deviations from the actual operating status of the equipment, the original value is retained, and a key focus indicator is added, such as by setting a special mark or label. After completing the matching, verification, and classification processing of all feature difference data, the corrected and labeled feature difference data is obtained.

[0120] Step S424: Calculate the importance score of each timestamp feature based on the matching verification results. Timestamps with feature differences within a reasonable range and consistent with the trend of the standard evolution template are given corresponding scores. Timestamps that exceed the matching range have their score weights adjusted according to preset rules.

[0121] Specifically, for timestamps whose feature differences are within a reasonable range and consistent with the trend of the standard evolution template, a corresponding score is given based on their proximity to the standard features and their contribution to the overall feature changes. For timestamps that exceed the matching range, the scoring weight is adjusted according to preset rules based on the degree of exceedance and the severity of the anomaly, such as increasing or decreasing the score.

[0122] Step S425: Normalize the importance scores of all timestamps so that the scores are within a uniform range, and determine the attention weight of each timestamp feature based on the normalized scores.

[0123] In practical normalization processing, normalization methods such as max-min normalization and Z-score normalization can be used to transform the importance scores of all timestamps into a unified numerical range, such as the [0,1] range, and the normalized scores are determined as the attention weights of each timestamp feature.

[0124] Step S426: Arrange the attention weights of all timestamps in the order of timestamps to obtain a time attention weight matrix that is consistent with the dimension of the time-related feature layer. Each element in the matrix corresponds to the attention weight of a timestamp feature.

[0125] Arranging the attention weights of all timestamps in time-stamp order creates a time attention weight matrix with the same dimension as the time-related feature layer. Each element in the matrix corresponds to the attention weight of a timestamp feature. This matrix can be used to weight the time-related feature layer, highlighting important timestamp features. In the actual arrangement process, the attention weights of each timestamp are sequentially arranged into a row or column of the matrix according to the order of the timestamps. This ensures that the matrix dimension is consistent with the time-related feature layer, and that each element accurately corresponds to the attention weight of a timestamp feature.

[0126] Step S430: Assign attention weights to the working condition related feature layer, analyze the contribution of different working condition feature dimensions to equipment anomaly identification, assign corresponding attention weights to each working condition feature dimension, and generate a working condition attention weight matrix.

[0127] Attention weight allocation for the operating condition-related feature layer aims to highlight those operating condition feature dimensions that contribute significantly to equipment anomaly identification. Different operating condition feature dimensions, such as operating load and ambient temperature and humidity, may reflect abnormal equipment conditions to varying degrees. By analyzing the contribution of each operating condition feature dimension to equipment anomaly identification, appropriate attention weights can be assigned to it. The operating condition attention weight matrix records the attention weight of each operating condition feature dimension, which is used for subsequent weighted processing of the operating condition-related feature layer.

[0128] In the actual allocation process, the correlation between each operating condition feature dimension and the abnormal state of the equipment is first analyzed. Statistical analysis methods, such as correlation analysis and regression analysis, can be used to evaluate the contribution of each operating condition feature dimension to the identification of equipment abnormalities. Based on the evaluation results, a corresponding attention weight is assigned to each operating condition feature dimension. The attention weights of all operating condition feature dimensions are arranged in order to form an operating condition attention weight matrix.

[0129] Step S440: Apply the time attention weight matrix to the time-related feature layer to perform weighted enhancement processing on the feature vector. At the same time, apply the working condition attention weight matrix to the working condition-related feature layer to obtain the weighted time feature layer and the weighted working condition feature layer.

[0130] Applying a time-related attention weight matrix to the time-related feature layer can perform weighted enhancement processing on the feature vectors in the time-related feature layer. Timestamp features with larger attention weights will receive greater weight in the weighting process, thus highlighting these important timestamp features. Similarly, applying a condition-related attention weight matrix to the condition-related feature layer can perform weighted enhancement processing on the feature vectors in the condition-related feature layer, highlighting the condition feature dimensions that contribute significantly to equipment anomaly identification. Finally, a weighted time feature layer and a weighted condition feature layer are obtained. After weighting processing, these two feature layers better reflect the key information of the equipment's operating status.

[0131] For example, for each feature vector in the time-related feature layer, it is multiplied by the attention weight of the corresponding timestamp in the time attention weight matrix to obtain a weighted feature vector. All weighted feature vectors are combined to form a weighted time feature layer. The same process is performed on the operating condition-related feature layer, multiplying it by the operating condition attention weight matrix to obtain a weighted operating condition feature layer.

[0132] Step S450: Perform cross-layer feature fusion on the weighted time feature layer, weighted working condition feature layer and cross-correlation feature layer, extract complementary information between each layer, and generate a fusion enhanced feature matrix.

[0133] Cross-layer feature fusion is a process of comprehensively processing weighted time feature layers, weighted operating condition feature layers, and cross-correlation feature layers. Each layer may contain different types of feature information; cross-layer feature fusion can extract the complementarity between these information, uncovering more comprehensive and in-depth feature information. The generated fusion-enhanced feature matrix contains comprehensive information from each layer, enabling it to more accurately reflect the operating status of the equipment.

[0134] For example, we first analyze the feature structure and information characteristics of the weighted time feature layer, the weighted operating condition feature layer, and the cross-correlation feature layer. We identify complementary information between each layer; for instance, the time dimension change information in the time-related feature layer can be combined with the equipment operating condition information in the operating condition-related feature layer, and the correlation information in the cross-correlation feature layer can further supplement and improve this information. Then, we use appropriate fusion methods, such as feature concatenation and weighted feature summation, to fuse the features from each layer, generating a fused enhanced feature matrix.

[0135] Step S460: Perform feature filtering on the fusion enhanced feature matrix, and separate the feature components that focus on the expression of time-dimensional anomalies and the feature components that focus on the expression of working condition-related anomalies according to their relevance to anomaly identification, to obtain the first key anomaly feature subset and the second key anomaly feature subset, respectively.

[0136] Feature filtering of the fused and enhanced feature matrix aims to extract the feature components most relevant to equipment anomaly identification. Separating feature components emphasizing time-dimensional anomalies and those emphasizing operating condition-related anomalies based on their relevance to anomaly identification allows for the classification of different types of key anomaly features. The first subset of key anomaly features primarily contains feature components emphasizing time-dimensional anomalies, which may reflect abnormal changes in the equipment over time. The second subset primarily contains feature components emphasizing operating condition-related anomalies, which may be related to the equipment's operating condition information and correlations. In the actual filtering process, key feature indicators relevant to equipment anomaly identification are first determined. For each feature component in the fused and enhanced feature matrix, its relevance to anomaly identification is evaluated. Based on the evaluation results, the feature components are divided into two categories: those emphasizing time-dimensional anomalies and those emphasizing operating condition-related anomalies. The feature components emphasizing time-dimensional anomalies are combined into the first subset of key anomalies, and the feature components emphasizing operating condition-related anomalies are combined into the second subset of key anomalies.

[0137] Step S500: Perform dynamic weighted fusion processing on the first key abnormal feature subset and the second key abnormal feature subset to generate a fused feature vector. Compare and analyze the energy distribution of the abnormal features in the fused feature vector with the preset abnormal threshold to determine whether the equipment has an abnormal operating state.

[0138] In one implementation, step S500 may specifically include the following steps S510-S560: Step S510: Calculate the sum of feature energies of the first key anomaly feature subset and the second key anomaly feature subset respectively. The sum of squaring the values ​​of each feature component is used to obtain the total energy, and the first feature total energy and the second feature total energy are generated.

[0139] The sum of feature energies reflects the combined strength of all feature components within a feature subset. Summing the squares of each feature component highlights the impact of its magnitude on the total energy. By calculating the sum of feature energies for the first and second key anomaly feature subsets, we can obtain the first and second total feature energies, which can be used for subsequent weight allocation and anomaly detection. In practice, for the first key anomaly feature subset, the value of each feature component is squared, and then all squared values ​​are summed to obtain the first total feature energy. The same calculation is performed on the second key anomaly feature subset to obtain the second total feature energy.

[0140] Step S520: Analyze the changing trends of the first feature total energy and the second feature total energy over time. Combined with the feature energy change law during the equipment operation phase, determine the energy contribution ratio of the two feature subsets in different time intervals and generate the energy contribution distribution.

[0141] Analyzing the trends of the total energy of the first and second features over time reveals the energy variations of the two feature subsets across different time periods. Combining this with the energy variation patterns during equipment operation allows for a more accurate assessment of the contribution of each feature subset to the overall anomaly characteristics across different time intervals. Determining the energy contribution ratio of the two feature subsets across different time intervals generates an energy contribution distribution, which can provide a basis for dynamic weighted fusion processing.

[0142] In the actual analysis process, firstly, curves showing the changes in the first and second characteristic total energies over time are plotted to observe their trends. Then, based on the characteristic energy change patterns during equipment operation, time is divided into different intervals. Within each time interval, the ratio of the first and second characteristic total energies is calculated to obtain the energy contribution ratio of the two characteristic subsets in that interval. The energy contribution ratios of all time intervals are arranged in chronological order to generate an energy contribution distribution.

[0143] Step S530: Dynamically adjust the fusion weights according to the energy contribution distribution. Assign the corresponding weights to the first key abnormal feature subset during the time interval when the total energy proportion of the first feature meets the preset dominant threshold, and assign the corresponding weights to the second key abnormal feature subset during the interval when the total energy proportion of the second feature meets the preset dominant threshold.

[0144] Dynamically adjusting the fusion weights based on the energy contribution distribution makes the fusion process more flexible and accurate. The preset dominance threshold is a standard value pre-set based on extensive experimental data and experience, used to determine whether a certain feature subset dominates within a specific time interval. In time intervals where the total energy proportion of the first feature meets the preset dominance threshold, the first key anomalous feature subset is given a higher weight, allowing it to play a greater role in the fusion; similarly, in time intervals where the total energy proportion of the second feature meets the preset dominance threshold, the second key anomalous feature subset is given a higher weight.

[0145] In one implementation, step S530 may specifically include the following steps S531 to S536: Step S531: Calculate the first feature total energy difference and the second feature total energy difference in adjacent time moments in the energy contribution distribution. Based on the preset interval division threshold, when any feature total energy difference reaches the threshold, it is marked as an interval boundary. The overall time range is divided into multiple continuous time intervals according to the boundaries, and there are no additional boundary markings in each continuous time interval.

[0146] Calculating the difference in total energy of the first and second features between adjacent time points can identify abrupt changes in energy. A preset interval division threshold is a pre-defined standard value used to determine whether the energy change is large enough to define the interval boundary. When the total energy difference of any feature reaches the threshold, that time point is marked as the interval boundary. Dividing the overall time range into multiple consecutive time intervals according to the boundaries divides time into different stages, within which energy changes are relatively stable.

[0147] For example, for each moment in the energy contribution distribution, the difference between its first characteristic total energy and the second characteristic total energy and the previous moment is calculated. These differences are compared with a preset interval division threshold. If the difference reaches the threshold, the moment is marked as an interval boundary. Based on the marked interval boundaries, the overall time range is divided into multiple continuous time intervals.

[0148] Step S532: Extract the complete numerical sequence of the total energy of the first feature and the total energy of the second feature within each time interval, obtain the cumulative energy value of the two within the interval through sequence integration, determine the energy ratio relationship within the interval based on the cumulative energy value, and generate the first energy ratio data and the second energy ratio data.

[0149] Extracting the complete numerical sequences of the total energy of the first and second features within each time interval reveals the energy changes of the two feature subsets within that interval. Sequence integration calculates the cumulative energy value of both features within the interval, reflecting their total energy contribution. Based on these cumulative energy values, the energy proportions within the interval are determined, yielding the first and second energy proportion data, which are used for subsequent weight allocation.

[0150] In practice, for each continuous time interval, the numerical sequences of the first feature total energy and the second feature total energy within that interval are extracted. These two sequences are then integrated to obtain the cumulative energy value within the interval. The proportion of the cumulative energy value of the first feature total energy in the total cumulative energy value is calculated to obtain the first energy proportion data; similarly, the proportion of the cumulative energy value of the second feature total energy in the total cumulative energy value is calculated to obtain the second energy proportion data.

[0151] Step S533: Compare the first energy percentage data of each time interval with the preset dominant threshold. If the threshold condition is met, the interval is determined as the first feature dominant interval. Use the same comparison logic to determine the second feature dominant interval and determine the unique dominant feature subset type of each interval.

[0152] By comparing the first energy percentage data for each time interval with a preset dominance threshold, it can be determined whether the first key anomaly feature subset dominates within that interval. If the first energy percentage data meets the threshold condition, the interval is determined as the first feature-dominant interval. The same comparison logic is used to judge the second energy percentage data to determine the second feature-dominant interval. Determining a unique dominant feature subset type for each interval provides a clear basis for subsequent weight allocation. Specifically, for the first energy percentage data of each time interval, it is compared with a preset dominance threshold. If the first energy percentage data is greater than or equal to the preset dominance threshold, the interval is determined as the first feature-dominant interval. The same comparison is performed on the second energy percentage data to determine the second feature-dominant interval. This ensures that each interval has only one dominant feature subset type.

[0153] Step S534: For the first feature-dominant interval, the first energy percentage data of the interval is directly used as the fusion weight of the first key anomaly feature subset, and the remaining percentage data is allocated to the second key anomaly feature subset; the fusion weight of the two is determined according to the same allocation logic for the second feature-dominant interval.

[0154] In the first feature-dominant region, directly using the first energy percentage data as the fusion weight of the first key anomaly feature subset allows this subset to play a dominant role in the fusion process. Allocating the remaining percentage data to the second key anomaly feature subset ensures that the sum of the weights of the two subsets is 1. In the second feature-dominant region, the same allocation logic is used: the second energy percentage data is used as the fusion weight of the second key anomaly feature subset, and the remaining percentage data is allocated to the first key anomaly feature subset. Specifically, for the first feature-dominant region, the first energy percentage data of this region is used as the fusion weight of the first key anomaly feature subset. The remaining percentage data is obtained by subtracting the first energy percentage data from 1 and used as the fusion weight of the second key anomaly feature subset. The same allocation operation is performed for the second feature-dominant region.

[0155] Step S535: Integrate the fusion weights of all time intervals in chronological order to obtain a dynamic weight sequence. Each weight data is bound to the start and end times of the corresponding time interval, the dominant feature type, and the original data of accumulated energy.

[0156] By integrating the fusion weights of all time intervals in chronological order, a dynamic weight sequence can be formed. This sequence records the changes in the fusion weights of the first and second key anomaly feature subsets within different time intervals. Binding each weight data point to the start and end times, dominant feature type, and accumulated energy raw data of its corresponding time interval makes the weight sequence more complete and traceable. For example, the fusion weights of each time interval are arranged in chronological order to form a dynamic weight sequence. Adding information such as the start and end times of the corresponding time interval, dominant feature type, and accumulated energy raw data to each weight data point ensures that each weight data point has a clear context and relevant information.

[0157] Step S536: Retrieve the weight change consistency template and verify the weight change amplitude of adjacent intervals in the dynamic weight sequence. If the change amplitude exceeds the allowable range specified by the template, adjust the weight of the transition interval based on the energy proportion trend of the preceding and following intervals to keep the overall weight change synchronized with the energy contribution distribution.

[0158] Retrieving the consistency template for weight changes allows us to obtain the standards and constraints for weight changes. Verifying the magnitude of weight changes between adjacent intervals in the dynamic weight sequence can identify cases of excessive weight changes, which may lead to instability in the fusion results. If the magnitude of the change exceeds the allowable range specified by the template, adjusting the weights of the transition interval based on the energy proportion trend of the preceding and following intervals can keep the overall weight change synchronized with the energy contribution distribution, ensuring the rationality and accuracy of the fusion process.

[0159] In one implementation, step S536 may specifically include the following steps S5361 to S5366: Step S5361: Retrieve the complete parameters of the weight change consistency template. This template includes the allowable change range of adjacent interval weights, the upper limit of the change rate, and the energy ratio correlation coefficient corresponding to each operating stage of the equipment. All parameters are generated based on historical normal operation data.

[0160] The weight change consistency template is a set of standard parameters derived from statistical analysis of a large amount of historical normal operation data. During different operating phases, the weight changes of equipment exhibit certain regularity and stability, and these parameters reflect this regularity and stability. The allowable change range of weights in adjacent intervals specifies the range within which the fused weight change cannot exceed in adjacent time intervals, limiting abrupt changes in weights and ensuring the smoothness of weight changes. The upper limit of the change rate further constrains the speed of weight changes, preventing excessively drastic changes in weights within a short period. The energy proportion correlation coefficient reflects the degree of correlation between weight changes and energy proportion, and is used to consider the impact of energy proportion when correcting weights.

[0161] Step S5362: Extract weight data pairs for all adjacent time intervals in the dynamic weight sequence. Each data pair contains the fusion weight value of the previous interval and the fusion weight value of the next interval, and associates the first energy proportion data, the second energy proportion data, and the energy contribution distribution trend of the corresponding interval.

[0162] The dynamic weight sequence records the changes in the fusion weights of the first and second key anomaly feature subsets within different time intervals. Extracting weight data pairs from adjacent time intervals facilitates the analysis of weight changes between adjacent intervals. The fusion weight values ​​of the preceding and following intervals in each data pair directly reflect the weight changes. Associating the first energy proportion data, second energy proportion data, and energy contribution distribution trend of corresponding intervals is to comprehensively consider the relationship between weight changes and energy proportions. The energy proportion data reflects the energy contribution of the two feature subsets in different time intervals, while the energy contribution distribution trend reflects the changing pattern of energy proportions over time. Specifically, adjacent time intervals are selected sequentially from the dynamic weight sequence, and the fusion weight values ​​of each adjacent interval are grouped into a data pair. Simultaneously, the first and second energy proportion data of the corresponding intervals are obtained from the previously generated energy contribution distribution data, and the energy contribution distribution trend of this interval and the intervals before and after it is analyzed. This information is then associated with the weight data pairs.

[0163] Step S5363: Calculate the absolute difference of each weight data pair to obtain the weight change range of adjacent intervals. Compare the change range with the allowable change range of the corresponding running stage in the weight change consistency template, and mark the abnormal change range and corresponding weight data pairs that exceed the allowable range.

[0164] Calculating the magnitude of weight changes between adjacent intervals quantifies the degree of weight variation between adjacent time intervals. By calculating the absolute difference of each weight data pair, a specific numerical value is obtained to represent the magnitude of change. Comparing this magnitude of change with the allowable magnitude of change for the corresponding operational stage in the weight change consistency template determines whether the weight change is within a reasonable range. If the magnitude of change exceeds the allowable range, the abnormal magnitude of change and the corresponding weight data pair are marked for subsequent correction.

[0165] In the actual calculation and comparison process, for each weighted data pair, the absolute value of the difference between the fused weight value of the previous interval and the fused weight value of the next interval is used to obtain the weight change range of the adjacent interval. Then, based on the running stage of the weighted data pair, the corresponding allowable change range is obtained from the weight change consistency template, and the two are compared. If the weight change range of the adjacent interval is greater than the allowable change range, the abnormal change range and the corresponding weighted data pair are marked, for example, by setting a special flag.

[0166] Step S5364: Analyze the energy contribution distribution trend corresponding to the abnormal change amplitude, extract the cumulative energy value change pattern of the interval and the adjacent intervals before and after, and determine whether the abnormal change amplitude of the weight is synchronized with the change in the energy proportion trend. If synchronized, retain the original weight data; if not synchronized, determine that the weight is abnormal and needs to be corrected.

[0167] Analyzing the energy contribution distribution trend corresponding to abnormal changes can provide insights into the intrinsic relationship between weight changes and energy proportions. Extract the cumulative energy value change patterns of the given interval and adjacent intervals, and observe the trend of energy proportion changes. If the abnormal weight change is synchronized with the energy proportion trend, it indicates that the weight change may be due to normal changes in energy proportions, and therefore the original weight data should be retained. If they are not synchronized, it suggests that the weight change may be unreasonable, and it should be identified as an abnormal weight that needs correction.

[0168] For example, for the abnormal change magnitude of the marked data and the corresponding weight data pair, examine the associated energy contribution distribution trend. Plot the cumulative energy value over time for the interval and the adjacent intervals before and after it, and analyze the trend of energy proportion change. Compare the weight change trend and the energy proportion change trend. If the direction and rate of change of the two are basically the same, they are considered to be synchronous; otherwise, the weight is determined to be abnormal and needs to be corrected.

[0169] Step S5365: For data pairs determined to have abnormal weights, calculate transition weight values ​​using linear interpolation based on the energy proportion data of the preceding and following intervals, the fused weight values, and the energy contribution distribution trend, so that the magnitude of the corrected weight change is within the allowable range and consistent with the energy proportion trend.

[0170] For example, first determine the fusion weight values ​​and corresponding time points for the preceding and following intervals, as well as the energy proportion data and energy contribution distribution trends for the preceding and following intervals. Based on this information, establish a linear interpolation model. In this model, the impact of energy proportion data on weight changes is considered, and the weight values ​​for the transition interval are calculated through linear interpolation. Adjust the interpolation parameters so that the magnitude of the corrected weight changes meets the allowable range, and the weight change trend matches the energy proportion trend.

[0171] Step S5366: Update the corrected weight data pairs to the dynamic weight sequence, recalculate the weight change magnitude of all adjacent intervals and compare it with the template's allowed range, repeat the verification and correction process until the weight change magnitude of all adjacent intervals meets the template requirements, and generate a dynamic weight sequence that is completely synchronized with the energy contribution distribution.

[0172] For example, replace the original abnormal weight data pairs in the dynamic weight sequence with the corrected weight data pairs. Then, recalculate the weight change magnitude of all adjacent intervals and compare it again with the allowable change magnitude for the corresponding running stage in the weight change consistency template. If there are still cases that exceed the allowable range, repeat steps S5364-S5365 for correction. Continue this process until the weight change magnitude of all adjacent intervals is within the allowable range, resulting in a dynamic weight sequence that is completely synchronized with the energy contribution distribution.

[0173] Step S540: Based on the dynamically adjusted fusion weights, perform weighted summation of each feature component on the first key anomaly feature subset and the second key anomaly feature subset. The weight of each component is consistent with the fusion weight of the corresponding time interval, and a fusion feature vector is generated.

[0174] For example, first determine the number and order of feature components in the first and second key anomaly feature subsets. Then, group the feature components according to time intervals. For each time interval, sum the corresponding feature components of the first and second key anomaly feature subsets according to the fusion weight in the dynamic weight sequence. Combine the weighted sums of all time intervals in order to obtain the fused feature vector.

[0175] Step S550: Calculate the energy distribution of the fused feature vector in the entire feature space, and calculate the cumulative distribution of feature energy in different feature dimensions and time intervals by integration, and generate the cumulative distribution data of fused feature energy.

[0176] The energy distribution of the fused feature vectors across the entire feature space reflects the contribution of different feature dimensions and time intervals to the overall anomalous features. By integrating the cumulative feature energy across different feature dimensions and time intervals, the cumulative distribution data of fused feature energy can be obtained. Specifically, the fused feature vectors are divided according to feature dimensions and time intervals. For each combination of feature dimensions and time intervals, the energy value of the feature vectors within that combination is calculated. Methods such as summing the squares of the feature vector values ​​can be used to calculate the energy value. Then, these energy values ​​are integrated to obtain the cumulative energy value across different feature dimensions and time intervals. All cumulative energy values ​​are arranged in order of feature dimensions and time intervals to generate the cumulative distribution data of fused feature energy.

[0177] Step S560: Compare the fused feature energy cumulative distribution data with the preset anomaly threshold. If the energy cumulative values ​​of multiple feature dimensions or time intervals exceed the preset anomaly threshold, it is determined that the device is in an abnormal operating state; otherwise, it is determined that the device is in a normal operating state.

[0178] A preset anomaly threshold can be a standard pre-set based on experimental data or experience, used to determine whether the equipment is in an abnormal operating state. Comparing the fused feature energy cumulative distribution data with the preset anomaly threshold provides a clear view of whether the abnormal feature energy exceeds the normal range across different feature dimensions and time intervals. If the cumulative energy values ​​of multiple feature dimensions or time intervals exceed the preset anomaly threshold, it indicates that the equipment exhibits abnormal characteristics in multiple aspects, and the equipment is determined to be in an abnormal operating state. If the cumulative energy values ​​of all feature dimensions and time intervals are within the preset anomaly threshold, the equipment is determined to be in a normal operating state. Specifically, each cumulative energy value in the fused feature energy cumulative distribution data is compared with the preset anomaly threshold. The number of feature dimensions and time intervals exceeding the preset anomaly threshold is counted. If the number exceeding the threshold reaches a certain standard (i.e., multiple feature dimensions or time intervals exceed the threshold), the equipment is determined to be in an abnormal operating state; otherwise, the equipment is determined to be in a normal operating state.

[0179] Please see Figure 2 , Figure 2 This is a schematic diagram of a computer system provided in an embodiment of the present invention. The computer system includes at least a processor 101, a communication interface 102, and a memory 103. The processor 101, communication interface 102, and memory 103 can be connected via a bus or other means. The processor 101 (or Central Processing Unit, CPU) is the computing and control core of the computer system, capable of parsing various instructions and processing various data within the computer system. The communication interface 102 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, mobile communication interface, etc.), and can be used to send and receive data under the control of the processor 101; the communication interface 102 can also be used for data transmission and interaction within the computer system. The memory 103 is a storage device in the computer system used to store programs and data. It is understood that the memory 103 here can include the computer system's built-in memory, or it can include extended memory supported by the computer system. The memory 103 provides storage space, which stores the computer system's operating system; this invention does not limit this storage space.

[0180] In one embodiment, the processor 101 executes the digital factory equipment anomaly detection method based on voiceprint recognition provided above in the embodiments of the present invention by running a computer program in the memory 103.

Claims

1. A method for detecting anomalies in digital factory equipment based on voiceprint recognition, characterized in that, The method includes: Acquire the acoustic signature signal stream generated by the digital factory equipment during operation, wherein the acoustic signature signal stream carries time-frequency domain fluctuation information and timestamp markers corresponding to the equipment operating status; The acoustic signature signal stream is subjected to joint time-frequency domain analysis to generate a time-frequency hybrid feature set containing time-dimensional fluctuation features and frequency-dimensional spectral features; The time-frequency hybrid feature set is input into a preset self-supervised pre-trained model for feature mapping processing to generate a voiceprint embedding vector. The equipment condition information corresponding to the voiceprint signal stream is retrieved, and the voiceprint embedding vector is associated with the equipment condition information to construct a feature expansion matrix with condition labels. Based on the aforementioned feature expansion matrix, a hierarchical attention mechanism is employed to perform key anomaly feature extraction processing. Through multi-level feature filtering, a first key anomaly feature subset and a second key anomaly feature subset are obtained. Specifically, this includes: inputting the feature expansion matrix into the input interface of the hierarchical attention mechanism, dividing it into a time-related feature layer, a condition-related feature layer, and a cross-correlation feature layer according to feature attributes; assigning attention weights to the time-related feature layer, analyzing the correlation between feature vectors corresponding to different timestamps and the evolution of equipment operating states, assigning corresponding attention weights to each timestamp feature, and generating a time attention weight matrix; and assigning attention weights to the condition-related feature layer, analyzing the contribution of different condition feature dimensions to equipment anomaly identification, and assigning attention weights to each... Attention weights are assigned to each working condition feature dimension to generate a working condition attention weight matrix. The time attention weight matrix is ​​applied to the time-related feature layer to perform weighted enhancement processing on the feature vectors. At the same time, the working condition attention weight matrix is ​​applied to the working condition-related feature layer to obtain a weighted time feature layer and a weighted working condition feature layer. Cross-layer feature fusion is performed on the weighted time feature layer, the weighted working condition feature layer, and the cross-related feature layer to extract complementary information between the layers and generate a fusion enhanced feature matrix. Feature filtering is performed on the fusion enhanced feature matrix to separate the feature components that emphasize the expression of time dimension anomalies and the feature components that emphasize the expression of working condition related anomalies according to their relevance to anomaly identification, to obtain the first key anomaly feature subset and the second key anomaly feature subset, respectively. The first key abnormal feature subset and the second key abnormal feature subset are dynamically weighted and fused to generate a fused feature vector. The energy distribution of the abnormal features in the fused feature vector is compared with a preset abnormal threshold to determine whether the device has an abnormal operating state. The step of comparing and analyzing the energy distribution of abnormal features in the fused feature vector with a preset abnormal threshold to determine whether the device is in an abnormal operating state includes: comparing the cumulative energy distribution data of the fused features with the preset abnormal threshold; if the cumulative energy values ​​of multiple feature dimensions or time intervals exceed the preset abnormal threshold, the device is determined to be in an abnormal operating state; otherwise, the device is determined to be in a normal operating state.

2. The method according to claim 1, characterized in that, The process of performing joint time-frequency domain analysis on the acoustic signature signal stream generates a time-frequency hybrid feature set containing both time-dimensional fluctuation features and frequency-dimensional spectral features, including: According to the timestamp of the voiceprint signal stream, the voiceprint signal stream is segmented into segments with continuous and non-overlapping fixed time windows to obtain multiple orderly arranged segmented voiceprint signal units. Each segmented voiceprint signal unit is bound to the corresponding timestamp start point and end point information. Each segment of the voiceprint signal unit is analyzed point by point in the time dimension, and the changes in the signal amplitude and the evolution trajectory of the phase over time are recorded at each sampling moment to obtain the original change sequence in the time dimension. At the same time, the boundary change data of adjacent segment voiceprint signal units are correlated. Each segment of the voiceprint signal unit is decomposed into frequency dimension, and the distribution range of various frequency components contained in the signal and their respective energy transfer are analyzed to obtain the original frequency dimension distribution sequence. The occurrence order and duration of different frequency components are recorded simultaneously. The original time dimension change sequence of each segmented voiceprint signal unit is timestamped with the original frequency dimension distribution sequence to establish a real-time correspondence between the two and generate a preliminary time-frequency feature association pair. Cross-segment feature fusion is performed on the preliminary time-frequency feature association pairs of all segmented voiceprint signal units to extract the evolution law of time-frequency features between adjacent segments, supplement boundary transition feature information, and obtain complete time-frequency feature association pairs; All time-frequency features are fully associated and integrated and arranged in the order of timestamps to obtain a time-frequency hybrid feature set containing time-dimensional fluctuation features and frequency-dimensional spectral features. Each feature component retains the corresponding segment identifier and timestamp information.

3. The method according to claim 2, characterized in that, The process involves point-by-point temporal analysis of each segmented acoustic signal unit, recording the changes in signal amplitude and phase evolution at each sampling moment to obtain the original temporal change sequence. Simultaneously, it correlates the boundary change data of adjacent segmented acoustic signal units, including: The amplitude and phase data of each segmented acoustic signal unit are extracted at all sampling times and arranged sequentially according to the sampling time to obtain the original amplitude sequence and the original phase sequence, so that the data order is consistent with the timestamp order; The original amplitude sequence is subjected to point-by-point difference calculation to obtain the amplitude change at adjacent sampling times, and the original phase sequence is subjected to point-by-point difference calculation to obtain the phase change at adjacent sampling times, thus obtaining the amplitude change sequence and the phase change sequence. The amplitude change sequence is divided into sub-intervals. The dispersion of the change in each sub-interval is calculated. Sub-intervals whose dispersion meets the preset standard are marked as amplitude stable intervals, and sub-intervals whose dispersion does not meet the preset standard are marked as amplitude fluctuation intervals. The starting and ending sampling points of each interval are recorded. The phase change sequence is divided into sub-intervals and the degree of dispersion is calculated in the same way. Sub-intervals that meet the preset standard are marked as phase stable intervals, and sub-intervals that do not meet the preset standard are marked as phase fluctuating intervals. The sampling point range of each interval is determined, and the time overlap range of the amplitude and phase stable intervals and fluctuating intervals is compared. Retrieve the last sampling point data of the previous segment of the current segment and the starting sampling point data of the next segment of the current segment, and calculate the amplitude difference and phase difference between the current segment and the segments before and after at the boundary sampling point. The original amplitude sequence, original phase sequence, amplitude change sequence, phase change sequence, and boundary difference data are integrated to obtain the original change sequence in the time dimension. Each sequence is bound to a corresponding interval label and boundary association information.

4. The method according to claim 1, characterized in that, The process of inputting the time-frequency hybrid feature set into a preset self-supervised pre-trained model for feature mapping to generate a voiceprint embedding vector, retrieving equipment condition information corresponding to the voiceprint signal stream, associating the voiceprint embedding vector with the equipment condition information, and constructing a feature expansion matrix with condition labels includes: For each time-frequency feature pair in the time-frequency hybrid feature set, a dimension verification is performed to check the dimensional consistency between the time dimension fluctuation feature and the frequency dimension spectrum feature. Features with inconsistent dimensions are aligned to generate a standardized time-frequency hybrid feature set. The standardized time-frequency hybrid feature set is input into the input layer of a preset self-supervised pre-trained model. The features are then mapped hierarchically through feature transformation within the model, gradually compressing the feature dimensions while retaining the core information, and generating a voiceprint embedding vector. Based on the timestamp range of the voiceprint signal stream, equipment operating condition information within the same time range is retrieved from the digital factory equipment operating condition database. The equipment operating condition information includes equipment operating load, cumulative working time, ambient temperature and humidity, and the sequence of executed operation instructions. The retrieved equipment operating condition information is structured and transformed, converting non-numerical operation instruction sequences into coded data in a unified format, and performing range normalization on numerical data to generate standardized equipment operating condition data. The acoustic embedding vector corresponding to each segmented acoustic signal unit is correlated with the standardized equipment operating condition data of the same period in a dimension-by-dimensional manner to explore the inherent correspondence between acoustic features and operating condition data and generate acoustic-operating condition correlation feature vector. Arrange the voiceprint-operating condition association feature vectors corresponding to all segmented voiceprint signal units in the order of timestamps to obtain a multi-row, multi-column matrix structure. Each row corresponds to a segmented association feature, and each column corresponds to a feature dimension, generating a feature expansion matrix with operating condition labels.

5. The method according to claim 4, characterized in that, The step of associating the acoustic embedding vector corresponding to each segmented acoustic signal unit with the standardized equipment operating condition data of the same period dimensionally, mining the inherent correspondence between acoustic features and operating condition data, and generating an acoustic-operating condition associated feature vector includes: The various feature dimensions of the voiceprint embedding vector are divided into amplitude-related dimensions, phase-related dimensions, and time-frequency-related dimensions according to their functional attributes. The standardized equipment operating condition data are divided into operating status dimensions, environmental influence dimensions, and operation command dimensions. Calculate the correspondence between amplitude correlation dimension and operating status dimension, analyze the variation law of amplitude characteristics under different operating loads, generate amplitude-operation correlation data, and calculate the correspondence between phase correlation dimension and environmental impact dimension, generate phase-environment correlation data; Calculate the correspondence between the time-frequency correlation dimension and the operation command dimension, analyze the response of voiceprint time-frequency features when different operation commands are executed, generate time-frequency-operation correlation data, and record the corresponding strength and stability of various correlation data; The amplitude-operation correlation data, phase-environment correlation data, and time-frequency-operation correlation data are fused across dimensions to extract common trends in different correlation dimensions, supplement cross-correlation features, and obtain comprehensive correlation data. Based on the feature distribution of the comprehensive association data, the feature dimensions of the voiceprint embedding vector and the standardized equipment operating condition data are filtered. Feature dimensions that match the corresponding association type in the comprehensive association data with preset conditions are retained, while feature dimensions that do not match any association type in the comprehensive association data with preset conditions are removed. The selected voiceprint embedding vector features and standardized equipment operating condition data features are arranged in order of correlation strength and integrated to obtain a voiceprint-operating condition correlation feature vector, in which each feature component is labeled with the corresponding correlation type and correlation strength information.

6. The method according to claim 5, characterized in that, The calculation establishes a correspondence between the amplitude correlation dimension and the operational status dimension, analyzes the variation patterns of amplitude characteristics under different operational loads, and generates amplitude-operation correlation data, including: Feature data of all amplitude-related dimensions in the voiceprint embedding vector are extracted and arranged in chronological order to obtain an amplitude feature sequence. Operational load data of the operational status dimension in the standardized equipment operating condition data are extracted to obtain an operational load sequence, so that the timestamps of the two are aligned. The amplitude feature sequence and the operating load sequence are simultaneously subjected to sliding window analysis, with each window covering the same time range. Statistical data of amplitude features and operating load within each window are calculated to obtain window-level amplitude statistics and window-level load statistics. Analyze how the window-level amplitude statistics change with the window-level load statistics within each window, record the relationship between the two—whether they rise synchronously, fall synchronously, or change in opposite directions—and mark the window range and magnitude of change corresponding to each relationship. The changes in all windows are categorized and statistically analyzed. Changes that meet the preset core screening criteria are identified, and changes that do not match the preset set of typical changes are recorded. This results in a set of change relationship categories, with each category containing the corresponding number of windows and time distribution. Based on the classification set of change relationships, a correspondence model between amplitude characteristics and operating load is constructed, which clarifies the amplitude characteristic range and change trend corresponding to different operating load intervals, and marks the applicable time range of the model; The change relationship classification set, correspondence model and related window statistics are integrated to generate amplitude-operation correlation data, which includes the correspondence between features and load, change patterns and applicable conditions.

7. The method according to claim 6, characterized in that, The analysis examines the changes in window-level amplitude statistics with window-level load statistics within each window, recording whether they rise synchronously, fall synchronously, or change in opposite directions. It also marks the window range and magnitude of change corresponding to each relationship, including: Calculate the rate of change of the amplitude of the window-level amplitude statistics within each window. The rate of change of amplitude is the difference between the amplitude statistics of the current window and the previous window divided by the time interval. Also, the rate of change of the load statistics of the window-level load is obtained using the same calculation method. Compare the signs of the amplitude change rate and load change rate for each window. If both signs are the same and the values ​​are both greater than zero, they are determined to be in a synchronous upward relationship; if both signs are the same and the values ​​are both less than zero, they are determined to be in a synchronous downward relationship; if the signs are opposite, they are determined to be in an inverse relationship. Calculate the ratio of amplitude change rate to load change rate within each window. The closer the ratio is to the preset benchmark value, the better the change amplitude matches. Record the deviation between the ratio and the benchmark value. Mark the window number, window start timestamp, and end timestamp corresponding to each change relationship, and record the maximum and minimum values ​​of amplitude statistics and load statistics within that window; For windows showing synchronous rise and synchronous fall, analyze whether the rate of change of amplitude and the rate of change of load are consistent, and record the cases where the rates of change are consistent and inconsistent, as well as the corresponding window information; The change relationship type, window range, change rate ratio, deviation, extreme value data, and rhythm synchronization of each window are integrated and recorded to obtain a complete record of window change relationships.

8. The method according to claim 1, characterized in that, The process involves assigning attention weights to the time-related feature layer, analyzing the correlation between the feature vectors corresponding to different timestamps and the evolution of the device's operating state, assigning corresponding attention weights to each timestamp feature, and generating a time attention weight matrix, including: Extract the feature vector corresponding to each timestamp in the time-related feature layer, calculate the degree of difference between each feature vector and the standard time feature vector when the device is running normally, and obtain the feature difference degree of each timestamp. The difference degree reflects the degree of deviation of the feature from the normal state. Analyze the characteristic difference data of all timestamps, identify the intervals where the difference evolves in a continuous and consistent direction and the time points where the difference deviates from the adjacent numerical range, record the range and direction of the difference change before and after the deviation time point, and mark the evolution stability information of the intervals where the difference evolves in a continuous and consistent direction. Retrieve the standard time feature evolution template corresponding to normal equipment operation. The feature evolution template records the standard evolution range and evolution rate of feature differences within the entire operating cycle of the equipment, and clarifies the reasonable range of differences corresponding to different operating stages. Match and verify each feature difference with the reasonable range of the corresponding stage in the standard time feature evolution template according to the timestamp order, mark the difference data that exceeds the preset matching range, and record the range that exceeds the range and the corresponding timestamp information. Based on the matching verification results, the importance score of each timestamp feature is calculated. Timestamps whose feature difference is within a reasonable range and consistent with the trend of the standard evolution template are given the corresponding score. Timestamps that exceed the matching range have their score weights adjusted according to preset rules. The importance scores of all timestamps are normalized to ensure that the scores are within a uniform range. The normalized scores are then used as the attention weights for each timestamp feature. Arrange the attention weights of all timestamps in the order of timestamps to obtain a time attention weight matrix that is consistent with the dimension of the time-related feature layer. Each element in the matrix corresponds to the attention weight of a timestamp feature.

9. A computer system, characterized in that, include: A memory, wherein a computer program is stored; A processor is configured to load the computer program to implement the digital factory equipment anomaly detection method based on voiceprint recognition as described in any one of claims 1-8.

Images