A data processing method and a computer device

Abstract

The application provides a data processing method and a computer device, which can be applied to the field of device predictive maintenance, and comprises the following steps: determining a weight matrix (which represents the importance proportion of m features in x historical data sets) according to the x historical data sets, and performing y times of solving on a target data set to obtain y target transformation matrices respectively; determining a target feature score vector (which represents the importance degree of the m features in the target data set) according to the weight matrix and the y target transformation matrices; and selecting k target features from the m features according to the target feature score vector. The application fully utilizes the information of the historical data sets, determines the importance proportion of each feature, and the calculation process comprises a nonlinear optimization algorithm such as matrix decomposition, so that the complex interaction between the features is fully considered. Moreover, the method has low calculation complexity, has no requirement for the data distribution (no fault sample is needed) and data volume of the target data set, and well solves the feature selection problem under the condition that the data sample distribution is uneven.

Description

Technical Field

[0001] This application relates to the field of data processing, and more particularly to a data processing method and a computer device. Background Technology

[0002] Industries such as coal mines, ports, and power plants contain a large number of electromechanical equipment. To avoid production losses caused by unplanned equipment downtime, predictive maintenance has become a crucial strategy for improving operational efficiency. The core of building a predictive maintenance strategy is to discover potential early signs of failure through data processing and analysis. The core of data processing and analysis is feature extraction, which involves transforming and calculating raw data (also known as raw signals) to obtain quantitative indicators that characterize the fault. Because different equipment operates in different environments and under different service conditions, the signal changes caused by faults vary significantly across different equipment, making it difficult to use the same features for fault identification. Therefore, after feature extraction, feature selection needs to be performed through analysis of field signals to adapt to changes in environment and operating conditions, thereby ensuring the accuracy of fault identification.

[0003] The purpose of feature selection is to remove irrelevant features to reduce the difficulty of the learning task. Current solutions mainly include: 1. Feature selection based on relevance. Specifically, statistical indicators are used to quantify the relationship between input and output variables, thereby selecting input variables with a high degree of correlation to the output variable. Common statistical indicators include Peason correlation coefficient, entropy index, and Shapley value. 2. Algorithms based on decision trees (e.g., random forests). In this approach, important features are more likely to appear in shallower nodes (the closer to the root node, the higher the importance), and they may appear more frequently. Features can then be ranked by importance based on indicators such as the frequency of feature occurrences in the tree model.

[0004] However, all of the above methods have some drawbacks: Method 1 assumes that features are independent of each other. However, in reality, features often have complex dependencies, leading to the selection of an inaccurate or ineffective feature set. Secondly, this method requires complete data, meaning that different types of samples (normal and abnormal) are evenly distributed. This constraint reduces deployment efficiency in predictive maintenance scenarios, and the results are unstable when the dataset contains only normal samples. In Method 2, when the number of features in the dataset is very large, building the decision tree and calculating feature importance requires a lot of time and computational resources, resulting in excessive computational overhead in the feature selection process, especially when dealing with large-scale datasets. Furthermore, this method is ineffective in handling imbalanced scenarios because imbalance has a significant negative impact on decision tree construction and feature selection, making it difficult to capture useful information from the data, and the results are unstable when the dataset contains only normal samples. Summary of the Invention

[0005] This application provides a data processing method and computer equipment for fully utilizing information from historical datasets (i.e., x historical datasets) to determine the importance percentage of each feature to be considered (i.e., m features). The calculation process includes nonlinear optimization algorithms such as matrix factorization, which fully considers the complex interactions between features. Furthermore, this method has low computational complexity and no requirements on the data distribution of the target dataset (no fault samples are required) or the amount of data (only a small number of normal samples are required), thus effectively solving the feature selection problem under the condition of unbalanced data sample distribution.

[0006] Based on this, the embodiments of this application provide the following technical solutions:

[0007] Firstly, this application provides a data processing method, which specifically includes: First, determining a weight matrix based on x (x≥1) historical datasets, where each historical dataset includes t (t≥1) historical data entries. This weight matrix, also called a feature score weight matrix or weight regression matrix, is used to characterize the importance percentage of m (m≥2) features in the x historical datasets. Each element in the weight matrix can be called a feature score regression weight or simply a regression weight, representing the importance percentage of the current feature in the current historical data entry. For example, the regression weight Qij represents the value of the importance percentage of the current feature i (m≥i≥1) in the j (x≥j≥1) historical data entry. In the embodiments of this application, the number of historical data entries included in each historical dataset can be the same or different, and this application does not limit this. For example, historical dataset 1 includes t1 historical data entries, historical dataset 2 includes t2 historical data entries, ..., historical dataset x includes tx historical data entries, and the values ​​of t1, t2, ..., tx can be the same or different. Furthermore, each historical dataset possesses m-dimensional features (e.g., mean, variance, kurtosis, harmonic ratio, etc.). The acquired target dataset undergoes y (y≥1) optimization iterations, yielding y target transformation matrices. Each target transformation matrix represents the statistical state of the target dataset in one aspect, and the y target transformation matrices represent the statistical state of the target dataset in each of the y aspects. Next, based on the obtained weight matrix and the y target transformation matrices, a feature score vector for the target dataset is determined. This feature score vector, also known as the target feature score vector, represents the importance of the m features in the target dataset. Finally, based on the determined target feature score vector, k (k≥1) features are selected from the m features as the final target features. For example, the features with the top k score values ​​can be selected as the target features.

[0008] In the above embodiments of this application, information from x historical datasets is fully utilized to determine the importance percentage of each feature to be considered (i.e., m features). The target feature score vector is obtained through the target transformation matrix and weight matrix, achieving feature optimization. This calculation process includes nonlinear optimization algorithms such as matrix factorization, fully considering the complex nonlinear relationships between features. Furthermore, the solution process reflects a data-driven approach, resulting in more stable results. This method has low computational complexity and no requirements on the data distribution (no fault samples are needed) or data volume (only a small number of normal samples are needed) of the target dataset, effectively solving the feature selection problem under conditions of imbalanced data sample distribution.

[0009] In one possible implementation of the first aspect, one way to determine the weight matrix based on x historical datasets is as follows: First, determine the historical representation matrix of the x historical datasets based on the x historical datasets, and determine the historical feature scoring matrix of the x historical datasets based on the x historical datasets, wherein the historical feature scoring matrix is ​​used to characterize the importance of the above m features in the x historical datasets. Finally, obtain the weight matrix based on the historical representation matrix and the historical feature scoring matrix.

[0010] In the above embodiments of this application, a weight matrix is ​​obtained based on the historical representation matrix and the historical feature scoring matrix, and feature optimization is achieved accordingly.

[0011] In one possible implementation of the first aspect, determining the historical representation matrices of x historical datasets based on x historical datasets can be achieved as follows: First, select one historical dataset from the x historical datasets as the target (referred to as the first historical dataset), and perform y (y≥1) statistical optimization solutions on it, obtaining y historical transformation matrices respectively. One historical transformation matrix represents the statistical state of the first historical dataset in one aspect, and the y historical transformation matrices represent the statistical state of the first historical dataset in each of the y aspects. Then, determine the historical representation vector of the first historical dataset based on the obtained y historical transformation matrices. For each of the x historical datasets, it can be used as the first historical dataset described above, and the above statistical optimization solution process can be repeated to obtain y historical transformation matrices corresponding to each historical dataset. For each historical dataset, a historical representation vector can be obtained based on the corresponding y historical transformation matrices, thus obtaining a total of x historical representation vectors. Finally, concatenate the obtained x historical representation vectors to obtain the historical representation matrix of the x historical datasets.

[0012] In the above embodiments of this application, the construction process of the historical characterization matrix is ​​specifically described. Based on statistical transformation decomposition, components that can characterize the device's characteristic state are extracted. By combining mechanistic knowledge, the stability of the feature optimization results is enhanced.

[0013] In one possible implementation of the first aspect, determining the historical representation vector of the first historical dataset based on y historical transformation matrices can be achieved as follows: First, calculate m features (e.g., mean, variance, kurtosis, harmonic ratio, etc.) of the historical data in the first historical dataset, which can then be expanded into corresponding time-series feature matrices (referred to as the first time-series feature matrices). Then, obtain y result matrices (referred to as the first result matrices) based on the aforementioned y historical transformation matrices and the first time-series feature matrices. For example, matrix multiplication can be performed between the aforementioned y historical transformation matrices and the first time-series feature matrix to obtain y first result matrices, where one historical transformation matrix corresponds to one first result matrix. Finally, obtain the historical representation vector based on the y first result matrices obtained above.

[0014] In the above embodiments of this application, it is specifically explained that the historical representation vector is obtained by multiplying each historical transformation matrix by the first temporal feature matrix to obtain each first result matrix, and finally by obtaining the y first result matrices, which has low computational complexity.

[0015] In one possible implementation of the first aspect, one way to obtain the historical representation vector based on y first result matrices is as follows: First, extract the first row of each of the y first result matrices to obtain y historical representation sub-vectors, with one first result matrix corresponding to one historical representation sub-vector. Finally, concatenate the y historical representation sub-vectors end to end to obtain the historical representation vector.

[0016] In the above embodiments of this application, the first row of each first result matrix is ​​extracted as a historical representation sub-vector, resulting in a total of y historical representation sub-vectors. Then, the y historical representation sub-vectors are concatenated to obtain the historical representation vector. This method has low computational complexity and embodies the idea of ​​data-driven approach, making the calculation results more stable.

[0017] In one possible implementation of the first aspect, the aforementioned y historical transformation matrices include, but are not limited to, any one or more of the following: historical steady-state transformation matrix, historical monotonic transformation matrix, and historical principal energy transformation matrix, wherein the historical steady-state transformation matrix is ​​used to characterize the data form that maintains a relatively stable state in x historical datasets, the historical monotonic transformation matrix is ​​used to characterize the data form that maintains a continuous increasing trend or a continuous decreasing trend in x historical datasets, and the historical principal energy transformation matrix is ​​used to characterize the data form that characterizes the energy magnitude in x historical datasets.

[0018] In the above embodiments of this application, the main components extracted are the time-series stable components, trend components, and energy components. By combining the three aspects of stability, trend, and energy, comprehensive coverage is achieved.

[0019] In one possible implementation of the first aspect, determining the historical feature score matrix of x historical datasets based on x historical datasets can be achieved as follows: First, select one historical dataset from the x historical datasets as the target (i.e., the first historical dataset), calculate the feature score of the first historical dataset, and obtain a historical feature score vector, where the historical feature score vector is used to characterize the importance of the aforementioned m features in the first historical dataset. For each of the x historical datasets, it can be used as the first historical dataset described above, and the above process can be repeated to obtain the historical feature score vector corresponding to each historical dataset (a total of x historical feature score vectors). Finally, the obtained x historical feature score vectors are concatenated to obtain the historical feature score matrix of the x historical datasets.

[0020] In the above embodiments of this application, the construction process of the historical feature scoring matrix is ​​specifically described. It has low computational complexity, is easy to operate, and is feasible.

[0021] In one possible implementation of the first aspect, the weight matrix obtained from the historical representation matrix and the historical feature scoring matrix can be as follows: First, the historical feature scoring matrix is ​​decomposed into a first decomposition matrix and a second decomposition matrix. The decomposition process is then iteratively optimized according to the constructed optimization algorithm and constraints until the iteration termination condition is met. The second decomposition matrix obtained in the last decomposition is the weight matrix. The optimization algorithm is related to the errors of the first and second matrices. The first matrix error is the error between the historical feature scoring matrix obtained in the current optimization round and the historical feature scoring matrix obtained in the previous round. The second matrix error is the error between the first decomposition matrix and the historical representation matrix in the current optimization round.

[0022] In the above embodiments of this application, a matrix factorization optimization method is specifically described to optimize a randomly initialized matrix to obtain a weight matrix. This optimization process is an unsupervised optimization process that does not require sample labeling, which greatly improves ease of use.

[0023] In one possible implementation of the first aspect, the constraint on the decomposition could be that the first decomposition matrix approximates the historical representation matrix.

[0024] In the above embodiments of this application, the constraints of the constructed optimization algorithm are specifically described to improve the accuracy of iterative optimization.

[0025] In one possible implementation of the first aspect, the optimization algorithm could be the sum of the squares of the first matrix error and the squares of the second matrix error.

[0026] In the above embodiments of this application, a specific implementation of the constructed optimization algorithm is described, which is easy to construct and flexible.

[0027] In one possible implementation of the first aspect, the conditions for reaching the termination of the iteration include, but are not limited to: the value of the optimization algorithm being less than a preset threshold, for example, when the value of the optimization algorithm is < 0.01, the optimization solution ends; or, the number of execution rounds of the iterative optimization reaches a preset number of rounds, for example, when the preset number of rounds is set to 200, the optimization solution ends when the execution round of the iterative optimization reaches the 200th round; or, the execution time of the iterative optimization reaches a preset time, for example, when the preset time is set to 2 hours, the optimization solution ends when the execution time of the iterative optimization reaches 2 hours.

[0028] In the above embodiments of this application, several types of iteration termination conditions are specifically described, which have wide applicability.

[0029] In one possible implementation of the first aspect, the method of determining the target feature score vector of the target dataset based on the weight matrix and y target transformation matrices can be as follows: First, determine the target representation vector of the target dataset based on the y target transformation matrices; then, determine the target feature score vector of the target dataset based on the weight matrix and the target representation vector.

[0030] In the above embodiments of this application, a method for obtaining a target feature score vector by means of a target representation vector and a weight matrix is ​​specifically described. This method integrates mechanistic knowledge into optimization modeling, and the solution process reflects a data-driven approach, realizing the fusion of mechanism and data, resulting in more stable results.

[0031] In one possible implementation of the first aspect, determining the target representation vector of the target dataset based on y target transformation matrices can be achieved as follows: First, calculate m features (e.g., mean, variance, kurtosis, harmonic ratio, etc.) for the target data in the target dataset, which can then be expanded into corresponding temporal feature matrices (referred to as second temporal feature matrices). Then, obtain y result matrices (referred to as second result matrices) based on the aforementioned y target transformation matrices and the second temporal feature matrices. For example, matrix multiplication can be performed between the aforementioned y target transformation matrices and the second temporal feature matrix to obtain y second result matrices, where one target transformation matrix corresponds to one second result matrix. Finally, the target representation vector is obtained based on the y second result matrices obtained above.

[0032] In the above embodiments of this application, it is specifically explained that the target representation vector is obtained by multiplying each target transformation matrix by the second temporal feature matrix to obtain each second result matrix, and finally by obtaining the y second result matrices. The computational complexity is low.

[0033] In one possible implementation of the first aspect, one way to obtain the target representation vector based on y second result matrices is as follows: First, extract the first row of each of the y second result matrices to obtain y target representation sub-vectors, with one second result matrix corresponding to one target representation sub-vector. Finally, concatenate the y target representation sub-vectors to obtain the target representation vector.

[0034] In the above embodiments of this application, the first row of each second result matrix is ​​extracted as the target representation sub-vector, resulting in a total of y target representation sub-vectors. Then, the y target representation sub-vectors are concatenated to obtain the target representation vector. This method has low computational complexity and embodies the idea of ​​data-driven approach, making the calculation results more stable.

[0035] In one possible implementation of the first aspect, one way to determine the target feature score vector of the target dataset based on the weight matrix and the target representation vector is to perform matrix multiplication on the weight matrix and the target representation vector to obtain the target feature score vector of the target dataset.

[0036] In the above embodiments of this application, the target feature scoring vector = target representation vector * weight matrix is ​​simple to calculate and feasible.

[0037] In one possible implementation of the first aspect, the aforementioned y target transformation matrices include, but are not limited to, any one or more of the following: target steady-state transformation matrix, target monotonic transformation matrix, and target principal energy transformation matrix, wherein the target steady-state transformation matrix is ​​used to characterize the data form in the target dataset that maintains a relatively stable state, the target monotonic transformation matrix is ​​used to characterize the data form in the target dataset that maintains a continuous increasing trend or a continuous decreasing trend, and the target principal energy transformation matrix is ​​used to characterize the data form in the target dataset that characterizes the energy magnitude.

[0038] In the above embodiments of this application, the main components extracted are the time-series stable components, trend components, and energy components. By combining the three aspects of stability, trend, and energy, comprehensive coverage is achieved.

[0039] In one possible implementation of the first aspect, one way to select k target features from m features based on the target feature scoring vector is to select the k features with the highest scores from the m features as target features, where k ≥ 1.

[0040] In the above embodiments of this application, it is feasible to select the k features with the highest scores as the feature selection results.

[0041] A second aspect of this application provides a computer device having the function of implementing the method of the first aspect or any possible implementation thereof. This function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described function.

[0042] A third aspect of this application provides a computer device that may include a memory, a processor, and a bus system. The memory is used to store a computer program (also referred to as a program or computer-readable instructions), and the processor is used to invoke the program stored in the memory to execute the method of the first aspect of the embodiments of this application or any possible implementation of the first aspect.

[0043] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, enable the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0044] The fifth aspect of this application provides a computer program or a computer program product containing instructions that, when the computer program or computer program product is run on a computer, causes the computer to perform the method described in the first aspect or any possible implementation of the first aspect.

[0045] A sixth aspect of this application provides a chip including at least one processor and at least one interface circuit coupled to the processor. The interface circuit performs transceiver functions and sends instructions to the at least one processor. The at least one processor runs a computer program or instructions, having the functionality to implement the methods described in the first aspect or any possible implementation of the first aspect. This functionality can be implemented in hardware, software, or a combination of hardware and software, including one or more modules corresponding to the described functions. Furthermore, the interface circuit is used to communicate with other modules outside the chip.

[0046] In some implementations of this application, some of the one or more processors may implement some steps of the above method through dedicated hardware. For example, the processing involving neural network models may be implemented by a dedicated neural network processor or graphics processor.

[0047] The method provided in this application embodiment can be implemented by a single chip or by multiple chips working together. Attached Figure Description

[0048] Figure 1 A schematic diagram of the system architecture provided in the embodiments of this application;

[0049] Figure 2 A flowchart illustrating the data processing method provided in this application embodiment;

[0050] Figure 3 A schematic diagram illustrating the vectorization of the historical representation matrix as provided in an embodiment of this application;

[0051] Figure 4 A flowchart illustrating the process of solving the weight matrix provided in this application embodiment;

[0052] Figure 5 A schematic diagram illustrating an example of an embodiment of this application;

[0053] Figure 6 This is another example diagram provided for an embodiment of this application;

[0054] Figure 7 A schematic diagram illustrating the principle of the method of this application embodiment provided for the purpose of this application embodiment;

[0055] Figure 8 A schematic diagram of a computer device provided in an embodiment of this application;

[0056] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0057] This application provides a data processing method and computer equipment for fully utilizing information from historical datasets (i.e., x historical datasets) to determine the importance percentage of each feature to be considered (i.e., m features). The calculation process includes nonlinear optimization algorithms such as matrix factorization, which fully considers the nonlinear and complex relationships between features. Furthermore, this method has low computational complexity and no requirements on the data distribution of the target dataset (no fault samples are required) or the amount of data (only a small number of normal samples are required), thus effectively solving the feature selection problem under the condition of unbalanced data sample distribution.

[0058] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0059] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0060] The embodiments of this application are mainly applicable to the monitoring of key rotating equipment in the industry (such as fans, pumps), and their system architecture is as follows: Figure 1 As shown, the method in this embodiment is mainly executed in modules such as the historical data processing component and the real-time data processing component in the industrial hub, and the execution result is displayed in the mechanism early warning engine of the application system.

[0061] It should be noted that, in the embodiments of this application, Figure 1 The system architecture described herein is for illustrative purposes only and does not limit the deployment method of each unit / module / component. In some system architectures, there may be more or fewer units / modules / components, and this application does not limit this.

[0062] The data processing method provided in the embodiments of this application is described below. Please refer to the following for details. Figure 2 , Figure 2 A flowchart illustrating the data processing method provided in this application embodiment specifically includes the following steps:

[0063] 201. Determine a weight matrix based on x historical datasets. This weight matrix is ​​used to characterize the importance ratio of m features in x historical datasets. The historical datasets include t historical data points, which include m features in m dimensions, x≥1, m≥2, t≥1.

[0064] First, a weight matrix is ​​determined based on x (x≥1) historical datasets, each containing t (t≥1) historical data points. This weight matrix represents the importance percentage of m (m≥2) features within these x historical datasets. Each element in the weight matrix can be called a feature score regression weight, or simply a regression weight, and represents the importance percentage of the current feature within the current historical data point. For example, the regression weight Qij represents the importance percentage of the current feature i (m≥i≥1) within the j (x≥j≥1) historical data point.

[0065] It should be noted that in the embodiments of this application, the number of historical data entries included in each historical dataset may be the same or different, and this application does not limit this. For example, historical dataset 1 includes t1 historical data entries, historical dataset 2 includes t2 historical data entries, ..., historical dataset x includes tx historical data entries, and the values ​​of t1, t2, ..., tx may be the same or different.

[0066] It should also be noted that in the embodiments of this application, each historical dataset has m-dimensional features (e.g., mean, variance, kurtosis, harmonic ratio, etc.). Thus, each historical dataset can be expanded into an m*t or t*m time series matrix. Specifically, if the time series matrix uses "rows" to represent the number of data items (i.e., t) and "columns" to represent the feature data (i.e., m), then the time series matrix is ​​represented in the form of a t*m matrix. If the time series matrix uses "rows" to represent the feature data (i.e., m) and "columns" to represent the number of data items (i.e., t), then the time series matrix is ​​represented in the form of an m*t matrix. This application does not limit the specific matrix form of the time series matrix, but for ease of explanation, the t*m matrix form will be used as an example in the following descriptions, and will not be elaborated further.

[0067] It should be noted that in some embodiments of this application, the x historical datasets can be datasets obtained through standardization. Specifically, target sensors deployed in devices (one or more, not limited) periodically collect raw data (the raw data collected by all target sensors on all devices within a preset range over a period of time constitutes the raw dataset). For example, the raw data can be waveform data such as vibration acceleration signals, audio signals, and current waveform signals, or other types of data, depending on the type of target sensor. The type of raw data is not limited here. Finally, the raw dataset is split into x raw subsets. For example, the raw dataset can be split into x raw subsets based on the computing power of the computer device, and each of the x raw subsets is standardized to obtain the x historical datasets. It should be noted that in the embodiments of this application, each historical dataset needs to include fault data, i.e., abnormal samples, also referred to as fault samples.

[0068] It should also be noted that, in some embodiments of this application, the above-mentioned standardized operations may include, but are not limited to:

[0069] (1) The sampling frequency is standardized.

[0070] Downsampling is performed on all other historical datasets, using the sampling frequency of the historical dataset with the lowest sampling frequency as a benchmark.

[0071] For example, suppose there are a total of 4 historical datasets (i.e., x = 4), with sampling frequencies of 5KHz, 8KHz, 10KHz and 10KHz respectively. Then, the latter three historical datasets are downsampled based on 5KHz, so that the sampling frequency of all historical datasets after processing is 5KHz.

[0072] (2) Normalize the sampling duration.

[0073] The sampling duration of the historical dataset with the lowest sampling duration is used as the benchmark to truncate the duration of all other historical datasets.

[0074] For example, suppose there are a total of 4 historical datasets (i.e., x = 4), with single sampling durations of 10s, 2s, 5s, and 2s respectively. Then, the first and third historical datasets are truncated with a time truncation of 2s, so that the sampling duration of all historical datasets after processing is 2s.

[0075] It should be noted that, in the embodiments of this application, the standardization operation on the original subset can be the above-mentioned sampling frequency standardization and / or sampling duration normalization, and this application does not limit it in this way.

[0076] It should be noted that, in some embodiments of this application, one way to determine the weight matrix based on x historical datasets is as follows: First, determine the historical representation matrix of the x historical datasets based on the x historical datasets, and determine the historical feature scoring matrix of the x historical datasets based on the x historical datasets, wherein the historical feature scoring matrix is ​​used to characterize the importance of the above m features in the x historical datasets. Finally, the weight matrix is ​​obtained based on the historical representation matrix and the historical feature scoring matrix.

[0077] As an example, the specific construction process of the historical representation matrix and the historical feature scoring matrix is ​​explained below:

[0078] (1) The process of constructing the historical representation matrix.

[0079] Select one historical dataset from the x historical datasets as the target (i.e., the first historical dataset), and perform y (y≥1) statistical optimizations on it to obtain y historical transformation matrices. One historical transformation matrix is ​​used to characterize the statistical state of the first historical dataset in one aspect, and the y historical transformation matrices are used to characterize the statistical state of the first historical dataset in each of the y aspects.

[0080] As an example, the aforementioned y historical transformation matrices include, but are not limited to, any one or more of the following:

[0081] a. Historical steady-state transformation matrix (which can be represented by g) is used to characterize the data form that maintains a relatively stable state in the above x historical datasets.

[0082] As an example, the historical steady-state transformation matrix g can be obtained by solving the following optimization problem 1:

[0083]

[0084] b. Historical monotonic transformation matrix (which can be represented by p) is used to characterize the data patterns of the above x historical datasets that maintain a continuous increasing or decreasing trend.

[0085] As an example, the historical monotonic transformation matrix p can be obtained by solving the following optimization problem 2:

[0086]

[0087] c. Historical principal energy transformation matrix (which can be represented by E), used to characterize the data form representing the energy magnitude in the above x historical datasets.

[0088] As an example, the historical principal energy transformation matrix E can be obtained by solving the following optimization problem 3:

[0089]

[0090] Then, the historical representation vector (which can be represented by V) of the first historical dataset is determined based on the y historical transformation matrices obtained from the solution. Specifically, in some embodiments of this application, m features (e.g., mean, variance, kurtosis, harmonic ratio, etc.) can be calculated first for the historical data in the first historical dataset, which can then be expanded into corresponding time-series feature matrices (i.e., the first time-series feature matrices). Then, y result matrices (i.e., the first result matrices) are obtained based on the aforementioned y historical transformation matrices and the first time-series feature matrices. For example, the aforementioned y historical transformation matrices can be multiplied by the first time-series feature matrix to obtain y first result matrices, where one historical transformation matrix corresponds to one first result matrix. Finally, the historical representation vector V can be obtained based on the aforementioned y first result matrices.

[0091] As an example, assuming the y historical transformation matrices include a historical steady-state transformation matrix g, a historical monotonic transformation matrix p, and a historical principal energy transformation matrix E, then the corresponding first result matrix J1 can be calculated based on the historical steady-state transformation matrix g and the first time-series characteristic matrix; the corresponding first result matrix J2 can be calculated based on the historical monotonic transformation matrix p and the first time-series characteristic matrix; and the corresponding first result matrix J3 can be calculated based on the historical principal energy transformation matrix E and the first time-series characteristic matrix. Finally, the historical representation vector V can be obtained based on the first result matrices J1, J2, and J3 obtained above.

[0092] In some embodiments of this application, the first row of each of the y first result matrices may be extracted to obtain y historical representation sub-vectors (which can be represented by v), with one first result matrix corresponding to one historical representation sub-vector. Finally, the y historical representation sub-vectors are concatenated to obtain the historical representation vector V.

[0093] As an example, suppose the y historical transformation matrices include a historical steady-state transformation matrix g, a historical monotonic transformation matrix p, and a historical principal energy transformation matrix E, and the historical steady-state transformation matrix g corresponds to the first result matrix J1, the historical monotonic transformation matrix p corresponds to the first result matrix J2, and the historical principal energy transformation matrix E corresponds to the first result matrix J3. Then, extract the first row from the first result matrix J1 to obtain the first historical representation sub-vector v1, extract the first row from the first result matrix J2 to obtain the second historical representation sub-vector v2, and extract the first row from the first result matrix J3 to obtain the third historical representation sub-vector v3. Concatenate these three historical representation sub-vectors v1, v2, and v3 to obtain the historical representation vector V, i.e., V = [v1, v2, v3].

[0094] It should be noted that the above example is only for illustration. For y historical transformation matrices, the historical representation vector of the first historical dataset can be expressed as: V = [v1, v2, ..., vy].

[0095] In this embodiment of the application, for each of the x historical datasets, it can be used as the first historical dataset described above, and the above statistical optimization solution process can be repeated to obtain y historical transformation matrices corresponding to each historical dataset. That is, for historical dataset 1, y historical transformation matrices Z1 can be obtained (e.g., one or more of historical steady-state transformation matrix g1, historical monotonic transformation matrix p1, and historical principal energy transformation matrix E1), for historical dataset 2, y historical transformation matrices Z2 can be obtained (e.g., one or more of historical steady-state transformation matrix g2, historical monotonic transformation matrix p2, and historical principal energy transformation matrix E2), ..., for historical dataset x, y historical transformation matrices Zx can be obtained (e.g., one or more of historical steady-state transformation matrix gx, historical monotonic transformation matrix px, and historical principal energy transformation matrix Ex). For each historical dataset, a historical representation vector can be obtained based on the corresponding y historical transformation matrices. Therefore, a total of x historical representation vectors can be obtained. For example, historical dataset 1 corresponds to historical representation vector V1, historical dataset 2 corresponds to historical representation vector V2, ..., historical dataset x corresponds to historical representation vector Vx.

[0096] Finally, the obtained x historical representation vectors V1, V2, ..., Vx are concatenated to obtain the historical representation matrix of the above x historical datasets. This historical representation matrix can be represented by V_M, i.e., V_M = [V1, V2, ..., Vx].

[0097] (2) The process of constructing the historical feature scoring matrix.

[0098] Select one historical dataset from the x historical datasets as the target (i.e., the first historical dataset), calculate the feature score of the first historical dataset, and obtain the historical feature score vector, where the historical feature score vector is used to characterize the importance of the above m features in the first historical dataset.

[0099] As an example, for this first historical dataset, the current score for each feature can be calculated using the following formula (4) based on each single feature (i.e., each of the m features):

[0100]

[0101] Where σ is the standard deviation of the t current features (assuming that the first historical dataset contains t historical data, and each historical data has a value of a current feature, that is, there are a total of t current features), and μ is the mean of the t current features.

[0102] Perform the above operation on each of the m features in the first historical dataset to obtain the feature score for each of the m features, which can be denoted as CV1, CV2, ..., CV1. m These m feature scores can constitute the historical feature score vector r, which can be expressed as: r = [CV1, CV2, ..., CV2] m ].

[0103] In this embodiment of the application, for each of the x historical datasets, it can be used as the first historical dataset described above, and the above process can be repeated to obtain the historical feature score vector corresponding to each historical dataset (a total of x historical feature score vectors). For example, historical dataset 1 corresponds to historical feature score vector r1, historical dataset 2 corresponds to historical feature score vector r2, ..., historical dataset x corresponds to historical feature score vector rx.

[0104] Finally, the obtained x historical representation vectors r1, r2, ..., rx are concatenated to obtain the historical feature scoring matrix of the above x historical datasets. This historical feature scoring matrix can be represented by R, i.e., R = [r1, r2, ..., rx].

[0105] After constructing the historical representation matrix V_M and the historical feature scoring matrix R as described above, the weight matrix can be obtained accordingly. Specifically, the historical feature scoring matrix R can be decomposed into a first decomposition matrix (denoted as R1) and a second decomposition matrix (denoted as R2). The decomposition process is iteratively optimized according to the constructed optimization algorithm (denoted as J(a)) and constraints until the optimization iteration termination condition is reached. Among them, the second decomposition matrix (denoted as R_2) obtained in the last decomposition is the weight matrix. The optimization algorithm J(a) is related to the first matrix error (denoted as e1) and the second matrix error (denoted as e2). The first matrix error e1 is the error between the historical feature scoring matrix (denoted as R′) obtained in the current optimization round and the historical feature scoring matrix (denoted as R″) obtained in the previous round. The second matrix error e2 is the error between the first decomposition matrix (denoted as R1′) in the current optimization round and the historical representation matrix (i.e., V_M).

[0106] It should be noted that in some embodiments of this application, the constraint condition for decomposition can be: the first decomposition matrix R1 approximates the historical representation matrix V_M, that is: R1≈V_M.

[0107] It should also be noted that in some other embodiments of this application, the constructed optimization algorithm J(a) can be as shown in equation (5):

[0108] J(a)=e12+e22(5)

[0109] It should be noted that in the embodiments of this application, the matrix decomposition solution can adopt the following strategy: during the optimization process, an initial value neighborhood search strategy is added to optimize in the neighborhood and reduce the search space, as shown in Equation (6):

[0110]

[0111] It should also be noted that, in some embodiments of this application, the iteration termination condition may include, but is not limited to:

[0112] a. The value of the optimization algorithm J(a) is less than the preset threshold.

[0113] For example, when J(a) < 0.01, the optimization solution ends. At this time, if the obtained matrix decomposition result R = R_1 * R_2, where R_1 is the first decomposition matrix obtained from the last decomposition, R_2 is the second decomposition matrix obtained from the last decomposition, and R_2 is the weight matrix.

[0114] b. The execution rounds of iterative optimization reach the preset number of rounds.

[0115] For example, if the preset number of rounds is set to 200, the optimization ends when the 200th round of iteration is executed. At this point, the matrix decomposition result R = R_1 * R_2, where R_1 is the first decomposition matrix obtained from the last decomposition, R_2 is the second decomposition matrix obtained from the last decomposition, and R_2 is the weight matrix.

[0116] c. The execution time of the iterative optimization reaches the preset time.

[0117] For example, if the preset duration is set to 2 hours, the optimization ends when the execution time of the iterative optimization reaches 2 hours. At this time, if the obtained matrix decomposition result is R = R_1 * R_2, where R_1 is the first decomposition matrix obtained from the last decomposition, R_2 is the second decomposition matrix obtained from the last decomposition, and R_2 is the weight matrix.

[0118] In summary, the above weight matrix is ​​obtained based on "historical representation matrix * weight matrix = historical feature scoring matrix", as shown below. Figure 3 As shown, for the historical representation matrix, the temporal feature matrices of each of the x historical datasets can first be reasonably vectorized and represented, and then the components of y aspects can be extracted ( Figure 3This illustrates the historical representation vector V for each historical dataset, representing the three aspects of steady state, dynamic state, and principal energy. V = [v1, v2, v3, ...], where v1, v2, and v3 are the respective historical representation sub-vectors for the steady state, dynamic state, and principal energy aspects. The historical representation vector V can fully represent the characteristic states of the device. Figure 4 As shown, for the weight matrix, matrix decomposition can be used to fully consider the integrity of historical representation vectors in the optimization objective and optimization process, and the weight matrix can be obtained by constructing an optimization algorithm.

[0119] To further understand the process of determining the weight matrix in step 201 above, a specific example will be used below for illustration. Please refer to [link to relevant documentation]. Figure 5 In this example, taking high-frequency vibration signals (e.g., sampling frequency > 5 kHz) and 4 (i.e., x = 4) historical datasets as an example, the process of determining the weight matrix based on the aforementioned procedure is as follows:

[0120] Step 1: Data set standardization.

[0121] Standardization operations are performed on historical data from multiple historical datasets (each historical dataset includes at least one fault sample, and there are no strict requirements for the division between different historical datasets), including sampling frequency standardization and sampling duration normalization.

[0122] In this example, there are four historical datasets, with sampling frequencies of 5kHz, 6kHz, 8kHz, and 10kHz, and single sampling durations of 9s, 3s, 6s, and 3s, respectively. We downsample the latter three historical datasets using 5kHz as the baseline, and truncate the first and third historical datasets using 3s as the baseline, ensuring that all four historical datasets have a sampling frequency of 5kHz and a sampling duration of 3s.

[0123] Step 2: Construct the historical representation matrix.

[0124] Each historical dataset contains t historical data points. For example, in this instance, there are 4 historical datasets: historical dataset 1 has t1 historical data points, historical dataset 2 has t2 historical data points, historical dataset 3 has t3 historical data points, and historical dataset 4 has t4 historical data points. Each historical data point can be used to calculate m features (e.g., mean, variance, kurtosis, harmonic ratio, etc.), forming an m*t time series matrix.

[0125] For each historical dataset, the historical transformation matrix is ​​calculated sequentially through three statistical optimizations to obtain the historical steady-state transformation matrix, the historical monotonic transformation matrix, and the historical principal energy transformation matrix. These three transformation matrices are then multiplied by the m*t time series matrix to obtain three result matrices. The first row of each result matrix is ​​extracted as a historical representation sub-vector, resulting in a total of three historical representation sub-vectors. These three historical representation sub-vectors are then concatenated to obtain the historical representation vector for the historical dataset.

[0126] The process of constructing the historical representation vectors described above will be introduced using a historical dataset as an example:

[0127] 1) After standardization, the historical dataset is expanded into a corresponding m*t time series matrix.

[0128] 2) Solve the optimization problem 1 as shown in equation (1) above to obtain the historical steady-state transformation matrix g, and multiply g with the m*t time series matrix to obtain the result matrix. Extract the first row of the result matrix to obtain the first historical characterization sub-vector v1.

[0129] 3) Solve the optimization problem 2 as shown in equation (2) above to obtain the historical monotonic transformation matrix p, and multiply p with the m*t time series matrix to obtain the result matrix. Extract the first row of the result matrix to obtain the second historical representation sub-vector v2.

[0130] 4) Solve the optimization problem 3 as shown in equation (3) above to obtain the historical principal energy transformation matrix E, and multiply E with the m*t time series matrix to obtain the result matrix. Extract the first row of the result matrix to obtain the third historical characterization sub-vector v3.

[0131] 5) Concatenate the first and last of the above three historical representation sub-vectors v1, v2, and v3 to obtain the historical representation vector V of the historical dataset, V = [v1, v2, v3].

[0132] 6) The historical representation vectors corresponding to multiple historical datasets are combined to obtain the historical representation matrix of the historical dataset. Taking the above four historical datasets as an example, we can obtain four historical representation vectors V1, V2, V3, and V4. These four historical representation vectors can form the historical representation matrix V_M, where V_M = [V1, V2, V3, V4].

[0133] Step 3: Construct a historical feature scoring matrix.

[0134] 1) In each historical dataset, based on each single feature (i.e. each of the m features) in the current historical dataset (assuming it is historical dataset 1), the feature score of the current feature (assuming it is feature m1) is calculated using the above formula (4), and is denoted as CV1.

[0135] 2) Perform the above operation on all features in historical dataset 1 to obtain the feature scores of all m features, denoted as CV1, CV2, ..., CV1. m The historical feature score vector r1 is composed of m feature scores, which can be represented as: r1=[CV1,CV2,……,CV m ].

[0136] 3) Repeat the above operation on historical datasets 2-4 to obtain historical feature score vectors r2, r3, and r4.

[0137] 4) Concatenate r1, r2, r3, and r4 to obtain the historical feature scoring matrix R, R = [r1, r2, r3, r4].

[0138] Step 4: Obtain the weight matrix through optimization.

[0139] The historical feature scoring matrix R is decomposed, with the constraint that the first decomposition matrix R1≈V_M. An optimization algorithm J(a) as shown in equation (5) is constructed. The following strategy is adopted when solving the matrix decomposition: During the optimization process, an initial value neighborhood search strategy is added to optimize in the neighborhood and reduce the search space. For details, please refer to equation (6) above, which will not be elaborated here.

[0140] When J(a) < a specified threshold (e.g., 0.01), the optimization ends. At this point, the obtained matrix decomposition result R = R_1 * R_2, where R_1 is the first decomposition matrix obtained from the last decomposition, R_2 is the second decomposition matrix obtained from the last decomposition, and R_2 is the weight matrix.

[0141] 202. Perform y optimizations on the obtained target dataset to obtain y target transformation matrices. Each target transformation matrix is ​​used to characterize the statistical state of the target dataset in one aspect, where y ≥ 1.

[0142] The target dataset is optimized y times (y≥1) to obtain y target transformation matrices. One target transformation matrix is ​​used to characterize the statistical state of the target dataset in one aspect, and the y target transformation matrices are used to characterize the statistical state of the target dataset in each of the y aspects.

[0143] It should be noted that, in the embodiments of this application, the target dataset includes target data that can be collected online, and this application does not limit the method of collecting target data.

[0144] It should also be noted that in some embodiments of this application, the data in the target dataset also needs to be standardized, such as sampling frequency standardization and / or sampling duration normalization. This process is similar to the standardization operation of the historical dataset mentioned above. Please refer to the above description for details, which will not be repeated here.

[0145] It should also be noted that, in some embodiments of this application, the aforementioned y target transformation matrices include, but are not limited to, any one or more of the following:

[0146] a. The target steady-state transformation matrix (which can be represented by g′) is used to characterize the data form that maintains a relatively stable state in the above target dataset.

[0147] b. The target monotonic transformation matrix (which can be represented by p′) is used to characterize the data pattern in the above target dataset that maintains a continuous increasing trend or a continuous decreasing trend.

[0148] c. The target principal energy transformation matrix (which can be represented by E′) is used to characterize the data form representing the energy magnitude in the above target dataset.

[0149] In the embodiments of this application, the solution process for the target steady-state transformation matrix g′, the target monotonic transformation matrix p′, and the target principal energy transformation matrix E′ is similar to that for the aforementioned y historical transformation matrices. For details, please refer to the above description, which will not be repeated here.

[0150] It should be noted that, in the embodiments of this application, steps 201 and 202 are not executed in any particular order. Step 201 can be executed first and then step 202, or step 202 can be executed first and then step 201, or steps 201 and 202 can be executed simultaneously. This application does not limit the specific execution order.

[0151] 203. Based on the weight matrix and the y target transformation matrices, determine the target feature score vector of the target dataset. This target feature score vector is used to characterize the importance of the m features in the target dataset.

[0152] Then, based on the obtained weight matrix (i.e., R_2 mentioned above) and y target transformation matrices, the feature score vector of the target dataset is determined. This feature score vector can be called the target feature score vector, which is used to characterize the importance of the m features in the target dataset.

[0153] It should be noted that, in some embodiments of this application, one way to determine the target feature score vector based on the weight matrix and y target transformation matrices is as follows: First, determine the target representation vector of the target dataset based on the y target transformation matrices; then, determine the target feature score vector of the target dataset based on the weight matrix and the target representation vector.

[0154] As an example, the specific construction process of the target representation vector is explained below:

[0155] First, m features (e.g., mean, variance, kurtosis, harmonic ratio, etc.) are calculated for the target data in the target dataset, which can then be expanded into corresponding temporal feature matrices (i.e., the second temporal feature matrix). Then, based on the aforementioned y target transformation matrices and the second temporal feature matrix, y result matrices (i.e., the second result matrices) are obtained. For example, the aforementioned y target transformation matrices can be multiplied by the second temporal feature matrix to obtain y second result matrices, where one target transformation matrix corresponds to one second result matrix. Finally, the target representation vector V′ is obtained based on the y second result matrices.

[0156] For example, assuming the y target transformation matrices include a target steady-state transformation matrix g′, a target monotonic transformation matrix p′, and a target principal energy transformation matrix E′, then the corresponding second result matrix J1′ can be calculated based on the target steady-state transformation matrix g′ and the second time-series characteristic matrix; the corresponding second result matrix J2′ can be calculated based on the target monotonic transformation matrix p′ and the second time-series characteristic matrix; and the corresponding second result matrix J3′ can be calculated based on the target principal energy transformation matrix E′ and the second time-series characteristic matrix. Finally, the target representation vector V′ can be obtained based on the second result matrices J1′, J2′, and J3′ obtained above.

[0157] In some embodiments of this application, the first row of each of the y second result matrices may be extracted to obtain y target representation sub-vectors (which can be represented by v′), with one second result matrix corresponding to one target representation sub-vector. Finally, the y target representation sub-vectors are concatenated to obtain the target representation vector V′.

[0158] Finally, the target feature score vector (which can be represented by Rq) of the target dataset can be determined based on the weight matrix R_2 and the target representation vector V′. In one implementation, the weight matrix R_2 and the target representation vector V′ can be multiplied together to obtain the target feature score vector Rq of the target dataset, i.e., Rq = V′*R_2.

[0159] 204. Based on the target feature scoring vector, select k target features from m features, where k ≥ 1.

[0160] Finally, based on the determined target feature score vector Rq, k (k≥1) features are selected from m features as the final target features. For example, the features with the top k scores can be selected as the target features.

[0161] To further understand the process of determining k target features in steps 202 to 204 above, a specific example is given below. For details, please refer to [link / reference needed]. Figure 6 In this example, taking a target dataset consisting of 100 target data points collected online as an example, the process of determining k target features based on the aforementioned procedure is as follows:

[0162] Step 1: Data standardization.

[0163] First, the 100 target data points collected online were standardized, including downsampling to 5kHz and truncating to 3s (for illustration only). In this target dataset, there are no negative samples (i.e., no faulty data).

[0164] Step 2: Construct the target representation vector.

[0165] After calculating m features from the target data, an m*100 time series matrix is ​​formed. For example, assuming the m features are 32-dimensional (i.e., m=32), such as mean, variance, kurtosis, and harmonic ratio, the resulting time series matrix is ​​32*100. Based on this 32*100 time series matrix, the above... Figure 5 The calculation process is similar to step 2 in the process to obtain the target representation vector V′.

[0166] Step 3: Determine the target feature scoring vector.

[0167] The target feature score vector Rq is determined according to Rq = V′*R_2. This target feature score vector Rq includes the score value of each of the above m features in the target device (i.e., the device from which the target dataset is obtained), which can be represented by r1′, r2′, r3′, ..., i.e., Rq = [r1′, r2′, r3′, ...].

[0168] Step 4: Determine k target features.

[0169] Since the target feature score vector Rq includes the score value of each of the above m features in the target dataset, the top k scores (e.g., the top 8) can be selected as the feature set of the target device to complete the feature optimization work.

[0170] In summary, the working principle of the method in the embodiments of this application is as follows: Figure 7 As shown, it includes two stages: offline training (i.e., training the weight matrix) and online inference (i.e., calculating the target feature score vector), as follows:

[0171] (1) Offline training phase

[0172] First, multiple offline datasets (i.e. x historical datasets) are vectorized to obtain historical representation vectors of the historical datasets. Multiple historical representation vectors form a historical representation matrix.

[0173] Then, the recognition accuracy of each feature is calculated (based on the comparison between the feature recognition results and the real labels) to obtain the historical feature score matrix of the historical dataset.

[0174] Finally, by performing matrix decomposition on the historical feature scoring matrix and iterating using the historical representation matrix as the initial value, the weight matrix is ​​calculated. Then, through two steps of inference data representation and regression calculation, the feature score ranking is obtained, thus achieving feature selection. The regression calculation only requires normal samples to complete.

[0175] (2) Online reasoning stage

[0176] First, the online dataset (which may contain only normal data) is vectorized to obtain the target representation vector.

[0177] Then, by multiplying the target representation vector by the weight matrix, the target feature score vector of the online dataset is obtained, and the top k scores are selected as the feature selection results.

[0178] In this embodiment, feature optimization results can be obtained without fault samples, significantly improving the efficiency of online model deployment. Furthermore, this embodiment fully utilizes information from historical datasets, and the calculation process includes nonlinear optimization problems such as matrix factorization, fully considering the complex interactions between features. After feature optimization is implemented through this embodiment, the accuracy of fault identification can be significantly improved. Actual measurements show that the accuracy is improved by at least 15%+ (82% -> 99.73%) after feature optimization.

[0179] Based on the above embodiments, in order to better implement the above solutions of this application, related equipment for implementing the above solutions is also provided below. See details. Figure 8 , Figure 8This is a schematic diagram of a computer device provided in an embodiment of this application. The computer device 800 specifically includes: a first determining module 801, an optimization solving module 802, a second determining module 803, and a feature selection module 804. The first determining module 801 is used to determine a weight matrix based on x historical datasets. This weight matrix represents the importance percentage of m features in the x historical datasets. The historical datasets include t historical data points, each containing m features, where x≥1, m≥2, and t≥1. The optimization solving module 802 is used to perform y optimization solving operations on the acquired target dataset, obtaining y target transformation matrices. Each target transformation matrix represents the statistical state of the target dataset in one aspect, where y≥1. The second determining module 803 is used to determine a target feature score vector for the target dataset based on the weight matrix and the y target transformation matrices. This target feature score vector represents the importance of the m features in the target dataset. The feature selection module 804 is used to select k target features from the m features based on the target feature score vector.

[0180] In one possible design, the first determining module 801 is specifically used to: determine the historical representation matrix of the x historical datasets based on the x historical datasets; determine the historical feature scoring matrix of the x historical datasets based on the x historical datasets, the historical feature scoring matrix being used to characterize the importance of the m features in the x historical datasets; and obtain the weight matrix based on the historical representation matrix and the historical feature scoring matrix.

[0181] In one possible design, the first determining module 801 is further configured to: perform y optimizations on the first historical dataset to obtain y historical transformation matrices, each historical transformation matrix representing the statistical state of the first historical dataset in one aspect, wherein the first historical dataset is one of the x historical datasets; determine the historical representation vector of the first historical dataset based on the y historical transformation matrices; take each of the x historical datasets as the first historical dataset and repeat the above steps to obtain a total of x historical representation vectors; and concatenate the x historical representation vectors to obtain the historical representation matrix of the x historical datasets.

[0182] In one possible design, the first determining module 801 is further configured to: obtain y first result matrices based on the y historical transformation matrices and the first time-series feature matrix corresponding to the first historical dataset, wherein one historical transformation matrix corresponds to one first result matrix, and the first time-series feature matrix is ​​obtained by calculating m features from the historical data in the first historical dataset; and obtain the historical representation vector based on the y first result matrices.

[0183] In one possible design, the first determining module 801 is further configured to: extract the first row of the first result matrix y to obtain y historical representation sub-vectors, with one first result matrix corresponding to one historical representation sub-vector; and concatenate the y historical representation sub-vectors to obtain the historical representation vector.

[0184] In one possible design, the y historical transformation matrices include at least one of the following: a historical steady-state transformation matrix, a historical monotonic transformation matrix, and a historical principal energy transformation matrix; the historical steady-state transformation matrix is ​​used to characterize the data form that maintains a relatively stable state in the x historical datasets; the historical monotonic transformation matrix is ​​used to characterize the data form that maintains a continuous increasing trend or a continuous decreasing trend in the x historical datasets; and the historical principal energy transformation matrix is ​​used to characterize the data form that represents the energy magnitude in the x historical datasets.

[0185] In one possible design, the first determining module 801 is further configured to: calculate the feature scores of the first historical dataset to obtain a historical feature score vector, which is used to characterize the importance of the m features in the first historical dataset, where the first historical dataset is one of the x historical datasets; take each of the x historical datasets as the first historical dataset, and repeat the above steps to obtain a total of x historical feature score vectors; and concatenate the x historical feature score vectors to obtain the historical feature score matrix of the x historical datasets.

[0186] In one possible design, the first determining module 801 is further configured to: decompose the historical feature scoring matrix into a first decomposition matrix and a second decomposition matrix, and iteratively optimize the decomposition according to the constructed optimization algorithm and constraints until the iteration termination condition is reached; wherein, the second decomposition matrix obtained in the last decomposition is the weight matrix, the optimization algorithm is related to the first matrix error and the second matrix error, the first matrix error is the error between the historical feature scoring matrix obtained in the current optimization round and the historical feature scoring matrix obtained in the previous round, and the second matrix error is the error between the first decomposition matrix in the current optimization round and the historical representation matrix.

[0187] In one possible design, the constraints of the decomposition include: the first decomposition matrix approximates the history representation matrix.

[0188] In one possible design, the optimization algorithm includes the sum of the squares of the first matrix error and the squares of the second matrix error.

[0189] In one possible design, the conditions for reaching the termination of the iteration include: the value of the optimization algorithm is less than a preset threshold; or, the number of execution rounds of the iterative optimization reaches a preset number of rounds; or, the execution time of the iterative optimization reaches a preset time.

[0190] In one possible design, the second determining module 803 is specifically used to: determine the target representation vector of the target dataset based on the y target transformation matrices; and determine the target feature score vector of the target dataset based on the weight matrix and the target representation vector.

[0191] In one possible design, the second determining module 803 is further configured to: obtain y second result matrices based on the y target transformation matrices and the second temporal feature matrix corresponding to the target dataset, wherein one target transformation matrix corresponds to one second result matrix, and the second temporal feature matrix is ​​obtained by calculating m features of the target data in the target dataset; and obtain the target representation vector based on the y second result matrices.

[0192] In one possible design, the second determining module 803 is further configured to: extract the first row of the second result matrix y to obtain y target representation sub-vectors, with one second result matrix corresponding to one target representation sub-vector; and concatenate the y target representation sub-vectors to obtain the target representation vector.

[0193] In one possible design, the second determining module 803 is further configured to: perform matrix multiplication on the weight matrix and the target representation vector to obtain the target feature score vector of the target dataset.

[0194] In one possible design, the y target transformation matrices include at least one of the following: target steady-state transformation matrix, target monotonic transformation matrix, and target principal energy transformation matrix; the target steady-state transformation matrix is ​​used to characterize the data form in the target dataset that maintains a relatively stable state; the target monotonic transformation matrix is ​​used to characterize the data form in the target dataset that maintains a continuous increasing trend or a continuous decreasing trend; and the target principal energy transformation matrix is ​​used to characterize the data form in the target dataset that represents the energy magnitude.

[0195] In one possible design, the feature selection module 804 is specifically used to: select the k features with the highest scores from the m features as target features based on the target feature score vector.

[0196] It should be noted that the information interaction and execution process between the modules / units in the computer device 800 are based on the same concept as the method embodiments described above in this application. For details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0197] Next, we will introduce another computer device provided in the embodiments of this application. Please refer to [link / reference]. Figure 9 , Figure 9This is a schematic diagram of a computer device provided in an embodiment of this application. The computer device 900 may be equipped with... Figure 8 The computer device 800 described in the corresponding embodiment is used to implement Figure 8 Corresponding to the functionality of the computer device 800 in the corresponding embodiment, specifically, the computer device 900 is implemented by one or more servers. The computer device 900 can vary significantly due to differences in configuration or performance, and may include one or more Central Processing Units (CPUs) 922 and memory 932, and one or more storage media 930 (e.g., one or more mass storage devices) for storing application programs 942 or data 944. The memory 932 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the computer device 900. Furthermore, the CPU 922 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the computer device 900.

[0198] The computer device 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input / output interfaces 958, and / or one or more operating systems 941, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, etc.

[0199] In this embodiment, the central processing unit 922 is used to execute... Figure 2The steps performed by the computer device in the corresponding embodiment. For example, the central processing unit 922 can be used to: First, determine a weight matrix based on x (x≥1) historical datasets, where the historical datasets include t (t≥1) historical data entries. This weight matrix, also called a feature score weight matrix or weight regression matrix, is used to characterize the importance proportion of m (m≥2) features in the x historical datasets. Each element in the weight matrix can be called a feature score regression weight, or simply a regression weight, and is used to represent the importance proportion of the current feature in the current historical data entry. For example, the regression weight Qij represents the value of the importance proportion of the current feature i (m≥i≥1) in the j (x≥j≥1) historical data entry. Then, perform y (y≥1) optimization solutions on the obtained target dataset to obtain y target transformation matrices. One target transformation matrix is ​​used to characterize the statistical state of the target dataset in one aspect, and the y target transformation matrices are used to characterize the statistical state of the target dataset in each of the y aspects. Next, based on the obtained weight matrix and y target transformation matrices, the target feature score vector of the target dataset is determined. This target feature score vector is used to characterize the importance of the m features in the target dataset. Finally, based on the determined target feature score vector, k (k≥1) features are selected from the m features as the final target features. For example, the features with the top k score values ​​can be selected as the target features.

[0200] It should be noted that the specific manner in which the central processing unit 922 executes the above steps is different from that described in this application. Figure 2 The corresponding method embodiments are based on the same concept, and the technical effects they bring are the same as those in the above embodiments of this application. For details, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0201] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0202] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0203] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0204] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A data processing method, characterized in that, include: A weight matrix is ​​determined based on x historical datasets. The weight matrix is ​​used to characterize the importance ratio of m features in the x historical datasets. The historical datasets include t historical data points and m features in m dimensions, where x≥1, m≥2, and t≥1. The target dataset is optimized y times to obtain y target transformation matrices. Each target transformation matrix is ​​used to characterize the statistical state of the target dataset in one aspect, where y ≥ 1. Based on the weight matrix and the y target transformation matrices, a target feature score vector is determined for the target dataset. The target feature score vector is used to characterize the importance of the m features in the target dataset. Based on the target feature scoring vector, k target features are selected from the m features, where k ≥ 1.

2. The method according to claim 1, characterized in that, The process of determining the weight matrix based on x historical datasets includes: Determine the historical representation matrix of the x historical datasets based on the x historical datasets; Based on the x historical datasets, a historical feature scoring matrix is ​​determined for each of the x historical datasets. The historical feature scoring matrix is ​​used to characterize the importance of the m features in the x historical datasets. The weight matrix is ​​obtained based on the historical representation matrix and the historical feature scoring matrix.

3. The method according to claim 2, characterized in that, The step of determining the historical representation matrix of the x historical datasets based on the x historical datasets includes: The first historical dataset is optimized y times to obtain y historical transformation matrices. Each historical transformation matrix is ​​used to characterize the statistical state of the first historical dataset in one aspect. The first historical dataset is one of the x historical datasets. The historical representation vector of the first historical dataset is determined based on the y historical transformation matrices; Take each of the x historical datasets as the first historical dataset, repeat the above steps, and obtain a total of x historical representation vectors; The x historical representation vectors are concatenated to obtain the historical representation matrix of the x historical datasets.

4. The method according to claim 3, characterized in that, Determining the historical representation vector of the first historical dataset based on the y historical transformation matrices includes: Based on the y historical transformation matrices and the first time-series feature matrix corresponding to the first historical dataset, y first result matrices are obtained. One historical transformation matrix corresponds to one first result matrix. The first time-series feature matrix is ​​obtained by calculating m features from the historical data in the first historical dataset. The historical representation vector is obtained based on the y first result matrices.

5. The method according to claim 4, characterized in that, The step of obtaining the historical representation vector based on the y first result matrices includes: Extract the first row of the first result matrix y to obtain y historical representation sub-vectors, with one first result matrix corresponding to one historical representation vector; The y historical representation sub-vectors are concatenated to obtain the historical representation vector.

6. The method according to any one of claims 3-5, characterized in that, The y historical transformation matrices include at least one of the following: Historical steady-state transformation matrix, historical monotonic transformation matrix, historical principal energy transformation matrix; The historical steady-state transformation matrix is ​​used to characterize the data form that maintains a relatively stable state in the x historical datasets; The historical monotonic transformation matrix is ​​used to characterize the data patterns in the x historical datasets that maintain a continuous increasing trend or a continuous decreasing trend. The historical master energy transformation matrix is ​​used to characterize the data form representing the energy magnitude in the x historical datasets.

7. The method according to any one of claims 2-6, characterized in that, The step of determining the historical feature scoring matrix of the x historical datasets based on the x historical datasets includes: Calculate the feature scores of the first historical dataset to obtain a historical feature score vector. The historical feature score vector is used to characterize the importance of the m features in the first historical dataset. The first historical dataset is one of the x historical datasets. Take each of the x historical datasets as the first historical dataset, repeat the above steps, and obtain a total of x historical feature score vectors; The x historical feature score vectors are concatenated to obtain the historical feature score matrix of the x historical datasets.

8. The method according to any one of claims 2-7, characterized in that, The step of obtaining the weight matrix based on the historical representation matrix and the historical feature scoring matrix includes: The historical feature scoring matrix is ​​decomposed into a first decomposition matrix and a second decomposition matrix, and the decomposition is iteratively optimized according to the constructed optimization algorithm and constraints until the iteration termination condition is reached. The second decomposition matrix obtained from the last decomposition is the weight matrix. The optimization algorithm is related to the first matrix error and the second matrix error. The first matrix error is the error between the historical feature scoring matrix obtained in the current optimization round and the historical feature scoring matrix obtained in the previous round. The second matrix error is the error between the first decomposition matrix in the current optimization round and the historical representation matrix.

9. The method according to claim 8, characterized in that, The constraints of the decomposition include: The first decomposition matrix approximates the historical representation matrix.

10. The method according to any one of claims 8-9, characterized in that, The optimization algorithm includes: The sum of the squares of the errors of the first matrix and the second matrix.

11. The method according to any one of claims 8-10, characterized in that, The conditions for reaching the iteration termination include: The value of the optimization algorithm is less than a preset threshold; or, The iteration optimization has been performed for a preset number of rounds. or, The execution time of the iterative optimization reaches the preset time.

12. The method according to any one of claims 1-11, characterized in that, Based on the weight matrix and the y target transformation matrices, the target feature scoring vector of the target dataset is determined as follows: The target representation vector of the target dataset is determined based on the y target transformation matrices; The target feature score vector of the target dataset is determined based on the weight matrix and the target representation vector.

13. The method according to claim 12, characterized in that, The step of determining the target representation vector of the target dataset based on the y target transformation matrices includes: Based on the y target transformation matrices and the second time-series feature matrix corresponding to the target dataset, y second result matrices are obtained, with one target transformation matrix corresponding to one second result matrix. The second time-series feature matrix is ​​obtained by calculating m features from the target data in the target dataset. The target representation vector is obtained based on the y second result matrices.

14. The method according to claim 13, characterized in that, The step of obtaining the target representation vector based on the y second result matrices includes: Extracting from the first row of the second result matrix y yields y target representation vectors, with one second result matrix corresponding to one target representation vector; The target representation vector is obtained by concatenating the y target representation sub-vectors.

15. The method according to any one of claims 12-14, characterized in that, Determining the target feature score vector of the target dataset based on the weight matrix and the target representation vector includes: The target feature score vector of the target dataset is obtained by performing matrix multiplication on the weight matrix and the target representation vector.

16. The method according to any one of claims 1-15, characterized in that, The y target transformation matrices include at least one of the following: Target steady-state transformation matrix, target monotonic transformation matrix, target principal energy transformation matrix; The target steady-state transformation matrix is ​​used to characterize the data form that maintains a relatively stable state in the target dataset; The target monotonic transformation matrix is ​​used to characterize the data pattern in the target dataset that maintains a continuous increasing trend or a continuous decreasing trend. The target principal energy transformation matrix is ​​used to characterize the data form representing the energy magnitude in the target dataset.

17. The method according to any one of claims 1-16, characterized in that, The step of selecting k target features from the m features based on the target feature scoring vector includes: Based on the target feature scoring vector, select the k features with the highest scores from the m features as target features, where k ≥ 1.

18. A computer device, characterized in that, include: The first determining module is used to determine a weight matrix based on x historical datasets. The weight matrix is ​​used to characterize the importance ratio of m features in the x historical datasets. The historical datasets include t historical data points and m features in m dimensions, where x≥1, m≥2, and t≥1. The optimization solution module is used to perform y optimization solutions on the acquired target dataset to obtain y target transformation matrices. Each target transformation matrix is ​​used to characterize the statistical state of the target dataset in one aspect, where y≥1. The second determining module is used to determine the target feature score vector of the target dataset based on the weight matrix and the y target transformation matrices. The target feature score vector is used to characterize the importance of the m features in the target dataset. The feature selection module is used to select k target features from the m features based on the target feature scoring vector.

19. The device according to claim 18, characterized in that, The first determining module is specifically used for: Determine the historical representation matrix of the x historical datasets based on the x historical datasets; Based on the x historical datasets, a historical feature scoring matrix is ​​determined for each of the x historical datasets. The historical feature scoring matrix is ​​used to characterize the importance of the m features in the x historical datasets. The weight matrix is ​​obtained based on the historical representation matrix and the historical feature scoring matrix.

20. The device according to claim 19, characterized in that, The first determining module is further configured to: The first historical dataset is optimized y times to obtain y historical transformation matrices. Each historical transformation matrix is ​​used to characterize the statistical state of the first historical dataset in one aspect. The first historical dataset is one of the x historical datasets. The historical representation vector of the first historical dataset is determined based on the y historical transformation matrices; Take each of the x historical datasets as the first historical dataset, repeat the above steps, and obtain a total of x historical representation vectors; The x historical representation vectors are concatenated to obtain the historical representation matrix of the x historical datasets.

21. The device according to claim 20, characterized in that, The first determining module is further configured to: Based on the y historical transformation matrices and the first time-series feature matrix corresponding to the first historical dataset, y first result matrices are obtained. One historical transformation matrix corresponds to one first result matrix. The first time-series feature matrix is ​​obtained by calculating m features from the historical data in the first historical dataset. The historical representation vector is obtained based on the y first result matrices.

22. The device according to claim 21, characterized in that, The first determining module is further configured to: Extract the first row of the first result matrix y to obtain y historical representation sub-vectors, with one first result matrix corresponding to one historical representation vector; The y historical representation sub-vectors are concatenated to obtain the historical representation vector.

23. The device according to any one of claims 20-22, characterized in that, The y historical transformation matrices include at least one of the following: Historical steady-state transformation matrix, historical monotonic transformation matrix, historical principal energy transformation matrix; The historical steady-state transformation matrix is ​​used to characterize the data form that maintains a relatively stable state in the x historical datasets; The historical monotonic transformation matrix is ​​used to characterize the data patterns in the x historical datasets that maintain a continuous increasing trend or a continuous decreasing trend. The historical master energy transformation matrix is ​​used to characterize the data form representing the energy magnitude in the x historical datasets.

24. The device according to any one of claims 19-23, characterized in that, The first determining module is further configured to: Calculate the feature scores of the first historical dataset to obtain a historical feature score vector. The historical feature score vector is used to characterize the importance of the m features in the first historical dataset. The first historical dataset is one of the x historical datasets. Take each of the x historical datasets as the first historical dataset, repeat the above steps, and obtain a total of x historical feature score vectors; The x historical feature score vectors are concatenated to obtain the historical feature score matrix of the x historical datasets.

25. The device according to any one of claims 19-24, characterized in that, The first determining module is further configured to: The historical feature scoring matrix is ​​decomposed into a first decomposition matrix and a second decomposition matrix, and the decomposition is iteratively optimized according to the constructed optimization algorithm and constraints until the iteration termination condition is reached. The second decomposition matrix obtained from the last decomposition is the weight matrix. The optimization algorithm is related to the first matrix error and the second matrix error. The first matrix error is the error between the historical feature scoring matrix obtained in the current optimization round and the historical feature scoring matrix obtained in the previous round. The second matrix error is the error between the first decomposition matrix in the current optimization round and the historical representation matrix.

26. The device according to claim 25, characterized in that, The constraints of the decomposition include: The first decomposition matrix approximates the historical representation matrix.

27. The device according to any one of claims 25-26, characterized in that, The optimization algorithm includes: The sum of the squares of the errors of the first matrix and the second matrix.

28. The device according to any one of claims 25-27, characterized in that, The conditions for reaching the iteration termination include: The value of the optimization algorithm is less than a preset threshold; or, The iteration optimization has been performed for a preset number of rounds. or, The execution time of the iterative optimization reaches the preset time.

29. The device according to any one of claims 18-28, characterized in that, According to the second determining module, it is specifically used for: The target representation vector of the target dataset is determined based on the y target transformation matrices; The target feature score vector of the target dataset is determined based on the weight matrix and the target representation vector.

30. The device according to claim 29, characterized in that, The second determining module is further used for: Based on the y target transformation matrices and the second time-series feature matrix corresponding to the target dataset, y second result matrices are obtained, with one target transformation matrix corresponding to one second result matrix. The second time-series feature matrix is ​​obtained by calculating m features from the target data in the target dataset. The target representation vector is obtained based on the y second result matrices.

31. The device according to claim 30, characterized in that, The second determining module is further used for: Extracting from the first row of the second result matrix y yields y target representation vectors, with one second result matrix corresponding to one target representation vector; The target representation vector is obtained by concatenating the y target representation sub-vectors.

32. The device according to any one of claims 29-31, characterized in that, The second determining module is further used for: The target feature score vector of the target dataset is obtained by performing matrix multiplication on the weight matrix and the target representation vector.

33. The device according to any one of claims 18-32, characterized in that, The y target transformation matrices include at least one of the following: Target steady-state transformation matrix, target monotonic transformation matrix, target principal energy transformation matrix; The target steady-state transformation matrix is ​​used to characterize the data form that maintains a relatively stable state in the target dataset; The target monotonic transformation matrix is ​​used to characterize the data pattern in the target dataset that maintains a continuous increasing trend or a continuous decreasing trend. The target principal energy transformation matrix is ​​used to characterize the data form representing the energy magnitude in the target dataset.

34. The device according to any one of claims 18-33, characterized in that, The feature selection module is specifically used for: Based on the target feature scoring vector, select the k features with the highest scores from the m features as target features.

35. A computer device comprising a processor and a memory, the processor being coupled to the memory, characterized in that, The memory is used to store programs; The processor is configured to execute a program in the memory, causing the computer device to perform the method as described in any one of claims 1-17.

36. A computer storage medium, characterized in that, The device stores computer-readable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-17.

37. A computer program product, characterized in that, The computer program product includes computer-readable instructions that, when executed by a processor, implement the method as described in any one of claims 1-17.

38. A chip, the chip comprising a processor and a data interface, characterized in that, The processor reads instructions stored in the memory through the data interface and executes the method as described in any one of claims 1-17.

Images