A forging equipment detection method and system based on multi-source data fusion

Abstract

The present invention provides a forging equipment detection method and system based on multi-source data fusion. Specifically, each time series signal is decomposed into a group of intrinsic mode functions, and the instantaneous amplitude sequence and instantaneous frequency sequence of each intrinsic mode function are calculated using Hilbert transform; the instantaneous amplitude sequence and instantaneous frequency sequence of all intrinsic mode functions obtained by decomposing each sensor source signal are combined to obtain a time-frequency spectrum feature map corresponding to the sensor; multiple parallel cyclic unit branches including phase gates are used to extract features of different time steps from each time-frequency spectrum feature map, a time step attention layer is connected after the cyclic unit of each parallel branch, and a sensor association map is constructed based on the time series features; a graph convolutional network with residual connections is used to aggregate the features of adjacent nodes in the graph to update the nodes to obtain fused features; and the detection results of the forging equipment are obtained based on the fused features.

Description

Technical Field

[0001] The present application relates to the field of equipment detection, and in particular to a forging equipment detection method and system based on multi-source data fusion. Background Art

[0002] As core equipment in modern heavy industry, forging equipment plays an indispensable role in key sectors such as automotive, aerospace, and energy. Its operating environment is often extremely harsh, subjecting it to immense shock loads, high temperatures, and intense vibration. This makes critical components such as crankshafts, bearings, and gears highly susceptible to fatigue damage, wear, and even fracture. Once a failure occurs, it not only leads to expensive repair costs and prolonged downtime, but can also cause serious production accidents. Traditional equipment inspection methods rely primarily on regular manual inspections and planned maintenance. The former is highly subjective, inefficient, and lacks real-time early warning, while the latter can lead to insufficient or excessive maintenance, resulting in low economic returns. With the advancement of sensor and signal processing technologies, data-based condition monitoring methods have become mainstream. Early research focused on analyzing a single signal source, such as using frequency domain analysis of vibration signals to diagnose bearing faults. However, the operation of forging equipment is an extremely complex multi-physics system coupled with mechanical, electrical, and hydraulic fields. A single signal source often only partially reflects the local state of the equipment, with limited information carrying capacity. In the presence of strong noise and fluctuating operating conditions, misjudgments and missed detections are highly likely. To overcome the limitations of a single information source, detection methods based on multi-source data fusion have emerged. By comprehensively analyzing data collected from sensors of different types and locations, such as vibration, acoustics, current, and temperature, a more comprehensive picture of the equipment's operating status can be generated. Currently, commonly used methods for feature extraction include Fourier transforms and wavelet transforms. However, forging signals exhibit typical non-stationary and nonlinear characteristics, making Fourier transforms difficult to process, while the wavelet transform's basis function selection lacks adaptability. The forging process has distinct stages. When traditional RNN models process such stage-by-stage signals, the hidden state of the previous stage interferes with feature extraction in subsequent stages, obscuring fault signatures closely associated with a specific stage. Furthermore, when fusing multi-source features, simple feature concatenation or weighted fusion based on attention mechanisms fail to explicitly model the inherent correlations between sensors. Improving the accuracy of forging equipment detection using multi-source data remains an urgent challenge. Summary of the Invention

[0003] In order to solve the above problems, this application proposes a forging equipment detection method based on multi-source data fusion, including:

[0004] Acquire multi-source time series signals collected by multiple sensors installed on the forging equipment, decompose each of the time series signals into a set of intrinsic mode functions using variational mode decomposition, and calculate the instantaneous amplitude sequence and instantaneous frequency sequence of each intrinsic mode function using Hilbert transform; combine the instantaneous amplitude sequence and instantaneous frequency sequence of all the intrinsic mode functions obtained by decomposing each sensor source signal to obtain a time-frequency spectrum characteristic diagram corresponding to the sensor;

[0005] Multiple parallel recurrent unit branches including phase gates are used to extract features at different time steps from each time-frequency spectrum feature map, wherein the phase gates are used to identify phase switching points during the forging process; a time step attention layer is connected after the recurrent unit in each parallel branch, and the attention layer obtains time series features by weighting the features at different time steps;

[0006] A sensor association graph is constructed based on the temporal features, wherein the nodes of the sensor association graph are the temporal features extracted by the branches of each attention-gated recurrent unit, and the edge weights of the sensor association graph are weighted according to the physical correlation and data correlation between the corresponding sensors; a graph convolutional network with residual connections is used to aggregate the features of adjacent nodes in the sensor association graph to update the nodes to obtain fused features; and a detection result of the forging equipment is obtained based on the fused features.

[0007] Preferably, the extracting features of different time steps from each time-frequency spectrum feature graph using multiple parallel cycle unit branches including phase gates comprises:

[0008] For each parallel branch, at the tth time step, the current input multi-channel time-frequency spectrum feature map vector and the recurrent unit hidden state at the previous time step Splicing is performed, the spliced ​​vector is input into the fully connected layer, and the phase gate output value is obtained by the activation function calculation ; The value range of is [0,1];

[0009] A gated recurrent unit is used as the recurrent unit to calculate the candidate hidden state at the t-th time step When , the reset gate used to adjust the hidden state of the previous moment in GRU is AND phase gate output value Perform element-by-element multiplication to obtain the corrected reset gate ; Using the Calculate the candidate hidden state , and update the hidden state of the current time step .

[0010] Preferably, the use of Calculate the candidate hidden state , and update the hidden state of the current time step ,include:

[0011] The hidden state of the previous time step With the revised reset gate Perform element-by-element multiplication to obtain the reset history information;

[0012] Multiply the reset historical information with the first trainable weight matrix to convert the input of the current time step Multiply it with the second trainable weight matrix, add the two multiplication results and the trainable bias vector to obtain an intermediate result, apply the hyperbolic tangent activation function to the intermediate result, and obtain the candidate hidden state ;

[0013] Calculate the update gate for the current time step , the hidden state of the previous time step and Multiply element by element to get the historical state part to be retained; and update gate Perform element-by-element multiplication to obtain the new information part to be updated; add the historical state part to be retained and the new information part to be updated element-by-element to obtain the hidden state of the current time step .

[0014] Preferably, the attention layer obtains temporal features by weighting features at different time steps, including:

[0015] The hidden state sequence Input to the time-step attention layer, which calculates the hidden state of each time step through a fully connected layer with a tanh activation function Attention scores, all scores are normalized to attention weights , the hidden state sequence According to the corresponding attention weight Perform weighted summation to obtain the parallel branch timing characteristics.

[0016] Preferably, the edge weights of the graph are obtained by weighting according to the physical association and data correlation between corresponding sensors, including:

[0017] Get the physical correlation coefficient matrix If sensor i and sensor j are installed on the same mechanical component or two components with direct force transmission relationship, then The value of is 1; if sensor i and sensor j are installed on independent components, then The value of is 0.1;

[0018] Extract the temporal features of all samples in the training data set output by each parallel branch attention layer, and calculate the Pearson correlation coefficient between the temporal feature sequence sets corresponding to the two sensors i and j And take the absolute value to get the correlation coefficient matrix ,in ;

[0019] The edge weight A(i,j) between node i and node j is calculated by the following formula:

[0020]

[0021] in is a hyperparameter with a value range between (0,1).

[0022] Preferably, the method of using a graph convolutional network with residual connections to aggregate features of adjacent nodes in a graph to update nodes to obtain fused features includes:

[0023] For the lth layer of the graph convolutional network, the node features are updated according to the following formula:

[0024]

[0025] in, is the node feature matrix of the l-1 layer, The adjacency matrix with self-loops is obtained by adding an identity matrix to the adjacency matrix A. for The diagonal matrix of is the trainable weight matrix of the l-th layer network, and ReLU is the activation function;

[0026] Node features output by the l-th layer network for:

[0027]

[0028] and is the input of the l+1th layer ;

[0029] After L layers of graph convolution with residual connections, the node feature matrix is ​​obtained. ;right The feature vectors of all nodes in the ensemble are averaged and pooled element by element to obtain the fused features.

[0030] This application also proposes a forging equipment detection system based on multi-source data fusion, including:

[0031] An acquisition and transformation unit is used to obtain multi-source time series signals collected by multiple sensors installed on the forging equipment, decompose each of the time series signals into a set of intrinsic mode functions using variational mode decomposition, and calculate the instantaneous amplitude sequence and instantaneous frequency sequence of each intrinsic mode function using Hilbert transform; and combine the instantaneous amplitude sequence and instantaneous frequency sequence of all the intrinsic mode functions obtained by decomposing each sensor source signal to obtain a time-frequency spectrum characteristic diagram corresponding to the sensor;

[0032] a feature extraction unit configured to extract features at different time steps from each time-frequency spectrum feature graph using multiple parallel recurrent unit branches including phase gates, wherein the phase gates are used to identify phase switching points during the forging process; a time step attention layer is connected after the recurrent unit in each parallel branch, wherein the attention layer obtains time series features by weighting the features at different time steps;

[0033] A fusion and detection unit is used to construct a sensor association graph based on the temporal features, wherein the nodes of the sensor association graph are the temporal features extracted by the branches of each attention-gated recurrent unit, and the edge weights of the sensor association graph are weighted according to the physical correlation and data correlation between the corresponding sensors; a graph convolutional network with residual connections is used to aggregate the features of adjacent nodes in the sensor association graph to update the nodes to obtain fusion features; and a detection result of the forging equipment is obtained based on the fusion features.

[0034] Preferably, the extracting features of different time steps from each time-frequency spectrum feature graph using multiple parallel cycle unit branches including phase gates comprises:

[0035] For each parallel branch, at the tth time step, the current input multi-channel time-frequency spectrum feature map vector and the recurrent unit hidden state at the previous time step Splicing is performed, the spliced ​​vector is input into the fully connected layer, and the phase gate output value is obtained by the activation function calculation ; The value range of is [0,1];

[0036] A gated recurrent unit is used as the recurrent unit to calculate the candidate hidden state at the t-th time step When , the reset gate used to adjust the hidden state of the previous moment in GRU is AND phase gate output value Perform element-by-element multiplication to obtain the corrected reset gate ; Using the Calculate the candidate hidden state , and update the hidden state of the current time step .

[0037] Preferably, the use of Calculate the candidate hidden state , and update the hidden state of the current time step ,include:

[0038] The hidden state of the previous time step With the revised reset gate Perform element-by-element multiplication to obtain the reset history information;

[0039] Multiply the reset historical information with the first trainable weight matrix to convert the input of the current time step Multiply it with the second trainable weight matrix, add the two multiplication results and the trainable bias vector to obtain an intermediate result, apply the hyperbolic tangent activation function to the intermediate result, and obtain the candidate hidden state ;

[0040] Calculate the update gate for the current time step , the hidden state of the previous time step and Multiply element by element to get the historical state part to be retained; and update gate Perform element-by-element multiplication to obtain the new information part to be updated; add the historical state part to be retained and the new information part to be updated element-by-element to obtain the hidden state of the current time step .

[0041] Preferably, the attention layer obtains temporal features by weighting features at different time steps, including:

[0042] The hidden state sequence Input to the time-step attention layer, which calculates the hidden state of each time step through a fully connected layer with a tanh activation function Attention scores, all scores are normalized to attention weights , the hidden state sequence According to the corresponding attention weight Perform weighted summation to obtain the parallel branch timing characteristics.

[0043] Preferably, the edge weights of the graph are obtained by weighting according to the physical association and data correlation between corresponding sensors, including:

[0044] Get the physical correlation coefficient matrix If sensor i and sensor j are installed on the same mechanical component or two components with direct force transmission relationship, then The value of is 1; if sensor i and sensor j are installed on independent components, then The value of is 0.1;

[0045] Extract the temporal features of all samples in the training data set output by each parallel branch attention layer, and calculate the Pearson correlation coefficient between the temporal feature sequence sets corresponding to the two sensors i and j And take the absolute value to get the correlation coefficient matrix ,in ;

[0046] The edge weight A(i,j) between node i and node j is calculated by the following formula:

[0047]

[0048] in is a hyperparameter with a value range between (0,1).

[0049] Preferably, the method of using a graph convolutional network with residual connections to aggregate features of adjacent nodes in a graph to update nodes to obtain fused features includes:

[0050] For the lth layer of the graph convolutional network, the node features are updated according to the following formula:

[0051]

[0052] in, is the node feature matrix of the l-1 layer, The adjacency matrix with self-loops is obtained by adding an identity matrix to the adjacency matrix A. for The diagonal matrix of is the trainable weight matrix of the l-th layer network, and ReLU is the activation function;

[0053] Node features output by the l-th layer network for:

[0054]

[0055] and is the input of the l+1th layer ;

[0056] After L layers of graph convolution with residual connections, the node feature matrix is ​​obtained. ;right The feature vectors of all nodes in the ensemble are averaged and pooled element by element to obtain the fused features.

[0057] In this application, a multi-channel time-frequency feature graph is composed of the instantaneous amplitude reflecting the signal energy transformation and the instantaneous frequency reflecting the fluctuation of the signal frequency over time, which can monitor the weak characteristics of the equipment during the evolution from normal to early failure; and by introducing a phase gate in the recurrent neural network, it is possible to identify the switching points of the working stages in the forging process, eliminating the interference of irrelevant information in the previous stage; the graph structure weightedly fuses the physical correlation of the mechanical structure with the data correlation reflecting the real-time working conditions, enhancing the expression ability of the fused features, thereby improving the accuracy and reliability of the detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of Example 1;

[0059] Figure 2 Schematic diagram of the time-frequency spectrum characteristic diagram corresponding to the sensor;

[0060] Figure 3 This is a structural diagram of the sensor association graph. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0062] Specific embodiments, such as Figure 1 As shown, this application proposes a forging equipment detection method based on multi-source data fusion, including:

[0063] S1, obtaining multi-source time series signals collected by multiple sensors installed on the forging equipment, decomposing each of the time series signals into a set of intrinsic mode functions using variational mode decomposition, and calculating the instantaneous amplitude sequence and instantaneous frequency sequence of each intrinsic mode function using Hilbert transform; combining the instantaneous amplitude sequence and instantaneous frequency sequence of all the intrinsic mode functions obtained by decomposing each sensor source signal to obtain a time-frequency spectrum characteristic diagram corresponding to the sensor;

[0064] For example, three signals are collected simultaneously from sensors installed at different locations on the forging equipment: a vibration sensor located on the main bearing seat, an acoustic sensor near the die area, and a current sensor on the drive motor. For example, the vibration signal is [0.1, 0.3, -0.2, ...]. Variational mode decomposition (VMD) is applied to the collected vibration signal. This decomposition decomposes the original complex vibration signal into multiple sub-signals, known as intrinsic mode functions (IMFs). Assume that VMD decomposes the original vibration signal into three IMF components: IMF1 represents the high-frequency vibration component, likely related to the impact of the bearing rolling elements passing over a defect; IMF2 represents the mid-frequency vibration component, likely related to the gear meshing frequency; and IMF3 represents the low-frequency vibration component, likely related to the spindle's rotational imbalance. Each signal—vibration, acoustic, and current—is decomposed to produce its own set of IMF components.

[0065] For each IMF component, a Hilbert transform is used to calculate the time-varying instantaneous information. For example, for IMF1 of the vibration signal, an instantaneous amplitude sequence and an instantaneous frequency sequence are calculated. Each point in the instantaneous amplitude sequence represents the energy of IMF1 at that moment. For example, when the device is impacted, the value of this sequence will increase instantaneously. Each point in the instantaneous frequency sequence represents the dominant vibration frequency of IMF1 at that moment. For example, a change in rotational speed will cause the value of this sequence to change. The above operation is performed on IMF1, IMF2, and IMF3 decomposed from the vibration signal, resulting in three instantaneous amplitude sequences and three instantaneous frequency sequences, for a total of six time series.

[0066] The instantaneous frequency sequence and instantaneous amplitude sequence of the IMF of all sensors are combined to obtain a multi-channel time-frequency spectrum feature map. In one embodiment, the instantaneous amplitude sequence of all IMFs of a sensor are spliced ​​row by row to obtain the instantaneous amplitude matrix of the sensor, and the instantaneous frequency sequence of all IMFs of the sensor are spliced ​​row by row to obtain the instantaneous frequency matrix of the sensor. The instantaneous amplitude matrix and instantaneous frequency matrix of each sensor are used as two channels of the time-frequency spectrum feature map corresponding to the sensor, such as Figure 2 shown.

[0067] S2, using multiple parallel recurrent unit branches including phase gates to extract features at different time steps from each time-frequency spectrum feature graph, wherein the phase gates are used to identify phase switching points during the forging process; a time step attention layer is connected after the recurrent unit in each parallel branch, and the attention layer obtains time series features by weighting the features at different time steps;

[0068] A separate processing branch is established for each sensor's time-frequency spectrum obtained in step S1. If the three sensors are vibration, acoustic, and current sensors, respectively, three processing branches are activated: Branch 1 processes the vibration sensor's multi-channel time-frequency spectrum, Branch 2 processes the acoustic sensor's multi-channel time-frequency spectrum, and Branch 3 processes the current sensor's multi-channel time-frequency spectrum. Assume the vibration time-frequency spectrum consists of 100 time points. At each time point, for example, time point t, the recurrent unit analyzes the characteristic data at time point t and, combined with information from time point t-1, outputs a phase gate value. This value, between 0 and 1, is used to determine whether a critical phase transition has occurred at that moment. For example, during the forging process, the signal characteristics change dramatically when the equipment switches from the mold closing phase to the pressing phase. The phase gate recognizes this change and outputs a value close to 1. When the recurrent unit receives a signal close to 1, it significantly clears or resets its memory of the mold closing phase, allowing it to more focusedly learn the characteristics of the subsequent pressing phase. On the contrary, if the device is still in a stable pressure-holding stage and the signal characteristics do not change much, the phase gate will output a value close to 0, and the cyclic unit will retain and continue to accumulate previous memories to analyze subtle changes within this stage.

[0069] After the recurrent unit completes preliminary processing of 100 time points, it generates 100 preliminary feature vectors, one for each time point. However, not all time points are equally important. The time-step attention layer initially examines these 100 feature vectors and calculates an importance score, or attention weight, for each. For example, for a fault detection task, the moment of metal-die contact at the start of forging and the moment of peak pressure contain more important information than information during periods of idle or constant motion. Therefore, the attention layer assigns higher weights, such as 0.8 or 0.9, to feature vectors corresponding to the moment of impact and peak pressure, while assigning lower weights, such as 0.01, to feature vectors during steady-state operation. The attention layer multiplies the 100 preliminary feature vectors by their corresponding importance weights and then sums the results to produce a fixed-length feature vector. This feature vector not only captures the dynamic changes in the signal but also highlights key time points. Branches two and three use the same approach, each generating a corresponding time series feature vector.

[0070] S3, constructing a sensor association graph based on the temporal features, wherein the nodes of the sensor association graph are the temporal features extracted by the branches of the attention gated recurrent units, and the edge weights of the sensor association graph are weighted according to the physical correlation and data correlation between the corresponding sensors; adopting a graph convolutional network with residual connections to aggregate the features of adjacent nodes in the sensor association graph to update the nodes to obtain fused features; obtaining the detection results of the forging equipment based on the fused features.

[0071] To construct a graph structure, using the example above, we'll use three nodes: node V representing vibration information, node A representing acoustic information, and node C representing current information. Based on the structural diagram of the forging equipment, a physical correlation matrix is ​​predefined. For example, a vibration sensor is located on the main bearing, and a current sensor monitors the drive motor. These are connected through the main shaft and drive system, resulting in a strong correlation, with a physical correlation coefficient of 0.9. The acoustic sensor is located near the die, some distance from the main bearing, but still part of the same equipment. This correlation is moderate, with a physical correlation coefficient of 0.5. Furthermore, through analysis of extensive historical data, we've found that whenever a specific vibration pattern occurs, the current pattern also exhibits another specific pattern, indicating a high correlation at the data level. By calculating the Pearson correlation coefficient between their eigenvectors, we obtain a value of, for example, 0.8. The physical and data correlations are weighted together to determine the final connection strength. Assuming the physical correlation weight is 0.4 and the data correlation weight is 0.6, the connection strength between nodes V and C is 0.84. After constructing the sensor association graph, graph convolution is performed. To prevent information loss during graph convolution, residual connections are introduced. After at least one round of graph convolution and residual connections, fused features are obtained. In one embodiment, the feature vectors of the three nodes are averaged to form a global feature vector representing the current overall state of the device. This global feature vector is then input into a pre-trained classifier, which outputs the final detection result, such as whether the device is normal or a bearing has cracks.

[0072] In an optional embodiment, the extracting features of different time steps from each time-frequency feature graph using multiple parallel cycle unit branches including phase gates includes:

[0073] For each parallel branch, at the tth time step, the current input multi-channel time-frequency spectrum feature map vector and the recurrent unit hidden state at the previous time step Splicing is performed, the spliced ​​vector is input into the fully connected layer, and the phase gate output value is obtained by the activation function calculation ; The value range of is [0,1];

[0074] A gated recurrent unit is used as the recurrent unit to calculate the candidate hidden state at the t-th time step When , the reset gate used to adjust the hidden state of the previous moment in GRU is AND phase gate output value Perform element-by-element multiplication to obtain the corrected reset gate ; Using the Calculate the candidate hidden state , and update the hidden state of the current time step .

[0075] Specifically, suppose the forging equipment has just completed a violent downward pressing action and entered a relatively stable pressure holding stage. At the moment when the downward pressing ends, that is, time point t, the cyclic unit receives a new time-frequency feature map vector representing the pressure holding stage , its value is [0.2, 0.1, 0.3]. And the memory information of the previous time step stored inside it , still describing the characteristics of the intense downward pressure stage, for example, [1.5, 2.8, 1.9]. The time step is the smallest processing unit into which the sequence signal is divided. These two sets of data are spliced ​​together and sent to the fully connected layer. The fully connected layer is trained to recognize the pattern mutation from high energy to low energy and calculate a phase gate output value close to 1. , for example 0.98, marks the phase switch.

[0076] Furthermore, the phase gate signal is used to update the memory, and the phase gate output value of 0.98 , used to intervene in the reset gate inside the gated recurrent unit GRU . High value It will amplify the effect of the reset gate, making the corrected reset gate It is also close to 1, which makes the cycle unit forget the information of the downward phase. . When calculating new candidate memory states When , it depends on the new input representing the packing phase . Update the current memory , clearing the new state of the old phase interference. On the contrary, if the device is still in the pressure-down phase, the new input is similar to the old memory, and the phase gate A value close to 0 will be output, allowing the cyclic unit to inherit and accumulate historical information, and then continue to track the dynamics of the downward pressure process.

[0077] In an optional embodiment, the use of Calculate the candidate hidden state , and update the hidden state of the current time step ,include:

[0078] The hidden state of the previous time step With the revised reset gate Perform element-by-element multiplication to obtain the reset history information;

[0079] Multiply the reset historical information with the first trainable weight matrix to convert the input of the current time step Multiply it with the second trainable weight matrix, add the two multiplication results and the trainable bias vector to obtain an intermediate result, apply the hyperbolic tangent activation function to the intermediate result, and obtain the candidate hidden state ;

[0080] Calculate the update gate for the current time step , the hidden state of the previous time step and Multiply element by element to get the historical state part to be retained; and update gate Perform element-by-element multiplication to obtain the new information part to be updated; add the historical state part to be retained and the new information part to be updated element-by-element to obtain the hidden state of the current time step .

[0081] Specifically, a calculated correction reset gate, for example, 0.1, is used to weaken the memory of the previous moment. If the old memory was [1.5, 2.8], it becomes [0.15, 0.28] after weakening. This cleared historical information and the new input at the current moment, for example [0.6, 0.7], are each transformed through separate neural network layers, added with a bias term, and finally processed through a hyperbolic tangent activation function to obtain a candidate hidden state representing the current moment, for example [0.8, 0.9]. The update gate, for example 0.9, multiplies the old memory [1.5, 2.8] by 1 minus the update gate value, or 0.1, to obtain [0.15, 0.28], the portion of the old memory to be retained. The resulting new memory [0.8, 0.9] is multiplied by the update gate value of 0.9 to obtain [0.72, 0.81], the new content. Adding these two parts together, we get [0.87, 1.09], which is the hidden state at the current moment. It retains some traces of the past and contains new information at the moment.

[0082] In an optional embodiment, the attention layer obtains temporal features by weighting features at different time steps, including:

[0083] The hidden state sequence Input to the time-step attention layer, which calculates the hidden state of each time step through a fully connected layer with a tanh activation function Attention scores, all scores are normalized to attention weights , the hidden state sequence According to the corresponding attention weight Perform weighted summation to obtain the parallel branch timing characteristics.

[0084] Specifically, the role of the attention layer is to evaluate which moment of information is most important in the entire forging cycle. Assume that after processing a complete forging cycle, the recurrent unit outputs a sequence of hidden states representing the state at each moment. For example, at the 10th millisecond, the device starts to press down, and the hidden state vector is [0.2, 0.3]; at 50 milliseconds, the mold is violently impacted, and the state vector is [3.4, 4.1]; at the 100th millisecond, the device is in a stable pressure-maintaining state, and the state vector The attention layer uses a fully connected network to score the states of these three moments. Since the feature value of the impact moment is the largest, it will be given a high original score, such as 5, while the other two moments will be given lower scores, such as 0.5 and 0.8. The original scores are converted into attention weights with a sum of 1 through the Softmax function. The weight of the moment is 0.9, and and The weights of are 0.05 and 0.05 respectively. The attention layer uses the weights to generate the final output. Specifically, the hidden state vector at each moment is multiplied by the corresponding attention weight, and then all the results are added together. In one example, the time series feature vector is calculated by weighted summation. The value of the calculated final feature vector is mainly determined by the impact moment. Decide.

[0085] In an optional embodiment, the edge weights of the graph are obtained by weighting according to the physical association and data correlation between corresponding sensors, including:

[0086] Get the physical correlation coefficient matrix If sensor i and sensor j are installed on the same mechanical component or two components with direct force transmission relationship, then The value of is 1; if sensor i and sensor j are installed on independent components, then The value of is 0.1;

[0087] Extract the temporal features of all samples in the training data set output by each parallel branch attention layer, and calculate the Pearson correlation coefficient between the temporal feature sequence sets corresponding to the two sensors i and j And take the absolute value to get the correlation coefficient matrix ,in ;

[0088] The edge weight A(i,j) between node i and node j is calculated by the following formula:

[0089]

[0090] in is a hyperparameter with a value range between (0,1).

[0091] Specifically, a physical correlation coefficient matrix is ​​constructed based on the drawings of the equipment. For example, suppose that sensor i is a vibration sensor installed on the motor housing, and sensor j is a current sensor that monitors the power supply of the motor. Since they both serve the same component motor, the physical correlation between them is strong, and their corresponding physical correlation correlation values ​​are set to 1. The other pressure sensor k installed on the hydraulic system far away from the motor has a weaker physical correlation with the motor vibration sensor i, and the physical correlation correlation value between them is set to 0.1. In order to make up for the lack of prior knowledge, the actual correlation between sensors is further mined from the massive historical data. Specifically, the Pearson correlation coefficient between the feature sequence of motor vibration sensor i and the feature sequence of motor current sensor j in all historical data is calculated. If the calculated absolute value is 0.85, it means that they are highly synchronized in data performance. The feature sequences of vibration sensor i and hydraulic sensor k may have no rules, and the calculated absolute value of the correlation coefficient is only 0.05. By balancing the hyperparameters, such as 0.5, these two correlations are combined. The edge weight between the vibration and current sensors is 0.925, and the edge weight between the vibration and pressure sensors is 0.075, as shown in Figure 3 As shown in Figure 2, the graph structure constructed contains both physical connection information and data statistics.

[0092] In an optional embodiment, the step of aggregating features of adjacent nodes in a graph using a graph convolutional network with residual connections to update nodes to obtain fused features includes:

[0093] For the lth layer of the graph convolutional network, the node features are updated according to the following formula:

[0094]

[0095] in, is the node feature matrix of the l-1 layer, The adjacency matrix with self-loops is obtained by adding an identity matrix to the adjacency matrix A. for The diagonal matrix of is the trainable weight matrix of the l-th layer network, and ReLU is the activation function;

[0096] Node features output by the l-th layer network for:

[0097]

[0098] and is the input of the l+1th layer ;

[0099] After L layers of graph convolution with residual connections, the node feature matrix is ​​obtained. ;right The feature vectors of all nodes in the ensemble are averaged and pooled element by element to obtain the fused features.

[0100] Specifically, assume there are three nodes: vibration, acoustics, and current. Each node's initial feature vector has a dimension of 10. During the first round of information exchange, the vibration node updates its features based on the edge weights between it and the acoustics and current nodes. This updates the original 10-dimensional feature vector of the vibration node to a new 10-dimensional vector that not only contains vibration information but also incorporates the related current and acoustics information. In one embodiment, a ReLU activation function is used to increase nonlinear representation capabilities. After the vibration node receives the updated new feature vector, to prevent the original information from being lost during multiple rounds of interaction, a residual connection adds the pre-update vibration feature vector back to the new feature vector, forming the final output of the first layer. This vector, containing both the fused and original information, serves as the input to the second layer of graph convolution, which again exchanges information with neighboring nodes. This process is repeated L times, for example, 3 times. An average pooling operation is performed, summing each dimension of the final feature vectors of the vibration, acoustics, and current nodes and then dividing by 3 to obtain a 10-dimensional fused feature vector representing the current overall state of the device.

[0101] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, it should be understood by those skilled in the art that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. In addition, the various different implementations of the embodiments of the present invention can also be arbitrarily combined, as long as they do not violate the ideas of the embodiments of the present invention, and they should also be regarded as the contents disclosed in the embodiments of the present invention.

Claims

1. A forging equipment detection method based on multi-source data fusion, characterized in that: include: Acquire multi-source time series signals collected by multiple sensors installed on the forging equipment, decompose each of the time series signals into a set of intrinsic mode functions using variational mode decomposition, and calculate the instantaneous amplitude sequence and instantaneous frequency sequence of each intrinsic mode function using Hilbert transform; combine the instantaneous amplitude sequence and instantaneous frequency sequence of all the intrinsic mode functions obtained by decomposing each sensor source signal to obtain a time-frequency spectrum characteristic diagram corresponding to the sensor; Multiple parallel recurrent unit branches including phase gates are used to extract features at different time steps from each time-frequency spectrum feature map, wherein the phase gates are used to identify phase switching points during the forging process; a time step attention layer is connected after the recurrent unit in each parallel branch, and the attention layer obtains time series features by weighting the features at different time steps; A sensor association graph is constructed based on the temporal features, wherein the nodes of the sensor association graph are the temporal features extracted by each attention gated recurrent unit branch, and the edge weights of the sensor association graph are weighted according to the physical correlation and data correlation between the corresponding sensors; a graph convolutional network with residual connections is used to aggregate the features of adjacent nodes in the sensor association graph to update the nodes to obtain fused features; and a detection result of the forging equipment is obtained based on the fused features; The method of extracting features of different time steps from each time-frequency spectrum feature graph using multiple parallel cycle unit branches including phase gates includes: For each parallel branch, at the tth time step, the current input multi-channel time-frequency spectrum feature map vector and the recurrent unit hidden state at the previous time step Splicing is performed, the spliced ​​vector is input into the fully connected layer, and the phase gate output value is obtained by the activation function calculation ; The value range of is [0,1]; A gated recurrent unit is used as the recurrent unit to calculate the candidate hidden state at the t-th time step When , the reset gate used to adjust the hidden state of the previous moment in GRU is AND phase gate output value Perform element-by-element multiplication to obtain the corrected reset gate ; Using the Calculate the candidate hidden state , and update the hidden state of the current time step .

2. The method according to claim 1, characterized in that The use of the Calculate the candidate hidden state , and update the hidden state of the current time step ,include: The hidden state of the previous time step With the revised reset gate Perform element-by-element multiplication to obtain the reset history information; Multiply the reset historical information with the first trainable weight matrix to convert the input of the current time step Multiply it with the second trainable weight matrix, add the two multiplication results and the trainable bias vector to obtain an intermediate result, apply the hyperbolic tangent activation function to the intermediate result, and obtain the candidate hidden state ; Calculate the update gate for the current time step , the hidden state of the previous time step and Multiply element by element to get the historical state part to be retained; and update gate Perform element-by-element multiplication to obtain the new information part to be updated; add the historical state part to be retained and the new information part to be updated element-by-element to obtain the hidden state of the current time step .

3. The method according to claim 1, characterized in that The attention layer obtains temporal features by weighting the features of different time steps, including: The hidden state sequence Input to the time-step attention layer, which calculates the hidden state of each time step through a fully connected layer with a tanh activation function Attention scores, all scores are normalized to attention weights , the hidden state sequence According to the corresponding attention weight Perform weighted summation to obtain the parallel branch timing characteristics.

4. The method according to claim 1, wherein The edge weights of the graph are obtained by weighting the physical associations and data correlations between the corresponding sensors, including: Get the physical correlation coefficient matrix If sensor i and sensor j are installed on the same mechanical component or two components with direct force transmission relationship, then The value of is 1; if sensor i and sensor j are installed on independent components, then The value of is 0.1; Extract the temporal features of all samples in the training data set output by each parallel branch attention layer, and calculate the Pearson correlation coefficient between the temporal feature sequence sets corresponding to the two sensors i and j And take the absolute value to get the correlation coefficient matrix ,in ; The edge weight A(i,j) between node i and node j is calculated by the following formula: in is a hyperparameter with a value range between (0,1).

5. The method according to claim 1, wherein The graph convolutional network with residual connections is used to aggregate the features of adjacent nodes in the graph to update the nodes to obtain fused features, including: For the lth layer of the graph convolutional network, the node features are updated according to the following formula: in, is the node feature matrix of the l-1 layer, The adjacency matrix with self-loops is obtained by adding an identity matrix to the adjacency matrix A. for The diagonal matrix of is the trainable weight matrix of the l-th layer network, and ReLU is the activation function; Node features output by the l-th layer network for: and is the input of the l+1th layer ; After L layers of graph convolution with residual connections, the node feature matrix is ​​obtained. ;right The feature vectors of all nodes in the ensemble are averaged and pooled element by element to obtain the fused features.

6. A forging equipment detection system based on multi-source data fusion, characterized in that: include: An acquisition and transformation unit is used to obtain multi-source time series signals collected by multiple sensors installed on the forging equipment, decompose each of the time series signals into a set of intrinsic mode functions using variational mode decomposition, and calculate the instantaneous amplitude sequence and instantaneous frequency sequence of each intrinsic mode function using Hilbert transform; and combine the instantaneous amplitude sequence and instantaneous frequency sequence of all the intrinsic mode functions obtained by decomposing each sensor source signal to obtain a time-frequency spectrum characteristic diagram corresponding to the sensor; a feature extraction unit configured to extract features at different time steps from each time-frequency spectrum feature graph using multiple parallel recurrent unit branches including phase gates, wherein the phase gates are used to identify phase switching points during the forging process; a time step attention layer is connected after the recurrent unit in each parallel branch, wherein the attention layer obtains time series features by weighting the features at different time steps; A fusion and detection unit is configured to construct a sensor association graph based on the temporal features, wherein the nodes of the sensor association graph are the temporal features extracted by each attention gated recurrent unit branch, and the edge weights of the sensor association graph are weighted according to the physical correlation and data correlation between the corresponding sensors; a graph convolutional network with residual connections is used to aggregate the features of adjacent nodes in the sensor association graph to update the nodes to obtain fused features; and a detection result of the forging equipment is obtained based on the fused features; The method of extracting features of different time steps from each time-frequency spectrum feature graph using multiple parallel cycle unit branches including phase gates includes: For each parallel branch, at the tth time step, the current input multi-channel time-frequency spectrum feature map vector and the recurrent unit hidden state at the previous time step Splicing is performed, the spliced ​​vector is input into the fully connected layer, and the phase gate output value is obtained by the activation function calculation ; The value range of is [0,1]; A gated recurrent unit is used as the recurrent unit to calculate the candidate hidden state at the t-th time step When , the reset gate used to adjust the hidden state of the previous moment in GRU is AND phase gate output value Perform element-by-element multiplication to obtain the corrected reset gate ; Using the Calculate the candidate hidden state , and update the hidden state of the current time step .

7. The system according to claim 6, characterized in that The use of the Calculate the candidate hidden state , and update the hidden state of the current time step ,include: The hidden state of the previous time step With the revised reset gate Perform element-by-element multiplication to obtain the reset history information; Multiply the reset historical information with the first trainable weight matrix to convert the input of the current time step Multiply it with the second trainable weight matrix, add the two multiplication results and the trainable bias vector to obtain an intermediate result, apply the hyperbolic tangent activation function to the intermediate result, and obtain the candidate hidden state ; Calculate the update gate for the current time step , the hidden state of the previous time step and Multiply element by element to get the historical state part to be retained; and update gate Perform element-by-element multiplication to obtain the new information part to be updated; add the historical state part to be retained and the new information part to be updated element-by-element to obtain the hidden state of the current time step .

8. The system according to claim 6, wherein: The attention layer obtains temporal features by weighting the features of different time steps, including: The hidden state sequence Input to the time-step attention layer, which calculates the hidden state of each time step through a fully connected layer with a tanh activation function Attention scores, all scores are normalized to attention weights , the hidden state sequence According to the corresponding attention weight Perform weighted summation to obtain the parallel branch timing characteristics.

Images