Forging and pressing equipment detection method and system based on multi-source data fusion

Abstract

The invention provides a forging and pressing equipment detection method and system based on multi-source data fusion, specifically, each time sequence signal is decomposed into a group of intrinsic mode functions, and the instantaneous amplitude sequence and the instantaneous frequency sequence of each intrinsic mode function are calculated through Hilbert transform; combining the instantaneous amplitude sequence and the instantaneous frequency sequence of all the intrinsic mode functions obtained by decomposing each sensor source signal to obtain a time-frequency spectrum characteristic pattern corresponding to the sensor; extracting features of different time steps from each time-frequency spectrum feature map by using a plurality of parallel circulation unit branches comprising phase gates, connecting a time step attention layer behind the circulation unit of each parallel branch, and constructing a sensor association map based on the time sequence features; using the features of the adjacent nodes in the graph convolution network aggregation graph with residual connection to update the nodes to obtain fusion features; and obtaining a detection result of the forging and pressing equipment based on the fusion features.

Description

Technical Field

[0001] The present application relates to the field of equipment detection, and in particular to a forging equipment detection method and system based on multi-source data fusion. Background Art

[0002] As the core equipment of modern heavy industry, forging equipment plays an indispensable role in key fields such as automobile, aerospace, and energy. Its working environment is usually extremely harsh, and it is subjected to huge impact loads, high temperatures, and severe vibrations, which makes its key components such as crankshafts, bearings, gears, etc. very prone to fatigue damage, wear, and even fracture. Once a failure occurs, it will not only lead to expensive maintenance costs and long downtime, but also may cause serious production safety accidents. Traditional equipment detection methods mainly rely on manual regular inspections and planned maintenance. The former is highly subjective, inefficient, and cannot provide real-time warnings, while the latter may lead to insufficient or excessive maintenance, and the economic benefits are not high. With the development of sensor technology and signal processing technology, data-based condition monitoring methods have become mainstream. Early studies focused on the analysis of a single signal source, such as using only vibration signals for frequency domain analysis to diagnose bearing faults. However, the operation process of forging equipment is an extremely complex multi-physical field coupling system of machinery, electricity, and fluid. A single signal source can often only partially reflect the local state of the equipment, and its information carrying capacity is limited. In the context of strong noise and changing working conditions, it is very easy to produce misjudgments and missed judgments. In order to overcome the limitations of a single information source, detection methods based on multi-source data fusion have emerged. By comprehensively analyzing data collected from sensors of different types and locations, such as vibration, acoustics, current, temperature, etc., more comprehensive information about the operating status of the equipment can be formed. At present, in the feature extraction link, commonly used methods include Fourier transform, wavelet transform, etc., but the forging signal has typical non-stationary and nonlinear characteristics, Fourier transform is difficult to process, and the basis function selection of wavelet transform lacks adaptability. The forging process has distinct stages. When the traditional RNN model processes such stage signals, the hidden state of the previous stage will interfere with the feature extraction of the subsequent stage, thereby blurring the fault features closely related to the specific stage. Moreover, when performing multi-source feature fusion, simple feature splicing or weighted fusion based on the attention mechanism fails to explicitly model the intrinsic correlation between sensors. How to use multi-source data to improve the accuracy of forging equipment detection is an urgent problem to be solved. Summary of the invention

[0003] In order to solve the above problems, this application proposes a forging equipment detection method based on multi-source data fusion, including: Obtain multi-source time series signals collected by multiple sensors installed on a forging equipment. Use variational mode decomposition to decompose each of the time series signals into a set of intrinsic mode functions, and calculate the instantaneous amplitude sequence and instantaneous frequency sequence of each intrinsic mode function using Hilbert transform; combine the instantaneous amplitude sequences and instantaneous frequency sequences of all the intrinsic mode functions obtained by decomposing each sensor source signal to obtain a time-frequency spectrum feature map corresponding to the sensor; Use multiple parallel recurrent unit branches including phase gates to extract features at different time steps from each time-frequency spectrum feature map respectively, where the phase gate is used to identify the stage switching points in the forging process; connect a time step attention layer after the recurrent unit in each parallel branch, and the attention layer obtains temporal features by weighting the features at different time steps; Construct a sensor association graph based on the temporal features. The nodes of the sensor association graph are the temporal features extracted by each attention gated recurrent unit branch, and the edge weights of the sensor association graph are weighted according to the physical relevance and data correlation between the corresponding sensors; use a graph convolutional network with residual connections to aggregate the features of adjacent nodes in the sensor association graph to update the nodes and obtain fused features; obtain the detection result of the forging equipment based on the fused features.

[0004] Preferably, the using multiple parallel recurrent unit branches including phase gates to extract features at different time steps from each time-frequency spectrum feature map respectively includes: For each parallel branch, at the t-th time step, splice the current input multi-channel time-frequency spectrum feature map vector with the hidden state of the recurrent unit at the previous time step and input the spliced vector into a fully connected layer, and calculate the output value of the phase gate through an activation function ; the value range of the is [0, 1]; Use a gated recurrent unit as the recurrent unit. When calculating the candidate hidden state at the t-th time step, element-wise multiply the reset gate in the GRU for adjusting the hidden state at the previous moment with the output value of the phase gate to obtain a corrected reset gate ; use the to calculate the candidate hidden state and update to obtain the hidden state at the current time step.

[0005] Preferably, the using the to calculate the candidate hidden state and update to obtain the hidden state , including: Multiply the hidden state of the previous time step element-wise with the corrected reset gate to obtain the reset historical information; Multiply the reset historical information with the first trainable weight matrix, multiply the input of the current time step with the second trainable weight matrix, add the two multiplication results and the trainable bias vector to obtain an intermediate result, and apply the hyperbolic tangent activation function to the intermediate result to obtain the candidate hidden state ; Calculate the update gate of the current time step , multiply the hidden state of the previous time step element-wise with to obtain the part of the historical state to be retained; multiply the candidate hidden state element-wise with the update gate to obtain the part of the new information to be updated; add the part of the historical state to be retained and the part of the new information to be updated element-wise to obtain the hidden state of the current time step .

[0006] Preferably, the attention layer obtains the temporal feature by weighting the features of different time steps, including: Input the hidden state sequence into the time step attention layer, and the time step attention layer calculates the attention score of the hidden state of each time step through a fully connected layer with a tanh activation function, and normalize all scores into attention weights , and weight-sum the hidden state sequence according to the corresponding attention weights to obtain the temporal feature of the parallel branch.

[0007] Preferably, the edge weights of the graph are weighted according to the physical relevance and data correlation between the corresponding sensors, including: Obtain the physical correlation coefficient matrix ; if sensor i and sensor j are installed on the same mechanical component or on two components with a direct force transmission relationship, then has a value of 1; if sensor i and sensor j are installed on independent components, then has a value of 0.1; Extract the temporal features output by all samples in the training dataset through each parallel branch attention layer, calculate the Pearson correlation coefficient between the temporal feature sequence sets corresponding to two sensors i and j and take the absolute value to obtain the correlation coefficient matrix , where ; The edge weight A(i,j) between node i and node j is calculated by the following formula:

[0008] where is a hyperparameter with a value range between (0,1).

[0009] Preferably, the method of aggregating the features of adjacent nodes in the graph by using a graph convolutional network with residual connections to update the nodes and obtaining the fused features includes: For the l-th layer of the graph convolutional network, the node features are updated according to the following formula:

[0010] where, is the node feature matrix of the (l - 1)-th layer, is the adjacency matrix A plus an identity matrix to obtain an adjacency matrix with self-loops, is the diagonal matrix of, is the trainable weight matrix of the l-th layer network, and ReLU is the activation function; The node features output by the l-th layer network are:

[0011] and is the input of the (l + 1)-th layer ; After L layers of graph convolutional calculations with residual connections, the node feature matrix is obtained; for in, element-wise average pooling operation is performed on the feature vectors of all nodes to obtain the fused features.

[0012] This application also proposes a forging equipment detection system based on multi-source data fusion, including: An acquisition and transformation unit, configured to obtain multi-source time series signals collected by a plurality of sensors installed on the forging equipment, decompose each of the time series signals into a group of intrinsic mode functions by using variational mode decomposition, and calculate the instantaneous amplitude sequence and instantaneous frequency sequence of each intrinsic mode function by using Hilbert transform; combine the instantaneous amplitude sequences and instantaneous frequency sequences of all the intrinsic mode functions decomposed from each sensor source signal to obtain a time-frequency spectrum feature map corresponding to the sensor; A feature extraction unit, which is used to extract features at different time steps from each time-frequency spectrum feature map respectively by using a plurality of parallel recurrent unit branches including phase gates, wherein the phase gates are used to identify the phase switching points in the forging process; a time step attention layer is connected after the recurrent unit in each parallel branch, and the attention layer obtains temporal features by weighting the features at different time steps; A fusion and detection unit, which is used to construct a sensor association graph based on the temporal features. The nodes of the sensor association graph are the temporal features extracted by each attention gated recurrent unit branch, and the edge weights of the sensor association graph are obtained by weighting according to the physical relevance and data correlation between the corresponding sensors; a graph convolutional network with residual connections is used to aggregate the features of adjacent nodes in the sensor association graph to update the nodes and obtain fusion features; a detection result of the forging equipment is obtained based on the fusion features.

[0013] Preferably, the step of extracting features at different time steps from each time-frequency spectrum feature map respectively by using a plurality of parallel recurrent unit branches including phase gates includes: For each parallel branch, at the t-th time step, the multi-channel time-frequency spectrum feature map vector input currently is concatenated with the hidden state of the recurrent unit at the previous time step , and the concatenated vector is input into a fully connected layer, and the output value of the phase gate is calculated through an activation function ; the value range of is [0, 1]; A gated recurrent unit is used as the recurrent unit. When calculating the candidate hidden state at the t-th time step, the reset gate in the GRU for adjusting the hidden state at the previous moment is multiplied element by element with the output value of the phase gate to obtain a corrected reset gate ; the candidate hidden state is calculated by using , and the hidden state at the current time step is updated.

[0014] Preferably, the step of calculating the candidate hidden state by using and updating the hidden state at the current time step includes: The hidden state at the previous time step is multiplied element by element with the corrected reset gate to obtain the reset historical information; The reset historical information is multiplied by the first trainable weight matrix, and the input Multiply by the second trainable weight matrix, add the two multiplication results and the trainable bias vector to obtain an intermediate result, and apply the hyperbolic tangent activation function to the intermediate result to obtain the candidate hidden state ; Calculate the update gate at the current time step , and multiply the hidden state at the previous time step element-wise with to obtain the part of the historical state to be retained; multiply the candidate hidden state element-wise with the update gate to obtain the part of the new information to be updated; add the part of the historical state to be retained and the part of the new information to be updated element-wise to obtain the hidden state at the current time step.

[0015] Preferably, the attention layer obtains the temporal feature by weighting the features at different time steps, including: Input the hidden state sequence into the time step attention layer, and the time step attention layer calculates the attention score of each time step hidden state through a fully connected layer with a tanh activation function, and normalizes all scores to attention weights , and weight-sum the hidden state sequence according to the corresponding attention weights to obtain the temporal feature of the parallel branch.

[0016] Preferably, the edge weights of the graph are weighted according to the physical correlation and data correlation between the corresponding sensors, including: Obtain the physical correlation coefficient matrix ; if sensor i and sensor j are installed on the same mechanical component or on two components with a direct force transmission relationship, then has a value of 1; if sensor i and sensor j are installed on independent components, then has a value of 0.1; Extract the temporal features output by all samples in the training dataset through each parallel branch attention layer, calculate the Pearson correlation coefficient between the temporal feature sequence sets corresponding to two sensors i and j, and take the absolute value to obtain the correlation coefficient matrix , where ; The edge weight A(i, j) between node i and node j is calculated by the following formula:

[0017] where is a hyperparameter with a value range between (0, 1).

[0018] Preferably, aggregating the features of adjacent nodes in the graph by using a graph convolutional network with residual connections to update the nodes to obtain the fused features includes: For the l-th layer of the graph convolutional network, the node features are updated according to the following formula:

[0019] where is the node feature matrix of the (l - 1)-th layer, is the adjacency matrix A plus an identity matrix to obtain an adjacency matrix with self-loops, is the diagonal matrix of is the trainable weight matrix of the l-th layer network, and ReLU is the activation function; The node features output by the l-th layer network are:

[0020] and is the input of the (l + 1)-th layer ; After L layers of graph convolutional calculations with residual connections, the node feature matrix is obtained; for the feature vectors of all nodes in

[0021] In this application, the instantaneous amplitude reflecting the signal energy transformation and the instantaneous frequency reflecting the signal frequency fluctuation over time are used to form a multi-channel time-frequency feature map, which can monitor the weak features in the evolution process of the device from normal to early failure; moreover, by introducing a phase gate in the recurrent neural network, it is possible to identify the working stage switching points in the forging process and eliminate the interference of irrelevant information in the previous stage; the graph structure weights and fuses the physical relevance of the mechanical structure and the data relevance reflecting the real-time working conditions, enhancing the expression ability of the fused features, and further improving the accuracy and reliability of detection. Description of the Drawings

[0022] Figure 1 is the flowchart of the first embodiment; Figure 2 is the schematic diagram of the time-frequency spectrum feature map corresponding to the sensor; Figure 3 is the structural schematic diagram of the sensor association graph. Detailed Embodiments

[0023] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0024] Specific embodiments are as follows Figure 1 As shown, the present application proposes a forging equipment detection method based on multi-source data fusion, including: S1. Obtain multi-source time series signals collected by multiple sensors installed on the forging equipment, decompose each of the time series signals into a set of intrinsic mode functions by variational mode decomposition, and calculate the instantaneous amplitude sequence and instantaneous frequency sequence of each intrinsic mode function using Hilbert transform; combine the instantaneous amplitude sequences and instantaneous frequency sequences of all the intrinsic mode functions obtained by decomposing each sensor source signal to obtain a time-frequency spectrum feature map corresponding to the sensor. Synchronously collect, for example, three signals from sensors installed at different positions on the forging equipment, namely, vibration sensor signals from the main bearing seat, acoustic sensor signals near the die area, and current sensor signals of the drive motor. Taking the vibration signal as an example, the vibration signal is [0.1, 0.3, -0.2,...]. Apply variational mode decomposition to the collected vibration signal. Variational mode decomposition decomposes the original complex vibration signal into multiple sub-signals, that is, intrinsic mode functions IMF. Assume that VMD decomposes the original vibration signal into 3 IMF components, where IMF1 represents the high-frequency vibration component, which may be related to the impact when the bearing rolling element passes through a defect, IMF2 represents the medium-frequency vibration component, which may be related to the gear meshing frequency, and IMF3 represents the low-frequency vibration component, which may be related to the rotational imbalance of the main shaft. Decompose each of the three signals, namely vibration, acoustic, and current signals, to obtain their respective sets of IMF components.

[0025] Use Hilbert transform to calculate the instantaneous information changing with time for each IMF component. For example, for IMF1 of the vibration signal, calculate the instantaneous amplitude sequence and instantaneous frequency sequence. Each point of the instantaneous amplitude sequence represents the energy magnitude of IMF1 at that moment. For example, when the equipment is impacted, the value of this sequence will increase instantaneously; each point of the instantaneous frequency sequence represents the main vibration frequency of IMF1 at that moment. For example, a change in the rotational speed will cause the value of this sequence to change. Perform the above operations on IMF1, IMF2, and IMF3 decomposed from the vibration signal, so as to obtain 3 instantaneous amplitude sequences and 3 instantaneous frequency sequences, a total of 6 time series.

[0026] Combine the instantaneous frequency sequences and instantaneous amplitude sequences of the IMFs of all sensors to obtain a multi-channel time-frequency spectrum feature map. In one embodiment, concatenate the instantaneous amplitude sequences of all IMFs of a sensor row by row to obtain the instantaneous amplitude matrix of this sensor, concatenate the instantaneous frequency sequences of all IMFs of this sensor row by row to obtain the instantaneous frequency matrix of this sensor, and use the instantaneous amplitude matrix and instantaneous frequency matrix of each sensor as two channels of the time-frequency spectrum feature map corresponding to this sensor, as Figure 2 shown.

[0027] S2. Use multiple parallel recurrent unit branches including phase gates to extract features at different time steps from each time-frequency spectrum feature map. Among them, the phase gate is used to identify the stage switching points in the forging process; connect a time step attention layer after the recurrent unit in each parallel branch, and the attention layer obtains the temporal features by weighting the features at different time steps; Establish an independent processing branch for each sensor time-frequency spectrum feature map obtained from step S1. If the three sensors are vibration, acoustic, and current sensors respectively, then start three processing branches. Branch one is used to process the multi-channel time-frequency spectrum feature map of the vibration sensor, branch two is used to process the multi-channel time-frequency spectrum feature map of the acoustic sensor, and branch three is used to process the multi-channel time-frequency spectrum feature map of the current sensor. Assume that the vibration time-frequency spectrum feature map contains 100 time points. At each time point, for example, time point t, the recurrent unit will analyze the feature data at the current time point t and combine the information about time point t - 1 to output a phase gate value. This value ranges between 0 and 1 and is used to determine whether a key stage switch occurs at the current moment. For example, during the forging process, when the equipment switches from the die closing stage to the pressing down stage, the signal features will change drastically. The phase gate identifies the change and outputs a value close to 1. When the recurrent unit receives a signal close to 1, it greatly clears or resets its memory about the die closing stage, so as to focus more on learning the features of the next pressing down stage. On the contrary, if the equipment is still in the stable pressure holding stage and the signal features change little, the phase gate will output a value close to 0, and the recurrent unit will retain and continue to accumulate the previous memory to analyze the subtle changes within this stage.

[0028] After the loop unit finishes the preliminary processing of 100 time points, 100 preliminary feature vectors are obtained, with each vector corresponding to a time point. However, the information at all time points is not equally important. The time step attention layer initially examines these 100 feature vectors all at once and calculates an importance score, i.e., an attention weight, for each vector. For example, in the case of a fault detection task, the moment when the metal contacts the die at the start of forging and the moment when the pressure reaches its peak contain more important information than the information during periods when the equipment is idle or moving at a constant speed. The attention layer will assign higher weights, such as 0.8 or 0.9, to the feature vectors corresponding to the impact moment and the pressure peak moment, while assigning lower weights, such as 0.01, to the feature vectors during steady operation. The attention layer multiplies the 100 preliminary feature vectors by their corresponding importance weights and then sums all the results to obtain a feature vector of a fixed length. The feature vector not only contains the dynamic changes of the signal but also highlights the key time points. Branches two and three also use exactly the same method and each generate a corresponding time series feature vector.

[0029] S3. Based on the time series features, construct a sensor association graph. The nodes of the sensor association graph are the time series features extracted by each branch of the attention gated recurrent unit. The edge weights of the sensor association graph are obtained by weighting according to the physical relevance and data correlation between the corresponding sensors; use a graph convolutional network with residual connections to aggregate the features of adjacent nodes in the sensor association graph to update the nodes and obtain the fused features; based on the fused features, obtain the detection result of the forging equipment.

[0030] Construct a graph structure. Still taking the above example, it includes three nodes, namely node V representing vibration information, node A representing acoustic information, and node C representing current information. According to the structure diagram of the forging equipment, a physical association matrix is predefined. For example, the vibration sensor is located on the main bearing, and the current sensor monitors the drive motor. They are associated through the main shaft and the transmission system, with a strong correlation, and the physical association coefficient is 0.9. The acoustic sensor is close to the mold, at a certain distance from the main bearing but still belonging to the same equipment, with a medium correlation, and the physical association coefficient is 0.5. Moreover, through the analysis of a large amount of historical data, it is found that whenever a certain specific pattern appears in the vibration characteristics, another specific pattern always appears synchronously in the current characteristics, and they are highly correlated at the data level. By calculating the Pearson correlation coefficient between their feature vectors, a value such as 0.8 is obtained. The physical association and the data association are weighted and combined to obtain the final connection strength. Assuming that the weight of the physical association is 0.4 and the weight of the data association is 0.6, the connection strength between node V and node C is 0.84. After constructing the sensor association graph, graph convolution operations are performed. To prevent information loss in the graph convolution, residual connections are introduced, and after at least one round of graph convolution and residual connections, the fused features are obtained. In one embodiment, the feature vectors of the three nodes are merged into a global feature vector representing the current overall state of the equipment through an averaging operation, and the global feature vector is input into a pre-trained classifier, and the classifier outputs the final detection result, for example: the equipment is normal, or there is a crack in the bearing, etc.

[0031] In an alternative embodiment, the extracting, by using multiple parallel recurrent unit branches including phase gates, features of different time steps from each time-frequency spectrum feature map respectively includes: For each parallel branch, at the t-th time step, the multi-channel time-frequency spectrum feature map vector of the current input is concatenated with the hidden state of the recurrent unit at the previous time step and the concatenated vector is input into a fully connected layer, and the output value of the phase gate is calculated through an activation function ; the value range of the is [0, 1]; A gated recurrent unit is used as the recurrent unit. When calculating the candidate hidden state at the t-th time step, the reset gate in the GRU for adjusting the hidden state at the previous moment is multiplied element-wise by the output value of the phase gate to obtain the corrected reset gate ; the is used to calculate the candidate hidden state , and the hidden state at the current time step is updated.

[0032] Specifically, assume that the forging equipment has just completed a drastic downward pressing action and entered a relatively stable pressure-holding stage. At the moment when the downward pressing ends, that is, at time point t, the cyclic unit receives a new spectrogram feature map vector representing the pressure-holding stage , and its value at this time is, for example, [0.2, 0.1, 0.3]. And the memory information of the previous time step stored inside it , still describes the characteristics of the drastic downward pressing stage, for example, it is [1.5, 2.8, 1.9]. Among them, the time step is the smallest processing unit obtained by splitting the sequence signal. Concatenate these two sets of data and send them into the fully connected layer. The fully connected layer is trained to recognize the pattern mutation from high energy to low energy, and calculate a phase gate output value close to 1 , for example, 0.98, indicating the stage transition.

[0033] Furthermore, use the phase gate signal to update the memory. The phase gate output value of 0.98 is used to intervene in the reset gate inside the gated recurrent unit GRU . A high value of will amplify the role of the reset gate, making the corrected reset gate also close to 1, so that the cyclic unit forgets the information of the downward pressing stage . When calculating the new candidate memory state , it depends on the new input representing the pressure-holding stage . The updated current memory is a new state that clears the interference of the old stage. On the contrary, if the equipment is still in the downward pressing stage, the new input is similar to the old memory, and the phase gate will output a value close to 0, so that the cyclic unit inherits and accumulates historical information, and then continuously tracks the dynamics of the downward pressing process.

[0034] In an optional embodiment, the calculating the candidate hidden state using the , and updating the hidden state of the current time step , includes: Element-wise multiply the hidden state of the previous time step with the corrected reset gate to obtain the reset historical information; Multiply the reset historical information by the first trainable weight matrix, multiply the input of the current time step by the second trainable weight matrix, add the two multiplication results and the trainable bias vector to obtain an intermediate result, and apply the hyperbolic tangent activation function to the intermediate result to obtain the candidate hidden state ; Calculate the update gate for the current time step , multiply the hidden state from the previous time step element-wise with to obtain the part of the historical state to be retained; multiply the candidate hidden state element-wise with the update gate to obtain the part of the new information to be updated; add the part of the historical state to be retained and the part of the new information to be updated element-wise to obtain the hidden state for the current time step.

[0035] Specifically, use the calculated modified reset gate, for example, with a value of 0.1, to weaken the memory of the previous moment. If the old memory is [1.5, 2.8], after weakening it becomes [0.15, 0.28]. Pass this cleared historical information and the new input at the current moment, such as [0.6, 0.7], through separate neural network layers for transformation, then add them with a bias term, and finally process them through a hyperbolic tangent activation function to obtain a candidate hidden state representing the current moment, such as [0.8, 0.9]. The update gate, such as 0.9, multiplies the old memory [1.5, 2.8] with the value of 1 minus the update gate, which is 0.1, to obtain [0.15, 0.28], the part of the old memory to be retained. Multiply the generated new [0.8, 0.9] with the value of the update gate 0.9 to obtain [0.72, 0.81], the new content. Add these two parts together to obtain [0.87, 1.09], which is the hidden state at the current moment, retaining a bit of the past trace and containing the current new information.

[0036] In an optional embodiment, the attention layer obtains the temporal feature by weighting the features of different time steps, including: Input the hidden state sequence into the time step attention layer, and the time step attention layer calculates the attention scores of the hidden state at each time step through a fully connected layer with a tanh activation function, and normalizes all scores into attention weights , and weighted sum the hidden state sequence according to the corresponding attention weights to obtain the temporal feature of the parallel branch.

[0037] Specifically, the role of the attention layer is to evaluate which moment's information is the most important during the entire forging cycle. Assume that after the recurrent unit processes a complete forging cycle, it outputs a hidden state sequence representing the state of each moment. For example, at the 10th millisecond, the device starts to press down, and the hidden state vector is [0.2, 0.3]; at the 50th millisecond, the mold has a violent impact, and the state vector is [3.4, 4.1]; at the 100th millisecond, the device is in a stable pressure holding state, and the state vector is [0.5, 0.6]. The attention layer scores the states at these three moments through a fully connected network. Since the characteristic value at the impact moment is the largest, it will give it a very high original score, such as 5, while giving lower scores to the other two moments, such as 0.5 and 0.8. The original scores are converted into attention weights with a sum of 1 through the Softmax function. Finally, the weight at the moment is 0.9, while and the weights of are 0.05 and 0.05 respectively. The attention layer uses the weights to generate the final output. Specifically, the hidden state vector at each moment is multiplied by the corresponding attention weight, and then all the results are added. In one example, the calculation method of the temporal feature vector is weighted summation, and the value of the finally calculated feature vector is mainly determined by the at the impact moment.

[0038] In an optional embodiment, the edge weights of the graph are weighted according to the physical relevance and data correlation between the corresponding sensors, including: Obtain the physical correlation coefficient matrix ; if sensor i and sensor j are installed on the same mechanical component or on two components with a direct force transmission relationship, then the value of is 1; if sensor i and sensor j are installed on independent components, then the value of is 0.1; Extract the temporal features output by each parallel branch attention layer for all samples in the training dataset, and calculate the Pearson correlation coefficient between the temporal feature sequence sets corresponding to two sensors i and j, and take the absolute value to obtain the correlation coefficient matrix , where ; The edge weight A(i, j) between node i and node j is calculated by the following formula:

[0039] where is a hyperparameter with a value range between (0, 1).

[0040] Specifically, a physical correlation coefficient matrix is constructed based on the device drawings. For example, assume that sensor i is a vibration sensor installed on the motor housing, and sensor j is a current sensor monitoring the power supply of the motor. Since they both serve the same component, the motor, their physical association is strong, and the corresponding physical correlation coefficient value between them is set to 1. Another pressure sensor k installed on a hydraulic system far from the motor has a weak physical association with the motor vibration sensor i, and the physical correlation coefficient value between them is set to 0.1. To make up for the lack of prior knowledge, the actual correlation between sensors is further mined from a large amount of historical data. Specifically, calculate the Pearson correlation coefficient between the feature sequences of the motor vibration sensor i and the motor current sensor j in all historical data. If the calculated absolute value is 0.85, it indicates that they are highly synchronized in data performance. The feature sequences of the vibration sensor i and the hydraulic sensor k may have no pattern, and the absolute value of the calculated correlation coefficient is only 0.05. By balancing hyperparameters such as 0.5, these two types of correlations are combined. The edge weight between the vibration and current sensors is 0.925, while the edge weight between the vibration and pressure sensors is 0.075, as Figure 3 shown. Furthermore, the constructed graph structure contains both physical connection information and reflects the statistical information of the data.

[0041] In an alternative embodiment, the method of using a graph convolutional network with residual connections to aggregate the features of adjacent nodes in the graph to update the nodes and obtain the fused features includes: For the l-th layer of the graph convolutional network, the node features are updated according to the following formula:

[0042] where, is the node feature matrix of the (l - 1)-th layer, is the adjacency matrix A plus an identity matrix to obtain a self-looped adjacency matrix, is 's diagonal matrix, is the trainable weight matrix of the l-th layer network, ReLU is the activation function; The node feature output by the l-th layer network is:

[0043] and is the input of the (l + 1)-th layer ; After L layers of graph convolutional calculations with residual connections, the node feature matrix is obtained; perform element-wise average pooling operation on the feature vectors of all nodes in to obtain the fused features.

[0044] Specifically, assume there are three nodes: vibration, acoustics, and current, and the initial feature vector dimension of each node is 10. In the first round of information exchange, the vibration node updates its features according to the edge weights between it and the acoustics and current nodes. The original 10-dimensional feature vector of the vibration node is updated into a new 10-dimensional vector, which now not only contains vibration information but also incorporates relevant current and acoustics information. In one embodiment, a ReLU activation function is used to enhance the non-linear representation ability. After the vibration node obtains the updated new feature vector, to prevent the loss of original information in multiple rounds of interaction, a residual connection adds the vibration feature vector before update back to the new feature vector to form the final output of the first layer. The vector containing the fused information and the original information is used as the input for the second-layer graph convolution and exchanges information with neighbor nodes again. The above process is repeated L times, for example, 3 times. An average pooling operation is performed, that is, each dimension of the final feature vectors of the vibration, acoustics, and current nodes is added separately and then divided by 3 to obtain a 10-dimensional fused feature vector representing the current overall state of the device.

[0045] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention. In addition, any arbitrary combination can be made between the various different embodiments of the embodiments of the present invention, as long as it does not violate the idea of the embodiments of the present invention, it should also be regarded as the content disclosed in the embodiments of the present invention.

Claims

1. A forging equipment detection method based on multi-source data fusion, characterized in that, Including: Obtain multi-source time series signals collected by multiple sensors installed on a forging press. Use variational mode decomposition to decompose each of the time series signals into a set of intrinsic mode functions, and use the Hilbert transform to calculate the instantaneous amplitude sequence and instantaneous frequency sequence of each intrinsic mode function; combine the instantaneous amplitude sequences and instantaneous frequency sequences of all the intrinsic mode functions obtained by decomposing each sensor source signal to obtain a time-frequency spectrum feature map corresponding to the sensor; Use multiple parallel recurrent unit branches including phase gates to extract features at different time steps from each time-frequency spectrum feature map respectively, where the phase gate is used to identify the stage switching points in the forging process; connect a time step attention layer after the recurrent unit in each parallel branch, and the attention layer obtains temporal features by weighting the features at different time steps; Construct a sensor association graph based on the temporal features. The nodes of the sensor association graph are the temporal features extracted by each attention gated recurrent unit branch, and the edge weights of the sensor association graph are weighted according to the physical correlation and data correlation between the corresponding sensors; use a graph convolutional network with residual connections to aggregate the features of adjacent nodes in the sensor association graph to update the nodes and obtain fused features; obtain the detection result of the forging press based on the fused features.

2. The method according to claim 1, wherein The step of using multiple parallel recurrent unit branches including phase gates to extract features at different time steps from each time-frequency spectrum feature map respectively includes: For each parallel branch, at the t-th time step, the multi-channel spectrogram feature map vector of the current input is concatenated with the hidden state of the recurrent unit at the previous time step , and the concatenated vector is input into the fully connected layer, and the output value of the phase gate is calculated through the activation function ; the value range of the is [0, 1]; Using a gated recurrent unit as the recurrent unit, when calculating the candidate hidden state at the t-th time step the reset gate in the GRU that is used to adjust the hidden state at the previous moment is element-wise multiplied by the output value of the phase gate to obtain a corrected reset gate ; the candidate hidden state is calculated using the above and the hidden state at the current time step is updated.

3. The method according to claim 2, wherein The use of the to calculate the candidate hidden state and update to obtain the hidden state at the current time step includes: The hidden state of the previous time step and the corrected reset gate are multiplied element-wise to obtain the reset historical information; Multiply the reset historical information by the first trainable weight matrix, and multiply the input at the current time step by the second trainable weight matrix, add the two multiplication results and the trainable bias vector to obtain an intermediate result, and apply the hyperbolic tangent activation function to the intermediate result to obtain the candidate hidden state ; Calculate the update gate for the current time step , and multiply the hidden state of the previous time step element-wise with to obtain the part of the historical state to be retained; multiply the candidate hidden state element-wise with the update gate to obtain the part of the new information to be updated; add the part of the historical state to be retained and the part of the new information to be updated element-wise to obtain the hidden state of the current time step .

4. The method according to claim 1, wherein The step that the attention layer obtains temporal features by weighting the features at different time steps includes: Input the hidden state sequence into the time-step attention layer, which calculates the attention scores of each time-step hidden state through a fully-connected layer with a tanh activation function and normalizes all scores into attention weights . Weight-sum the hidden state sequence according to the corresponding attention weights to obtain the parallel-branch temporal features.

5. The method according to claim 1, characterized in that, The step that the edge weights of the graph are weighted according to the physical correlation and data correlation between the corresponding sensors includes: Obtain the physical correlation coefficient matrix ; If sensor i and sensor j are installed on the same mechanical component or on two components with a direct force transmission relationship, then the value of is 1; If sensor i and sensor j are installed on independent components, then the value of is 0.1; Extract the temporal features output by all samples in the training dataset through each parallel branch attention layer, and calculate the Pearson correlation coefficient between the temporal feature sequence sets corresponding to two sensors i and j And take the absolute value to obtain the correlation coefficient matrix , where ; The edge weight A(i, j) between node i and node j is calculated by the following formula: wherein is a hyperparameter with a value range between (0, 1).

6. The method according to claim 1, wherein The step of using a graph convolutional network with residual connections to aggregate the features of adjacent nodes in the graph to update the nodes and obtain fused features includes: For the l-th layer of the graph convolutional network, the node features are updated according to the following formula: Among them, is the node feature matrix of the (l - 1)-th layer, is the adjacency matrix with self-loops obtained by adding an identity matrix to the adjacency matrix A, is the diagonal matrix of, is the trainable weight matrix of the l-th layer network, and ReLU is the activation function; The node features output by the l-th layer network are as follows: and is the input of the (l + 1)-th layer ; After L layers of graph convolution calculations with residual connections, a node feature matrix is obtained ; For The feature vectors of all nodes in are subjected to element-wise average pooling operation to obtain the fusion feature.

7. A forging equipment detection system based on multi-source data fusion, characterized in that, Including: An acquisition and transformation unit, which is used to obtain multi-source time series signals collected by multiple sensors installed on a forging press, use variational mode decomposition to decompose each of the time series signals into a set of intrinsic mode functions, and use the Hilbert transform to calculate the instantaneous amplitude sequence and instantaneous frequency sequence of each intrinsic mode function; combine the instantaneous amplitude sequences and instantaneous frequency sequences of all the intrinsic mode functions obtained by decomposing each sensor source signal to obtain a time-frequency spectrum feature map corresponding to the sensor; A feature extraction unit, which is used to use multiple parallel recurrent unit branches including phase gates to extract features at different time steps from each time-frequency spectrum feature map respectively, where the phase gate is used to identify the stage switching points in the forging process; connect a time step attention layer after the recurrent unit in each parallel branch, and the attention layer obtains temporal features by weighting the features at different time steps; The fusion and detection unit is used to construct a sensor association graph based on the timing features. The nodes of the sensor association graph are the timing features extracted by each branch of the attention gated recurrent unit, and the edge weights of the sensor association graph are obtained by weighting according to the physical relevance and data correlation between the corresponding sensors; A graph convolutional network with residual connections is used to aggregate the features of adjacent nodes in the sensor association graph to update the nodes and obtain the fusion features; Based on the fusion features, the detection result of the forging equipment is obtained.

8. The system according to claim 7, wherein The utilization of multiple parallel branches of recurrent units including phase gates to extract features of different time steps from each time-frequency spectrogram respectively includes: For each parallel branch, at the t-th time step, the multi-channel time-frequency spectrum feature map vector of the current input is concatenated with the hidden state of the recurrent unit at the previous time step , and the concatenated vector is input into the fully connected layer, and the output value of the phase gate is calculated through the activation function ; the ranges from [0, 1]; The gated recurrent unit is used as the recurrent unit. When calculating the candidate hidden state at the t-th time step the reset gate in the GRU for adjusting the hidden state at the previous moment is element-wise multiplied by the output value of the phase gate to obtain the corrected reset gate ; the candidate hidden state is calculated using the above and the hidden state at the current time step is updated.

9. The system according to claim 8, wherein The use of the to calculate the candidate hidden state , and update to obtain the hidden state at the current time step , including: The hidden state of the previous time step and the corrected reset gate are multiplied element-wise to obtain the reset historical information; Multiply the reset historical information by the first trainable weight matrix, and multiply the input at the current time step by the second trainable weight matrix, add the two multiplication results and the trainable bias vector to obtain an intermediate result, and apply the hyperbolic tangent activation function to the intermediate result to obtain the candidate hidden state ; Calculate the update gate for the current time step , multiply the hidden state of the previous time step element-wise with to obtain the part of the historical state to be retained; multiply the candidate hidden state element-wise with the update gate to obtain the part of the new information to be updated; add the part of the historical state to be retained and the part of the new information to be updated element-wise to obtain the hidden state of the current time step .

10. The system according to claim 7, characterized in that, The attention layer obtains the timing features by weighting the features of different time steps, including: Input the hidden state sequence into the time-step attention layer, which calculates the attention scores of each time-step hidden state through a fully connected layer with a tanh activation function and normalizes all scores into attention weights . Weight-sum the hidden state sequence according to the corresponding attention weights to obtain the parallel-branch temporal features.

Images