Equipment health state identification method based on gradient dynamic integration

Abstract

The invention discloses an equipment health state recognition method based on gradient dynamic integration. The method comprises the following steps: acquiring a state data set of target equipment; optimizing parameters of the multi-mode recognizer are initialized, feature extraction and classification are carried out, and multi-mode joint loss and single-mode auxiliary loss are obtained; in each optimization iteration, gradients generated by the two losses respectively are calculated, and direction conflict is evaluated; according to an evaluation result, executing corresponding dynamic gradient integration to generate an integrated gradient; optimizing parameters of the multi-mode recognizer are updated; iteration is stopped until a preset condition is met, and the multi-mode recognizer with updated parameters is obtained; and performing application detection on the equipment of which the health state needs to be detected by adopting the multi-mode identifier with the updated parameters to obtain an identification result corresponding to the health state. According to the method, the gradient conflict problem in multi-modal learning is solved, optimization and convergence of the multi-modal recognizer are effectively guided, and the recognition precision and robustness are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent operation and maintenance of mechanical equipment, and more particularly to a device health state recognition method based on gradient dynamic integration. BACKGROUND

[0002] Mechanical equipment is the core of modern industrial production, and its health state is directly related to the stability and economic benefits of the entire production system. In actual operation, the equipment is inevitably subjected to fatigue, wear and other health problems under complex working conditions of high load, variable load and strong vibration. Once a fault occurs, it will not only cause costly equipment downtime and production interruption, but also may trigger a chain reaction, threatening personnel safety and production environment. The traditional equipment operation and maintenance mode mainly relies on manual inspection and regular maintenance. This mode has a lagging response and is difficult to capture early signs of failure, resulting in high operation and maintenance costs and low efficiency. With the advancement of artificial intelligence technology, health state intelligent recognition methods based on deep learning have been widely applied. Such methods can automatically learn complex fault features from massive amounts of monitoring data, overcoming the limitations of traditional methods that rely on expert experience and signal processing knowledge, and achieving accurate and real-time evaluation of equipment health state, providing strong technical support for predictive maintenance and significantly improving the reliability and safety of equipment operation.

[0003] With the increasing complexity of mechanical equipment structure and operating environment, traditional diagnosis methods based on a single signal source have shown obvious limitations. Due to the inherent one-sidedness and vulnerability of single modal information, such methods not only have difficulty in fully describing complex fault mechanisms, but also have poor robustness in strong noise environments and face the risk of single-point failure. To overcome the above bottlenecks, multi-modal health state intelligent recognition methods have emerged, which complementarily capture equipment state information from different physical dimensions by fusing multi-source sensor data, significantly improving the accuracy and robustness of recognition.

[0004] However, in the joint optimization process of multi-modal recognizers, the multi-modal joint learning objective and the single-modal auxiliary objective will conflict in direction during optimization, i.e., "gradient conflict". To address this multi-task optimization conflict problem, existing methods usually focus on adjusting the loss weight or gradient norm, but these methods do not fundamentally solve the inherent contradiction of gradient direction conflict. For multi-modal learning, traditional multi-objective optimization algorithms often fail to achieve ideal results when dealing with such tasks with significant learning difficulty differences, and even may harm the final performance of the recognizer.

[0005] Therefore, it is a problem that needs to be solved by those skilled in the art to propose a multi-modal intelligent recognition method based on gradient dynamic optimization and solve the gradient conflict problem in multi-modal learning. SUMMARY

[0006] In view of the above problems, this invention is proposed to provide a device health status identification method based on gradient dynamic integration to overcome or at least partially solve the above problems. By dynamically evaluating and integrating the direction and magnitude of gradient updates, the gradient conflict problem in multimodal learning is resolved, thereby improving the generalization ability and final identification accuracy of the recognizer.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: This invention provides a device health status identification method based on gradient dynamic integration, comprising: S1. Obtain the status dataset of the target device under various historical operating conditions; the status dataset includes multiple sets of multimodal data; each set of multimodal data includes: vibration, sound, current and rotational speed data, as well as the device health label corresponding to the set of modal data; S2. Initialize the optimizable parameters of the multimodal recognizer; use the multimodal recognizer to extract features and classify the multimodal data to obtain the multimodal joint loss and the auxiliary loss of each single mode; S3. In each optimization iteration, calculate the gradients generated by the multimodal joint loss and each single-mode auxiliary loss, and evaluate the directional conflict between the multimodal gradient and the single-mode gradient. S4. Based on the evaluation results of directional conflict, perform the corresponding dynamic gradient ensemble to generate the ensemble gradient; S5. Update the optimizable parameters of the multimodal recognizer according to the integrated gradient and re-extract and classify the state dataset; stop iterating until a preset condition is met, and obtain the multimodal recognizer with updated parameters; S6. The updated multimodal recognizer is used to perform application detection on the device whose health status needs to be detected, and the corresponding health status recognition result is obtained.

[0008] Furthermore, in step S1, the state dataset is expressed by the formula:

[0009]

[0010] Where Xi represents the first One optimized sample, Indicates the first The binary classification category labels corresponding to each optimized sample This indicates the total number of optimized samples; Indicates the first The first sample Modal data, This represents the total number of multimodal data points included in each sample.

[0011] Further, in step S2, the multi-modal identifier comprises: one single-modal feature extraction module, one feature fusion module and one decision module; Each single-modal feature extraction module extracts single-modal raw data in each sample to obtain a high-dimensional feature vector, which is expressed by the formula as:

[0012] The feature fusion module integrates all single-modal high-dimensional feature vectors into a single fusion feature vector , which is expressed by the formula as:

[0013] The decision module maps the fusion feature vector into a prediction probability vector of the health state , which is expressed by the formula as:

[0014] wherein, is the feature vector of the i-th modality, is the feature extraction module network of the i-th modality, is the i-th modality, is the i-th modality, is the corresponding optimizable parameter; is the fusion feature vector, is the fusion function, is the corresponding optimizable parameter of the fusion module; is the prediction probability vector output by the multi-modal identifier, is the decision module network, is the corresponding optimizable parameter. Further, in step S2, the multi-modal joint loss is a main task loss of classification after modal fusion , which is used to supervise the overall health state classification of the device;

[0015] The single-modal auxiliary loss is a supervision loss of individual classification of each modality , which is used to supervise the enhanced feature learning of each modality branch; The loss functions of the multi-modal joint loss and the single-modal auxiliary loss are both cross-entropy loss functions, and the mathematical formula is:

[0016] wherein, is the total number of categories; ​​is a symbol function, is the true class of the sample 1 or 0; is the probability that the sample belongs to class predicted by the multi-modal recognizer.

[0017] Further, in step S3, the gradient generated by the multi-modal joint loss and each single-modal auxiliary loss is calculated in each optimization iteration, specifically comprising: In the first iteration, for the parameters shared with other modalities in the m-th modal feature extraction module, the gradient is calculated by the back propagation algorithm; the gradient generated by the multi-modal joint loss is expressed as: The gradient generated by the single-modal auxiliary loss is expressed as:

[0018]

[0019] wherein, is the gradient operator of the parameter ; and are the predicted outputs of the multi-modal and single-modal, respectively; is the true label.

[0020] Further, in step S3, the direction conflict between the multi-modal gradient and the single-modal gradient is evaluated, specifically comprising: The direction conflict between the multi-modal and single-modal gradient vectors and is quantified by calculating the cosine value of the included angle between them, which is expressed as:

[0021] wherein, denotes the dot product of two gradient vectors; denotes the norm of a vector.

[0022] Further, the step S4 specifically comprises: If , it is determined as no conflict; the integrated gradient is obtained by equal-weight addition as:

[0023] ​​​If , it is determined as a conflict case; the directions and magnitudes of the two gradients are decoupled and reconstructed, and an integrated gradient is obtained .

[0024] Further, the directions and magnitudes of the two gradients are decoupled and reconstructed, and an integrated gradient is obtained ; specifically comprising: A common descending direction beneficial to both conflict targets is found; weights are assigned to the two gradients and , and the weights and are solved to minimize the norm of the combined gradient, which is expressed by the formula as:

[0025] The gradient magnitude is reconstructed to obtain the integrated gradient , which is expressed by the formula as:

[0026] wherein, and are the weights of the multi-modal gradient and the single-modal gradient to be solved, respectively.

[0027] Further, in step S5, the optimizable parameters of the multi-modal recognizer are updated according to the integrated gradient, specifically comprising: Based on the integrated gradient, the optimizable parameters of the multi-modal recognizer are updated by using the gradient descent method, which is expressed by the formula as:

[0028] wherein, and are the , and parameters of the multi-modal recognizer at the th and the th iteration, respectively.

[0029] According to the above technical solution, compared with the prior art, the present application provides a device health state recognition method based on gradient dynamic integration, which has the following beneficial effects: The application fundamentally solves the conflict and interference of the multi-modal and single-modal targets in the optimization process by evaluating the gradient direction between the multi-modal and single-modal targets in real time, and ensures the stability and efficiency of the multi-modal recognizer optimization. The direction and value of the gradient update are decoupled and reconstructed innovatively, the noise intensity of the stochastic gradient descent is strategically enhanced while ensuring the common descending direction, the multi-modal recognizer is guided to converge to a flat minimum value region with stronger generalization ability, and the health state recognition precision and robustness of the recognizer under complex working conditions are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0031] Figure 1 A device health state recognition method flow chart based on gradient dynamic integration provided in the embodiments of the present application; Figure 2 A smart recognition method framework diagram based on gradient dynamic optimization provided in the embodiments of the present application; Figure 3 A smart recognition method schematic diagram based on gradient dynamic optimization provided in the embodiments of the present application; Figure 4 A performance comparison diagram of the method of the present application and the existing multi-modal learning optimization method provided in the embodiments of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly and completely in the embodiments of the present application combined with the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0033] The embodiments of the present application disclose a device health state recognition method based on gradient dynamic integration, as shown in Figure 1 The method comprises the following steps. S1, obtaining state data sets of a target device under various historical working conditions; the state data set comprises a plurality of groups of multi-modal data; each group of multi-modal data comprises vibration, sound, current and speed data, and a device health label corresponding to the group of modal data; S2, initialize the optimizable parameters of the multi-modal recognizer; perform feature extraction and classification on the multi-modal data using the multi-modal recognizer to obtain a multi-modal joint loss and each single-modal auxiliary loss; S3, in each optimization iteration, calculate the gradients generated by the multi-modal joint loss and each single-modal auxiliary loss respectively, and evaluate the directional conflict between the multi-modal gradient and the single-modal gradient; S4, according to the evaluation result of the directional conflict, perform corresponding dynamic gradient integration to generate an integrated gradient; S5, update the optimizable parameters of the multi-modal recognizer according to the integrated gradient and re-perform feature extraction and classification on the state data set; stop iteration until a preset condition is met to obtain a multi-modal recognizer with updated parameters; S6, apply the multi-modal recognizer with updated parameters to the equipment whose health status needs to be detected to obtain the recognition result of the corresponding health status.

[0034] The embodiment is applied to a certain factory, and the health status of a large numerical control machine tool on the production line needs to be monitored to find problems in time and perform maintenance. First, a numerical control machine tool equipment health status data set is constructed, and a multi-task type recognition framework including a multi-modal joint loss and a single-modal auxiliary loss is constructed as a multi-modal recognizer. Second, the parameters of the multi-modal recognizer for feature extraction and classification are initialized. In each optimization iteration, the gradients generated by the multi-modal joint loss and each single-modal auxiliary loss are calculated. The directional conflict between the multi-modal gradient and the single-modal gradient is evaluated. According to the evaluation result of the directional conflict, a corresponding dynamic gradient integration strategy is executed to generate a final integrated gradient. Finally, the network parameters are updated using the integrated gradient, and the iteration process of forward propagation, loss calculation and parameter update is repeated until a preset convergence condition is met to stop iteration, and a multi-modal recognizer with updated parameters is obtained. During the model iteration optimization process, the directional consistency between the gradients is evaluated in real time, and a differentiated integration strategy is used to decouple and reconstruct the direction and value of the gradient according to the directional consistency, so that the auxiliary learning of the single-modal can harmlessly promote the main task, thereby improving the generalization ability and recognition accuracy of the recognizer. When monitoring the current health status of the machine tool equipment, the current state data of the machine tool equipment is obtained, and the above multi-modal recognizer with updated parameters is used for feature extraction, fusion and judgment to obtain the recognition result of the current health status of the equipment.

[0035] The implementation process of the embodiment will be described in detail below, including the construction, optimization and application of the multi-modal recognizer.

[0036] For the construction of the multi-modal recognizer, refer to Figure 2As shown, firstly, following step S1, the status dataset of the target device under various historical operating conditions is obtained. In this embodiment, the multimodal data of each sample in the status dataset includes: vibration, sound, current, and rotational speed data, as well as the device health tag corresponding to that set of modal data.

[0037] This embodiment defines a multimodal device health status dataset. .

[0038] in, Indicates the first Each sample contains optimized samples. Data for each modality, For the first The first sample Modal data; The category label corresponding to this sample is represented by a One-Hot vector in the multimodal health category identification task, which is a binary vector with a length equal to the total number of categories. To optimize the total number of samples.

[0039] This embodiment also constructs a multi-task recognition framework that includes multimodal joint loss and single-modal auxiliary loss as a multimodal recognizer.

[0040] Overall loss function of the identifier By a multimodal joint loss and Single-mode auxiliary loss Together they constitute the following, expressed by the formula:

[0041] in, Let the overall loss function of the identifier be , For multimodal joint loss, For the first Single-mode auxiliary loss for each mode, This represents the total number of modes.

[0042] For the health status identification classification task in this embodiment, all loss functions adopt the cross-entropy loss function, and its mathematical formula is:

[0043] in, Total number of categories; For the sign function, if the true class of the sample equals The value is 1 if it is 1, otherwise it is 0. Predict the category to which a sample belongs for the recognizer The probability of.

[0044] The multimodal recognizer constructed in this embodiment consists of a set of single-modal feature extraction modules. A feature fusion module and a decision module Composition. To facilitate low-level feature learning and knowledge sharing among different modalities, the shallow network parameters of some or all single-modal feature extraction modules are set to cross-modal shared parameters.

[0045] Each single-modal feature extraction module in this embodiment Responsible for obtaining raw data from the corresponding single modality. Extracting high-dimensional feature vectors This process can be represented as:

[0046] in, For the first Feature vectors of each modality For the first A feature extraction module network for each modality, These are the corresponding optimizable parameters.

[0047] The feature fusion module in this embodiment All single-modal feature vectors are combined using post-processing fusion methods such as splicing. Integrate into a single fusion feature vector This process can be represented as:

[0048] in, The fused feature vector For fusion function, These are the optimizable parameters that may be included in the fusion module.

[0049] Finally, the determination module of this embodiment fuse feature vectors Mapped to a predicted probability vector of health status This process can be represented as:

[0050] in, This is the predicted probability vector output by the recognizer. To determine the module network, These are the parameters that can be optimized.

[0051] For optimization of the multimodal recognizer, refer to... Figure 3 As shown, the optimizable parameters of the multimodal recognizer are first initialized according to S2.

[0052] For the optimizable parameters , and , a standard random initialization method (He initialization) is used to assign small random values; the method specifically samples from a Gaussian distribution with mean 0 and variance ( is the number of input nodes) to maintain the consistency of the output variance of each layer of the network, thereby establishing the initial parameter state, preventing gradient vanishing or explosion in deep networks, and avoiding the problem of network symmetry failure caused by all-zero initialization.

[0053] Secondly, a multi-modal recognizer is used to extract features and classify multi-modal data in the state data set, obtaining a multi-modal joint loss.

[0054] At the same time, the multi-modal recognizer is used to extract features and classify each single modal data in the multi-modal data, obtaining each single modal auxiliary loss. Specifically, after each single modal feature extraction module, an auxiliary single modal classifier is connected to input the extracted high-dimensional feature vector into the auxiliary classifier to obtain a single modal prediction result . Since the single modal data and the multi-modal data both represent the state of the same device at the same time, the real labels in the data set are shared by each single modal branch, and by calculating the cross-entropy loss between and , the single modal auxiliary loss for supervising independent feature learning of each modal can be obtained.

[0055] Next, according to S3, in each optimization iteration, the gradients generated by the multi-modal joint loss and each single modal auxiliary loss are calculated, and the directional conflict between the multi-modal gradient and the single modal gradient is evaluated.

[0056] The multi-modal recognizer of the present embodiment uses a stochastic gradient descent method based on mini-batch for optimization. In the first iteration, for the parameters shared by the modal feature extraction module and other modalities, the gradient is calculated by the back propagation algorithm. The gradient generated by the multi-modal joint loss and the gradient generated by the corresponding single modal auxiliary loss are respectively represented as:

[0057]

[0058] wherein represents the parameter the gradient operator of and are the prediction outputs of multi-modal and single-modal, respectively; is the true label.

[0059] This embodiment quantifies the directional conflict between two gradient vectors and by calculating the cosine value of the included angle between them. The calculation formula is:

[0060] where denotes the dot product of two gradient vectors; denotes the norm of a vector. If , it is determined that there is no conflict; if , it is determined that there is a conflict.

[0061] Then, according to S4, the integrated gradient is generated according to the evaluation result of the directional conflict.

[0062] The gradient integration strategy in the non-conflict case in this embodiment. When , it indicates that the directions of the two gradients are basically consistent. To enhance the random gradient descent noise in the optimization process to improve the generalization ability of the recognizer, the equal-weighted addition method is directly used for integration.

[0063] The final integrated gradient is defined as the sum of the two gradients: .

[0064] The gradient integration strategy in the conflict case in this embodiment. When , it indicates that there is a conflict in the gradient direction, and the gradient needs to be decoupled and reconstructed in terms of direction and magnitude. The purpose of this strategy is to project the conflicting gradient onto a common descent direction that is beneficial to both targets and restore its proper update strength to ensure the stability and efficiency of the optimization. This strategy specifically includes: 1. Solving the balanced common descent direction.

[0065] To find a common descent direction that is beneficial to both conflicting targets, a balanced optimization problem needs to be solved, that is, to find weights and to minimize the norm of the combined gradient:

[0066] where and are the weights of the multi-modal gradient and the single-modal gradient respectively. The Pareto optimization is performed, and for this optimization problem, it is essentially to find the vector with the minimum norm in the convex hull of the two gradient vectors. According to the convex optimization theory, this problem belongs to the quadratic programming problem, and there is a unique closed-form solution. The combined vector solved is not only the mathematical optimal solution, but in the physical sense, it represents the steepest common descent direction that can simultaneously meet the requirements of the descent of the two loss functions, thereby effectively avoiding the oscillation caused by gradient competition in multi-task learning.

[0067] 2. Reconstruct the gradient magnitude to maintain the optimization intensity.

[0068] Directly using the combined gradient solved in the last step will reduce the update step due to the decrease of the norm of the gradient, which may cause the optimization stagnation. To solve this problem, the present application reconstructs the gradient magnitude to restore it to the reference value under the non-conflict scenario, i.e., the norm of the combined gradient is equal to the norm of the single-modal gradient, to ensure sufficient optimization exploration ability. The significance of this step is to decouple the "direction" and "magnitude" of the gradient, and to retain the update intensity while correcting the direction. The final update of the integrated gradient is calculated as follows:

[0069] wherein, is the final integrated gradient vector used for updating. To further enhance the generalization performance, the present embodiment also introduces a scaling factor greater than 1. The final gradient is adjusted as follows: wherein is the scaling factor, which is a preset hyperparameter, and its specific value can be determined by experiments on the validation set.

[0070] Finally, according to S5, the multi-modal recognizer is updated according to the integrated gradient, which specifically includes: The gradient descent method is used to update the network parameters that can be optimized:

[0071] wherein, and are the sets of the parameters that can be optimized of the multi-modal recognizer at the th and the th iteration, which specifically include the aforementioned feature extraction parameters , fusion parameters and decision parameters . The formula indicates that the calculated integrated gradient is used to update the parameters that can be optimized of the multi-modal recognizer, combined with the learning rate All network weights at the current moment are corrected in reverse, so that the model parameters move to the flat minimum area of the loss function surface, thereby reducing the training loss and improving the generalization ability of the model.

[0072] After parameter optimization, the state data is re-extracted and classified, and the iteration is stopped until the preset maximum iteration number is reached. The preset iteration number is determined according to the statistical results of the model reaching the loss convergence standard in the previous experiment, that is, when the loss value fluctuation amplitude is continuously lower than the preset threshold, the iteration round corresponding to the fixed stop condition is set to ensure that the model reaches the optimal convergence state while considering the calculation efficiency, and finally the multi-modal recognizer after parameter update is obtained.

[0073] The embodiment solves the gradient conflict problem in multi-modal learning by dynamically evaluating and integrating the direction and magnitude of gradient update, and improves the generalization ability and final recognition accuracy of the recognizer.

[0074] After the multi-modal recognizer is optimized, the embodiment applies the multi-modal recognizer to identify the current health status of the device according to S6.

[0075] At a certain monitoring moment, a segment of time series data of the device is synchronously collected as state data to be identified, and is input into the corresponding branches of the trained multi-modal recognizer respectively. Each branch shares the network structure and parameters determined in the training stage, and outputs a deep feature vector of fixed dimension. The multi-modal features are spliced through a feature fusion module to obtain a joint feature representation and input into a judgment module to obtain a prediction result of the current health status of the device.

[0076] The recognition process of the embodiment is based on the trained multi-modal recognizer, which realizes real-time, accurate and robust intelligent discrimination of the current running state of the device by synchronously collecting multi-source signals, parallel feature extraction, fusion classification and outputting the health status, thereby providing a key decision basis for predictive maintenance.

[0077] Finally, the health state evaluation of the electromechanical hybrid transmission system is taken as an example to verify the method. The experimental platform is built according to the ratio of 1:2, and different rotating speeds are applied by using a three-phase asynchronous motor, and different load conditions are simulated by using a hydraulic loading device. The data acquisition covers 24 channels, the sampling frequency is 64 kHz, and the channel types include three-axis vibration, three-phase current, rotating speed and sound. The experimental health state settings include five types of motors (normal state, bearing fault, short circuit, rotor bending and rotor broken bar). There are 9 kinds of operating conditions. 100 long sequence samples are collected for each type and each condition, and the sampling length is 640,000. The method proposed in the present application is used to identify various health states, and compared with four existing multi-modal learning optimization methods (GradNorm, PCGrad, IMTL, DWA). In order to ensure the fairness of comparison, all methods use the same feature extraction network structure and basic hyperparameter configuration, and the difference is only in the loss function or gradient optimization strategy. The accuracy (ACC), average precision mean (MAP) and macro F1 score (Macro-F1) are used as the evaluation indexes of the identifier performance. The basic structure of the feature extraction network used by each method is shown in Table 1, and the unified hyperparameter setting is shown in Table 2. The final experimental results are shown in Table 3.

[0078] Table 1 Summary of basic structure of feature extraction network

[0079] Table 2 Summary of hyperparameters

[0080] Table 3 Accuracy summary table in the final stage

[0081] The experimental results are compared with those shown in Figure 4 The health state recognition effect of the method is significantly better than that of the four comparison methods, which is due to the unique "direction decoupling and value reconstruction" strategy. The strategy not only resolves the conflict of the gradient direction, but also maintains the effective optimization exploration intensity, so that the identifier can converge to the optimal solution with stronger generalization performance. The experimental results fully demonstrate the effectiveness and advancement of the method in improving the accuracy of multi-modal electromechanical equipment health state recognition.

[0082] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between each embodiment can be referred to each other.

[0083] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying device health status based on gradient dynamic integration, characterized in that, include: S1. Obtain the status dataset of the target device under various historical operating conditions; The state dataset includes multiple sets of multimodal data; Each set of multimodal data includes: vibration, sound, current and rotational speed data, as well as the device health tag corresponding to that set of modal data; S2. Initialize the optimizable parameters of the multimodal recognizer; use the multimodal recognizer to extract features and classify the multimodal data to obtain the multimodal joint loss and the auxiliary loss of each single mode; S3. In each optimization iteration, calculate the gradients generated by the multimodal joint loss and each single-mode auxiliary loss, and evaluate the directional conflict between the multimodal gradient and the single-mode gradient. S4. Based on the evaluation results of directional conflict, perform the corresponding dynamic gradient ensemble to generate the ensemble gradient; S5. Update the optimizable parameters of the multimodal recognizer according to the integrated gradient and re-extract and classify the state dataset; stop iterating until a preset condition is met, and obtain the multimodal recognizer with updated parameters; S6. The updated multimodal recognizer is used to perform application detection on the device whose health status needs to be detected, and the corresponding health status recognition result is obtained.

2. The method as described in claim 1, characterized in that, In step S1, the state dataset is expressed by the formula: in, X i Indicates the first One optimized sample, Indicates the first The binary classification category labels corresponding to each optimized sample This indicates the total number of optimized samples; Indicates the first The first sample Modal data, This represents the total number of multimodal data points included in each sample.

3. The method as described in claim 2, characterized in that, In step S2, the multimodal recognizer includes: One single-modal feature extraction module, one feature fusion module, and one decision module; Each single-modal feature extraction module extracts the original data of a single modality from each sample, obtaining a high-dimensional feature vector; expressed by the formula: The feature fusion module will combine all single-modality high-dimensional feature vectors Integrate into a single fusion feature vector It can be expressed by the formula: The determination module The fused feature vector Mapped to a predicted probability vector of health status It can be expressed by the formula: in, For the first Feature vectors of each modality For the first A feature extraction module network for each modality, for Corresponding optimizable parameters; The fused feature vector For fusion function, These are the optimizable parameters corresponding to the fusion module; This is the predicted probability vector output by the multimodal recognizer. To determine the module network, for The corresponding optimizable parameters.

4. The method as described in claim 2, characterized in that, In step S2, the multimodal joint loss is the main task loss classified after mode fusion. Used to classify and monitor the overall health status of equipment; The single-mode auxiliary loss is a supervised loss that is classified separately for each mode. This is used to supervise the learning of enhanced features in each modality branch; The loss functions for both the multimodal joint loss and each single-mode auxiliary loss are the cross-entropy loss function, and its mathematical formula is as follows: in, Total number of categories; For symbolic functions, The sample represents the true class 1 or 0; Predicting the category to which a sample belongs for a multimodal recognizer The probability of.

5. The method as described in claim 1, characterized in that, In step S3, calculating the gradient generated by the multimodal joint loss and each single-mode auxiliary loss in each optimization iteration specifically includes: In the In the nth iteration, for the th Parameters shared with other modalities in each modality feature extraction module The gradient is calculated using the backpropagation algorithm; the multimodal joint loss The generated gradient It can be expressed by the formula: The single-mode auxiliary loss The generated gradient It can be expressed by the formula: in, For parameters The gradient operator; and These are the prediction outputs for multimodal and unimodal modes, respectively. This is a real label.

6. The method as described in claim 5, characterized in that, Step S3, specifically evaluating the directional conflict between multimodal gradients and single-modal gradients, includes: By calculating the gradient vectors of multimodal and unimodal modes and The angle between The cosine value, used to quantify the directional conflict between gradients, is expressed by the formula: in, This represents the dot product of two gradient vectors; Representing vectors Norm.

7. The method as described in claim 6, characterized in that, Step S4 specifically includes: like If the result is negative, it is considered a conflict-free case; then, the integration is performed using an equal-weighted summation method to obtain the integration gradient. for: like If the two gradients are not in conflict, then it is considered a conflict situation; the directions and magnitudes of the two gradients are decoupled and reconstructed to obtain the integrated gradient. .

8. The method as described in claim 7, characterized in that, The process involves decoupling and reconstructing the directions and magnitudes of the two gradients to obtain the integrated gradient. Specifically, it includes: Find a common descent direction that benefits both conflicting objectives; assign weights to the two gradients. and And solve for the weights and To minimize the combined gradient Norm, expressed by the formula: Reconstruct the gradient magnitudes to obtain the integrated gradient. It can be expressed by the formula: in, and These are the weights of the multimodal gradient and the single-modal gradient to be solved, respectively.

9. The method as described in claim 8, characterized in that, In step S5, updating the optimizable parameters of the multimodal recognizer based on the integrated gradient specifically includes: Based on the integrated gradient, the optimizable parameters of the multimodal recognizer are updated using the gradient descent method, as expressed by the following formula: in, and The multimodal recognizer is in the first... Second and third During the next iteration , and parameter.

Images