Motor defect identification and compensation method and device based on reinforcement learning, equipment and storage medium

Abstract

The application discloses a motor defect identification and compensation method and device based on reinforcement learning, equipment and a storage medium, relates to the technical field of motor fault diagnosis, and comprises the following steps: acquiring electrical parameter information, vibration information, temperature information, acoustic information and visual information; determining corresponding space-time composite representation vectors based on the electrical parameter information, the vibration information, the temperature information, the acoustic information and the visual information; inputting the space-time composite representation vectors into a reinforcement learning model to predict corresponding motor defects and compensation strategies, determine defect prediction results and compensation strategy information, the reinforcement learning model comprises a policy network and a value network, and the policy network and the value network are obtained by adjusting parameters of the policy network and the value network according to a reward function corresponding to the space-time composite representation vectors; and compensating defects corresponding to the defect prediction results according to the compensation strategy information to generate motor defect compensation results.

Description

Technical Field

[0001] This application relates to the field of motor fault diagnosis technology, and in particular to a method, apparatus, device and storage medium for motor defect identification and compensation based on reinforcement learning. Background Technology

[0002] As a core power source in modern industrial systems, the reliability of electric motors directly impacts production safety and efficiency. Failure to promptly identify and compensate for motor defects such as bearing wear, winding short circuits, and rotor demagnetization will lead to decreased equipment performance, increased energy consumption, unplanned downtime, and significant economic losses. Therefore, accurate diagnosis and adaptive compensation of motor operating conditions are essential.

[0003] Currently, existing methods employ offline diagnostic models based on statistical models and machine learning, adjusting them using preset PID parameters or feedforward control strategies based on fixed models. However, these methods, based on a single signal source, struggle to comprehensively capture the multifaceted characteristics of complex defects. They have limited capabilities in fusing multi-source heterogeneous data and modeling spatiotemporal correlations, and lack adaptive and collaborative capabilities. Consequently, diagnostic results cannot directly and intelligently drive the optimization of compensation strategies. Therefore, how to more accurately and effectively identify and compensate for motor defects has become an urgent problem to be solved.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a method, apparatus, device, and storage medium for motor defect identification and compensation based on reinforcement learning, aiming to solve the technical problem of how to more accurately and effectively identify and compensate for motor defects.

[0006] To achieve the above objectives, this application proposes a method for motor defect identification and compensation based on reinforcement learning, the method comprising: Acquire electrical parameter information, vibration information, temperature information, acoustic information, and visual information; The corresponding spatiotemporal composite representation vector is determined based on the electrical parameter information, the vibration information, the temperature information, the acoustic information, and the visual information. The spatiotemporal composite representation vector is input into the reinforcement learning model to predict the corresponding motor defects and compensation strategies, and to determine the defect prediction results and compensation strategy information. The reinforcement learning model includes a policy network and a value network, which are obtained by adjusting the parameters of the policy network and the value network according to the reward function corresponding to the spatiotemporal composite representation vector. The defects corresponding to the defect prediction results are compensated according to the compensation strategy information to generate motor defect compensation results.

[0007] In one embodiment, the step of determining the corresponding spatiotemporal composite representation vector based on the electrical parameter information, the vibration information, the temperature information, the acoustic information, and the visual information includes: Based on the electrical parameter information, vibration information, temperature information, acoustic information, and visual information, time-domain features, frequency-domain features, time-frequency features, and image features are extracted; Multi-head attention is used to analyze the dependencies between the temporal features, the frequency features, the time-frequency features, and the image features to obtain dependency information; A graph attention network is used to construct a spatial topology graph from the spatial topological relationships between different sensor measurement points corresponding to the dependency information. The spatiotemporal composite representation vector is obtained by weighted summation of the feature vectors corresponding to each central node and neighboring nodes in the spatial topology graph.

[0008] In one embodiment, the step of using multi-head attention to resolve the dependencies between the temporal features, the frequency features, the time-frequency features, and the image features to obtain dependency information includes: Multi-head attention is used to analyze the dependencies between the time-domain features, the frequency-domain features, the time-frequency features, and the image features to determine the query vector, key vector, and value vector. Calculate attention weights based on the query vector, the key vector, and the value vector; The attention weights are used to aggregate features at different time steps to obtain dependency information.

[0009] In one embodiment, the step of obtaining a spatiotemporal composite representation vector by weighted summation of the feature vectors corresponding to each central node and neighboring nodes in the spatial topology graph includes: Calculate the unnormalized attention coefficient based on each central node and neighboring node in the spatial topology graph; The attention coefficients are normalized to obtain normalized attention weights; Based on the normalized attention weights, the feature vectors corresponding to the neighbor nodes in the spatial topology graph are weighted and summed to determine the weighting value; The feature vector corresponding to the central node in the spatial topology graph is updated using the weighted value to obtain the spatiotemporal composite representation vector.

[0010] In one embodiment, the step of inputting the spatiotemporal composite representation vector into the reinforcement learning model to predict the corresponding motor defects and compensation strategies, and determining the defect prediction results and compensation strategy information, includes: Obtain historical fault data; The historical fault data and the spatiotemporal composite representation vector are input into the reinforcement learning model for training to determine the target reinforcement learning model. The spatiotemporal composite representation vector is input into the target reinforcement learning model to predict the corresponding motor defects and compensation strategies, thereby obtaining defect prediction results and compensation strategy information. The defect prediction results include emergency defects, moderate defects, and minor defects. The compensation strategy information includes reducing power or shutting down the machine, adjusting control parameters, and recording defect trends.

[0011] In one embodiment, the step of inputting the historical fault data and the spatiotemporal composite representation vector into a reinforcement learning model for training, and determining the target reinforcement learning model, includes: The historical fault data is input into the reinforcement learning model to train the policy network and value network with the corresponding classification task and state evaluation as the objectives, and the initial reinforcement learning model is determined. Based on the spatiotemporal composite representation vector, corresponding decision information and compensation state information are generated; The instant reward is calculated based on the compensation status information and the preset reward function; The spatiotemporal composite representation vector, the decision information, the compensation state information, and the instant reward are integrated in the form of quadruples to determine the interaction experience sample; The policy network and value network of the initial reinforcement learning model are updated based on the interaction experience samples to obtain the target reinforcement learning model.

[0012] In one embodiment, the step of updating the policy network and value network of the initial reinforcement learning model based on the interaction experience samples to obtain the target reinforcement learning model includes: Sampling priorities are dynamically allocated based on the time difference error corresponding to the interactive experience samples. The sampling batch of empirical data is determined based on the sampling priority. The policy network and value network of the initial reinforcement learning model, which are based on the gradient loss function of the sampled batch of empirical data and satisfy a predefined learning rate, are updated using the gradient descent algorithm to obtain the target reinforcement learning model.

[0013] Furthermore, to achieve the above objectives, this application also proposes a reinforcement learning-based motor defect identification and compensation device, which includes: The acquisition module is used to acquire electrical parameter information, vibration information, temperature information, acoustic information, and visual information. The processing module is used to determine the corresponding spatiotemporal composite representation vector based on the electrical parameter information, the vibration information, the temperature information, the acoustic information, and the visual information; The processing module is also used to input the spatiotemporal composite representation vector into the reinforcement learning model to predict the corresponding motor defects and compensation strategies, and to determine the defect prediction results and compensation strategy information. The reinforcement learning model includes a policy network and a value network, which are obtained by adjusting the parameters of the policy network and the value network according to the reward function corresponding to the spatiotemporal composite representation vector. The execution module is used to compensate for the defects corresponding to the defect prediction results according to the compensation strategy information, and generate motor defect compensation results.

[0014] Furthermore, to achieve the above objectives, this application also proposes a reinforcement learning-based motor defect identification and compensation device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the reinforcement learning-based motor defect identification and compensation method described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the reinforcement learning-based motor defect identification and compensation method described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: This embodiment proposes a motor defect identification and compensation method based on reinforcement learning. It acquires electrical parameter information, vibration information, temperature information, acoustic information, and visual information. Based on these information, a corresponding spatiotemporal composite representation vector is determined. This vector is then input into a reinforcement learning model to predict the corresponding motor defect and compensation strategy, determining the defect prediction result and compensation strategy information. The reinforcement learning model includes a policy network and a value network, whose parameters are adjusted according to the reward function corresponding to the spatiotemporal composite representation vector. Finally, the defect corresponding to the defect prediction result is compensated according to the compensation strategy information, generating a motor defect compensation result. This application integrates information from multiple heterogeneous sensors to construct a corresponding spatiotemporal composite representation vector, which is then input into a reinforcement learning model. The reward function drives the online collaborative optimization of the policy network and value network, simultaneously outputting defect types and adaptive compensation strategies. Based on these adaptive compensation strategies, hierarchical compensation control is executed in real time, achieving integrated operation from comprehensive perception and intelligent decision-making to precise compensation. This significantly improves the detection rate of early, minor faults. Furthermore, through continuous learning, the system acquires the ability to adapt to changes in operating conditions and defect evolution, effectively improving the reliability, lifespan, and energy efficiency of the motor. This promotes a shift in motor health management from passive maintenance to proactive early warning and intelligent maintenance. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the reinforcement learning-based motor defect identification and compensation method of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the reinforcement learning-based motor defect identification and compensation method of this application. Figure 3 This is a schematic diagram of the module structure of the motor defect identification and compensation device based on reinforcement learning according to an embodiment of this application; Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the reinforcement learning-based motor defect identification and compensation method in the embodiments of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of this application embodiment is as follows: Acquire electrical parameter information, vibration information, temperature information, acoustic information, and visual information; determine the corresponding spatiotemporal composite representation vector based on the electrical parameter information, vibration information, temperature information, acoustic information, and visual information; input the spatiotemporal composite representation vector into a reinforcement learning model to predict the corresponding motor defects and compensation strategies, and determine the defect prediction result and compensation strategy information. The reinforcement learning model includes a policy network and a value network, which are obtained by adjusting the parameters of the policy network and the value network according to the reward function corresponding to the spatiotemporal composite representation vector; compensate for the defects corresponding to the defect prediction result according to the compensation strategy information to generate a motor defect compensation result.

[0024] In this embodiment, for ease of description, the following description will focus on identifying a reinforcement learning-based motor defect identification and compensation device.

[0025] Because existing technologies based on single signal sources are unable to fully capture the multi-dimensional representation of complex defects, have limited ability to fuse multi-source heterogeneous data and model spatiotemporal correlations, and lack adaptive and collaborative capabilities, the diagnostic results cannot directly and intelligently drive the optimization of compensation strategies.

[0026] This application provides a solution for acquiring electrical parameter information, vibration information, temperature information, acoustic information, and visual information; determining a corresponding spatiotemporal composite representation vector based on the electrical parameter information, vibration information, temperature information, acoustic information, and visual information; inputting the spatiotemporal composite representation vector into a reinforcement learning model to predict the corresponding motor defects and compensation strategies, determining the defect prediction result and compensation strategy information, wherein the reinforcement learning model includes a policy network and a value network, the parameters of which are adjusted according to the reward function corresponding to the spatiotemporal composite representation vector; and compensating for the defects corresponding to the defect prediction result according to the compensation strategy information to generate a motor defect compensation result.

[0027] As can be seen from the above embodiments, this application constructs a corresponding spatiotemporal composite representation vector by fusing information from multiple heterogeneous sensors and inputting it into a reinforcement learning model. Through the reward function, the policy network and value network are driven to perform online collaborative optimization, and the defect type and adaptive compensation strategy are output simultaneously. Thus, hierarchical compensation control is executed in real time according to the adaptive compensation strategy, realizing integrated operation from comprehensive perception and intelligent decision-making to precise compensation. This significantly improves the detection rate of early minor faults, and through continuous learning, the system has the ability to adapt to changes in operating conditions and the evolution of defects, effectively improving the reliability, lifespan, and energy efficiency of the motor. This promotes the transformation of motor health management from passive maintenance to proactive early warning and intelligent maintenance.

[0028] Based on this, embodiments of this application provide a method for motor defect identification and compensation based on reinforcement learning, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the reinforcement learning-based motor defect identification and compensation method of this application.

[0029] In this embodiment, the reinforcement learning-based motor defect identification and compensation method includes steps S10 to S40: Step S10: Obtain electrical parameter information, vibration information, temperature information, acoustic information, and visual information; It should be noted that the electrical parameter information is electrical signal data of the internal electromagnetic state and energy conversion process of the motor; the vibration information is structural vibration signal generated by the movement or interaction of the motor's mechanical parts; the temperature information is thermal state data of different parts of the motor; the acoustic information is sound radiated into the air during motor operation and ultrasonic signals generated by internal discharge; and the visual information is image data of the shape and state of the external and observable internal components of the motor obtained through optical imaging.

[0030] In specific embodiments, the electrical parameter information is acquired through high-precision current transformers and voltage sensors (sampling rate ≥10kHz) to directly identify electrical defects such as winding short circuits, rotor bar breakage, and power supply imbalance. The vibration information is acquired through triaxial accelerometers (frequency range 0-10kHz) installed on the bearing housing or end cover, which can sensitively reflect mechanical defects such as bearing wear, rotor imbalance, misalignment, and mechanical loosening. The temperature information is measured through contact sensors (such as PT100 platinum resistance thermometers) and non-contact sensors (such as infrared thermal imagers) to detect local or overall overheating defects caused by overload, insulation aging, cooling failure, or increased friction. The acoustic information is acquired through microphone arrays and ultrasonic sensors. The air acoustic signal is sensitive to abnormal noise caused by bearing damage, uneven air gap between stator and rotor, etc., while the ultrasonic signal is specifically used to detect partial discharge, i.e., insulation defects. The visual information can be acquired using an industrial camera with appropriate light source, and after image processing, it can be used to automatically identify appearance abnormalities such as surface damage, oil leakage, dust accumulation, component displacement, or missing parts.

[0031] Step S20: Determine the corresponding spatiotemporal composite representation vector based on the electrical parameter information, the vibration information, the temperature information, the acoustic information, and the visual information; It should be noted that the spatiotemporal composite representation vector is a high-dimensional feature vector formed by deep fusion of time-domain, frequency-domain, time-frequency, and image features extracted from multi-source sensor information using a multi-head attention mechanism and a graph attention network.

[0032] In a specific embodiment, time-domain features, frequency-domain features, time-frequency features, and image features are extracted based on the electrical parameter information, vibration information, temperature information, acoustic information, and visual information. Specifically, for the electrical parameter information, such as three-phase current and voltage, time-domain and frequency-domain features are extracted. The time-domain features of the electrical parameter information can be statistical quantities such as mean, RMS value, peak value, kurtosis, and waveform factor. The frequency-domain features of the electrical parameter information can be obtained by using a fast Fourier transform to obtain the fundamental frequency, harmonic amplitudes, and characteristic frequency components, such as for rotor failure... For the slip frequency sideband of the strip, in addition to extracting similar time-domain statistical features for the vibration and acoustic information, time-frequency features need to be extracted through short-time Fourier transform or wavelet packet transform, such as wavelet packet node energy and amplitude of the spectrum at the bearing fault characteristic frequency, to characterize the transient characteristics of non-stationary signals. For the temperature information, its time-domain trend features and spatial distribution features are mainly extracted, such as temperature rise rate and steady-state fluctuation variance. For the visual information, depth image features are automatically extracted using a pre-trained convolutional neural network, such as texture, shape, and edge features.

[0033] Multi-head attention is used to analyze the dependencies between the temporal features, frequency domain features, time-frequency features, and image features, determining the query vector, key vector, and value vector. Attention weights are calculated based on these vectors. These weights are then used to aggregate features from different time steps to obtain dependency information. Specifically, the temporal features, frequency domain features, time-frequency features, and image features are aligned and concatenated along the time dimension to form a multi-dimensional feature sequence. This sequence undergoes a linear transformation, generating corresponding query vector Q, key vector K, and value vector V through three independent and trainable fully connected layers. The query vector retrieves information relevant to the current state, the key vector identifies the feature at each time step, and the value vector carries specific information about the feature itself.

[0034] In the multi-head attention mechanism, Q, K, and V are uniformly divided into multiple subspaces. Each head independently calculates its attention weights within its own subspace. The outputs of each head are concatenated and fused through a linear layer, as shown below:

[0035] in, The similarity is calculated, where dk (usually dk=D) is a scaling factor to prevent the dot product from becoming too large and causing the gradient to vanish in the softmax function. The softmax function converts the similarity into a weight distribution, and finally, the values ​​V are summed using the weights to obtain the attention output. In motor diagnostics, this means that the model can assess the correlation strength between current fluctuations at a certain moment and temperature anomalies or specific noise patterns a few seconds earlier. Taking a specific fault as an example, the single-head attention mechanism can be clearly defined to focus on specific motor parameters and time points. For a given fault type, the model learns to assign high weights to signal components that exhibit significant physical anomalies and their corresponding time points. For example, in bearing failure, attention weights may be highly focused on the timing of the occurrence of harmonic components in the current signal that match the bearing failure characteristic frequency (such as BPFO), as well as the moment of sudden energy increase in the vibration signal; for rotor bar breakage, the focus may be on the timing of the rise of the (1±2s)f1 sideband component in the current spectrum and the moment of speed fluctuation; in the diagnosis of stator inter-turn short circuit, the model will simultaneously focus on the timing of abnormal three-phase current imbalance and local temperature rise; in the diagnosis of insulation aging / overheating, the model will focus on multiple consecutive time points where the temperature sensor readings continuously exceed the limit, and correlate them with small leakage changes in the current signal. This mechanism enables the model to autonomously capture complex dependencies across sensors and across time without relying on manually preset rules, thereby achieving intelligent and interpretable fusion of multi-source time-series information, and thus enabling the parallel and multi-angle capture of complex dependencies between different features to form dependency information.

[0036] By using a graph attention network, the spatial topological relationships between different sensor measurement points corresponding to the dependency information are constructed into a spatial topological graph. That is, GAT (Graph Attention Network) can use dependency information to construct motor parameters into a spatial topological graph. In the spatial topological graph, the nodes are sensor measurement points, and the edges represent the connection relationships between components. GAT learns the importance weights of different nodes in the graph through the attention mechanism, captures the spatial correlation between local features and global state, and can adaptively learn the importance weights of different nodes in the graph to their neighboring nodes through the attention mechanism.

[0037] The unnormalized attention coefficients are calculated for each central node and its neighboring nodes in the spatial topology graph. Specifically, the GAT layer first performs a linear transformation on the features of each node, projecting them into a new feature space to enhance expressive power. For node i and its neighboring node j, GAT calculates the unnormalized attention coefficients eij through a learnable attention mechanism a, expressed as:

[0038] Where W is the weight matrix, and hi and hj are the feature vectors of nodes i and j, respectively. This represents vector concatenation, where 'a' is a learnable weight vector and LeakyReLU is a non-linear activation function.

[0039] The attention coefficients are normalized to obtain normalized attention weights. To make the weights comparable across different nodes, the softmax function is typically used to normalize the attention coefficients, as shown below:

[0040] Where Ni is the set of neighbors of node i (usually including itself), and the normalized attention coefficients are... This represents the importance weight of neighbor node j to center node i.

[0041] The feature vectors corresponding to neighboring nodes in the spatial topology graph are weighted and summed according to the normalized attention weights to determine the weight value. The feature vector corresponding to the center node in the spatial topology graph is updated using the weight value to obtain a spatiotemporal composite representation vector. That is, after obtaining the normalized attention coefficients, the information of neighboring nodes is aggregated by weighted summation, and a nonlinear activation function (such as ELU or ReLU) is applied to update the feature representation of the center node i. In this way, the new feature of each node is the weighted average of the features of all its neighboring nodes. The weights are adaptively learned by the attention mechanism, enabling the model to dynamically assign different importance to different neighbors. Multi-source data can only play its maximum value through effective fusion. Therefore, a weighted feature fusion method can be used to weight and sum the feature vectors of each modality to generate a fused comprehensive feature vector, which contains the key information of each modality and highlights the most relevant feature in the current state. At this time, the system assigns learnable weights to different sensor data to form a spatiotemporal composite representation vector, thereby automatically adjusting the contribution of each sensor data according to the defect type. For example, when detecting electrical defects, current features are given higher weights, while vibration features are emphasized when detecting mechanical defects.

[0042] In one feasible implementation, step S10 may include steps A11 to A14: Step A11: Extract time-domain features, frequency-domain features, time-frequency features, and image features based on the electrical parameter information, vibration information, temperature information, acoustic information, and visual information; It should be noted that time-domain features are statistical quantities or morphological descriptors extracted directly from the waveform of the sensor signal changing over time, including mean, variance, peak value, kurtosis, skewness, waveform factor, impulse factor, etc. Frequency-domain features are features extracted after the time-domain signal is transformed to the frequency domain by Fourier transform (FFT), including characteristic frequency amplitude, harmonic components, frequency centroid, frequency standard deviation, etc. Time-frequency features are features extracted to characterize the evolution of non-stationary and transient signals with both time and frequency, obtained through short-time Fourier transform (STFT), wavelet transform, or wavelet packet transform, such as the energy of each node of the wavelet packet, energy accumulation areas in the time-spectrum graph, instantaneous frequency, etc. Image features are digital features extracted from motor images acquired by a visual sensor to describe their appearance and texture structure.

[0043] It is understood that the time-domain features can characterize the overall strength, fluctuation range, impact, and distribution pattern of the signal in the time dimension without the need for frequency domain transformation, resulting in high computational efficiency. They are suitable for preliminary judgment of the energy level and abnormal mutation components of the signal and are highly sensitive to amplitude changes caused by faults such as bearing wear and rotor imbalance. The frequency-domain features can characterize the distribution of periodic or quasi-periodic components in the signal and can effectively identify the enhancement or change of specific frequency components (such as rotational frequency, sidebands, and resonance peaks) caused by faults such as rotor bar breakage, bearing damage, and misalignment. They are suitable for analyzing fault characteristics under steady-state operation. The time-frequency features can simultaneously retain time and frequency information, making them particularly suitable for analyzing faults in transient processes such as startup, speed change, and impact load, or for capturing early, weak, and transient abnormal events. The image features can characterize visually identifiable defects such as surface damage, oil leakage, corrosion, and assembly abnormalities of the motor, thereby realizing image-based intelligent inspection and condition assessment.

[0044] Step A12: Use multi-head attention to analyze the dependencies between the temporal features, the frequency features, the time-frequency features, and the image features to obtain dependency information; It is understood that the dependency information is generated by using a multi-head attention mechanism to analyze the time domain, frequency domain, time-frequency and image features, and generates weights that can quantify the strength and influence of the association between different features and between different historical time steps.

[0045] In one feasible implementation, step A12 may include steps B11-B13: Step B11: Use multi-head attention to analyze the dependencies between the time-domain features, the frequency-domain features, the time-frequency features, and the image features to determine the query vector, key vector, and value vector; It should be noted that the query vector is generated from the current feature representation to be analyzed, representing the system's proactive inquiry into information. It can be understood as a question posed during the diagnostic process, such as which features from historical moments are most relevant to diagnosing the fault. It is used to perform matching calculations with feature identifiers from all time steps. The key vector is an index representing the feature at each time step, generated from the features at each time step. The value vector is the original information content representing the feature, generated from the features at each time step. After determining the attention weights, the model does not directly use the key vectors, but instead uses these weights to perform a weighted summation of the value vectors.

[0046] Step B12: Calculate attention weights based on the query vector, the key vector, and the value vector; It should be noted that the attention weights are a set of probability distribution values ​​obtained by normalization through the softmax function in the multi-head attention mechanism. They are used to quantify the relative importance or attention that the model should give to features at different historical moments when generating the feature representation at the current moment.

[0047] It is understood that the attention weight is a normalized expression of the similarity calculation results between the query vector and the key vector. The higher the weight, the stronger the correlation between the features at the corresponding moment and the current diagnostic task. In motor defect identification, the attention weight enables the model to dynamically and adaptively focus on the physical parameters most relevant to a specific fault and the time point of their occurrence, such as the instant when current harmonic anomalies occur or the period when the temperature continues to rise. This enables intelligent and selective information fusion of multi-source time-series signals, thereby improving the accuracy of diagnosis and the interpretability of the model.

[0048] Step B13: Aggregate the features at different time steps using the attention weights to obtain dependency information.

[0049] It is understood that the dependency information can be represented as an attention weight matrix, where each element of the matrix represents the degree of dependence of a feature at a certain moment on another feature at another moment when diagnosing a specific state. This is automatically learned by the model from the data, thereby revealing complex causal or symbiotic dependencies across modalities and time, such as how long after an abnormal current occurs the temperature begins to rise, or whether a specific vibration mode is always accompanied by changes in the acoustic spectrum.

[0050] Step A13: Use a graph attention network to construct a spatial topology graph from the spatial topological relationships between different sensor measurement points corresponding to the dependency information; It should be noted that the spatial topology graph is a graph-structured data representation that abstracts multiple sensor measurement points distributed in the motor system into nodes and the relationships between sensors that characterize the physical structure, functional associations, or signal transmission paths of the motor into edges. In this graph, each node carries the feature vector extracted by the corresponding sensor, and each edge defines whether there is a connection between nodes and the attributes of the connection. By constructing the spatial topology graph, the spatial correlation and dependence between state parameters of different parts of the motor can be formally represented. This allows the modeling of how local anomalies propagate through the system structure and affect the global state through message passing and attention mechanisms, thereby achieving an upgrade from isolated point monitoring to system-level correlation diagnosis.

[0051] Step A14: Perform a weighted summation of the feature vectors corresponding to each central node and neighboring nodes in the spatial topology graph to obtain the spatiotemporal composite representation vector.

[0052] It is understood that the spatiotemporal composite representation vector is obtained by analyzing the temporal dependencies between features at different time steps through multi-head attention, capturing the evolution pattern of defects over time, and constructing a spatial topology graph of sensor measurement points through a graph attention network. The spatial correlation weights between sensor nodes in the graph are learned to represent the mutual influence of signals from different parts. At the same time, through adaptive weighted fusion, a composite feature representation that can reflect both the dynamic temporal changes of defects and the spatial state correlation of the system is generated.

[0053] In one feasible implementation, step A14 may include steps C11 to C14: Step C11: Calculate the unnormalized attention coefficient based on each central node and neighboring node in the spatial topology graph; It should be noted that the unnormalized attention coefficient is an original scalar score calculated in the graph attention network (GAT) for any pair of adjacent nodes in the spatial topology graph through a shared, learnable attention mechanism. It quantifies the relative importance and correlation strength of the information provided by the neighboring nodes when updating the features of the central node. It is obtained by performing linear transformations on the features of the two nodes respectively, concatenating them, taking a dot product with a learnable attention weight vector, and then passing it through the LeakyReLU nonlinear activation function.

[0054] Step C12: Normalize the attention coefficients to obtain normalized attention weights; It should be noted that the normalized attention weights are probability distribution values ​​with a sum of 1 obtained by processing the unnormalized attention coefficients calculated for any center node and all its neighbor nodes in the graph attention network (GAT) through the softmax function. The normalized weights represent the exact contribution ratio of the features of the neighbor nodes when updating the features of the center node, enabling the model to adaptively aggregate information from different sensor nodes in a stable and interpretable manner.

[0055] Step C13: Based on the normalized attention weights, the feature vectors corresponding to the neighbor nodes in the spatial topology graph are weighted and summed to determine the weighting value; It should be noted that the weighted value is an aggregated feature vector obtained by weighting and summing the feature vectors of all neighboring nodes of the central node using normalized attention weights in the graph attention network (GAT). It is a comprehensive information that aggregates information from neighbors, filtered and weighted by the attention mechanism, and realizes the cross-node transmission and adaptive fusion of information in the graph structure, so that the new features of each node can dynamically and selectively represent the overall state of its local topological environment.

[0056] Step C14: Update the feature vector corresponding to the central node in the spatial topology graph using the weighted value to obtain the spatiotemporal composite representation vector.

[0057] It is understandable that by using the weighted values ​​to update the feature vector corresponding to each central node in the spatial topology graph, that is, by inputting the weighted values ​​that aggregate the weighted information of all neighboring nodes into a nonlinear activation function to generate the updated feature representation of the node, the adaptive fusion and transmission of information between nodes in the graph can be achieved.

[0058] Step S30: Input the spatiotemporal composite representation vector into the reinforcement learning model to predict the corresponding motor defects and compensation strategies, and determine the defect prediction results and compensation strategy information. The reinforcement learning model includes a policy network and a value network. The policy network and value network are obtained by adjusting the parameters of the policy network and value network according to the reward function corresponding to the spatiotemporal composite representation vector. It should be noted that the defect prediction result is a judgment on the current health status of the motor predicted by the policy network in the reinforcement learning model based on the input spatiotemporal composite representation vector. It can include defect type, defect level, and evolution trend prediction. The defect type can be a specific fault mode such as bearing inner ring wear, rotor bar breakage, stator inter-turn short circuit, permanent magnet demagnetization, etc. The defect level can correspond to different response priorities, such as emergency, moderate, and minor. The compensation strategy information is a set of optimal control instructions or parameter adjustment schemes generated synchronously by the policy network corresponding to the defect prediction result. That is, the specific compensation actions to be taken to deal with the identified defects. Its content matches the defect level. For emergency defects, the strategy can be to immediately reduce power or shut down for protection. For moderate defects, the strategy can be to adjust the current loop PID parameters, inject compensation current, or activate a specific filter. For minor defects, the strategy can be to record the trend and optimize the maintenance plan without real-time intervention, so as to bring the motor state back to normal or safe range or suppress the further development of the defect through active intervention.

[0059] It is understood that the policy network is a module used for direct decision-making in the actor-critic architecture reinforcement learning model. Its input is a spatiotemporal composite representation vector describing the overall state of the motor, and its output is a probability distribution or deterministic action value for the action to be performed in a given state. In the motor defect identification and compensation task, the output action space of the policy network can include the classification selection of defect type and severity level, as well as specific compensation control parameters corresponding to the diagnostic results. Its goal is to find a mapping strategy from state to optimal action through learning to maximize long-term cumulative reward. The value network is used for negative... The module responsible for evaluating state value receives a spatiotemporal composite representation vector and outputs a scalar value, namely the state value function V(s). This function represents the expected long-term cumulative reward that can be obtained starting from the current state and following the current policy. The value network does not directly generate actions, but rather acts as a critic to evaluate the merits of the decisions made by the policy network. It also provides direction for the parameter updates of the policy network through the calculated value gradient, which greatly stabilizes the training process and helps the policy network to more accurately determine which state and action sequences are more valuable. Thus, in motor health management, it achieves the optimization goal of considering both the accuracy of immediate diagnosis and the stability of long-term operation.

[0060] In a specific embodiment, historical fault data is acquired; the historical fault data and the spatiotemporal composite representation vector are input into a reinforcement learning model for training to determine the target reinforcement learning model; the spatiotemporal composite representation vector is input into the target reinforcement learning model to predict the corresponding motor defects and compensation strategies, obtaining defect prediction results and compensation strategy information. The defect prediction results include critical defects, moderate defects, and minor defects. The compensation strategy information includes reducing power or shutting down, adjusting control parameters, and recording defect trends. That is, historical fault data with multiple operating conditions and multiple defect types can be collected, cleaned, and labeled from historical motor operation records and fault simulation tests to form a labeled training set, thereby constructing and initializing the actor-critic framework. The reinforcement learning model is trained using historical data and its corresponding spatiotemporal composite representation vectors, i.e., supervised pre-training. A policy network (Actor) is trained using defect-labeled data to perform defect classification tasks, while a value network (Critic) is trained using normal state data to evaluate state values. Online reinforcement learning training is then initiated, inputting the real-time generated spatiotemporal composite representation vectors into the model. The model executes the output action corresponding to the current policy generated by the policy network, and receives an immediate reward according to a pre-defined reward function. This interaction experience is stored in an experience replay buffer. The model periodically samples from this buffer, using temporal difference error as a guide, and alternately updates the parameters of the policy and value networks using a gradient descent algorithm. Training continues until the model's performance on the independent validation set converges and meets a pre-defined standard. This converged model is then determined as the deployable target reinforcement learning model. During the inference phase, the real-time generated spatiotemporal composite representation vectors are input into this target model, and the policy network directly outputs specific defect prediction results and their corresponding compensation policy information, thus completing end-to-end intelligent output from state perception to diagnostic decision-making and control suggestions.

[0061] Step S40: Compensate for the defects corresponding to the defect prediction results according to the compensation strategy information to generate motor defect compensation results.

[0062] It should be noted that the motor defect compensation result is a comprehensive feedback of the operating state adjustment effect achieved by the system after performing actual control operations on the motor based on the compensation strategy information output by the reinforcement learning model.

[0063] It is understood that the motor defect compensation result includes the specific implementation of the compensation action, the parameter changes after execution, and the final evaluation conclusion of this compensation intervention. The specific implementation may be injecting the d-axis current compensation amount into the control system or activating a notch filter of a specific frequency. The parameter changes after execution may be a reduction in vibration amplitude, a decrease in three-phase current imbalance, and a suppression of the temperature rise trend. The final evaluation conclusion of this compensation intervention may be that the compensation was successful and the state was restored to within the safe threshold, or that the compensation effect was limited and an upgrade and maintenance were recommended.

[0064] In a specific embodiment, based on the defect prediction results and compensation strategy information, the system can generate specific compensation instructions to adjust the motor operating parameters. The decision-making mechanism adopts a hierarchical decision-making strategy to distinguish defect types with different levels of urgency, including emergency defects, moderate defects, and minor defects. The emergency defect may be a winding short circuit or bearing jamming. In this case, protective actions are immediately executed, such as reducing output power or stopping the machine, to prevent the fault from escalating. Its response time must be less than 10ms. The moderate defect may be bearing wear or insulation aging. In this case, control parameters are adjusted for compensation, such as modifying the current loop PID parameters or increasing the limit value. Its decision response time is about 50-100ms. The minor defect may be slight imbalance or early wear. In this case, the defect trend is recorded, the maintenance plan is optimized, and compensation intervention is not performed immediately.

[0065] The compensation mechanism employs specific strategies for different types of defects. For permanent magnet synchronous motors, when rotor demagnetization defects are detected, the system automatically adjusts the d-axis current component id to enhance the excitation magnetic field and compensate for the demagnetization effect. The compensation amount is calculated as follows:

[0066] Where Kp is the compensation coefficient, Δψ is the estimated flux linkage loss, and Δψ is the identification parameter used in the previous operation.

[0067] To address the vibration problem caused by bearing wear, a notch filter is added to the speed loop controller to generate phase lead near the resonant frequency, thereby suppressing the vibration amplitude. The filter parameters are adaptively adjusted based on the vibration spectrum analysis results, thus obtaining the motor defect compensation result.

[0068] To achieve a complete closed loop from monitoring to control, the inference system needs to be seamlessly integrated with the motor control system, providing a user-friendly human-machine interface and ensuring that each level operates independently yet collaboratively. Specifically, the acquisition module acquires multi-source parameters in real time, the processing module performs preprocessing such as filtering and alignment, and extracts key features using signal processing techniques, while the execution module performs fault diagnosis and status assessment based on these features. The system also provides an intuitive human-machine interface that dynamically displays key motor parameters, defect identification results, compensation actions during execution, and overall health status. It also features alarm management capabilities, automatically triggering audible and visual alarms and sending information notifications when serious defects are detected. Furthermore, by integrating digital twin technology, the system constructs a virtual model synchronized with the physical motor. This model not only simulates motor behavior under different operating conditions, providing a safe environment for strategy testing and verification for reinforcement learning agents, but also supports hypothesis analysis, thereby optimizing decision-making and reducing the operational risks of the actual system.

[0069] This embodiment proposes a motor defect identification and compensation method based on reinforcement learning. It acquires electrical parameter information, vibration information, temperature information, acoustic information, and visual information. Based on these information, a corresponding spatiotemporal composite representation vector is determined. This vector is then input into a reinforcement learning model to predict the corresponding motor defect and compensation strategy, determining the defect prediction result and compensation strategy information. The reinforcement learning model includes a policy network and a value network, whose parameters are adjusted according to the reward function corresponding to the spatiotemporal composite representation vector. Finally, the defect corresponding to the defect prediction result is compensated according to the compensation strategy information, generating a motor defect compensation result. This invention addresses the technical challenge of more accurate and effective motor defect identification and compensation. Compared to existing technologies, this application integrates information from multiple heterogeneous sensors to construct a corresponding spatiotemporal composite representation vector, which is then input into a reinforcement learning model. The reward function drives online collaborative optimization of the policy network and value network, simultaneously outputting defect types and adaptive compensation strategies. Based on these adaptive compensation strategies, hierarchical compensation control is executed in real time, achieving integrated operation from comprehensive perception and intelligent decision-making to precise compensation. This significantly improves the detection rate of early, minor faults. Furthermore, through continuous learning, the system acquires the ability to adapt to changes in operating conditions and defect evolution, effectively improving the reliability, lifespan, and energy efficiency of the motor. This promotes a shift in motor health management from passive maintenance to proactive early warning and intelligent maintenance.

[0070] Furthermore, after the motor defect identification and compensation system is developed, it must undergo rigorous testing and verification before it can be put into practical application. The system's performance, reliability, and safety must be evaluated to ensure that it meets the stringent requirements of industrial scenarios. Specifically, model performance needs to be comprehensively measured by multiple dimensions: accuracy (the overall correctness of model predictions, which is usually required to be >90% for industrial applications), recall (the detection coverage of actual defects, with high recall avoiding missed detections of serious faults), F1 score (the harmonic mean of accuracy and recall, with a good standard of overall performance generally being >0.85), and response time (the delay from data acquisition to outputting compensation commands, which should be <20ms for edge-side inference and can be relaxed to within 5 seconds for cloud analysis). In terms of evaluation methods, cross-validation is used to ensure the model's generalization ability under different data distributions, and performance is compared with traditional methods through comparative experiments to highlight the technical advantages in defect identification and response speed.

[0071] To comprehensively and objectively verify system performance, a dedicated motor test bench capable of simulating real-world operating conditions and typical defects needs to be constructed. This bench should include the following core components: the motor under test (selecting an appropriate type and power, such as a permanent magnet synchronous motor or induction motor, depending on the application scenario); a load simulation system (using an eddy current dynamometer or magnetic powder brake to simulate actual loads to verify the system's performance under different load conditions); a high-precision data acquisition system (including a sensor network and data acquisition card, with a sampling rate of no less than 100kHz and an accuracy class ≤0.5); and a controllable fault injection device (e.g., simulating rotor imbalance by adding an unbalanced mass block, or simulating inter-turn short circuits in the windings by connecting an external short-circuit resistor). The testing process begins by running the motor under normal operating conditions and collecting baseline data. Subsequently, various preset defects are injected sequentially, and the complete system response is recorded. Each defect state must be maintained for a sufficient duration to ensure statistically significant data collection. The entire test should cover the motor's full operating range, including various typical operating conditions such as startup, steady-state operation, speed change, load change, and shutdown, thereby comprehensively evaluating the system's defect identification and compensation capabilities.

[0072] After completing laboratory testing, the system needs to undergo long-term validation in industrial settings to comprehensively evaluate its performance and robustness under real, complex operating conditions. Emphasis is placed on reliability (system fault-free operation time should be ≥99.9%, false alarm rate <5%), stability (performance should not significantly degrade over time, and the model must be resistant to concept drift), and adaptability (effectively coping with environmental changes such as temperature fluctuations and electromagnetic interference). During validation, the system will fully record all identification results and compensation actions, and compare them with actual maintenance records to calculate accuracy and false alarm rate. Simultaneously, the downtime and maintenance costs avoided by defect warnings will be quantified to objectively assess its economic value. In summary, the reinforcement learning-based motor defect identification and compensation technology, through the organic synergy of detection, training, and inference, achieves full-process intelligent control from multi-source perception and strategy learning to precise control, significantly improving motor operational reliability and effectively reducing unplanned downtime.

[0073] Based on the first embodiment of this application, in the second embodiment of this application, the same or similar content as the first embodiment can be referred to the above description, and will not be repeated hereafter.

[0074] In this embodiment, refer to Figure 2 , Figure 2 This is a flowchart illustrating Embodiment 2 of the reinforcement learning-based motor defect identification and compensation method of this application. Step S30 specifically includes steps S31 to S33: Step S31: Obtain historical fault data; It should be noted that the historical fault data is a time-series dataset that was collected and recorded by the sensor system during the past operation of the motor, and includes a variety of typical defect states and their corresponding normal operating states.

[0075] It is understood that the historical fault data includes multi-source sensor information of the motor under different operating conditions, clearly marked fault types, occurrence time, severity, handling measures taken, and controlled defect samples obtained by artificially injecting faults on the test bench, which are used to characterize the accuracy and generalization ability of the trained defect identification and compensation model.

[0076] In a specific embodiment, a multi-source sensor network deployed on a motor test bench and in an industrial site can collect raw time-series data of the motor under normal and various preset fault conditions in a controlled experimental environment and real-world operating scenarios. The collected raw data is then cleaned, aligned, and labeled, i.e., abnormal noise points are removed, timestamps of data from various sensors are standardized, and experts label the data segments with specific fault types, occurrence time windows, and severity levels based on maintenance records, fault injection logs, and synchronously recorded multimodal signal characteristics. At the same time, to address the data imbalance problem, a generative adversarial network is used to synthesize rare fault samples, and time-series data augmentation techniques are applied to expand the dataset, thereby obtaining historical fault data.

[0077] Step S32: Input the historical fault data and the spatiotemporal composite representation vector into the reinforcement learning model for training, and determine the target reinforcement learning model; It should be noted that the target reinforcement learning model is the final reinforcement learning model that has reached the preset performance standard and can be used for actual deployment after offline pre-training and online reinforcement learning optimization.

[0078] In a specific embodiment, the historical fault data is input into a reinforcement learning model to train the policy network and value network with the corresponding classification task and state evaluation as the objectives. This determines the initial reinforcement learning model, i.e., data preparation and augmentation. Historical data under normal motor conditions and various fault states are collected and cleaned. After labeling, a generative adversarial network (GAN) is used to generate scarce fault samples to balance the data distribution. Noise addition and window sliding operations are applied to the time-series data to expand the dataset. Supervised pre-training is then conducted. The labeled historical data is used for time-series slicing and feature extraction, which are then used as training samples input into the policy network. Classification loss functions such as cross-entropy loss drive the network to learn the mapping between defect types and severity. Simultaneously, the normal state... Historical data samples are input into the value network and trained using the mean squared error loss function to accurately assess the value of normal states. After completing the above supervised pre-training, the policy network has a preliminary defect identification capability, and the value network has also established a basic state assessment capability. Together, they constitute an initial reinforcement learning model with reasonable initial parameters. Model convergence is judged by comparing the model's prediction results with the real fault labels to evaluate whether the model is stable. If it has not converged, it is switched to the reinforcement learning paradigm, using the reward function to guide the iterative optimization of model parameters. Through model validation and selection, performance indicators such as accuracy, recall, and F1 score are comprehensively examined on the independent validation set, and the best-performing model is selected as the initial model, i.e., the initial reinforcement learning model.

[0079] Based on the spatiotemporal composite representation vector, corresponding decision information and compensation state information are generated; an immediate reward is calculated based on the compensation state information and a preset reward function; the spatiotemporal composite representation vector, the decision information, the compensation state information, and the immediate reward are integrated in the form of a four-tuple to determine the interaction experience sample. That is, based on the spatiotemporal composite representation vector, the policy network can output the corresponding decision information, namely the defect classification result and the specific compensation instruction, for example, adjusting the d-axis current by +5A, thereby executing the compensation instruction, and the compensated motor operation data is collected in real time through the sensor network, and then subjected to the same feature extraction and fusion. The process generates a spatiotemporal composite representation vector reflecting the new state. This vector is the compensated state information. At this point, the reward function is triggered. By quantifying the comprehensive deviation changes between the compensated state and the normal state in multidimensional physical parameters, a scalar value is obtained as the immediate reward. A positive value indicates state improvement, and a negative value indicates deterioration or ineffectiveness. The four elements—the original state vector (input) before compensation, the decision information (action), the new state vector after compensation (new state), and the calculated immediate reward—are encapsulated in the standard quadruple form (s, a, r, s′) to form a complete interactive experience sample, which is then stored in the experience playback buffer.

[0080] Sampling priorities are dynamically allocated based on the temporal difference error corresponding to the interactive experience samples; a batch of sampled experience data is determined based on the sampling priorities; the policy network and value network of the initial reinforcement learning model, which satisfy the predefined learning rate, are updated using the loss function of the gradient descent algorithm under the loss function gradient corresponding to the batch of sampled experience data, to obtain the target reinforcement learning model. Specifically, during the model update phase, the system employs a priority experience replay mechanism to improve learning efficiency. The absolute value of the temporal difference error is calculated for each interactive experience sample in the experience replay buffer, serving as an indicator of the sample's learning value. Sampling priorities are dynamically allocated accordingly, using priority-based random sampling, such as proportional priority or rank priority, to extract a fixed batch of experience data from the buffer. The loss functions of the policy network and value network are calculated separately: the loss of the value network is based on the mean squared error; the loss of the policy network combines the policy gradient and the advantage function estimation. Under the condition of satisfying the predefined learning rate scheduling rules, the network parameters are updated using the gradient descent algorithm, as shown below:

[0081] Where θ represents the network parameters, α is the learning rate, and L(θ) is the loss function.

[0082] The model continuously learns from high-value experiences, eventually causing the initial reinforcement learning model to converge and be optimized into a target reinforcement learning model that meets the performance requirements.

[0083] The online learning phase also introduces a transfer learning mechanism, which allows the system to quickly adapt to new conditions and avoid retraining when the motor's operating environment or load characteristics change.

[0084] In one feasible implementation, step S32 may include steps D11 to D15: Step D11: Input the historical fault data into the reinforcement learning model and train the policy network and value network with the corresponding classification task and state evaluation as the objectives to determine the initial reinforcement learning model. It should be noted that the initial reinforcement learning model is a reinforcement learning model obtained only through supervised pre-training during the offline training phase.

[0085] It is understood that the initial reinforcement learning model is a model with basic decision-making and evaluation capabilities obtained by training the policy network and value network in a supervised manner after taking labeled historical fault data as input. The parameters of this model have not yet been optimized online with actual interaction with the environment and reward function. Its policy is relatively initial and conservative, but it already has the ability to map from state features to preliminary defect identification and compensation suggestions.

[0086] Step D12: Generate corresponding decision information and compensation state information based on the spatiotemporal composite representation vector; It should be noted that the decision information refers to the specific action instructions generated by inputting the spatiotemporal composite representation vector into the policy network of the reinforcement learning model. It may include the defect judgment result of the current motor state and the specific compensation control parameters or operation instructions suggested for the judgment result.

[0087] It is understood that the compensation state information is a spatiotemporal composite representation vector of the new state after compensation, generated by the system collecting motor operation data again through the sensor network after performing compensation actions based on decision information, and after feature extraction and fusion. It is used to quantitatively evaluate the immediate effect of the compensation actions. By comparing the differences between the state vectors before and after compensation, the reward function can determine the effectiveness of the actions, such as whether the vibration is reduced or whether the current is restored to balance.

[0088] Step D13: Calculate the immediate reward based on the compensation status information and the preset reward function; It should be noted that the immediate reward is a scalar feedback value calculated based on the compensation state information and a preset reward function within the reinforcement learning framework, representing the immediate effect of a single decision and compensation action.

[0089] Understandably, the reward function quantifies and evaluates the motor state parameters before and after compensation and maps them to potential energy values. The reward value is then obtained by the difference between the potential energy after compensation and the potential energy before compensation. If the compensation improves the state, i.e., the potential energy decreases, the reward is positive. If the state deteriorates or does not meet expectations, i.e., the potential energy increases, the reward is negative or zero.

[0090] Step D14: Integrate the spatiotemporal composite representation vector, the decision information, the compensation state information, and the instant reward in the form of a quadruple to determine the interaction experience sample; It should be noted that the interaction experience samples are training data units recorded and encapsulated during the online training process of reinforcement learning. They are stored in the form of quadruples to fully capture the information of the corresponding interactions. By storing the interaction experience samples in the experience replay buffer, the system can break the temporal correlation between data and repeatedly learn important experiences through the priority sampling mechanism, thereby achieving stable and efficient iterative optimization of the policy network and the value network.

[0091] Step D15: Update the policy network and value network of the initial reinforcement learning model based on the interaction experience samples to obtain the target reinforcement learning model.

[0092] It is understood that the target reinforcement learning model is based on an actor-critic architecture, and the parameters of its policy network and value network have been deeply tuned through supervised learning with historical data and online interactive trial and error, demonstrating superior and stable performance on the validation set.

[0093] In one possible implementation, step D15 may include steps E11 to E13: Step E11: Dynamically allocate sampling priorities based on the time difference error corresponding to the interactive experience samples; It should be noted that the sampling priority is a metric assigned to each interactive experience sample stored in the experience replay buffer in the reinforcement learning priority experience replay mechanism. It is positively correlated with the absolute value of the temporal difference error of the sample. The higher the priority, the greater the learning value of the sample and the higher the potential benefit of the model learning from it. The system can determine the probability of each sample being drawn into the training batch based on this priority value, thereby achieving non-uniform sampling. This allows the model to learn more frequently from high-error, high-information experiences, effectively accelerating the convergence process and improving the overall efficiency and stability of policy optimization.

[0094] Step E12: Determine the sample batch experience data according to the sampling priority; It should be noted that the sampled batch of experience data is a set of interactive experience samples dynamically extracted from the experience replay buffer according to the sampling priority based on the preset batch size during the online training process of reinforcement learning. It serves as the input for a single parameter update of the model. The sampling process is not uniform and random, but prioritizes the inclusion of experiences with high temporal difference errors to improve learning efficiency. The samples in each batch are diverse in terms of state, action, reward, and new state, which together provide statistically reliable and information-rich gradient estimates for the loss function calculation of the policy network and value network, thereby achieving stable and efficient network parameter updates in a single optimization step.

[0095] Step E13: The policy network and value network of the initial reinforcement learning model under the gradient of the loss function corresponding to the sampled batch of empirical data that satisfies the predefined learning rate are updated using the loss function of the gradient descent algorithm to obtain the target reinforcement learning model.

[0096] Understandably, the resulting target reinforcement learning model can accurately map the input spatiotemporal composite representation vector to the optimal defect classification result and corresponding compensation strategy. It also has the ability to generalize to unseen working conditions and an interface for online continuous learning, thus enabling intelligent diagnosis and decision-making.

[0097] Step S33: Input the spatiotemporal composite representation vector into the target reinforcement learning model to predict the corresponding motor defects and compensation strategies, and obtain defect prediction results and compensation strategy information. The defect prediction results include emergency defects, moderate defects and minor defects. The compensation strategy information includes reducing power or stopping the machine, adjusting control parameters and recording defect trends.

[0098] Understandably, the trained target reinforcement learning model can be deployed in the inference stage. It receives real-time generated spatiotemporal composite representation vectors as input and outputs defect prediction results and compensation strategy information strictly corresponding to the defect level by the policy network. The defect prediction results are classified according to the severity of the fault, including emergency defects that require immediate handling, moderate defects that can be adjusted, and minor defects that only require monitoring. The compensation strategy information strictly corresponding to the defect level includes power reduction or shutdown protection commands in emergency situations, optimization commands such as adjusting current loop PID parameters or activating dynamic filters in moderate defects, and observation commands for recording defect trends and planning maintenance in minor defects. This ensures real-time matching and accurate execution of diagnostic conclusions and compensation actions.

[0099] In specific embodiments, to ensure that the trained reinforcement learning model meets the stringent real-time, stability, and resource constraints of industrial scenarios, specific optimization and deployment strategies need to be designed for the inference stage. These strategies may include model lightweighting, multi-rate inference mechanisms, and adaptive inference. Model lightweighting involves removing redundant connections through model pruning, reducing storage and computational overhead through weight quantization, and transferring knowledge to a more compact model using knowledge distillation. This is accelerated by dedicated inference frameworks such as TensorRT or OpenVINO, enabling efficient inference on edge devices. The multi-rate inference mechanism sets the inference cycle differently based on the update frequency of different physical characteristics. For example, it infers every 10ms for high-frequency vibration signals, every 50ms for medium-frequency current signals, and every 1s for low-frequency temperature signals, thus balancing the computational load while ensuring system response speed. Adaptive inference dynamically adjusts the inference frequency based on the real-time operating status of the motor. Under normal operating conditions, the frequency is reduced to save computing power, and once abnormal signs are detected, the frequency is immediately increased for intensive monitoring and diagnosis, thereby achieving intelligent scheduling and optimization of computing resources while maintaining overall system performance.

[0100] This embodiment proposes a motor defect identification and compensation method based on reinforcement learning. The method involves acquiring historical fault data; inputting the historical fault data and the spatiotemporal composite representation vector into a reinforcement learning model for training to determine a target reinforcement learning model; inputting the spatiotemporal composite representation vector into the target reinforcement learning model to predict the corresponding motor defects and compensation strategies, thereby obtaining defect prediction results and compensation strategy information. The defect prediction results include critical defects, moderate defects, and minor defects, and the compensation strategy information includes reducing power or shutting down the machine, adjusting control parameters, and recording defect trends. This application addresses the technical challenge of more accurately and effectively identifying and compensating for motor defects. Compared to existing technologies, it inputs historical fault data into a reinforcement learning model. During supervised pre-training, fault data is used to train a policy network for defect classification, while normal data is used to train a value network for state evaluation. After establishing an initial model, the policy and value networks are continuously optimized through a reward function-driven online interaction and experience replay mechanism, resulting in a stable target reinforcement learning model. The real-time generated spatiotemporal composite representation vector is then input into this model, enabling its policy network to simultaneously output defect predictions at three levels: emergency, moderate, and minor, and generate precisely matched compensation strategy information. This significantly improves identification accuracy and early fault detection capabilities. Through reward mechanism and environmental interaction, the model can autonomously optimize compensation strategies, achieving intelligent adaptation to different operating conditions and defect evolution. Furthermore, the fully trained model can complete the integrated output from state perception to hierarchical decision-making online, ensuring that emergency defects are addressed immediately, moderate defects are optimized and adjusted, and minor defects are effectively monitored. This improves motor operational reliability while minimizing unplanned downtime and maintenance costs.

[0101] This application also provides a motor defect identification and compensation device based on reinforcement learning, please refer to... Figure 3 The reinforcement learning-based motor defect identification and compensation device includes: The acquisition module 10 is used to acquire electrical parameter information, vibration information, temperature information, acoustic information and visual information; Processing module 20 is used to determine the corresponding spatiotemporal composite representation vector based on the electrical parameter information, the vibration information, the temperature information, the acoustic information, and the visual information; The processing module 20 is further configured to input the spatiotemporal composite representation vector into the reinforcement learning model to predict the corresponding motor defects and compensation strategies, and determine the defect prediction results and compensation strategy information. The reinforcement learning model includes a policy network and a value network, which are obtained by adjusting the parameters of the policy network and the value network according to the reward function corresponding to the spatiotemporal composite representation vector. The execution module 30 is used to compensate for the defects corresponding to the defect prediction results according to the compensation strategy information, and generate motor defect compensation results.

[0102] The processing module 20 is further configured to extract time-domain features, frequency-domain features, time-frequency features, and image features based on the electrical parameter information, the vibration information, the temperature information, the acoustic information, and the visual information; Multi-head attention is used to analyze the dependencies between the temporal features, the frequency features, the time-frequency features, and the image features to obtain dependency information; A graph attention network is used to construct a spatial topology graph from the spatial topological relationships between different sensor measurement points corresponding to the dependency information. The spatiotemporal composite representation vector is obtained by weighted summation of the feature vectors corresponding to each central node and neighboring nodes in the spatial topology graph.

[0103] The processing module 20 is further configured to use multi-head attention to parse the dependencies between the time-domain features, the frequency-domain features, the time-frequency features, and the image features, and to determine the query vector, key vector, and value vector; Calculate attention weights based on the query vector, the key vector, and the value vector; The attention weights are used to aggregate features at different time steps to obtain dependency information.

[0104] The processing module 20 is also used to calculate the unnormalized attention coefficient based on each central node and neighboring node in the spatial topology graph; The attention coefficients are normalized to obtain normalized attention weights; Based on the normalized attention weights, the feature vectors corresponding to the neighbor nodes in the spatial topology graph are weighted and summed to determine the weighting value; The feature vector corresponding to the central node in the spatial topology graph is updated using the weighted value to obtain the spatiotemporal composite representation vector.

[0105] The processing module 20 is also used to acquire historical fault data; The historical fault data and the spatiotemporal composite representation vector are input into the reinforcement learning model for training to determine the target reinforcement learning model. The spatiotemporal composite representation vector is input into the target reinforcement learning model to predict the corresponding motor defects and compensation strategies, thereby obtaining defect prediction results and compensation strategy information. The defect prediction results include emergency defects, moderate defects, and minor defects. The compensation strategy information includes reducing power or shutting down the machine, adjusting control parameters, and recording defect trends.

[0106] The processing module 20 is further configured to input the historical fault data into the reinforcement learning model to train the policy network and value network with the corresponding classification task and state evaluation as the target, and to determine the initial reinforcement learning model. Based on the spatiotemporal composite representation vector, corresponding decision information and compensation state information are generated; The instant reward is calculated based on the compensation status information and the preset reward function; The spatiotemporal composite representation vector, the decision information, the compensation state information, and the instant reward are integrated in the form of quadruples to determine the interaction experience sample; The policy network and value network of the initial reinforcement learning model are updated based on the interaction experience samples to obtain the target reinforcement learning model.

[0107] The processing module 20 is also used to dynamically allocate sampling priority based on the time difference error corresponding to the interactive experience sample; The sampling batch of empirical data is determined based on the sampling priority. The policy network and value network of the initial reinforcement learning model, which are based on the gradient loss function of the sampled batch of empirical data and satisfy a predefined learning rate, are updated using the gradient descent algorithm to obtain the target reinforcement learning model.

[0108] The reinforcement learning-based motor defect identification and compensation device provided in this application, employing the reinforcement learning-based motor defect identification and compensation method described in the above embodiments, can solve the technical problem of how to more accurately and effectively identify and compensate for motor defects. Compared with the prior art, the beneficial effects of the reinforcement learning-based motor defect identification and compensation device provided in this application are the same as those of the reinforcement learning-based motor defect identification and compensation method provided in the above embodiments, and other technical features in the reinforcement learning-based motor defect identification and compensation device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0109] This application provides a reinforcement learning-based motor defect identification and compensation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the reinforcement learning-based motor defect identification and compensation method in Embodiment 1 above.

[0110] The following is for reference. Figure 4The diagram illustrates a structural schematic suitable for implementing the reinforcement learning-based motor defect identification and compensation device in the embodiments of this application. The reinforcement learning-based motor defect identification and compensation device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The reinforcement learning-based motor defect identification and compensation device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0111] like Figure 4 As shown, the reinforcement learning-based motor defect identification and compensation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the reinforcement learning-based motor defect identification and compensation device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the reinforcement learning-based motor defect identification and compensation device to exchange data wirelessly or via wired communication with other devices. Although the figure shows a reinforcement learning-based motor defect identification and compensation device with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0112] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0113] The reinforcement learning-based motor defect identification and compensation device provided in this application, employing the reinforcement learning-based motor defect identification and compensation method described in the above embodiments, can solve the technical problem of how to more accurately and effectively identify and compensate for motor defects. Compared with the prior art, the beneficial effects of the reinforcement learning-based motor defect identification and compensation device provided in this application are the same as those of the reinforcement learning-based motor defect identification and compensation method provided in the above embodiments, and other technical features in this reinforcement learning-based motor defect identification and compensation device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0114] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0115] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0116] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the reinforcement learning-based motor defect identification and compensation method in the above embodiments.

[0117] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0118] The aforementioned computer-readable storage medium may be included in a reinforcement learning-based motor defect identification and compensation device; or it may exist independently and not assembled into the reinforcement learning-based motor defect identification and compensation device.

[0119] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a reinforcement learning-based motor defect identification and compensation device, the device performs the following actions: acquires electrical parameter information, vibration information, temperature information, acoustic information, and visual information; determines a corresponding spatiotemporal composite representation vector based on the electrical parameter information, vibration information, temperature information, acoustic information, and visual information; inputs the spatiotemporal composite representation vector into a reinforcement learning model to predict the corresponding motor defect and compensation strategy, determining the defect prediction result and compensation strategy information. The reinforcement learning model includes a policy network and a value network, which are obtained by adjusting the parameters of the policy network and the value network according to the reward function corresponding to the spatiotemporal composite representation vector; and compensates for the defect corresponding to the defect prediction result according to the compensation strategy information, generating a motor defect compensation result.

[0120] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0121] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0122] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0123] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described reinforcement learning-based motor defect identification and compensation method, thereby solving the technical problem of how to more accurately and effectively identify and compensate for motor defects. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the reinforcement learning-based motor defect identification and compensation method provided in the above embodiments, and will not be repeated here.

[0124] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for motor defect identification and compensation based on reinforcement learning, characterized in that, The method includes: Acquire electrical parameter information, vibration information, temperature information, acoustic information, and visual information; The corresponding spatiotemporal composite representation vector is determined based on the electrical parameter information, the vibration information, the temperature information, the acoustic information, and the visual information. The spatiotemporal composite representation vector is input into the reinforcement learning model to predict the corresponding motor defects and compensation strategies, and to determine the defect prediction results and compensation strategy information. The reinforcement learning model includes a policy network and a value network, which are obtained by adjusting the parameters of the policy network and the value network according to the reward function corresponding to the spatiotemporal composite representation vector. The defects corresponding to the defect prediction results are compensated according to the compensation strategy information to generate motor defect compensation results.

2. The method as described in claim 1, characterized in that, The step of determining the corresponding spatiotemporal composite representation vector based on the electrical parameter information, the vibration information, the temperature information, the acoustic information, and the visual information includes: Based on the electrical parameter information, vibration information, temperature information, acoustic information, and visual information, time-domain features, frequency-domain features, time-frequency features, and image features are extracted; Multi-head attention is used to analyze the dependencies between the temporal features, the frequency features, the time-frequency features, and the image features to obtain dependency information; A graph attention network is used to construct a spatial topology graph from the spatial topological relationships between different sensor measurement points corresponding to the dependency information. The spatiotemporal composite representation vector is obtained by weighted summation of the feature vectors corresponding to each central node and neighboring nodes in the spatial topology graph.

3. The method as described in claim 2, characterized in that, The step of using multi-head attention to analyze the dependencies between the time-domain features, the frequency-domain features, the time-frequency features, and the image features to obtain dependency information includes: Multi-head attention is used to analyze the dependencies between the time-domain features, the frequency-domain features, the time-frequency features, and the image features to determine the query vector, key vector, and value vector. Calculate attention weights based on the query vector, the key vector, and the value vector; The attention weights are used to aggregate features at different time steps to obtain dependency information.

4. The method as described in claim 2, characterized in that, The step of obtaining the spatiotemporal composite representation vector by weighted summation of the feature vectors corresponding to each central node and neighboring nodes in the spatial topology graph includes: Calculate the unnormalized attention coefficient based on each central node and neighboring node in the spatial topology graph; The attention coefficients are normalized to obtain normalized attention weights; Based on the normalized attention weights, the feature vectors corresponding to the neighbor nodes in the spatial topology graph are weighted and summed to determine the weighting value; The feature vector corresponding to the central node in the spatial topology graph is updated using the weighted value to obtain the spatiotemporal composite representation vector.

5. The method as described in claim 1, characterized in that, The step of inputting the spatiotemporal composite representation vector into the reinforcement learning model to predict the corresponding motor defects and compensation strategies, and determining the defect prediction results and compensation strategy information, includes: Obtain historical fault data; The historical fault data and the spatiotemporal composite representation vector are input into the reinforcement learning model for training to determine the target reinforcement learning model. The spatiotemporal composite representation vector is input into the target reinforcement learning model to predict the corresponding motor defects and compensation strategies, thereby obtaining defect prediction results and compensation strategy information. The defect prediction results include emergency defects, moderate defects, and minor defects. The compensation strategy information includes reducing power or shutting down the machine, adjusting control parameters, and recording defect trends.

6. The method as described in claim 5, characterized in that, The step of inputting the historical fault data and the spatiotemporal composite representation vector into the reinforcement learning model for training, and determining the target reinforcement learning model, includes: The historical fault data is input into the reinforcement learning model to train the policy network and value network with the corresponding classification task and state evaluation as the objectives, and the initial reinforcement learning model is determined. Based on the spatiotemporal composite representation vector, corresponding decision information and compensation state information are generated; The instant reward is calculated based on the compensation status information and the preset reward function; The spatiotemporal composite representation vector, the decision information, the compensation state information, and the instant reward are integrated in the form of quadruples to determine the interaction experience sample; The policy network and value network of the initial reinforcement learning model are updated based on the interaction experience samples to obtain the target reinforcement learning model.

7. The method as described in claim 6, characterized in that, The step of updating the policy network and value network of the initial reinforcement learning model based on the interaction experience samples to obtain the target reinforcement learning model includes: Sampling priorities are dynamically allocated based on the time difference error corresponding to the interactive experience samples. The sampling batch of empirical data is determined based on the sampling priority. The policy network and value network of the initial reinforcement learning model, which are based on the gradient loss function of the sampled batch of empirical data and satisfy a predefined learning rate, are updated using the gradient descent algorithm to obtain the target reinforcement learning model.

8. A device for identifying and compensating for motor defects based on reinforcement learning, characterized in that, The device includes: The acquisition module is used to acquire electrical parameter information, vibration information, temperature information, acoustic information, and visual information. The processing module is used to determine the corresponding spatiotemporal composite representation vector based on the electrical parameter information, the vibration information, the temperature information, the acoustic information, and the visual information; The processing module is also used to input the spatiotemporal composite representation vector into the reinforcement learning model to predict the corresponding motor defects and compensation strategies, and to determine the defect prediction results and compensation strategy information. The reinforcement learning model includes a policy network and a value network, which are obtained by adjusting the parameters of the policy network and the value network according to the reward function corresponding to the spatiotemporal composite representation vector. The execution module is used to compensate for the defects corresponding to the defect prediction results according to the compensation strategy information, and generate motor defect compensation results.

9. A motor defect identification and compensation device based on reinforcement learning, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the reinforcement learning-based motor defect identification and compensation method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the reinforcement learning-based motor defect identification and compensation method as described in any one of claims 1 to 7.

Images