Equipment real-time monitoring and fault prediction method and system based on digital factory

Abstract

The invention relates to the technical field of digital factories, and discloses an equipment real-time monitoring and fault prediction method and system based on a digital factory. The method comprises the following steps: carrying out iterative filtering interpolation processing on equipment operation data collected by a multi-modal sensor in a digital factory to obtain smooth time sequence data, and constructing a multi-modal feature matrix based on the smooth time sequence data; inputting the multi-modal feature matrix into a joint training generalized Siamese-like wide learning network for model training to obtain an equipment state monitoring model; executing system-level state evaluation and subsystem-level fault prediction calculation based on the equipment state monitoring model, and generating an equipment operation and maintenance strategy; according to the equipment operation and maintenance strategy, a fault risk heat map is generated, a digital twin visualization system is constructed, and an equipment maintenance scheduling scheme is output, the production efficiency and equipment health management are balanced, and collaborative optimization of a maintenance plan and a production plan is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of digital factories, and in particular to a method and system for real-time monitoring and fault prediction of equipment based on digital factories. Background Art

[0002] Digital factories have become the core driving force for the transformation and upgrading of the manufacturing industry, among which real-time equipment monitoring and fault prediction are key links to ensure the stable operation of production lines. Most of the current factory equipment monitoring systems use a single sensor or a simple threshold detection method, which makes it difficult to fully capture the complex operating status changes of the equipment, resulting in insufficient fault prediction accuracy and unscientific maintenance decisions, causing frequent unplanned downtime and production losses. Especially in high-load production environments, the dynamic changes in equipment status are coupled with multiple factors, further increasing the difficulty of monitoring.

[0003] Traditional fault prediction technology mainly relies on algorithms such as deep neural networks. Although it has high prediction accuracy, the network structure is complex and requires back-propagation iterative calculations, which leads to time-consuming model training and large computing resource requirements, making it difficult to deploy in real time on edge computing devices. At the same time, most existing technologies monitor devices as independent individuals, ignoring the interdependence between production line equipment, lacking the ability to coordinate optimization at the system and subsystem levels, and unable to optimize the overall operation efficiency of the factory. Especially under the condition of scarce labeled data, the prediction ability of existing monitoring systems is severely restricted. Summary of the invention

[0004] The present invention provides a method and system for real-time monitoring and fault prediction of equipment based on a digital factory. The present invention balances production efficiency and equipment health management, and realizes coordinated optimization of maintenance plans and production plans.

[0005] In a first aspect, the present invention provides a method for real-time monitoring and fault prediction of equipment based on a digital factory, the method comprising: Iterative filtering and interpolation processing is performed on the equipment operation data collected by the multimodal sensors in the digital factory to obtain smooth time series data, and a multimodal feature matrix is ​​constructed based on the smooth time series data; Inputting the multimodal feature matrix into a joint training generalized Siamese-like extensive learning network for model training to obtain a device status monitoring model; Perform system-level status evaluation and subsystem-level fault prediction calculation based on the equipment status monitoring model to generate equipment operation and maintenance strategies; Generate a fault risk heat map based on the equipment operation and maintenance strategy, build a digital twin visualization system, and output the equipment maintenance scheduling plan.

[0006] In a second aspect, the present invention provides a digital factory-based equipment real-time monitoring and fault prediction system, the digital factory-based equipment real-time monitoring and fault prediction system comprising: An acquisition module is used to perform iterative filtering and interpolation processing on the equipment operation data collected by the multimodal sensors in the digital factory to obtain smooth time series data, and to construct a multimodal feature matrix based on the smooth time series data; A model training module is used to input the multimodal feature matrix into a joint training generalized Siamese-like extensive learning network for model training to obtain a device status monitoring model; A prediction calculation module, used to perform system-level status evaluation and subsystem-level fault prediction calculation based on the equipment status monitoring model, and generate equipment operation and maintenance strategies; The digital twin module is used to generate a fault risk heat map according to the equipment operation and maintenance strategy and build a digital twin visualization system to output an equipment maintenance scheduling plan.

[0007] In the technical solution provided by the present invention, the present invention uses an iterative filtering interpolation algorithm to smooth the multimodal sensor data, effectively eliminates random noise interference, retains the trend of equipment state changes, and significantly improves the data quality of subsequent feature extraction. In particular, the adaptive weighted moving average method can automatically adjust the sliding window size according to the local change rate of the data, and achieve a stronger smoothing effect while ensuring the state mutation characteristics. The key feature points are extracted by an improved extreme value detection algorithm, combined with the minimum interval constraint and local curvature analysis, to accurately capture the key moments of equipment state changes. This method reduces the data dimension while retaining key information, greatly improves the feature extraction efficiency and feature expression capabilities, and uses a Siamese-like structure to effectively utilize the complementarity of different modal data of the same device, significantly improving the learning ability of the model in the case of scarce labeled data. Taking equipment state evaluation as a system-level problem and fault prediction as a subsystem-level problem, the global optimal solution is solved by coupling the Lagrange multiplier method, balancing production efficiency and equipment health management. The maintenance scheduling optimization model based on the failure risk heat map comprehensively considers multiple constraints such as production task priority, maintenance personnel availability, spare parts inventory status, etc., generates the optimal maintenance schedule and equipment load adjustment plan, avoids production interruptions, improves equipment availability, and reduces maintenance costs. The digital twin visualization system provides interactive operations in four dimensions: equipment topology view, detail view, predictive analysis view, and maintenance plan view. Factory managers can intuitively understand the equipment status, foresee potential risks, and make timely decisions. At the same time, it integrates with external systems such as enterprise resource planning and manufacturing execution systems through standardized API interfaces, realizing the coordinated optimization of maintenance plans and production plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0009] Figure 1 A flow chart of a method for real-time monitoring and fault prediction of equipment based on a digital factory provided in an embodiment of the present application; Figure 2 A schematic block diagram of the structure of a digital factory-based equipment real-time monitoring and fault prediction system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0010] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0011] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change based on actual conditions.

[0012] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0013] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0014] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0015] See also Figure 1 , Figure 1 A flow chart of a method for real-time monitoring and fault prediction of equipment in a digital factory provided in an embodiment of the present application, such as Figure 1As shown, the method for real-time equipment monitoring and fault prediction based on a digital factory provided in an embodiment of the present application includes steps S100 to S400.

[0016] Step S100, performing iterative filtering and interpolation processing on the equipment operation data collected by the multimodal sensor in the digital factory to obtain smooth time series data, and constructing a multimodal feature matrix based on the smooth time series data; It is understandable that the execution subject of the present invention may be a real-time monitoring and fault prediction system for equipment based on a digital factory, or a terminal or a server, which is not specifically limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0017] Specifically, sensor signals such as vibration, temperature, sound and current are collected. In the digital factory environment, sensors acquire these physical quantities in real time and generate raw sensor data. Linear normalization mapping is performed on the raw sensor data to adjust each physical quantity to the same numerical range, thereby eliminating the calculation deviation caused by data dimension differences. Considering the differences in sampling frequencies of different sensors, the normalized sensor data is timestamped to ensure that the data points of all sensors are aligned so that the multimodal information in the same time step is accurately associated to form time-synchronized data. Based on the time-synchronized data, statistical methods or machine learning-based anomaly detection methods are used to identify data points that deviate from the normal distribution to obtain a data set marked with outliers. The missing parts in the data set marked with outliers are filled to restore the integrity of the data. The filling methods include mean filling, interpolation filling or deep learning-based prediction and completion methods. The missing values ​​are inferred based on the time series characteristics and historical trends of the data to obtain complete and continuous sensor data. The complete and continuous sensor data is organized in JSON format, and metadata information such as device ID, sensor ID and acquisition time are attached to form standardized multi-sensor data. The standardized multi-sensor data is iteratively filtered and interpolated. This processing step combines a variety of filtering methods, such as sliding average filtering, exponentially weighted moving average filtering, wavelet denoising, etc., to remove high-frequency noise and improve signal smoothness. At the same time, the interpolation method is used to make up for the slight loss of data to obtain smooth time series data. Feature point extraction is performed on the smoothed time series data based on time domain, frequency domain and time-frequency domain analysis. The time domain feature extraction includes indicators such as mean, variance, peak factor, skewness and kurtosis, while the frequency domain feature analyzes the spectral information of the signal through fast Fourier transform to extract the amplitude and phase of the main frequency components. Time-frequency feature extraction combines methods such as wavelet transform and Hilbert-Huang transform to analyze the changes of the signal on different time scales, so as to capture richer equipment status information. All extracted feature points are integrated into a multimodal feature matrix.

[0018] The standardized multi-sensor data is processed by sliding window segmentation, so that the subsequent analysis can fully capture the local characteristics of the time series data, while avoiding excessive consumption of computing resources due to excessive data volume. The size of the sliding window is dynamically adjusted according to the operating status of the equipment and the sampling frequency to ensure that the data in the window covers a complete vibration, temperature, sound or current signal cycle, and multiple signal window data segments are obtained. Fast Fourier transform is performed on multiple signal window data segments to convert the original time domain data into frequency domain representation data. The complex signal is effectively decomposed by Fourier transform, and its main frequency components are extracted to identify the potential laws of the equipment operating status. Frequency domain data is subjected to frequency band decomposition, the entire frequency domain range is divided into multiple frequency bands, and the signal characteristics of different frequency bands are analyzed separately. The frequency band decomposition method uses a bandpass filter group or wavelet transform, etc., so that different frequency components are processed separately to improve the resolution of the data and the accuracy of subsequent feature extraction. Through this process, multi-band decomposition data is obtained. De-noising is performed on the multi-band decomposition data to eliminate errors introduced by non-equipment operating characteristics. Denoising methods include wavelet threshold denoising, adaptive filtering and empirical mode decomposition. Wavelet threshold denoising effectively suppresses high-frequency noise, while empirical mode decomposition removes irrelevant components by decomposing different mode components of the signal. After denoising, the frequency domain data after filtering is obtained. The inverse transform is performed on the frequency domain data after filtering, and the signal is converted from the frequency domain back to the time domain to obtain preliminary smoothed data. The adaptive weighted moving average method is applied to the preliminary smoothed data, and the sliding window size is automatically adjusted according to the local change rate of the data, so that a larger window is used in a stable state to enhance the smoothing effect, and a smaller window is used in areas with large data fluctuations to retain key features. In order to ensure the time series consistency of the data, the smoothed data is resampled so that different sensor data maintain the same time step and avoid time series misalignment caused by different sampling rates, so as to obtain smoothed time series data. Based on the smoothed time series data, key feature points are extracted to construct a multimodal feature vector and form a multimodal feature matrix. Feature extraction mainly includes time domain features, frequency domain features and time-frequency features. Time domain features include mean, standard deviation, skewness, kurtosis and peak-to-peak values, which reflect the overall trend and fluctuation of the signal. Frequency domain features reveal the periodicity and stability of the equipment's operating status by analyzing parameters such as the main frequency, bandwidth, and harmonic components of the signal. Time-frequency features obtain signal features on different time scales through short-time Fourier transform or continuous wavelet transform, thereby enhancing the ability to identify early signs of equipment failure. The features of different modes are integrated into a multimodal feature matrix.

[0019] Construct a data difference sequence for the smooth time series data, and calculate the standard deviation of the sequence to determine the adaptive threshold parameters. The construction of the data difference sequence helps to highlight the change trend in the data, while the calculation of the standard deviation reflects the overall distribution of the data change, so that the threshold is adaptively adjusted to adapt to the fluctuation characteristics under different working conditions. According to the adaptive threshold parameters, extreme value detection is performed on the data difference sequence to identify feature points. At the same time, in order to avoid too dense feature points or too many invalid points, the minimum interval constraint is applied to make the final set of candidate feature points more representative. Calculate the local curvature of the smooth time series data, and identify the points where the curvature changes significantly to obtain a set of supplementary feature points. The calculation of the local curvature reflects the degree of nonlinear change of the data, and the points where the curvature changes significantly correspond to sudden changes or key events in the signal. Calculate the sum of squares of the signal in the sliding window for the vibration signal in the smooth time series data, and use this method to detect the energy mutation point, because the mutation of the signal energy is often a direct reflection of the change in the operating state of the equipment. In order to extract frequency domain feature information, short-time Fourier transform is performed to analyze the frequency distribution of the vibration signal in different time periods, obtain the frequency domain feature data of the vibration signal, and capture the periodic changes in the operation of the equipment. At the same time, for the current signal, since the operating state of the equipment under different load conditions will affect the harmonic components of the current, harmonic analysis is performed on the current signal in the smoothed time series data, and the total harmonic distortion rate is calculated to extract the current feature data. The total harmonic distortion rate quantifies the proportion of high-order harmonic components in the current signal, thereby reflecting whether the equipment has problems such as load imbalance, drive system abnormality, or motor aging. Based on the set of candidate feature points, the set of supplementary feature points, the frequency domain feature data of the vibration signal, and the harmonic feature data of the current signal, a multimodal feature matrix is ​​constructed.

[0020] Step S200, input the multimodal feature matrix into the joint training generalized Siamese-like extensive learning network for model training to obtain a device status monitoring model; Specifically, the multimodal feature matrix is ​​decomposed into four modal branch data, namely vibration features, temperature features, sound features and current features, to ensure that the data of different modalities are separately feature-learned and finally fused to form a unified device state representation. Since data of different modalities have different physical properties and distribution characteristics, feature mapping is performed on each modal branch data to learn its potential feature representation and obtain four modal mapping features. In order to enhance the model's feature extraction capability for different modal data, M groups of enhanced node groups are applied to each modal branch for feature processing, so that different modal features can learn more discriminative information through nonlinear mapping, and four modal enhanced features are obtained. On this basis, the four modal enhanced features are fused through a connection operation to form a unified feature representation, so that the model comprehensively considers the impact of different sensor data on the device state and improves the prediction accuracy. Moore-Penrose generalized inverse matrix calculation is performed on the unified feature representation to solve the output layer weights, and an initial network model is created based on the output layer weights. The Moore-Penrose generalized inverse matrix method effectively optimizes the computational complexity of the extensive learning network and improves the generalization ability of the model under limited sample conditions, thereby ensuring that the model can be efficiently calculated on high-dimensional multimodal data. Based on the initial network model, a Siamese-like structure is constructed to enable comparative learning of different modal data to improve the distinguishing ability of the model. The Siamese-like structure includes four parallel sub-networks that share structural parameters. Each sub-network contains feature mapping layer parameters and enhanced node parameters, so that different modal data can be learned in the same parameter space. In order to ensure the alignment ability of the model on multimodal data, the vibration modal branch is set as the anchor branch, and the four modal mapping features are mapped through their respective parallel sub-networks to obtain four encoding features. These encoding features not only retain the key information of each modality, but also establish cross-modal associations through the shared network structure. On the basis of constructing the Siamese-like structure, in order to optimize the representation ability of the model, a pairwise distance metric is constructed between the encoding features of the four parallel sub-networks, and different modal data of the same device are regarded as positive samples to learn the consistency features between them. A consistency loss function is designed so that different modal features of the same device are as close as possible in the feature space, while features of different devices are kept at a large distance. By optimizing this loss function, the discrimination ability of the model is improved, so that it can more effectively distinguish different states of the device. During the training process, in order to improve the optimization efficiency of the model, the training sample and cosine annealing learning rate scheduling strategy are adopted so that the learning rate can be dynamically adjusted during the training process to meet the needs of different training stages. Specifically, a larger learning rate is used in the early stages of training to quickly converge to a better solution, and the learning rate is gradually reduced in the later stages of training to stabilize the model parameters and prevent overfitting.By minimizing the consistency loss function, the model parameters are fine-tuned to form a semi-supervised training model, which can still have high prediction ability in the case of a small amount of labeled data. After completing the construction of the semi-supervised training model, the generalization ability of the model on unlabeled data is improved. Therefore, the semi-supervised training model is applied to unlabeled data, and pseudo labels are generated based on the model prediction results. Unlabeled data is predicted, and the prediction results with confidence higher than the preset target value are screened out as pseudo labels combined with labeled data to construct an enhanced training set. On this basis, iterative training is performed so that the model gradually learns more equipment operation modes and continuously optimizes its own discrimination ability. Through this self-training process, the model uses existing labeled data for supervised learning, while effectively using unlabeled data, thereby improving its adaptability in real industrial environments. After multiple rounds of iterative training, the equipment status monitoring model is finally obtained, which can monitor the operating status of the equipment in real time and predict equipment failures in combination with multimodal sensor data.

[0021] Step S300: Perform system-level status assessment and subsystem-level fault prediction calculation based on the equipment status monitoring model to generate equipment operation and maintenance strategies; Specifically, the real-time prediction results of the equipment status monitoring model are decomposed hierarchically to construct the top-level control unit and the bottom-level execution unit. In this hierarchical structure, the health status optimization problem of the entire equipment group is defined as a system-level problem, while the fault prediction and decision-making of a single device are divided into subsystem-level problems, so that problems at different levels can be optimized at different granularities. The system-level problem focuses on the overall operating status of the equipment group to ensure the global optimal management strategy, while the subsystem-level problem focuses on the state changes of specific devices so as to carry out personalized maintenance and optimization for the operating conditions of different devices. On the basis of establishing a hierarchical structure, an information transmission mechanism between the system-level problem and the subsystem-level problem is constructed to ensure the coordinated optimization of the two. The information interaction relationship is established through the equipment state vector and the coordination parameter vector, so that the system-level management strategy affects the decision-making of a single device, and the operating status of a single device can also be fed back to the system-level optimization model to form an adaptive optimization framework. In this process, the health index of each device is calculated to quantify its current operating status, and the health index of the individual device is comprehensively calculated by combining the weighted geometric mean analysis method to obtain the system-level status evaluation result of the entire equipment group. Define the optimization goal of the system-level problem, and build a sequential fault propagation model for each device on this basis. The sequential fault propagation model is used to describe the state changes of the device at different time steps and predict its future fault risk. In the modeling process, the state space and action space are defined, where the state space includes the operating parameters, health status and external environmental factors of the device, and the action space defines the executable maintenance or scheduling operations, such as adjusting the operating load, performing preventive maintenance or performing fault repair. Under this modeling framework, the subsystem-level decision model is optimized using the reinforcement learning algorithm to generate the optimal device management strategy under different operating conditions. In order to optimize the subsystem-level decision model, the Q-learning algorithm is used to update its Q value so that the model can learn the optimal maintenance strategy in the process of continuous interaction. The Q-learning algorithm calculates the optimal action value under different states, so that the model learns the optimal maintenance decision based on historical data and dynamically adjusts it under new equipment operating conditions. In order to improve the convergence speed and stability of the algorithm, the experience replay mechanism is used for batch update, that is, historical decision data is randomly extracted for learning in each training stage to avoid the model from falling into the local optimal solution and improve its adaptability to different working conditions. Through this process, a preliminary equipment management strategy, namely the original equipment management strategy, is obtained. This strategy dynamically adjusts the maintenance plan according to the historical operation data and current status of the equipment, thereby improving the overall reliability of the equipment. The Lagrange multiplier method is used to jointly optimize the system-level problem and the subsystem-level problem.By introducing Lagrange multipliers, the system-level objective function is combined with the constraints of the subsystem-level maintenance decision, so that the maintenance strategy of each device can be dynamically adjusted during the entire optimization process to achieve the global optimum. By iteratively solving this optimization problem, the global optimal solution of the equipment operation and maintenance strategy is obtained, and the equipment operation and maintenance strategy is generated.

[0022] Step S400: Generate a fault risk heat map based on the equipment operation and maintenance strategy, build a digital twin visualization system, and output an equipment maintenance scheduling plan.

[0023] Specifically, the equipment operation and maintenance strategy is decomposed in time to obtain multiple operation and maintenance time slots, so as to clarify the health status and maintenance requirements of the equipment in different time periods. Based on the historical operation data, the prediction results of the equipment status monitoring model and the maintenance history records, the equipment failure probability distribution of each operation and maintenance time slot is calculated to ensure the accurate quantification of future failure risks. According to the spatial distribution of failure probability, a failure risk heat map is generated to intuitively show the failure risk level of different equipment in different time periods, so that operation and maintenance personnel can quickly identify high-risk equipment and formulate corresponding maintenance strategies. Threshold detection is performed on the failure risk heat map to identify the trigger conditions for maintenance scheduling. By setting the failure risk threshold value suitable for different equipment types and working conditions, it is determined which equipment needs immediate maintenance, which equipment delays maintenance, and which equipment continues to operate without intervention. In order to ensure the reliability of fault detection, a dynamic threshold adjustment strategy is adopted, so that the system dynamically adjusts the maintenance trigger conditions according to the operation history and failure trend of the equipment to avoid maintenance too early or too late, thereby optimizing the utilization efficiency of maintenance resources. Based on the maintenance scheduling trigger conditions, the constraint condition data of production task priority, maintenance personnel availability, spare parts inventory status, equipment interdependence and expected maintenance time are collected. The production task priority determines the flexibility of the maintenance time window, the availability of maintenance personnel affects the actual executable maintenance tasks, the spare parts inventory status determines whether maintenance can be performed immediately, and the interdependence of equipment requires ensuring that the maintenance of some equipment does not affect the normal operation of other equipment. Based on these constraint data, a maintenance scheduling optimization model is constructed to find the optimal maintenance decision-making plan in a complex production environment. The maintenance scheduling optimization model is solved to obtain the optimal maintenance schedule. The optimization problem is solved using integer linear programming, mixed integer programming or heuristic algorithms (such as genetic algorithms, particle swarm optimization, etc.) to minimize maintenance costs, reduce unplanned downtime, and maximize production efficiency. In the solution process, multi-objective optimization strategies are considered so that the maintenance schedule can reduce the risk of equipment failure and minimize interference with the production plan. Based on the optimal maintenance schedule, the production plan is integrated to form an equipment load adjustment plan to ensure the continuity of the production plan and optimize the maintenance cycle and workload distribution of the equipment. Based on the optimal maintenance schedule and equipment load adjustment plan, a digital twin visualization system is constructed. The system contains four core views, namely, the equipment topology view, the equipment details view, the prediction analysis view, and the maintenance plan view. In the device topology view, all devices are visualized according to their physical connections and functional relationships. In the device details view, the current operating status, historical maintenance records, and fault prediction information of each device are reflected. In the predictive analysis view, the prediction results of the machine learning model are used to provide trend analysis of the future device operating status, and combined with the fault risk heat map, the health index of the device is presented in the form of color or value.In the maintenance plan view, the optimal maintenance schedule is combined with the production plan, and the maintenance task arrangement is displayed in the form of a Gantt chart or timeline, so that maintenance personnel can arrange their working time reasonably and ensure the execution of the maintenance plan. Through the digital twin visualization system, the optimized equipment maintenance scheduling plan is output and fed back to the production management system to realize intelligent operation and maintenance management.

[0024] In the embodiment of the present invention, the present invention uses an iterative filtering interpolation algorithm to smooth the multimodal sensor data, effectively eliminates random noise interference, retains the trend of equipment state changes, and significantly improves the data quality of subsequent feature extraction. In particular, the adaptive weighted moving average method can automatically adjust the sliding window size according to the local change rate of the data, and achieve a stronger smoothing effect while ensuring the state mutation characteristics. The key feature points are extracted by an improved extreme value detection algorithm, and the key moments of equipment state changes are accurately captured by combining the minimum interval constraint and local curvature analysis. This method reduces the data dimension while retaining key information, greatly improves the feature extraction efficiency and feature expression ability, and uses the Siamese-like structure to effectively utilize the complementarity of different modal data of the same device, significantly improving the learning ability of the model in the case of scarce labeled data. The equipment state assessment is regarded as a system-level problem, and the fault prediction is regarded as a subsystem-level problem. The global optimal solution is solved by coupling the Lagrange multiplier method, balancing production efficiency and equipment health management. The maintenance scheduling optimization model based on the failure risk heat map comprehensively considers multiple constraints such as production task priority, maintenance personnel availability, spare parts inventory status, etc., generates the optimal maintenance schedule and equipment load adjustment plan, avoids production interruptions, improves equipment availability, and reduces maintenance costs. The digital twin visualization system provides interactive operations in four dimensions: equipment topology view, detail view, predictive analysis view, and maintenance plan view. Factory managers can intuitively understand the equipment status, foresee potential risks, and make timely decisions. At the same time, it integrates with external systems such as enterprise resource planning and manufacturing execution systems through standardized API interfaces, realizing the coordinated optimization of maintenance plans and production plans.

[0025] In a specific embodiment, the process of executing step S100 may specifically include the following steps: Perform signal acquisition on vibration, temperature, sound and current data collected by multimodal sensors in digital factories to obtain raw sensor data; Performing linear normalization mapping on the raw sensor data to obtain normalized sensor data, and performing time stamp calibration on the normalized sensor data to obtain time synchronized data; Identify data points that deviate from normal distribution in the time-synchronized data to obtain a data set marked with outliers, and fill in the missing data segments in the data set marked with outliers to obtain complete and continuous sensor data; Organize the complete and continuous sensor data in JSON format and add metadata information such as device ID, sensor ID, and acquisition time to obtain standardized multi-sensor data; The standardized multi-sensor data is iteratively filtered and interpolated to obtain smooth time series data, and feature points are extracted from the smooth time series data to construct a multimodal feature matrix.

[0026] Specifically, the vibration, temperature, sound and current data collected by the multimodal sensor are collected to obtain the original sensor data. The original sensor data is linearly normalized to adjust the data of different physical quantities to the same scale range. The mathematical expression of linear normalization is as follows: in, represents the raw sensor data, is the normalized data, and Represent the minimum and maximum values ​​of the sensor data, respectively. Normalized data ensures that different types of sensor data are compared and modeled in the same numerical range (usually [0,1] or [-1,1]) to avoid numerical deviations caused by scale differences. Timestamp calibration is performed on the normalized sensor data to ensure that all data points can be aligned along a unified time axis. The time synchronization process uses interpolation methods to fill in uneven time intervals. Assume that the time series of a sensor is , and the time series of the other sensor is ,if Then the data values ​​of the missing time points are calculated by linear interpolation: in, is the interpolated data point, and are known adjacent data points, is the time point that needs to be interpolated. After completing time synchronization, identify the data points that deviate from the normal distribution, that is, outliers. Outlier detection is based on statistical methods, such as mean and standard deviation analysis. Assuming that the sensor data follows a normal distribution, the data point The conditions to be considered as outliers are: in, is the mean value of the sensor data, is the standard deviation, is the threshold value (usually ). If a data point meets this condition, the point is marked as an outlier and stored in the dataset marked with outliers. For the dataset marked with outliers, fill in the missing data segments to obtain complete and continuous sensor data. The filling method uses local mean filling, that is: in, is the filled data point, and The previous and next valid data points respectively. Organize the complete and continuous sensor data in JSON format, and add metadata information such as device ID, sensor ID and acquisition time to generate standardized multi-sensor data. Perform iterative filtering and interpolation processing on the standardized multi-sensor data to obtain smooth time series data. Use sliding average filtering and exponentially weighted moving average filtering. The formula for sliding average filtering is: in, is the sliding window size, is the smoothed data value. Reduce high-frequency noise and improve data stability. The expression of exponential weighted moving average filtering is: in, is the smoothing factor, usually Larger Makes data response faster and smaller Make the data smoother. Based on the smoothed time series data, feature points are extracted to construct a multimodal feature matrix. Feature extraction methods include time domain features, frequency domain features, and time-frequency features. Time domain features include mean, standard deviation, skewness, peak factor, etc. The calculation formula is as follows: in, Indicates the peak factor. The larger the value, the stronger the impact component in the signal, which means abnormal vibration or failure. Representation data In a given time window The maximum value within reflects the extreme peak value of the signal; Indicates the number of data points in the window; Represents a time window The first data points; It means calculating the signal energy, which is the sum of the squares of all data points; Represents the root mean square of the signal, which is used to measure the overall energy level of the signal. The frequency domain features are extracted through Fourier transform to extract the main frequency components, and the calculation formula is: in, is the frequency domain signal, is an imaginary unit. The time-frequency features are combined with short-time Fourier transform or wavelet transform to obtain feature information on different time scales. All extracted features are organized into a multimodal feature matrix for equipment status analysis and fault prediction. For example, a simplified multimodal feature matrix is ​​represented as follows: in, Represents the vibration characteristics, Represents the temperature characteristics, Indicates the sound characteristics, Represents the current characteristics, and each row represents the feature vector at a time point.

[0027] In a specific embodiment, the execution step performs iterative filtering and interpolation processing on the standardized multi-sensor data to obtain smooth time series data, and extracts feature points from the smooth time series data. The process of constructing a multimodal feature matrix can specifically include the following steps: Perform sliding window segmentation processing on the standardized multi-sensor data to obtain multiple signal window data segments; Performing fast Fourier transform on multiple signal window data segments to obtain frequency domain representation data, and performing frequency band decomposition on the frequency domain representation data to obtain multi-band decomposition data; De-noising the multi-band decomposition data to obtain frequency domain data after filtering; Perform inverse transformation on the filtered frequency domain data to reconstruct it into a time domain signal to obtain preliminary smoothed data; Apply the adaptive weighted moving average method to the preliminary smoothed data, automatically adjust the sliding window size according to the local change rate of the data, and perform resampling processing to obtain smooth time series data; Based on the smoothed time series data, key feature points are extracted and multimodal feature vectors are constructed to obtain a multimodal feature matrix.

[0028] Specifically, the standardized multi-sensor data is segmented by sliding windows to obtain multiple signal window data segments, thereby forming a local stable analysis unit in the time series. Directly affects the resolution of the signal, using a fixed window method or dynamic adjustment based on the data change rate to ensure that the signal information in different time segments is balanced. Next, for sensor data , construct multiple window data segments : in is the window index. Fast Fourier transform is performed on multiple signal window data segments to convert the time domain signal into frequency domain representation data. The complex time series signal is decomposed into the superposition of multiple sine waves. The calculation formula is as follows: in, represents the transformed frequency domain data, is the signal length, is an imaginary unit, is the frequency index. Perform frequency band decomposition on the frequency domain representation data to obtain multi-band decomposition data. The frequency band decomposition adopts a bandpass filter bank or a wavelet transform method, where the design of the bandpass filter follows: in, and are the starting frequency and ending frequency of the frequency band respectively. Yes, decompose the signal information of different frequency bands so that different frequency components can be analyzed independently, thus avoiding the aliasing problem of global frequency domain information. De-noise the multi-band decomposition data to obtain more reliable frequency domain features. De-noising methods include wavelet threshold denoising, empirical mode decomposition, and adaptive filtering. The mathematical expression of wavelet threshold denoising is as follows: in, is the noise threshold, which is determined by analyzing the statistical characteristics of the noise, so that the denoising process can effectively remove irrelevant interference without losing key frequency components. After denoising, the frequency domain data after filtering is inversely transformed to reconstruct the time domain signal. The formula is as follows: This process is equivalent to the inverse Fourier transform, which converts the denoised frequency information back to time series data to obtain preliminary smoothed data. The adaptive weighted moving average method is applied to the preliminary smoothed data, and the sliding window size is automatically adjusted according to the local change rate of the data, so that a larger window is used in a stable state to enhance the smoothing effect, and a smaller window is used in the mutation area to retain the key signal fluctuations. Its mathematical expression is as follows: in, For the current time point Adaptive window size at is the weighting coefficient, which is adjusted according to the local change rate of the data, for example: in, is the base window size, is the standard deviation of the data, which is used to measure the degree of local change. When the data changes dramatically, the window size is reduced to retain the mutation information, and when the data changes steadily, the window size is increased to enhance the smoothing effect. Resampling is performed to align the data points of all sensors and ensure that the time intervals of the data are uniform. Linear interpolation or spline interpolation methods are used to ensure the time continuity of the data. After obtaining the smoothed time series data, key feature points are extracted to construct a multimodal feature vector and form a multimodal feature matrix. The feature extraction process includes time domain features, frequency domain features and time-frequency features, among which the time domain features include mean, standard deviation, peak factor, etc. The calculation formula is as follows: The frequency domain features are based on fast Fourier transform to extract the main frequency components, and the calculation formula is: The time-frequency features obtain signal features on different time scales through short-time Fourier transform or wavelet transform, so that the device state changes can be accurately captured. All feature points are integrated into a multimodal feature matrix: in, Represents the vibration characteristics, Represents the temperature characteristics, Indicates the sound characteristics, Represents the current characteristics, and each row represents the feature vector at a time point.

[0029] In a specific embodiment, the execution step extracts key feature points based on the smoothed time series data and constructs a multimodal feature vector to obtain a multimodal feature matrix, which may specifically include the following steps: Construct a data difference sequence for the smoothed time series data and calculate the standard deviation to obtain the adaptive threshold parameter; Perform extreme value detection on the data difference sequence according to the adaptive threshold parameter and apply the minimum interval constraint to obtain a set of candidate feature points; Calculate the local curvature of the smoothed time series data, identify the points where the curvature changes significantly, and obtain a set of supplementary feature points; Calculate the signal square sum in the sliding window for the vibration signal in the smoothed time series data, identify the energy mutation point, and perform short-time Fourier transform to obtain frequency domain feature data; Perform harmonic analysis on the current signal in the smoothed time series data, calculate the total harmonic distortion rate, and obtain current characteristic data; A multimodal feature matrix including vibration, temperature, sound and current feature components is constructed based on the candidate feature point set, the supplementary feature point set, the frequency domain feature data and the current feature data.

[0030] Specifically, in order to highlight the changing trend of data and enhance the sensitivity to sudden anomalies, a data difference sequence is constructed for the smoothed time series data, and the increments of adjacent data points are calculated. The mathematical expression is as follows: in, For the data points at time steps, is the difference between adjacent data points. The data difference sequence can eliminate the global trend and make short-term fluctuations more obvious. In order to set the adaptive threshold for anomaly detection, the standard deviation of the data difference sequence is calculated, and the calculation formula is: in, is the total number of data points, is the mean of the data difference sequence, is the standard deviation of data fluctuations. Based on the standard deviation, define the adaptive threshold parameters In order to perform extreme value detection later: in, is an adjustable scaling factor, set to 3, to detect abnormal points outside the standard deviation range. After obtaining the adaptive threshold parameters, extreme value detection is performed on the data difference sequence to screen out points with significant changes as candidate feature points. satisfy: The point is marked as a candidate feature point. A minimum interval constraint is applied to ensure that the distance between two adjacent feature points is not less than the threshold. : in, and Represents the timestamps of two adjacent feature points. This constraint can effectively reduce the redundancy of feature points and improve the representativeness of feature points. Calculate the local curvature of smoothed time series data and identify points where the curvature changes significantly to obtain a set of supplementary feature points. The calculation of local curvature can measure the nonlinear changes of data, and its mathematical expression is as follows: in, For the The curvature value at each time step, A small constant to prevent the denominator from approaching zero. When the set threshold is exceeded, the point is added to the set of supplementary feature points. This method can identify inflection points or points where the data curvature changes suddenly, thereby enhancing the ability to capture signal mutations. After completing the extraction of time domain feature points, extract frequency domain features, such as performing sliding window square sum calculation on the vibration signal to identify energy mutation points. The calculation formula for the sliding window square sum is as follows: in, For the The signal energy in the window of time steps is is the size of the sliding window. If the energy change rate If the set threshold is exceeded, it is considered that an energy mutation occurs: In order to further analyze the frequency components of the vibration signal, a short-time Fourier transform is performed, and the calculation formula is: in, Indicates time The spectrum information at is a window function, is an imaginary unit, The short-time Fourier transform result provides the frequency component of the signal that changes over time, thereby extracting key information about the operating status of the equipment. For the current signal, harmonic analysis is performed and the total harmonic distortion (THD) is calculated to evaluate the operating status of the motor. The calculation formula for THD is as follows: in, For the The amplitude of the harmonic component, is the fundamental component, is the harmonic order. The higher the THD, the more serious the harmonic interference of the motor is. Based on the candidate feature point set, the supplementary feature point set, the frequency domain feature data of the vibration signal, and the harmonic feature data of the current signal, a multimodal feature matrix is ​​constructed. Assume that the time step is , the feature matrix is ​​in the following form: in, represents the vibration characteristics, Represents the temperature characteristics, Represents the sound characteristics, Represents the current characteristics, each row represents the data of a time step, and each column represents the characteristics of a mode.

[0031] In a specific embodiment, the process of executing step S200 may specifically include the following steps: Decomposing the multi-modal feature matrix into four modal branch data, the four modal branch data include vibration features, temperature features, sound features and current features; Perform feature mapping on each modal branch data to obtain four modal mapping features; The four modal mapping features are processed by M groups of enhanced node groups respectively to obtain four modal enhanced features, and the four modal enhanced features are fused into a unified feature representation through a connection operation; Perform Moore-Penrose generalized inverse matrix calculation on the unified feature representation, solve the output layer weights, and create an initial network model based on the output layer weights; A Siamese-like structure is constructed based on the data of four modal branches. The Siamese-like structure includes four parallel sub-networks that share structural parameters. Each parallel sub-network contains feature mapping layer parameters to enhance node parameters. The vibration modal branch is set as the anchor branch, and the four modal mapping features are mapped through their respective parallel sub-networks to obtain four encoding features. A pairwise distance metric is constructed between the encoding features of the four parallel sub-networks, and different modal data of the same device are regarded as positive samples to construct a consistency loss function. Apply training samples and cosine annealing learning rate scheduling strategy to the initial network model, and fine-tune the model parameters by minimizing the consistency loss function to obtain a semi-supervised training model; The semi-supervised training model performs predictions on unlabeled data, and the training prediction results with confidence levels higher than the preset target values ​​are combined with labeled data as pseudo-labels for iterative training to obtain an equipment status monitoring model.

[0032] Specifically, based on the data collected from the multimodal sensor, a multimodal feature matrix containing vibration, temperature, sound and current features is constructed, and its mathematical expression is as follows: in, Representative The vibration characteristics of the time step, Represents the temperature characteristics, Represents the sound characteristics, represents the current characteristics, and each row represents the sensor data at different time steps. In order to ensure that data of different modes are processed independently in the model, Decomposed into four modal branch data, namely Perform feature mapping on each modal branch data to extract more discriminative features. Assume that the feature mapping function is , then the modal mapping feature is expressed as: in, They are the mapping features of vibration, temperature, sound, and current respectively. The group of enhancement nodes performs feature enhancement on each modal mapping feature, and each enhancement node is represented as a nonlinear transformation Combination of: in, Represents the enhanced modal features. All modal enhancement features are fused through the concatenation operation to form a unified feature representation: In getting In order to solve the output layer weights, the Moore-Penrose generalized inverse matrix calculation is performed, which is mathematically expressed as follows: in, is the output layer weight, for The Moore-Penrose pseudo-inverse matrix, is the corresponding device status label. After that, the initial network model is created. In order to enhance the representation ability of multimodal features, a Siamese-like structure is constructed based on the four modal branch data. The structure contains four parallel sub-networks that share structural parameters. Each sub-network contains a feature mapping layer and enhanced node parameters. In this structure, the vibration modal branch is set as the anchor branch, that is: in, are the encoding features of the four parallel sub-networks, A neural network map representing shared parameters. Based on this, a pairwise distance metric is constructed to calculate the similarity of different modal data of the same device and treat them as positive pairs. The Euclidean distance calculation method is used: And define the consistency loss function: in, Representing modality and From the same device, is the preset boundary value. By minimizing this loss function, the model optimizes the alignment relationship between modalities. During the training process, the cosine annealing learning rate scheduling strategy is adopted so that the learning rate increases with the number of training rounds. Gradually decrease: in, is the initial learning rate, is the total number of training rounds. By minimizing the consistency loss function and optimizing the parameters and , and a semi-supervised training model is obtained. In order to use unlabeled data to improve model performance, a pseudo-label method is used, that is, prediction is performed on unlabeled data, and the confidence level is higher than a preset threshold. A sample of: in, A set of pseudo labels representing high-confidence predictions. The pseudo-label data is merged with the labeled data, and iterative training is performed again to finally obtain the equipment status monitoring model.

[0033] In a specific embodiment, the process of executing step S300 may specifically include the following steps: The real-time prediction results of the equipment status monitoring model are hierarchically decomposed to construct the top-level control unit and the bottom-level execution unit. The overall equipment group health status optimization is defined as a system-level problem, and the single equipment fault prediction and decision-making are defined as subsystem-level problems. The information transmission mechanism between system-level problems and subsystem-level problems is established through the device state vector and coordination parameter vector, and the health index calculation and weighted geometric mean analysis are performed on each device to obtain the system-level status assessment result; Define the system-level optimization goals for system-level problems, build a sequential fault propagation model for each device, define the state space and action space, and obtain a subsystem-level decision model; The Q-value of the subsystem-level decision model is updated through the Q-learning algorithm, and the experience replay mechanism is used for batch update to obtain the original equipment management strategy; The system-level problem is coupled with the subsystem-level problem through the Lagrange multiplier method, and the global optimal solution of the original equipment management strategy is iteratively solved to generate the equipment operation and maintenance strategy.

[0034] Specifically, the real-time prediction results of the equipment status monitoring model are hierarchically decomposed, and the top-level control unit and the bottom-level execution unit are constructed. Under this framework, the health status optimization of the entire equipment group is defined as a system-level problem, while the fault prediction and maintenance decision of a single device is defined as a subsystem-level problem. In order to achieve this hierarchical structure, the prediction results of the monitoring model are decomposed so that the top-level control unit evaluates the health status from a global perspective, while the bottom-level execution unit makes optimization decisions based on the specific status of a single device. Under this hierarchical architecture, an information transfer mechanism between system-level problems and subsystem-level problems is established through the equipment state vector and the coordination parameter vector. Assume that the equipment state vector is ,in Representative The state vector of each device includes information such as health index and abnormality of sensor data. The coordination parameter vector It is used to describe the relationship between devices, such as load sharing, energy consumption impact and other constraints, so that the system-level control can coordinate the subsystems based on the principle of global optimization. On this basis, the health index of each device is calculated to measure its current status. The health index is calculated using the weighted geometric average method: in, For the The health index of each device, Represents the device state characteristics, is the importance weight of the feature. By calculating the health index of all devices and based on weighted geometric mean analysis, the health status assessment result of the entire system is obtained: in, represents the system-level health assessment value, and Represents the contribution weight of each device to the overall health status. In order to achieve system-level optimization, the optimization goal of the system-level problem is defined, and a sequential fault propagation model is built for each device to describe the evolution of the device status over time. For the Devices at time The health state of is, then its fault propagation model is expressed as: in, For maintenance or scheduling actions taken on the equipment, Indicates that the equipment status is affected by the maintenance operation, is an external interference factor. On this basis, the state space and action space are defined, where the state space Contains all possible device health states, and the action space This includes possible maintenance actions, such as adjusting loads, performing preventive maintenance, and performing fault repair. In order to optimize the subsystem-level decision model, the Q-learning algorithm is used to update its Q value, so that each device can autonomously learn the optimal maintenance strategy. The core of Q-learning is to update the state-action value function. : in, is the learning rate, is the discount factor, is the current reward value, To perform actions The new state after Represents the optimal action that may be taken in the future. In order to improve learning efficiency, the experience replay mechanism is adopted, that is, in each training step, a batch of samples are randomly selected from the past experience pool for learning to reduce data correlation and improve the stability of the model. After obtaining the original equipment management strategy, the system-level problem is coupled with the subsystem-level problem through the Lagrange multiplier method to solve the global optimal equipment operation and maintenance strategy. The system-level optimization goal is defined as: in, For maintenance costs, is the Lagrange multiplier, Represents system-level constraints, such as dependencies between devices, resource allocation, etc. The Lagrangian function is: in, represents the constraints in the subsystem level problem, is the corresponding Lagrange multiplier. and Perform iterative updates: The globally optimal equipment management strategy is gradually solved so that the system-level health status and subsystem-level maintenance decisions are coordinated with each other to achieve the goal of global optimization.

[0035] In a specific embodiment, the process of executing step S400 may specifically include the following steps: Decompose the equipment operation and maintenance strategy in time to obtain multiple operation and maintenance time slots, calculate the equipment failure probability distribution of each operation and maintenance time slot, and generate a failure risk heat map; Perform threshold detection on the fault risk heat map to obtain maintenance scheduling trigger conditions; Based on the maintenance scheduling trigger conditions, the data on production task priority, maintenance personnel availability, spare parts inventory status, equipment interdependencies, and estimated maintenance duration constraints are collected to build a maintenance scheduling optimization model. Solve the maintenance scheduling optimization model to obtain the optimal maintenance schedule, and integrate the optimal maintenance schedule with the production plan to obtain the equipment load adjustment plan; Based on the optimal maintenance schedule and equipment load adjustment plan, a digital twin visualization system is built that includes equipment topology view, equipment details view, predictive analysis view, and maintenance plan view, and the equipment maintenance scheduling plan is output.

[0036] Specifically, the equipment operation and maintenance strategy is decomposed into multiple operation and maintenance time slots, and the failure probability distribution of the equipment in each time slot is calculated to generate a failure risk heat map. Assume that time is divided into non-overlapping time slots, each time slot Device Health Index characterizes, and the failure probability is predicted using historical data and current status. Indicates Devices in time slot The failure probability within is estimated by the exponentially weighted moving average method: in, is the smoothing factor, is a mapping function between the equipment health index and the probability of failure, based on historical data fitting, such as through a logistic regression model: in, is the regression parameter. After calculating the failure probability of all devices in all time slots, map it to a two-dimensional space, with the device location as the horizontal and vertical coordinates and the failure probability as the color weight, to form a failure risk heat map. The high-risk area in the heat map corresponds to the device with a high failure probability, and the low-risk area indicates that the device is relatively stable. Perform threshold detection on the failure risk heat map to determine the triggering conditions for maintenance scheduling. Set a failure probability threshold , when the failure probability of a certain device Exceed When the device enters the maintenance schedule trigger list: in, Represents a collection of equipment that needs maintenance. The threshold is set based on experience or adjusted through optimization methods to balance maintenance costs and failure risks. For equipment on the maintenance trigger list, its maintenance needs are analyzed, including production task priority, maintenance personnel availability, spare parts inventory status, equipment interdependencies, and expected maintenance duration constraints, and a maintenance scheduling optimization model is constructed. In the maintenance scheduling optimization model, is a decision variable. If the device In the time slot For maintenance, , otherwise it is 0. The objective function is to minimize the sum of maintenance cost and production loss: in, Representative equipment In the time slot The cost of carrying out maintenance, Indicates production tasks Losses due to maintenance, is the degree to which the task is affected by maintenance. Constraints include: Maintenance resource constraints: The number of maintenance personnel cannot exceed the available number : Spare parts inventory constraint: For the Class spare parts in time The inventory volume, Indicates the device The number of spare parts required for maintenance is: Equipment interdependency constraints: If the equipment Device dependent , then they cannot simultaneously maintain: After solving the maintenance scheduling optimization model, the optimal maintenance schedule is obtained ,Right now: In order to further reduce the impact of maintenance on production, the optimal maintenance schedule is integrated with the production plan to generate an equipment load adjustment plan. For production tasks In time The equipment demand is Indicates the device For maintenance, adjust the equipment load: in, Representative equipment To the task The contribution of. By adjusting the equipment allocation of production tasks, the impact of the production plan can be minimized. After obtaining the optimal maintenance schedule and equipment load adjustment plan, a digital twin visualization system is built so that operation and maintenance personnel can intuitively understand the equipment status, failure risk and maintenance plan. The digital twin system contains four core views: Equipment topology view: displays the connection relationship between devices, so that operation and maintenance personnel can intuitively understand the impact of maintenance operations on the entire system; Equipment details view: provides status information of a single device, including health index, failure probability, maintenance records, etc.; Prediction analysis view: displays equipment failure trends and prediction results, and combines failure risk heat maps to highlight high-risk areas; Maintenance plan view: displays the optimal maintenance schedule in the form of a Gantt chart or timeline, and combines the production task adjustment results to provide maintenance decision support.

[0037] See also Figure 2 , Figure 2 A schematic block diagram of the structure of a digital factory-based equipment real-time monitoring and fault prediction system 200 provided in an embodiment of the present application, such as Figure 2 As shown, the equipment real-time monitoring and fault prediction system 200 based on the digital factory includes: The acquisition module 210 is used to perform iterative filtering and interpolation processing on the equipment operation data collected by the multimodal sensor in the digital factory to obtain smooth time series data, and construct a multimodal feature matrix based on the smooth time series data; A model training module 220 is used to input the multimodal feature matrix into a joint training generalized Siamese-like extensive learning network for model training to obtain a device status monitoring model; Prediction calculation module 230, used to perform system-level status evaluation and subsystem-level fault prediction calculation based on the equipment status monitoring model, and generate equipment operation and maintenance strategies; The digital twin module 240 is used to generate a fault risk heat map according to the equipment operation and maintenance strategy and build a digital twin visualization system to output the equipment maintenance scheduling plan.

[0038] Through the cooperation of the above-mentioned components, the present invention uses an iterative filtering and interpolation algorithm to smooth the multimodal sensor data, effectively eliminate random noise interference, retain the trend of equipment state changes, and significantly improve the data quality of subsequent feature extraction. In particular, the adaptive weighted moving average method can automatically adjust the sliding window size according to the local change rate of the data, and achieve a stronger smoothing effect while ensuring the state mutation characteristics. The key feature points are extracted by the improved extreme value detection algorithm, combined with the minimum interval constraint and local curvature analysis, to achieve accurate capture of the key moments of equipment state changes. This method reduces the data dimension while retaining key information, greatly improves the feature extraction efficiency and feature expression ability, and uses the Siamese-like structure to effectively utilize the complementarity of different modal data of the same device, significantly improving the learning ability of the model in the case of scarce labeled data. Taking equipment state evaluation as a system-level problem and fault prediction as a subsystem-level problem, the global optimal solution is solved by coupling the Lagrange multiplier method, balancing production efficiency and equipment health management. The maintenance scheduling optimization model based on the failure risk heat map comprehensively considers multiple constraints such as production task priority, maintenance personnel availability, spare parts inventory status, etc., generates the optimal maintenance schedule and equipment load adjustment plan, avoids production interruptions, improves equipment availability, and reduces maintenance costs. The digital twin visualization system provides interactive operations in four dimensions: equipment topology view, detail view, predictive analysis view, and maintenance plan view. Factory managers can intuitively understand the equipment status, foresee potential risks, and make timely decisions. At the same time, it integrates with external systems such as enterprise resource planning and manufacturing execution systems through standardized API interfaces, realizing the coordinated optimization of maintenance plans and production plans.

[0039] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0040] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0041] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for real-time monitoring and fault prediction of equipment based on a digital factory, characterized in that: include: Iterative filtering and interpolation processing is performed on the equipment operation data collected by the multimodal sensors in the digital factory to obtain smooth time series data, and a multimodal feature matrix is ​​constructed based on the smooth time series data; Inputting the multimodal feature matrix into a joint training generalized Siamese-like extensive learning network for model training to obtain a device status monitoring model; Perform system-level status evaluation and subsystem-level fault prediction calculation based on the equipment status monitoring model to generate equipment operation and maintenance strategies; Generate a fault risk heat map based on the equipment operation and maintenance strategy, build a digital twin visualization system, and output the equipment maintenance scheduling plan.

2. The method for real-time monitoring and fault prediction of equipment based on digital factory according to claim 1 is characterized in that: The iterative filtering and interpolation processing is performed on the equipment operation data collected by the multimodal sensor in the digital factory to obtain smooth time series data, and a multimodal feature matrix is ​​constructed based on the smooth time series data, including: Perform signal acquisition on vibration, temperature, sound and current data collected by multimodal sensors in digital factories to obtain raw sensor data; Performing linear normalization mapping on the raw sensor data to obtain normalized sensor data, and performing time stamp calibration on the normalized sensor data to obtain time synchronization data; Identifying data points that deviate from normal distribution in the time synchronization data to obtain a data set marked with abnormal values, and filling missing data segments in the data set marked with abnormal values ​​to obtain complete and continuous sensor data; Organizing the complete and continuous sensor data in JSON format and appending metadata information of device ID, sensor ID and collection time to obtain standardized multi-sensor data; The standardized multi-sensor data is iteratively filtered and interpolated to obtain smoothed time series data, and feature points are extracted from the smoothed time series data to construct a multimodal feature matrix.

3. The method for real-time monitoring and fault prediction of equipment based on digital factory according to claim 2 is characterized in that: The iterative filtering and interpolation processing is performed on the standardized multi-sensor data to obtain smooth time series data, and feature points are extracted from the smooth time series data to construct a multimodal feature matrix, including: Performing sliding window segmentation processing on the standardized multi-sensor data to obtain a plurality of signal window data segments; Performing fast Fourier transform on the multiple signal window data segments to obtain frequency domain representation data, and performing frequency band decomposition on the frequency domain representation data to obtain multi-band decomposition data; Performing denoising on the multi-band decomposition data to obtain frequency domain data after filtering; Performing inverse transformation on the frequency domain data after the filtering process to reconstruct it into a time domain signal to obtain preliminary smoothed data; Applying an adaptive weighted moving average method to the preliminary smoothed data, automatically adjusting the sliding window size according to the local change rate of the data, and performing resampling processing to obtain smoothed time series data; Key feature points are extracted based on the smoothed time series data and a multimodal feature vector is constructed to obtain a multimodal feature matrix.

4. The method for real-time monitoring and fault prediction of equipment based on digital factory according to claim 3 is characterized in that: The step of extracting key feature points based on the smoothed time series data and constructing a multimodal feature vector to obtain a multimodal feature matrix includes: Constructing a data difference sequence for the smoothed time series data and calculating the standard deviation to obtain an adaptive threshold parameter; Performing extreme value detection on the data difference sequence according to the adaptive threshold parameter and applying a minimum interval constraint to obtain a set of candidate feature points; Calculating the local curvature of the smoothed time series data, and identifying points where the curvature changes significantly to obtain a set of supplementary feature points; Calculating the signal square sum in the sliding window for the vibration signal in the smoothed time series data, identifying the energy mutation point, and performing short-time Fourier transform to obtain frequency domain feature data; Performing harmonic analysis on the current signal in the smoothed time series data, calculating the total harmonic distortion, and obtaining current characteristic data; A multimodal feature matrix including vibration, temperature, sound and current feature components is constructed based on the candidate feature point set, the supplementary feature point set, the frequency domain feature data and the current feature data.

5. The method for real-time monitoring and fault prediction of equipment based on digital factory according to claim 1 is characterized in that: The multimodal feature matrix is ​​input into a joint training generalized Siamese-like extensive learning network for model training to obtain a device status monitoring model, including: Decomposing the multimodal feature matrix into four modal branch data, wherein the four modal branch data include vibration features, temperature features, sound features, and current features; Perform feature mapping on each modal branch data to obtain four modal mapping features; The four modal mapping features are processed by M groups of enhanced node groups respectively to obtain four modal enhanced features, and the four modal enhanced features are fused to form a unified feature representation through a connection operation; Performing Moore-Penrose generalized inverse matrix calculation on the unified feature representation, solving output layer weights, and creating an initial network model based on the output layer weights; A Siamese-like structure is constructed based on the four modal branch data, wherein the Siamese-like structure includes four parallel sub-networks that share structural parameters, and each parallel sub-network includes feature mapping layer parameters to enhance node parameters; the vibration modal branch is set as an anchor branch, and the four modal mapping features are respectively mapped through respective parallel sub-networks to obtain four encoding features; a pairwise distance metric is constructed between the encoding features of the four parallel sub-networks, and different modal data of the same device are regarded as positive samples, and a consistency loss function is constructed; Applying training samples and a cosine annealing learning rate scheduling strategy to the initial network model, fine-tuning model parameters by minimizing the consistency loss function, and obtaining a semi-supervised training model; The semi-supervised training model performs prediction on unlabeled data, and the training prediction results with confidence levels higher than a preset target value are combined with labeled data as pseudo labels for iterative training to obtain an equipment status monitoring model.

6. The method for real-time monitoring and fault prediction of equipment based on digital factory according to claim 1 is characterized in that: The performing of system-level status evaluation and subsystem-level fault prediction calculation based on the equipment status monitoring model to generate equipment operation and maintenance strategies includes: The real-time prediction results of the equipment status monitoring model are hierarchically decomposed to construct a top-level control unit and a bottom-level execution unit, and the overall equipment group health status optimization is defined as a system-level problem, and the single equipment failure prediction and decision-making is defined as a subsystem-level problem; The information transmission mechanism between system-level problems and subsystem-level problems is established through the device state vector and coordination parameter vector, and the health index calculation and weighted geometric mean analysis are performed on each device to obtain the system-level status assessment result; Define the system-level optimization goals for system-level problems, build a sequential fault propagation model for each device, define the state space and action space, and obtain a subsystem-level decision model; The Q value of the subsystem-level decision model is updated by using a Q-learning algorithm, and an experience replay mechanism is used for batch updating to obtain an original equipment management strategy; The system-level problem is coupled with the subsystem-level problem through the Lagrange multiplier method, and the global optimal solution of the original equipment management strategy is iteratively solved to generate the equipment operation and maintenance strategy.

7. The method for real-time monitoring and fault prediction of equipment based on digital factory according to claim 1 is characterized in that: The method generates a fault risk heat map according to the equipment operation and maintenance strategy, builds a digital twin visualization system, and outputs an equipment maintenance scheduling plan, including: Decomposing the equipment operation and maintenance strategy in time to obtain multiple operation and maintenance time slots, and calculating the equipment failure probability distribution of each operation and maintenance time slot to generate a failure risk heat map; Performing threshold detection on the fault risk heat map to obtain maintenance scheduling trigger conditions; Based on the maintenance scheduling trigger conditions, data on production task priorities, maintenance personnel availability, spare parts inventory status, equipment interdependencies, and estimated maintenance duration constraints are collected to build a maintenance scheduling optimization model; Solving the maintenance scheduling optimization model to obtain an optimal maintenance schedule, and integrating the optimal maintenance schedule with the production plan to obtain an equipment load adjustment plan; Based on the optimal maintenance schedule and the equipment load adjustment plan, a digital twin visualization system including an equipment topology view, an equipment detail view, a predictive analysis view, and a maintenance plan view is constructed, and an equipment maintenance scheduling plan is output.

8. A real-time monitoring and fault prediction system for equipment based on a digital factory, characterized in that: The method for real-time monitoring and fault prediction of equipment based on a digital factory according to any one of claims 1 to 7 comprises: An acquisition module is used to perform iterative filtering and interpolation processing on the equipment operation data collected by the multimodal sensors in the digital factory to obtain smooth time series data, and to construct a multimodal feature matrix based on the smooth time series data; A model training module is used to input the multimodal feature matrix into a joint training generalized Siamese-like extensive learning network for model training to obtain a device status monitoring model; A prediction calculation module, used to perform system-level status evaluation and subsystem-level fault prediction calculation based on the equipment status monitoring model, and generate equipment operation and maintenance strategies; The digital twin module is used to generate a fault risk heat map according to the equipment operation and maintenance strategy and build a digital twin visualization system to output an equipment maintenance scheduling plan.

Images