Equipment multi-physics field modeling method based on industrial process and multi-modal fusion

Abstract

The invention belongs to the technical field of intelligent operation and maintenance of a nuclear power plant, and particularly relates to an equipment multi-physics field modeling method based on an industrial process and multi-modal fusion. Comprising the following steps: step 1, carrying out equipment multi-physics field modeling; 2, equipment upstream and downstream information, the environment and man-made influence modeling are carried out; and step 3, performing model fusion and corresponding multi-task training. The method has the beneficial effects that a neural network can be used for simulating a complex physical field mechanism according to a neural network simulation physical field technology, and for any smooth and local Lipschitz ordinary differential equation, a neural network can be used for approaching the equation at any approaching precision according to the Chen et al.1995. The method comprises the following steps of: performing multi-modal fusion, performing mutual mapping and learning on each physical field and vectors for expressing external information of equipment mainly through a pre-training mode, and enabling data of each dimension to form a unified manifold in a high-dimensional space.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent operation and maintenance of nuclear power plants, and specifically relates to a multi-physical field modeling method for equipment based on industrial processes and multi-modal fusion. Background Art

[0002] Multiphysics covers many disciplines in science and engineering. It is an applied subject that combines mathematics, physics, science and engineering applications, and numerical analysis. Mathematics usually involves partial differential equations and tensor analysis, while physics refers to common types of physical fields or physical processes. Multiphysics applications involve one or more physical processes or fields. Typical applications include soil consolidation theory, fluid dynamics simulation, electrodynamics applications, computational electromagnetic fields, sensor design (such as piezoelectric materials), fluid-structure interaction, energy and climate change research in porous materials, and other multimodal fusion.

[0003] Multimodal fusion refers to the integration of information obtained from different perception modes (such as vision, speech, touch, etc.) to improve the performance and intelligence of the system. Multimodal fusion methods can be divided into model-independent methods and model-based methods.

[0004] The use of traditional partial differential equation solvers to model the physical field of equipment mainly relies on finite interpolation, finite element method and pseudo-spectral method. These traditional methods are already quite mature and can strictly give the upper and lower bounds of the error and accurately give the algorithm complexity. These methods are already quite mature, but the main disadvantage is that the amount of calculation is particularly large. As long as there is a change in parameters or multi-physics field fusion, repeated and massive calculations are required, and the speed is difficult to meet the requirements of real-time calculations.

[0005] The use of deep neural networks to simulate partial differential equation operators for fitting has been applied in industry and academia, but it is basically limited to the device's own information. In actual scenarios, the operating data of the device is affected by information such as the upstream and downstream of the device, the environment in which the device is located, and the operation history. Modeling based solely on the device's own information cannot accurately reflect the actual status of the device in actual scenarios. Summary of the invention

[0006] The purpose of the present invention is to provide a multi-physical field modeling method for equipment based on industrial processes and multimodal fusion, which not only includes the multi-physical field information and corresponding physical mechanisms of the equipment itself, but also includes other factors affecting the operation of the equipment, thereby improving the accuracy of the equipment model.

[0007] The technical solution of the present invention is as follows: A method for multi-physical field modeling of equipment based on industrial process and multi-modal fusion, comprising the following steps:

[0008] Step 1: Equipment multi-physics modeling;

[0009] Step 2: Modeling of upstream and downstream information of the equipment, the environment in which it is located, and human influence;

[0010] Step 3: Model fusion and corresponding multi-task training.

[0011] The step 1 includes the following:

[0012] Step 11: Data processing;

[0013] Step 12: Neural network structure design.

[0014] The step 11 includes obtaining relevant data from the device's own sensors, external sensors, device environment sensors, device upstream and downstream information, and device operation history data, and performing general data processing. The relevant data processing process is as follows:

[0015] AData Cleaning

[0016] Eliminate obviously erroneous data; fill in blank data, fill in uncollected data points, use N / A to fill, or use algorithms for interpolation fitting; remove duplicate data, that is, reasonably remove duplicates of exactly the same data items;

[0017] B. Data Processing

[0018] According to the design manual, design drawings, and standard specifications, standard curves for each physical field of the equipment under various working conditions are established; all collected data are standardized; based on the actual collected standardized physical field data, the deviation value at each time point is calculated by comparing it with the ideal data under normal circumstances; the final single physical field data is composed of standardized sequence data and deviation sequence data.

[0019] The step 12 includes using a neural network to model the partial differential equation operator and designing a corresponding network architecture. The overall network architecture is divided into two parts: the device's own information and the device's external environment. The device model network models the device's own information, and models it separately according to whether the device-related information is scalar or non-scalar. The scalar information is extracted through a multi-layer perceptron, and the non-scalar is extracted through a unidirectional recurrent neural network. The scalar features and non-scalar features need to be dimensionally compressed after extraction so that the data dimensions of the two types of features are the same and merged. Finally, the connection between each other is constructed through the attention mechanism, that is, the merged data is used as a query, and the correlation between each position data and other position data is traversed and calculated so that the model can simultaneously consider the relationship between the two types of data.

[0020] The step 2 includes the following:

[0021] Step 21: Data processing;

[0022] Step 22: Neural network structure design.

[0023] The step 21 includes processing the data obtained from various sensors, historical information, manuals, videos, pictures, and other system outputs as follows:

[0024] A. Data cleaning

[0025] Eliminate obviously erroneous data; fill in blank data and fill in uncollected data points, which can be filled with N / A or interpolated using algorithms; remove duplicate data, that is, reasonably remove duplicate data entries that are exactly the same, and perform content recognition and result collation for data that requires text recognition; for video data, it is necessary to extract frames of effective content and remove the majority of repeated video content; for text data, remove common stop words and reduce the input of meaningless text;

[0026] B. Data Processing

[0027] First, it is necessary to establish standard curves for each physical field of the equipment under various working conditions based on the design manual, design drawings, and standard specifications; standardize all collected data; and calculate the deviation value for each time point based on the actual collected standardized physical field data and the ideal data under normal circumstances. At the same time, for text, video, and image data, the data must be processed according to the input format requirements of the feature extraction model used.

[0028] The step 22 includes designing different feature extraction structures for different types of data, using a recurrent neural network for non-scalar data and a multi-layer perceptron for scalar data to extract features; performing the same dimensional compression after feature extraction to make the dimensions of each type of feature data the same, and then building connections between each other through an attention mechanism, that is, taking the merged data as a query, and traversing and calculating the correlation between each position data and other position data so that the model can simultaneously consider the relationship between the two types of data.

[0029] The step 3 includes integrating the multi-physics field model of the device itself with the upstream, downstream, environment, and operation history associated with the device, and performing corresponding multi-task learning. The main process is as follows:

[0030] A. Fusion of the equipment’s own multi-physics model and upstream and downstream models

[0031] The device's own multi-physics field model and upstream and downstream models extract features respectively, and build their own attention layers respectively. The dimensions of query, key, and value in each attention layer are consistent. When fused, three attention heads are built according to the multi-head attention mechanism, namely the device's own multi-physics field model attention, the upstream and downstream model attention, and the added attention of the two. These three attention heads are spliced ​​into the final mining information;

[0032] B. Pre-training of the device's own multi-physics model

[0033] Pre-train the multi-physics field model of the device itself, that is, randomly select a group of non-scalar data as the prediction target, use other scalar and non-scalar data as input for prediction, and use the historical operation data of the device under different working conditions to allow the multi-physics field model of the device itself to fully learn the device information;

[0034] C. Multi-task learning mechanism

[0035] For the final model, after the device's own multi-physics field model and the upstream and downstream models are integrated in the attention mechanism, the final multi-task learning is carried out. First, it is necessary to identify the relevant data range on which the data to be predicted depends; secondly, construct the prediction target, which consists of the final target to be predicted and the auxiliary prediction target; when it is finally realized, the device's own multi-physics field model first gives the relevant data prediction of the data to be predicted, and then combines the upstream and downstream information to finally obtain the data to be predicted.

[0036] The beneficial effects of the present invention are: the neural network simulation physical field technology can use neural networks to simulate complex physical field mechanisms. According to the Universal Approximation Theorem for Operator (Chen et al. 1995), for any smooth, local Lipschitz ordinary differential equation, there is a neural network that approximates the equation with arbitrary approximation accuracy. The classic deeponet network (arXiv:1910.03193[cs.LG]) is its representative. Multimodal fusion, mainly through pre-training, maps and learns each physical field and the vectors that express the external information of the device to each other, so that data of each dimension forms a unified manifold in high-dimensional space. Its idea is similar to the current graphic mapping of multimodal large models.

[0037] Specifically:

[0038] MAX(G <y a |y b ,y c ...y n >)

[0039] Where y is the sequence data of each physical field, that is, the sequence data of a physical field is predicted using the data of other physical fields, so that the model can deeply learn the relationship between various physical fields.

[0040] Multi-task learning, here by splitting the device model into the device's own multi-physics field modeling and the device's external information modeling, the model can learn both its own and external information during training, which improves the effect of the fused model. The specific loss function is:

[0041] ι(θ)={ι i (θ)+ι v (θ)+ι p (θ)}+ι all (θ, γ)

[0042] Among them, ι i (θ) represents the initial conditions that need to be met, ι v (θ) represents the predicted value needs to be close to the true value, ι p (θ) represents the partial differential condition to be satisfied, ι all (θ, γ) means that the predicted value after combining the device's own information and external information should be close to the true value. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is the overall network architecture diagram;

[0044] Figure 2 Schematic diagram of the training target disassembly. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] A multi-physics field modeling method for equipment based on industrial process and multi-modal fusion includes the following steps:

[0047] Step 1: Multiphysics Modeling of the Device

[0048] Step 11: Data Processing

[0049] Obtain relevant data from the device's own sensors, external sensors, device environment sensors, device upstream and downstream information, device operation history data, etc., and perform general data processing; when there is a lack of real monitoring data or numerical simulation, simulation software and simulation models can be used to obtain the corresponding device physical field data. (Data types can be found in Table 1)

[0050]

[0051] Because the physical information and sensor signals in each dimension may have different dimensions and ranges, the relevant data flow is as follows:

[0052] A. Data cleaning

[0053] After obtaining the data, it is necessary to remove the erroneous data, that is, to eliminate the obviously erroneous data; fill the blank data, to fill the data points that have not been collected, you can choose to fill in N / A, or use algorithms for interpolation fitting, etc.; remove duplicate data, that is, to reasonably deduplicate the completely identical data entries.

[0054] B. Data Processing

[0055] According to the design manual, design drawings, standard specifications, etc., standard curves for each physical field of the equipment under various working conditions are established; all collected data are standardized; based on the actual collected standardized physical field data, the deviation value of each time point is calculated by comparing it with the ideal data under normal circumstances; the final single physical field data is composed of standardized sequence data and deviation sequence data.

[0056] Step 12: Neural network structure design

[0057] Use neural networks to model partial differential equation operators and design the corresponding network architecture. The overall network architecture is divided into two parts: the device's own information and the device's external environment. The device model network models the device's own information, and models it separately according to whether the device-related information is scalar (i.e., there is no concept of order, such as device model, production date, manufacturer, etc.) or non-scalar (i.e., there is a concept of order, such as real-time device temperature, rotation speed, production rate, sound, etc.). Scalar information is extracted through a multi-layer perceptron (MLP), and non-scalar information is extracted through a unidirectional recurrent neural network (RNN) to extract sequential features. After the scalar features and non-scalar features are extracted, they need to be dimensionally compressed so that the data dimensions of the two types of features are the same (for example, both are n*K) and merged. Finally, the connection between them is constructed through the attention mechanism (standard attention layer), that is, the merged data is used as a query, and the correlation between each position data and other position data is calculated by traversing to enable the model to consider the relationship between the two types of data at the same time. See for details. Figure 1 .

[0058] Step 2: Modeling of equipment upstream and downstream information, environment, and human impact

[0059] Step 21: Data Processing

[0060] Process data obtained from various sensors, historical information, manuals, videos, pictures, other system outputs, etc. The main process is still divided into data cleaning and data processing.

[0061] A. Data cleaning

[0062] First, the acquired data needs to remove erroneous data, that is, the obviously erroneous data needs to be eliminated; fill in blank data, fill in the data points that have not been collected, and you can choose to use N / A to fill, or use algorithms for interpolation fitting, etc.; remove duplicate data, that is, reasonably remove the duplicate data entries that are exactly the same. At the same time, for the data that needs to be recognized by text, content recognition and result collation are performed (for example, the scanned version of the manual is converted into an editable document); for video data, it is necessary to extract frames of effective content (by identifying the percentage of pixel changes between the previous and next frames) to remove the duplicate video content that occupies the vast majority; for text data, it is necessary to remove common stop words to reduce the input of meaningless text.

[0063] B. Data Processing

[0064] First, it is necessary to establish standard curves for each physical field of the equipment under various working conditions based on design manuals, design drawings, standard specifications, etc.; standardize all collected data; and calculate the deviation value at each time point based on the actual collected standardized physical field data and the ideal data under normal conditions. At the same time, for text, video, and image data, it is necessary to process the data according to the input format requirements of the feature extraction model used. For example, for the BERT model, it is necessary to prepare a corresponding vocabulary so that the input is converted into objects that the model can recognize.

[0065] Step 22: Neural network structure design

[0066] It is necessary to design different feature extraction structures for different types of data (scalar or non-scalar). Recurrent neural networks are used for non-scalar data, and multi-layer perceptrons are used for scalar data to extract features. Note that when facing text, video, and image content, considering the wide application of large-scale pre-trained models in the text and image fields, using the corresponding pre-trained model to extract features will have better results (for example, using the bert model to extract text data features, and using yoloV8 to extract image and video data features). After feature extraction, the same dimensional compression is required to make the dimensions of various feature data the same, and then the connection between each other is established through the attention mechanism (standard attention layer), that is, the merged data is used as a query, and the correlation between each position data and other position data is calculated by traversing so that the model can consider the relationship between the two types of data at the same time. For details of the network architecture, see Figure 1 .

[0067] Step 3: Model fusion and corresponding multi-task training

[0068] The main task is to integrate the multi-physics field model of the device itself with the upstream, downstream, environment, operation history and other models associated with the device, and perform corresponding multi-task learning. The main process is as follows

[0069] D. Fusion of the equipment’s own multi-physics model and upstream and downstream models

[0070] Fusion is mainly performed at the attention layer. The device's own multi-physics field model and upstream and downstream models perform feature extraction respectively, and build their own attention layers respectively. Note that the dimensions of the query, key, and value in each attention layer are consistent (for example, they are all n*k). During fusion, three attention heads are built in imitation of the multi-head attention mechanism, namely the device's own multi-physics field model attention, the upstream and downstream model attention, and the added attention of the two. These three attention heads are spliced ​​into the final mining information.

[0071] E. Pre-training of the device's own multi-physics model

[0072] Pre-train the multi-physics field model of the device itself, that is, randomly select a group from the non-scalar data as the prediction target, and use other scalar and non-scalar data as input for prediction (for example, use scalars such as device model and production date, plus real-time temperature, real-time power consumption, real-time amplitude and other data to predict real-time speed). By using the historical operating data of the device under different working conditions, the multi-physics field model of the device itself can fully learn the device information.

[0073] F. Multi-task learning mechanism

[0074] For the final model, after fusing the device's own multi-physics field model and the upstream and downstream models in the attention mechanism, the final multi-task learning can be performed. First, it is necessary to identify the relevant data range on which the data to be predicted depends (for example, when predicting the device speed, the device's real-time temperature, real-time power consumption, etc. are relevant data); secondly, construct a prediction target, which consists of the final target to be predicted and auxiliary prediction targets (for example, when predicting the device speed, the overall prediction target is to optimize the predicted device speed + optimize the predicted device temperature + optimize the device power consumption); when finally realized, the device's own multi-physics field model first gives the relevant data prediction of the data to be predicted, and then combines the upstream and downstream information to finally obtain the data to be predicted (for example, the device's own multi-physics field model first gives the prediction of the device's real-time power consumption and temperature, and then combines the upstream and downstream information to give the prediction of the device speed).

[0075] Example:

[0076] A multi-physics field modeling method for equipment based on industrial process and multi-modal fusion includes the following steps:

[0077] Data collection: Obtain the physical field data of existing equipment, equipment external data, equipment environment data, etc., and perform data cleaning and data processing to finally obtain the final data input.

[0078] Network structure design: The overall model is divided into the device's own multi-physics field model and upstream and downstream models. The device's own multi-physics field model uses a multi-layer perceptron for scalars and a recurrent neural network for non-scalars, depending on the type of data related to the device. The extracted features are merged using an attention layer. The upstream and downstream models of the device are similar, except that for text, image, and video input data, the corresponding pre-trained model is selected for feature extraction and the extraction results are dimensional compressed. After the device's own multi-physics field model and upstream and downstream models are built, a similar multi-head attention mechanism is used to merge the attention outputs of the two and the accumulated outputs of the two to obtain the final model output layer.

[0079] Selection of text vector model and visual vector model: According to the actual situation of the device, select appropriate language or visual models, such as BERT and YOLO.

[0080] Data filtering: Filter after invalid data is determined.

[0081] Data deduplication: remove duplicate data.

[0082] Data standardization: Reasonably standardize data of different dimensions and ranges.

[0083] Pre-training of the device's own physical field model: For the pre-training of the device's own multi-physical field model, it is first necessary to freeze the other model structures (that is, the relevant parameters are not updated), and train the device's own physical field model alone. The training goal is to use other physical field data to make reasonable predictions about the remaining physical field data.

[0084] Training of all models: Because the device's own physical field model has been pre-trained, the device's own physical field model part is frozen at this time, and the device's upstream and downstream model parts are trained in conjunction with the device's external information. The goal is to use all the data to make reasonable predictions about the remaining physical field data.

Claims

1. A multi-physics field modeling method for equipment based on industrial process and multi-modal fusion, characterized in that: The steps include: Step 1: Equipment multi-physics modeling; Step 2: Modeling of upstream and downstream information of the equipment, the environment in which it is located, and human influence; Step 3: Model fusion and corresponding multi-task training.

2. The equipment multi-physics field modeling method based on industrial process and multi-modal fusion as claimed in claim 1, characterized in that: The step 1 includes the following: Step 11: Data processing; Step 12: Neural network structure design.

3. The equipment multi-physics field modeling method based on industrial process and multi-modal fusion as claimed in claim 2, characterized in that: The step 11 includes obtaining relevant data from the device's own sensors, external sensors, device environment sensors, device upstream and downstream information, and device operation history data, and performing general data processing. The relevant data processing process is as follows: AData Cleaning Eliminate obviously erroneous data; fill in blank data, fill in uncollected data points, use N / A to fill, or use algorithms for interpolation fitting; remove duplicate data, that is, reasonably remove duplicates of exactly the same data items; B. Data Processing According to the design manual, design drawings, and standard specifications, standard curves for each physical field of the equipment under various working conditions are established; all collected data are standardized; based on the actual collected standardized physical field data, the deviation value at each time point is calculated by comparing it with the ideal data under normal circumstances; the final single physical field data is composed of standardized sequence data and deviation sequence data.

4. The equipment multi-physics field modeling method based on industrial process and multi-modal fusion as claimed in claim 2, characterized in that: The step 12 includes using a neural network to model the partial differential equation operator and designing a corresponding network architecture. The overall network architecture is divided into two parts: the device's own information and the device's external environment. The device model network models the device's own information, and models it separately according to whether the device-related information is scalar or non-scalar. The scalar information is extracted through a multi-layer perceptron, and the non-scalar is extracted through a unidirectional recurrent neural network. The scalar features and non-scalar features need to be dimensionally compressed after extraction so that the data dimensions of the two types of features are the same and merged. Finally, the connection between each other is constructed through the attention mechanism, that is, the merged data is used as a query, and the correlation between each position data and other position data is traversed and calculated so that the model can simultaneously consider the relationship between the two types of data.

5. The equipment multi-physics field modeling method based on industrial process and multi-modal fusion as claimed in claim 1, characterized in that: The step 2 includes the following: Step 21: Data processing; Step 22: Neural network structure design.

6. The equipment multi-physics field modeling method based on industrial process and multi-modal fusion as claimed in claim 5, characterized in that: The step 21 includes processing the data obtained from various sensors, historical information, manuals, videos, pictures, and other system outputs as follows: A. Data cleaning Eliminate obviously erroneous data; fill in blank data and fill in uncollected data points, which can be filled with N / A or interpolated using algorithms; remove duplicate data, that is, reasonably remove duplicate data entries that are exactly the same, and perform content recognition and result collation for data that requires text recognition; for video data, it is necessary to extract frames of effective content and remove the majority of repeated video content; for text data, remove common stop words and reduce the input of meaningless text; B. Data Processing First, it is necessary to establish standard curves for each physical field of the equipment under various working conditions based on the design manual, design drawings, and standard specifications; standardize all collected data; and calculate the deviation value for each time point based on the actual collected standardized physical field data and the ideal data under normal circumstances. At the same time, for text, video, and image data, the data must be processed according to the input format requirements of the feature extraction model used.

7. The equipment multi-physics field modeling method based on industrial process and multi-modal fusion as claimed in claim 5, characterized in that: The step 22 includes designing different feature extraction structures for different types of data, using a recurrent neural network for non-scalar data and a multi-layer perceptron for scalar data to extract features; After extraction, the same dimensional compression is performed to make the dimensions of various feature data the same. Then, the connection between them is established through the attention mechanism. That is, the merged data is used as a query, and the correlation between each position data and other position data is traversed and calculated so that the model can consider the relationship between the two types of data at the same time.

8. The equipment multi-physics field modeling method based on industrial process and multi-modal fusion as claimed in claim 1, characterized in that: The step 3 includes integrating the multi-physics field model of the device itself with the upstream, downstream, environment, and operation history associated with the device, and performing corresponding multi-task learning. The main process is as follows: A. Fusion of the equipment’s own multi-physics model and upstream and downstream models The device's own multi-physics field model and upstream and downstream models extract features respectively, and build their own attention layers respectively. The dimensions of query, key, and value in each attention layer are consistent. When fused, three attention heads are built according to the multi-head attention mechanism, namely the device's own multi-physics field model attention, the upstream and downstream model attention, and the added attention of the two. These three attention heads are spliced ​​into the final mining information; B. Pre-training of the device's own multi-physics model Pre-train the multi-physics field model of the device itself, that is, randomly select a group of non-scalar data as the prediction target, use other scalar and non-scalar data as input for prediction, and use the historical operation data of the device under different working conditions to allow the multi-physics field model of the device itself to fully learn the device information; C. Multi-task learning mechanism For the final model, after fusing the device’s own multi-physics field model with the upstream and downstream models in the attention mechanism, the final multi-task learning is performed. First, it is necessary to identify the relevant data range that the data to be predicted depends on; Secondly, the prediction target is constructed, which consists of the final target to be predicted and the auxiliary prediction target. When it is finally realized, the multi-physical field model of the device itself first gives the relevant data prediction of the data to be predicted, and then combines the upstream and downstream information to finally get the data to be predicted.

Images