Design intent extraction method for large-scale engineering equipment based on multimodal data fusion

Abstract

The present invention discloses a method for extracting design intent of large-scale engineering equipment based on multimodal data fusion, which belongs to the technical field of engineering design intent extraction, and comprises the following steps: classifying historical project design resources to form multimodal resources, extracting multimodal feature vectors through a multi-branch unimodal feature extraction network based on a convolutional neural network and an encoder, fusing component model feature vectors in the multimodal vector through a unimodal feature fusion network based on a self-attention mechanism to obtain a component fusion feature vector, and fusing the multimodal feature vectors through a multimodal feature fusion network to obtain a design intent feature vector. The present invention combines the characteristics of multimodal design resources, retains the association information between components, improves the accuracy of extraction, can assist design tasks in the design process of large-scale engineering machinery equipment, reduces design overhead in the design process of large-scale engineering machinery equipment, and reduces design change losses caused by design errors.

Description

Technical Field

[0001] The present invention belongs to the technical field of engineering design intent extraction, and in particular relates to a method for extracting design intent of large-scale engineering equipment based on multimodal data fusion. Background Art

[0002] In today's industrial field, the design and manufacturing of large-scale engineering machinery and equipment are crucial to the country's economic and technological development. With the widespread application of advanced manufacturing technology, 3D models and CNC process data are growing exponentially, and the degree of digitization of design resources is also deepening. However, how to effectively integrate multimodal data to adapt to personalized needs and market changes remains a challenge. Statistics show that about 80% of large-scale engineering machinery and equipment designs can be completed by reusing or modifying existing designs. However, design resource data is usually unstructured, and designers need to spend a lot of time mining knowledge. Therefore, analyzing and mining the design intent and experience in design resource data is an urgent problem to be solved.

[0003] The invention patent application with application publication number CN104899242 A for design intent extraction method of traditional large-scale engineering equipment discloses a two-dimensional knowledge push method for mechanical product design based on design intent, which mainly includes: establishing a mechanical product design intent database, establishing a design intent acquisition and decomposition method, using rough set theory to construct a design intent attribute table and simplify it to obtain the simplest design intent set; establishing a mechanical product design knowledge ontology database; using a sequential traversal method to calculate the similarity between the intent element and the compared knowledge ontology using an improved similarity matching algorithm based on the semantic distance of knowledge ontology concepts and concept attributes, for the intent element with a similarity greater than a threshold, matching each intent element in the intent element set in order from large to small according to the similarity until all matches are completed, and using a text semantic similarity calculation method to complete the knowledge ontology matching.

[0004] The above method belongs to a single-mode method. For the design of large-scale engineering equipment, due to the existence of numerous components, the design solution obtained by using the above single-mode method is not effective. Summary of the invention

[0005] The present invention provides a method for extracting design intent of large-scale engineering equipment based on multimodal data fusion, so as to solve the problem that the method of extracting design intent of large-scale engineering equipment using a single-modal strategy has poor effect.

[0006] In order to solve the above technical problems, the technical solution provided by the present invention is:

[0007] The present invention relates to a method for extracting design intent of large-scale engineering equipment based on multimodal data fusion, which comprises the following steps:

[0008] S1. Obtain historical engineering project design resources, classify the project design resources into multimodal resources, and the multimodal resources include 3D Mesh overall model, 3D Mesh component model, component list, 2D drawings and complex environment information;

[0009] S2. For the 3D Mesh overall model, 3D Mesh component model, 2D drawing and complex environment information, the overall model feature vector, component model feature vector, image feature vector and environment feature vector are extracted respectively through a multi-branch single-modal feature extraction network based on a convolutional neural network and an encoder;

[0010] S3. concatenate the component model feature vector with the position code in the component list, and fuse the component model feature vector through a unimodal feature fusion network based on a self-attention mechanism to obtain a component fusion feature vector;

[0011] S4. The overall model feature vector, component fusion feature vector, image feature vector and environment feature vector are fused through a multimodal feature fusion network based on the Transformer structure to obtain the design intent feature vector of the engineering project;

[0012] S5. Based on the design intent feature vectors of all historical engineering projects, the design intent extraction network is used to obtain the historical engineering project clusters based on design intent, and the design intent center of each cluster is obtained.

[0013] Preferably, the specific steps of S2 for extracting the overall model feature vector, the component model feature vector and the image feature vector respectively through a multi-branch single-modal feature extraction network based on a convolutional neural network and an encoder for the three-dimensional Mesh overall model, the three-dimensional Mesh component model and the two-dimensional drawing are:

[0014] S2.1. Use MeshNet to extract the overall model feature vector of the 3D Mesh overall model and the component model feature vector of the 3D Mesh component model; use VggNet to extract the image feature vector of the 2D drawing;

[0015] S2.2. A multi-layer stacked Transformer encoder structure is used to learn the associated information in the overall model feature vector, the component model feature vector, and the image feature vector respectively, and to update the overall model feature vector, the component model feature vector, and the image feature vector.

[0016] Preferably, each layer of the Transformer encoder in S2.2 includes a self-attention layer and a feedforward network, and the self-attention layer includes a fully connected layer and a normalized point-product attention mechanism;

[0017] The specific steps of S2.2 learning the associated information of each feature vector and updating each feature vector are:

[0018] S2.2.1. The input feature vector is linearly transformed through the fully connected layer to obtain three different types of sub-vectors, namely the query vector Q , key vector K and the value vector V ;

[0019] S2.2.2. Calculate the product between the query vector and the key vector, scale the product, apply the attention mask, further normalize the Softmax function, and generate an attention weight matrix;

[0020] S2.2.3. Multiply the attention weight matrix by the value vector to update the input feature vector. The calculation formula is:

[0021] (1),

[0022] In the formula, Attention scaled is the normalized point-wise attention mechanism, Q , K , V denote the query vector, key vector and value vector respectively, X ' is the feature vector updated by the attention mechanism, d is the dimension of the feature vector, Softmax represents the Softmax function, T Represents matrix transpose;

[0023] S2.2.4. The feedforward network performs a nonlinear transformation on the feature vector updated by the attention mechanism, thereby updating the important components. The calculation formula for the operation of the feedforward network is:

[0024] (2),

[0025] In the formula, W 1. W 2. b 1 and b 2 are the learnable parameters in the feedforward network, X '' Represents the output feature vector obtained after the feedforward network processing, max(0, XW 1+ b 1) Indicates how the ReLU activation function is calculated.

[0026] Preferably, S2 uses a One-Hot coding strategy to process the complex environmental information involved in the design requirements for complex environmental information. The calculation formula of the One-Hot coding strategy is:

[0027] (3),

[0028] In the formula, Fn is the feature vector obtained after One-Hot encoding, p is the preamble code, a For subsequent encoding, OneHot This is a One-Hot encoding operation.

[0029] Preferably, the specific steps of S3 concatenating the component model feature vector with the position code in the component list, and fusing the component model feature vector through a unimodal feature fusion network based on a self-attention mechanism to obtain a component fusion feature vector are:

[0030] S3.1. construct the parts list into an array with dimension N, where N is the total number of parts in the parts list;

[0031] S3.2. According to the order of the parts list, concatenate the model feature vectors of each component with the position code in the parts list to obtain the adjusted component model feature vectors;

[0032] S3.3. The adjusted component model feature vectors are combined into a matrix as the input of the Transformer model. The Transformer model captures the complex relationships and interdependencies between components through its self-attention mechanism and multi-head attention strategy, and outputs a component fusion feature vector that integrates the features of all component models.

[0033] Preferably, the fusion formula for fusing the overall model feature vector, the component fusion feature vector, the image feature vector and the environment feature vector in S4 is:

[0034] (4),

[0035] In the formula, M represents the number of feature fusion module layers in the multimodal feature fusion network, Attention scaled is the normalized point-wise attention mechanism, V M Represents the output feature vector generated in the M-th layer feature fusion module, V M-1 is the output result of the M-1th layer, FM is the input vector sent to the Mth layer feature fusion module, d is the dimension of each feature vector, T Represents matrix transpose.

[0036] Preferably, the specific steps of obtaining historical engineering project clusters based on design intent through spectral clustering design intent extraction network in S5 and obtaining the design intent center of each cluster are:

[0037] S5.1. Calculate the cosine similarity of the design intent feature vector and construct a similarity matrix. The calculation formula of cosine similarity is:

[0038] (5),

[0039] In the formula, β is the similarity, A and B represent the descriptor vectors of the two design intent feature vectors, n is the dimension of the vector, and i represents the iteration value;

[0040] S5.2. Based on the constructed similarity matrix, further construct a Laplacian matrix;

[0041] S5.3. Perform eigendecomposition on the Laplace matrix, calculate its eigenvalues ​​and corresponding eigenvectors, select the eigenvectors corresponding to the smallest k eigenvalues, where k is the number of cluster categories, and integrate these eigenvectors into a new matrix;

[0042] S5.4. Apply the K-means clustering algorithm to cluster the new matrix and obtain the cluster division results of historical engineering projects and the design intent center.

[0043] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0044] The invention relates to a large-scale engineering equipment design intent extraction method based on multimodal data fusion, which classifies historical project design resources into multimodal resources, extracts multimodal feature vectors through a multi-branch single-modal feature extraction network based on a convolutional neural network and an encoder, fuses component model feature vectors in the multimodal vector through a single-modal feature fusion network based on a self-attention mechanism to obtain a component fusion feature vector, and fuses the multimodal feature vectors through a multimodal feature fusion network to obtain a design intent feature vector. The feature vector not only includes multi-dimensional information such as the geometric shape, physical properties and functional requirements of the equipment, but also reflects the inherent connection and interaction between these information through the processing of a deep learning model, and combines spectral clustering to perform similarity calculation to realize the extraction of the design intent of large-scale engineering equipment. Compared with the single-modal method, the method has better effect and can better assist designers in the design process of large-scale engineering equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is the network structure diagram of the overall model of the present invention.

[0046] Figure 2 Extract network structure graph for convolutional neural network.

[0047] Figure 3 Single-modal feature fusion network structure diagram. DETAILED DESCRIPTION

[0048] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with examples. The following examples are used to illustrate the present invention but are not used to limit the scope of the present invention.

[0049] Refer to the attached Figure 1 As shown, the present invention relates to a method for extracting design intent of large-scale engineering equipment based on multimodal data fusion, comprising the following steps:

[0050] S1. Export historical engineering project information from the system of large engineering equipment manufacturing enterprises, and use automated scripts and query tools to screen and classify by project work number to ensure the integrity and accuracy of all data, and then obtain historical engineering project design resources, classify project design resources into multi-modal resources, and multi-modal resources include 3D Mesh overall model, 3D Mesh component model, parts list, 2D drawings and complex environment information. Retrieve the corresponding design resources of the engineering project according to the historical engineering project work number, and save the design resource association information of different modal design resources in the form of project-resource according to the drawing number;

[0051] S2. For the 3D Mesh overall model, 3D Mesh component model, 2D drawing and complex environment information, the overall model feature vector, component model feature vector, image feature vector and environment feature vector are extracted respectively through a multi-branch single-modal feature extraction network based on a convolutional neural network and an encoder;

[0052] The single-modal feature extraction network is specifically divided into two extraction networks: a convolutional neural network extraction network and an encoder extraction network, which are responsible for extracting feature vectors of different modes respectively. The convolutional neural network extraction network is mainly responsible for the feature extraction of three-dimensional mesh models and two-dimensional drawings. Its structure is as follows: Figure 2 As shown. Convolutional neural network (CNN) is a deep learning algorithm that processes image data by simulating the mechanism of the human visual system. Therefore, it performs well in processing three-dimensional mesh models and two-dimensional drawings. CNN can automatically and effectively extract visual features from these design resources without complex pre-processing. In this embodiment, the single-modal feature extraction network is divided into three branches, aiming to apply corresponding convolutional neural network strategies to design resources of different modalities (three-dimensional mesh overall model, three-dimensional mesh component model, two-dimensional drawing).

[0053] As attached Figure 2As shown, the overall model feature vector, component model feature vector and image feature vector are extracted respectively through a multi-branch single-modal feature extraction network based on a convolutional neural network and an encoder. The specific steps are as follows:

[0054] S2.1. Use MeshNet to extract the overall model feature vector of the 3D Mesh overall model and the component model feature vector of the 3D Mesh component model; use VggNet to extract the image feature vector of the 2D drawing;

[0055] S2.2. A multi-layer stacked Transformer encoder structure is used to learn the association information in the overall model feature vector, the component model feature vector, and the image feature vector respectively, and to update the overall model feature vector, the component model feature vector, and the image feature vector; each layer of the Transformer encoder includes a self-attention layer and a feedforward network. The self-attention layer is a core component of the Transformer architecture, which allows the model to consider all other elements in the sequence when processing each element of the sequence, thereby capturing the complex dependencies within the sequence. This mechanism is achieved by calculating the attention score of each element in the sequence to all other elements, and then generating a weighted output representation based on these attention scores; the self-attention layer includes two key parts: a fully connected layer and a scaled dot-product attention mechanism; based on the above structure, the specific steps of learning the association information of each feature vector and updating each feature vector are as follows:

[0056] S2.2.1. The input feature vector is linearly transformed through the fully connected layer to obtain three different types of sub-vectors, namely the query vector Q , key vector K and the value vector V , the query vector, key vector, and value vector play their respective roles in the dot product attention mechanism;

[0057] S2.2.2. Map the input data to the query vector Q, key vector K, and value vector V respectively, so as to provide the required input parameters for the execution of the attention mechanism. Specifically, the product between the query vector and the key vector is calculated, the product result is scaled, and then the attention mask is applied, and the Softmax function is further normalized to generate an attention weight matrix;

[0058] S2.2.3. Multiply the attention weight matrix by the value vector to update the input feature vector. The calculation formula is:

[0059] (1),

[0060] In the formula, Attention scaled is the normalized point-wise attention mechanism, Q , K , V denote the query vector, key vector and value vector respectively, X ' is the feature vector updated by the attention mechanism, d is the dimension of the feature vector, Softmax represents the Softmax function, which ensures that all elements of the generated attention weight distribution sum to 1, allowing these weights to be interpreted as a probability distribution. T Represents the matrix transpose; the dot product result is divided by The purpose of is to adjust the scale of the dot product value to avoid the gradient vanishing problem during training;

[0061] This operation essentially assigns different weights to each input feature, where the distribution of weights depends on the correlation between the feature and other features in the sequence. The feature vector updated by the attention mechanism can better reflect the complex relationships and dependencies between the elements within the sequence; by adopting the normalized point product attention mechanism, the model can effectively capture long-distance dependencies and dynamically update the features on this basis.

[0062] S2.2.4. The feedforward network constitutes an important component for further updating the feature vector processed by the normalized dot product attention mechanism; the network consists of multiple fully connected layers and corresponding activation functions. The feedforward network performs nonlinear transformation on the feature vector updated by the attention mechanism, and then updates the important components. The calculation formula of the feedforward network operation is:

[0063] (2),

[0064] In the formula, W 1. W 2. b 1 and b 2 are the learnable parameters in the feedforward network, X '' Represents the output feature vector obtained after the feedforward network processing, max(0, XW 1+ b 1) Indicates the calculation method of the ReLU activation function, which is used to introduce nonlinear transformations and improve the model's ability to capture complex features;

[0065] By introducing the feedforward network, the model can further refine and strengthen the feature representation while retaining the relationship between the elements within the sequence; the application of the ReLU activation function ensures the nonlinear characteristics of the network, allowing the model to capture more complex data patterns. In addition, each layer of the feedforward network is independent, which means that the output of each position depends only on the input of that position. This design makes the feedforward network highly flexible and efficient when processing sequence data.

[0066] For complex environmental information, the One-Hot encoding strategy is used to process the complex environmental information involved in the design requirements. This type of information widely includes key geological survey data such as surrounding rock grade, rock composition, and weathering degree. According to the experience of experts in the field, when considering changes within a certain numerical range, these environmental information have little effect on the design decisions of large-scale engineering machinery and equipment. Therefore, the detailed environmental data in the geological survey report can be discretized and effectively classified. Specifically, the environmental information is converted through a carefully designed encoding system, in which the preamble code p and the subsequent code a are used to represent the different attributes and characteristics of the environmental information. By using One-Hot encoding for the environmental information, each environmental attribute is converted into a one-dimensional vector, in which only the index position corresponding to the attribute is 1, and the other positions are 0. This method greatly simplifies the representation of complex environmental information while retaining the core characteristics of the data, which facilitates subsequent data processing and analysis. The calculation formula of the above One-Hot encoding strategy is:

[0067] (3),

[0068] In the formula, Fn is the feature vector obtained after One-Hot encoding, which effectively captures the key characteristics of geological environment information. p is the preamble code, a For subsequent encoding, OneHot This is a One-Hot encoding operation.

[0069] S3. The component model feature vector is concatenated with the position code in the component list, and the component model feature vector is fused through a unimodal feature fusion network based on the self-attention mechanism to obtain a component fusion feature vector for subsequent multimodal feature fusion. The structure of the unimodal feature fusion network is as follows: Figure 3 As shown in the figure, the network is based on the Transfromer self-attention mechanism. It combines the input component model feature vector with the position encoding obtained by encoding the main component list, and fuses all the component model feature vectors in the engineering project through a multi-head attention mechanism and a feedforward neural network to obtain the output fused feature vector for subsequent multimodal feature fusion.

[0070] Specifically, in the field of large-scale engineering equipment design, the complexity of the design process and the high requirements for accuracy have prompted the pursuit of stability and consistency in the configuration of component modules; considering that these design projects often involve numerous components and complex assembly processes, the configuration changes of the main component modules are relatively small, and there are not many new components, so they can be summarized by a fixed parts list. The specific steps to obtain the component fusion feature vector are:

[0071] S3.1. Construct the parts list into an array of dimension N, denoted as C=[C1,C2,...,CN], where N is the total number of parts in the parts list and each element corresponds to a part, where 1 indicates that the part is included in the project and 0 indicates that it is not included;

[0072] S3.2. For the component model feature vector set V output by the component model branch, the original dimension of each feature vector Vi is d. According to the order of the component list, concatenate each component model feature vector with the position code in the component list to obtain the adjusted component model feature vector V ' i ;

[0073] S3.3. The adjusted component model feature vectors are combined into a matrix as the input of the Transformer model. The Transformer model captures the complex relationships and interdependencies between components through its self-attention mechanism and multi-head attention strategy, and outputs a component fusion feature vector that integrates the features of all component models. The component fusion feature vector contains information about the features of each component and their interactions, which is used for subsequent multimodal data fusion.

[0074] S4. The overall model feature vector, component fusion feature vector, image feature vector and environment feature vector are fused through a multimodal feature fusion network based on the Transformer structure to obtain the design intention feature vector of the engineering project; the multimodal feature fusion network contains several levels of feature fusion units, which are improved based on the Transformer decoder and are designed to integrate feature vectors of two different modes. The number of levels of feature fusion units is determined by the number of input modes, and the fusion formula for feature fusion is:

[0075] (4),

[0076] In the formula, M represents the number of feature fusion module layers in the multimodal feature fusion network, Attention scaled is the normalized point-wise attention mechanism, V M Represents the output feature vector generated in the M-th layer feature fusion module, V M-1is the output result of the M-1th layer, FM is the input vector sent to the Mth layer feature fusion module, d is the dimension of each feature vector, T Represents matrix transpose.

[0077] S5. Based on the design intent feature vectors of all historical engineering projects, the design intent extraction network is used to obtain the historical engineering project clusters based on the design intent, and the design intent center of each cluster is obtained. The specific steps are as follows:

[0078] S5.1. Calculate the cosine similarity of the design intent feature vector and construct a similarity matrix. The calculation formula of cosine similarity is:

[0079] (5),

[0080] In the formula, β is the similarity, A and B represent the descriptor vectors of the two design intent feature vectors, n is the dimension of the vector, and i represents the iteration value;

[0081] S5.2. Based on the constructed similarity matrix, further construct a Laplacian matrix;

[0082] S5.3. Perform eigendecomposition on the Laplace matrix, calculate its eigenvalues ​​and corresponding eigenvectors, select the eigenvectors corresponding to the smallest k eigenvalues, where k is the number of cluster categories, specified according to the specific situation, generally greater than or equal to 3, and integrate these eigenvectors into a new matrix;

[0083] S5.4. Apply K-means clustering algorithm to cluster the new matrix and obtain the cluster division results of historical engineering projects and the design intent center;

[0084] The spectral clustering design intent extraction network is responsible for clustering the multimodal features of historical engineering projects to form engineering project clusters and extract the design intent center. The spectral clustering algorithm is a clustering method based on graph theory. It clusters by studying the spectral (eigenvalue) properties of data. Its core idea is to use the similarity matrix of the data to construct a graph, and then cluster by analyzing the eigenvectors of the graph. Different from traditional clustering algorithms, spectral clustering can discover structural information on data sets of various shapes, has the ability to process nonlinear separable data sets, and uses a variety of similarity functions and graph construction techniques to adapt to various data distributions. Compared with other clustering techniques, this method has more significant processing effects on sparse data and non-spherical data families. For complex multimodal design resources of large engineering equipment, they usually have high similarity but are complex and difficult to distinguish in feature space distribution. The characteristics of the spectral clustering algorithm enable it to perform better in processing complex multimodal design resource data of complex large engineering equipment.

[0085] The present invention is described in detail above in conjunction with the embodiments, but the contents described are only preferred embodiments of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of application of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A method for extracting design intent of large-scale engineering equipment based on multimodal data fusion, characterized by: It includes the following steps: S1. Obtain historical engineering project design resources, classify the project design resources into multimodal resources, and the multimodal resources include 3D Mesh overall model, 3D Mesh component model, component list, 2D drawings and complex environment information; S2. For the 3D Mesh overall model, 3D Mesh component model, 2D drawing and complex environment information, the overall model feature vector, component model feature vector, image feature vector and environment feature vector are extracted respectively through a multi-branch single-modal feature extraction network based on a convolutional neural network and an encoder; S3. Concatenate the component model feature vector with the position code in the component list, and fuse the component model feature vector through a unimodal feature fusion network based on the self-attention mechanism to obtain a component fusion feature vector. The specific steps are as follows: S3.

1. construct the parts list into an array with dimension N, where N is the total number of parts in the parts list; S3.

2. According to the order of the parts list, concatenate the model feature vectors of each component with the position code in the parts list to obtain the adjusted component model feature vectors; S3.

3. The adjusted component model feature vectors are combined into a matrix as the input of the Transformer model. The Transformer model captures the complex relationships and interdependencies between components through its self-attention mechanism and multi-head attention strategy, and outputs a component fusion feature vector that integrates the features of all component models. S4. The overall model feature vector, component fusion feature vector, image feature vector and environment feature vector are fused through a multimodal feature fusion network based on the Transformer structure to obtain the design intent feature vector of the engineering project; S5. Based on the design intent feature vectors of all historical engineering projects, the design intent extraction network is used to obtain the historical engineering project clusters based on design intent, and the design intent center of each cluster is obtained.

2. The method for extracting design intent of large-scale engineering equipment based on multimodal data fusion according to claim 1, characterized in that: The specific steps of S2 for extracting the overall model feature vector, the component model feature vector and the image feature vector respectively for the 3D Mesh overall model, the 3D Mesh component model and the 2D drawing through a multi-branch single-modal feature extraction network based on a convolutional neural network and an encoder are as follows: S2.

1. Use MeshNet to extract the overall model feature vector of the 3D Mesh overall model and the component model feature vector of the 3D Mesh component model; use VggNet to extract the image feature vector of the 2D drawing; S2.

2. A multi-layer stacked Transformer encoder structure is used to learn the associated information in the overall model feature vector, the component model feature vector, and the image feature vector respectively, and to update the overall model feature vector, the component model feature vector, and the image feature vector.

3. The method for extracting design intent of large-scale engineering equipment based on multimodal data fusion according to claim 2 is characterized in that: Each layer of the Transformer encoder in S2.2 includes a self-attention layer and a feedforward network, wherein the self-attention layer includes a fully connected layer and a normalized point-product attention mechanism; The specific steps of S2.2 learning the associated information of each feature vector and updating each feature vector are: S2.2.

1. The input feature vector is linearly transformed through the fully connected layer to obtain three different types of sub-vectors, namely the query vector Q , key vector K and the value vector V ; S2.2.

2. Calculate the product between the query vector and the key vector, scale the product, apply the attention mask, further normalize the Softmax function, and generate an attention weight matrix; S2.2.

3. Multiply the attention weight matrix by the value vector to update the input feature vector. The calculation formula is: （1）， In the formula, Attention scaled is the normalized point-wise attention mechanism, Q , K , V denote the query vector, key vector and value vector respectively, X ' is the feature vector updated by the attention mechanism, d is the dimension of the feature vector, Softmax represents the Softmax function, T Represents matrix transpose; S2.2.

4. The feedforward network performs a nonlinear transformation on the feature vector updated by the attention mechanism, thereby updating the important components. The calculation formula for the operation of the feedforward network is: （2）， In the formula, W 1. W 2. b 1 and b 2 are the learnable parameters in the feedforward network, X '' Represents the output feature vector obtained after the feedforward network processing, max(0, X ' W 1+ b 1) Indicates how the ReLU activation function is calculated.

4. The method for extracting design intent of large-scale engineering equipment based on multimodal data fusion according to claim 1, characterized in that: For complex environmental information, S2 adopts a One-Hot encoding strategy to process the complex environmental information involved in the design requirements. The calculation formula of the One-Hot encoding strategy is: （3）， In the formula, Fn is the feature vector obtained after One-Hot encoding, p is the preamble code, a For subsequent encoding, OneHot This is a One-Hot encoding operation.

5. The method for extracting design intent of large-scale engineering equipment based on multimodal data fusion according to claim 1, characterized in that: The fusion formula of S4 for fusing the overall model feature vector, the component fusion feature vector, the image feature vector and the environment feature vector is: （4）， In the formula, M represents the number of feature fusion module layers in the multimodal feature fusion network, Attention scaled is the normalized point-wise attention mechanism, V M Represents the output feature vector generated in the M-th layer feature fusion module, V M-1 is the output result of the M-1th layer, FM is the input vector sent to the Mth layer feature fusion module, d is the dimension of each feature vector, T Represents matrix transpose.

6. The method for extracting design intent of large-scale engineering equipment based on multimodal data fusion according to claim 1, characterized in that: The specific steps of S5 to obtain historical engineering project clusters based on design intent through spectral clustering design intent extraction network and obtain the design intent center of each cluster are as follows: S5.

1. Calculate the cosine similarity of the design intent feature vector and construct a similarity matrix. The calculation formula of cosine similarity is: （5）， In the formula, β is the similarity, A and B represent the descriptor vectors of the two design intent feature vectors, n is the dimension of the vector, and i represents the iteration value; S5.

2. Based on the constructed similarity matrix, further construct a Laplacian matrix; S5.

3. Perform eigendecomposition on the Laplacian matrix, calculate its eigenvalues ​​and corresponding eigenvectors, select the eigenvectors corresponding to the smallest k eigenvalues, where k is the number of cluster categories, and integrate these eigenvectors into a new matrix; S5.

4. Apply the K-means clustering algorithm to cluster the new matrix and obtain the cluster division results of historical engineering projects and the design intent center.

Images