Multi-modal large model-based ship and ocean engineering aided design method

Abstract

The invention discloses a ship and ocean engineering aided design method based on a multi-modal large model, and the method comprises the following steps: S1, taking image features and text features of a ship engineering drawing as input, and constructing a drawing and text fusion model of ship engineering; s2, a multi-head attention mechanism is adopted to calculate attention weights between the image features and the text features, and fusion features are obtained; s3, optimizing the ship design drawing or design scheme based on the generative adversarial network; s4, predicting the performance of the ship based on a neural network; s5, performing cost prediction on the cost of the ship based on the decision tree; s6, performing standard evaluation based on AI reasoning regulations; according to the method, the working efficiency can be improved through intelligent means, the human error occurrence probability is reduced, and the ship construction project can be smoothly promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and particularly relates to a ship and ocean engineering auxiliary design method based on a multi-modal large model. Background Art

[0002] Since the birth of the computer, computer-aided design and manufacturing technologies have gradually entered all walks of life and have been widely applied in ship engineering. The application of this technology not only improves the automation degree of ship manufacturing, improves product quality, but also greatly shortens the product production cycle and significantly enhances production efficiency. With the continuous progress of computer technology, computer-aided design and manufacturing technologies are also constantly updated and iterated, providing a solid technical foundation for the ship and ocean engineering auxiliary design method based on a multi-modal large model.

[0003] The ship and ocean engineering industry is characterized by complex technology, frequent design changes, small batch sizes, and a large number of scattered professional knowledge, which makes it difficult to form a universal professional knowledge base in this industry, thus affecting the overall work efficiency and competitiveness. Therefore, the industry's demand for intelligence is becoming increasingly urgent. The auxiliary design method based on a multi-modal large model is proposed to solve this industry pain point. It can improve work efficiency through intelligent means, reduce the probability of human errors, and ensure the smooth progress of projects. Summary of the Invention

[0004] To solve the above problems, the present invention proposes a ship and ocean engineering auxiliary design method based on a multi-modal large model. This method can optimize ship design drawings or design schemes, predict the performance of ships based on neural networks, predict the cost of ship construction based on decision trees, and evaluate regulations and specifications based on AI reasoning. It can improve work efficiency through intelligent means, reduce the probability of human errors, and is conducive to the smooth progress of ship construction projects.

[0005] To achieve the above object, the present invention adopts the following technical solutions: A ship and ocean engineering auxiliary design method based on a multi-modal large model, comprising the following steps: S1. Using the image features and text features of ship engineering drawings as inputs to construct a drawing and text fusion model for ship engineering; The specific process of step S1 is as follows: S11. The image features of ship engineering drawings are , where n is the number of image features, is the dimension of image features; the text features of ship engineering drawings are , where m is the number of text features, is the dimension of text features; S12. Attention of Image to Text: Calculate the attention weight of the image features of the ship engineering drawings to the text features. The calculation formula is: , where is the attention weight of the image features of the ship engineering drawings to the text features; is the summation index when calculating the image representation of the text features; is the summation index when calculating the text representation of the image features; is the image feature of the ship engineering drawings in the th feature vector, ; is the linear transformation matrix from image to text; is the text feature of the ship engineering drawings in the th feature vector transposed, ; is the summation index when calculating the attention weight; is the text feature of the ship engineering drawings in the th feature vector transposed; Calculate the text representation of the image. The calculation formula is: , where is the text representation of the image features; is the text feature of the ship engineering drawings in the th feature vector; S13. Attention of Text to Image: Calculate the attention weight of the text features of the ship engineering drawings to the image features. The calculation formula is: , where is the attention weight of the text features of the ship engineering drawings to the image features; is the linear transformation matrix from text to image; is the image feature of the ship engineering drawings in the th feature vector transposed; is the image feature of the ship engineering drawings in the th feature vector transposed; Calculate the image representation of the text. The calculation formula is: , where is the image representation of the text features; S14. Fuse the text representation of the image and the image representation of the text with the original features respectively. Among them, the calculation formula for fusing the image features is: , where To fuse image features; For vector concatenation or summation operations; The calculation formula for fusing text features is: , where in the formula, is the fused text feature; S15. Input the image feature and the text feature as the query and key-value pair into the multi-head attention mechanism respectively; the image feature is used as the query , and the text feature is used as the key and value ; S2. Use the multi-head attention mechanism to calculate the attention weights between the image feature and the text feature, and obtain the fused feature; S3. Optimize the ship design drawings or design schemes based on the generative adversarial network; S4. Predict the performance of the ship based on the neural network; S5. Predict the cost of the ship's construction cost based on the decision tree.

[0006] Preferably, the specific process of step S2 is as follows: S21. Define the parameters of the multi-head attention mechanism for ship engineering images. The parameters of the multi-head attention mechanism include the number of heads h, the dimension of the query , the dimension of the key and the dimension of the value ; S22. Construct the linear transformation matrix and the output linear transformation matrix of the randomly initialized multi-head attention mechanism; S23. Based on the formula of the multi-head attention mechanism, perform the following calculations: , , , , where in the formula, is the query vector of the th attention head; is the linear transformation matrix used to map the image feature to the query vector in the th attention head, , is the dimension of the query; is the key vector of the th attention head; is the linear transformation matrix used to map the text feature to the key vector in the th attention head, , is the dimension of the key; is the value vector of the th attention head; is the linear transformation matrix used to map the text feature to the value vector in the th attention head, , is the dimension of the value; is the output result of the -th attention head; is the activation function; is the -th attention head's key vector transposed; The multi-head concatenation formula is: , where is the output feature that fuses the image feature and the text feature; is the function for concatenating strings; is the output result of attention heads in the multi-head attention mechanism; is the matrix for linearly transforming the output of the multi-head attention heads, ; The output feature that fuses the image feature and the text feature is used as the input for subsequent tasks, and the subsequent tasks include the classification, retrieval, equipment description, and equipment component query of ship engineering drawings.

[0007] Preferably, the specific process of step S3 is as follows: S31. Generator input: Represent the image feature of the ship engineering drawing as I and the text feature as T; then fuse the image feature I and the text feature T as the input of the generator. The generator receives the fused feature and the random noise z and generates a new ship engineering drawing or design scheme G(I, T, z); S32. Discriminator input: The discriminator receives the pair of the real ship engineering drawing and the corresponding text feature or the pair generated by the generator; The discriminator outputs a probability value, representing the probability of the real data distribution; For the GAN model of ship engineering drawing and text fusion, the objective function is adjusted to: , where is the objective function of the generative adversarial network; is the expected operation on the joint distribution of the real ship engineering drawing and the corresponding text feature pair is the joint distribution of the real ship engineering drawing and the corresponding text feature pair; is the discriminator function, which inputs real data or generated data and outputs a probability value, indicating the probability that the input data belongs to the real data distribution; is the expected operation on the prior distribution and the random noise ; is the prior distribution; is the generator According to the input ship engineering drawings , text features and random noise , generate a ship engineering drawing or related design scheme, and then input the generation result into the discriminator . The discriminator outputs a probability indicating that the discriminator believes the generation result is a pair of real ship engineering drawings and text features; S33. Optimization of the discriminator D: Take the derivative of the objective function with respect to D, and the calculation formula is: , where is the generator generates a result based on the input random noise , and then input the generation result into the discriminator . The discriminator judges the result generated only by noise and outputs a probability value indicating the probability that the judgment result is a pair of real ship engineering drawings and text features; Take the derivatives of the two items respectively to get: , . Update the discriminator parameters by gradient ascent according to the derivative to maximize the objective function, which is used to improve the ability to distinguish real samples and generated samples; S34. Optimization of the generator G: Take the derivative of the objective function with respect to G, and the calculation formula is: , where is the generator generates a result according to the ship engineering drawing , text features and random noise , and then input the output result of the generator and the text features into the discriminator together. The discriminator makes a judgment based on the two inputs and outputs a probability indicating that the judgment input is a pair of real ship engineering drawings and text features; Let u = G(I, T, z), and get: . Update the generator parameters by gradient descent according to the derivative to minimize the objective function, which is used to make the generated samples more realistic and be able to deceive the discriminator; S35. By continuously iteratively optimizing the discriminator and the generator, the generator fuses the image features and text features of the ship engineering drawing to generate high-quality design drawings or design schemes.

[0008] Preferably, the specific process of step S4 is: S41. Construct a ship design scheme database, which includes the dimensions, structural parameters, power system parameters, loading conditions, and performance indicators of the ship. The dimensions of the ship include the length between perpendiculars, the beam, and the draft. The performance indicators of the ship include the speed, stability, and maneuverability. Then, select the features related to performance prediction and perform feature transformation based on the multi-layer perceptron model. The features related to performance prediction include the main engine power, block coefficient, displacement, and resistance. S42. Assume that there are q neurons in the input layer, corresponding to the q features selected in ship performance prediction. Denote the input feature vector as x = ; there are r neurons in the hidden layer and p neurons in the output layer, corresponding to the p performance indicators to be predicted. S43. From the input layer to the hidden layer: The input weighted sum of the hidden layer neurons is: , where is the input weighted sum of the -th neuron in the hidden layer; is the connection weight from the i-th neuron in the input layer to the j-th neuron in the hidden layer; is the bias of the j-th neuron in the hidden layer; The output of the hidden layer neurons is obtained through non-linear transformation based on the activation function σ(·): , where is the output of the -th neuron in the hidden layer; S44. From the hidden layer to the output layer: The input weighted sum of the output layer neurons is: , where is the input weighted sum of the -th neuron in the output layer; is the connection weight from the j-th neuron in the hidden layer to the k-th neuron in the output layer; is the bias of the k-th neuron in the output layer; The result of the output layer represents the performance indicator as: , where is the output of the -th neuron in the output layer; σ(·) is the activation function; The overall multi-layer perceptron model is represented as a function mapping: , where is the vector of the predicted ship performance indicators; W is the matrix of all weights; b is the vector of all biases; is a multi-layer perceptron model function, describing the entire mapping relationship from the input feature vector , through the action of the weight matrix and the bias vector , to the output predicted ship performance indicator vector .

[0009] Preferably, the specific process of step S5 is as follows: S51. Construct a ship cost database, which includes the steel plate size, structural materials, equipment configuration, construction technology, welding materials, man-hours, and corresponding cost data of the ship; S52. Construct a shipbuilding cost model based on decision trees: For each decision tree, the input feature vector is , where t is the number of ship cost-related features; the decision tree recursively selects the best splitting feature and splitting point to divide the data set into different subsets until the stopping condition is met; assume that the decision tree finally divides the samples into the leaf node set { }}, for the input feature vector the reached leaf node is denoted as L( ). In the regression problem of ship cost budget, the predicted value of the decision tree for the input feature vector is the average value of all cost data within the leaf node, that is: , where is the predicted value of the decision tree for the input feature vector ; is the number of samples within the leaf node L( ); is the true cost value of each single sample ; is a single sample, that is, a single sample is the steel plate size, structural materials, equipment configuration, construction technology, welding materials, or man-hours of the ship; S53. The random forest consists of M decision trees. According to the total cost budget requirement of the newly built shipbuilding cost, the steel plate size, structural materials, equipment configuration, construction technology, welding materials, and man-hour features of the ship are combined into a feature vector ; The feature vector is input into the random forest model to obtain the predicted value of the ship cost for each decision tree in the random forest. The calculation formula is: , where is the total predicted value of the ship cost of the random forest model for the newly input feature vector ; is the predicted value of the a-th decision tree for the input feature vector ; a is the serial number of the decision tree; M is the number of decision trees.

[0010] Preferably, a ship and ocean engineering auxiliary design method based on a multimodal large model further includes step S6, and the specific process of step S6 is as follows: S61. Let S be the set of feature vectors of the ship design scheme, S = { }, where It is the feature vector of each ship design plan. The feature vector of the ship design plan includes the ship length, ship width, draft, and deadweight. S62. Let R be the set of requirements of regulations and specifications, R = { }, where is the specific requirement of each regulation and specification. The specific requirement of the regulation and specification includes the minimum safety distance and the maximum load limit. S63. Let M c be a derivative model variant of the GPT series or a large language model, which receives the set S of feature vectors of the input ship design plan and the set R of requirements of regulations and specifications, and outputs the adaptability evaluation result A. The adaptability evaluation result A is expressed as: A = M c (S, R). In the formula, A is a numerical value, indicating the adaptability degree of the ship design plan to the regulations and specifications.

[0011] After adopting the above technical solution, the present invention has the following beneficial effects: 1. The present invention can improve the design efficiency and quickly generate design plans. Traditional CAD software requires designers to manually create and modify design plans one by one. The algorithm of the present invention can generate multiple design options in a short time, and can automate routine tasks, such as automatically completing routine and cumbersome tasks such as dimensioning, annotation, and symbol placement, which can effectively save design time.

[0012] 2. The present invention can optimize the design plan, explore numerous design possibilities based on predefined constraints, help engineers optimize the design, and achieve higher efficiency, lower cost, and better performance. Traditional CAD software often relies on designers' experience and trial and error in design optimization, with low efficiency and difficult to achieve global optimality. At the same time, the present invention supports agile design. Through functions such as natural language interaction, designers can modify and adjust the design more conveniently, quickly respond to market demands and design changes, achieve more agile design iteration and testing, and shorten the product launch time.

[0013] 3. The present invention can predict the performance of the ship through a neural network. The performance prediction of the ship is a key link in the design process, including the ship speed, stability, maneuverability, etc. Based on actual ship data and simulation results, a performance prediction model is established to improve the design speed.

[0014] 4. The present invention can perform cost control analysis through a decision tree, and can analyze the impact of the design plan on the construction cost to realize the cost performance analysis of the design plan.

[0015] 5. The present invention can perform regulations and specifications analysis, and realize the analysis of whether the design plan meets the relevant regulations and requirements according to the regulations and specification standards.

[0016] 6. The present invention can provide instant feedback. During the design process, it can provide real-time feedback and suggestions to help users discover and solve problems in a timely manner, avoid errors and defects in later manufacturing or use, and improve the quality and reliability of the entire design process. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0019] like Figure 1 As shown, a ship and ocean engineering auxiliary design method based on a multi-modal large model includes the following steps: S1, taking the image features and text features of ship engineering drawings as input, and constructing a ship engineering drawing and text fusion model; The specific process of step S1 is: S11. The image features of ship engineering drawings are , where n is the number of image features, is the dimension of image features; the text features of ship engineering drawings are , where m is the number of text features, is the dimension of text features; S12. Image-to-text attention: Calculate the attention weight of the image features of the ship engineering drawings to the text features. The calculation formula is: , where The attention weight of the image features of the ship engineering drawings to the text features; is the sum index used to calculate the image representation of text features; is the sum index used to calculate the text representation of image features; Image features of ship engineering drawings The feature vectors, ; is the linear transformation matrix from image to text; Text features for ship engineering drawings The The transpose of the eigenvectors, ; is the sum index used to calculate the attention weight; Text features for ship engineering drawings The The transpose of a feature vector; Calculate the text representation of the image features, and the calculation formula is: , where is the text representation of the image features; is the text feature of the ship engineering drawing in the th feature vector; S13. Attention of text to image: Calculate the attention weight of the text feature of the ship engineering drawing to the image feature, and the calculation formula is: , where is the attention weight of the text feature of the ship engineering drawing to the image feature; is the linear transformation matrix from text to image; is the image feature of the ship engineering drawing in the transpose of the th feature vector; is the image feature of the ship engineering drawing in the transpose of the th feature vector; Calculate the image representation of the text feature, and the calculation formula is: , where is the image representation of the text feature; S14. Fuse the text representation of the image and the image representation of the text with the original features respectively. Among them, the calculation formula for fusing the image features is: , where is the fused image feature; is the vector concatenation or summation operation; The calculation formula for fusing the text features is: , where is the fused text feature; S15. Input the image features and text features as query and key-value pairs into the multi-head attention mechanism respectively; the image features are used as the query , and the text features are used as the key and value ; S2. Use the multi-head attention mechanism to calculate the attention weight between the image features and the text features, and obtain the fused features; The specific process of step S2 is as follows: S21. Determine the parameters of the multi-head attention mechanism for ship engineering images. The parameters of the multi-head attention mechanism include the number of heads h, the dimension of the query , the dimension of the key and the dimension of the value ; S22. Construct a linear transformation matrix for the randomly initialized multi-head attention mechanism and an output linear transformation matrix; S23. Based on the formula of the multi-head attention mechanism, perform the following calculations: , , , , where in the formula, is the query vector of the -th attention head; is the linear transformation matrix in the -th attention head for mapping image features to query vectors, , is the dimension of the query; is the key vector of the -th attention head; is the linear transformation matrix in the -th attention head for mapping text features to key vectors, , is the dimension of the key; is the value vector of the -th attention head; is the linear transformation matrix in the -th attention head for mapping text features to value vectors, , is the dimension of the value; is the output result of the -th attention head; is the activation function; is the -th attention head's key vector transposed; The multi-head concatenation formula is: , where in the formula, is the output feature fusing image features and text features; is the function for concatenating strings; is the output results of attention heads, is the matrix for linearly transforming the outputs of the multi-head attention heads, ; The output feature fusing image features and text features is used as the input for subsequent tasks, and the subsequent tasks include classification, retrieval, equipment description, and equipment component query of ship engineering drawings; S3. Optimize the ship design drawings or design schemes based on the generative adversarial network; The specific process of step S3 is: S31. Generator Input: Represent the image features of the ship engineering drawings as I and the text features as T. Then fuse the image features I and the text features T as the input of the generator. The generator receives the fused features and random noise z and generates a new ship engineering drawing or design scheme G(I, T, z). S32. Discriminator Input: The discriminator receives pairs of real ship engineering drawings and corresponding text features or pairs generated by the generator . The discriminator outputs a probability value representing the probability of the real data distribution. For the GAN model of ship engineering drawing and text fusion, the objective function is adjusted to: , where is the objective function of the generative adversarial network; is the expected operation on the joint distribution of real ship engineering drawings and corresponding text feature pairs; is the joint distribution of real ship engineering drawings and corresponding text features; is the discriminator function that inputs real data or generated data and outputs a probability value indicating the probability that the input data belongs to the real data distribution; is the expected operation on the prior distribution and random noise ; is the prior distribution; is the generator which, according to the input ship engineering drawing , text features and random noise , generates a ship engineering drawing or related design scheme, and then inputs the generated result into the discriminator . The discriminator outputs a probability for indicating the probability that the discriminator believes the generated result is a pair of real ship engineering drawing and text features; S33. Optimization of Discriminator D: Take the derivative of the objective function with respect to D. The calculation formula is: , where is the generator which generates a result based on the input random noise , and then inputs the generated result into the discriminator . The discriminator judges the result generated only by noise and outputs a probability value for indicating the probability that the judgment result is a pair of real ship engineering drawing and text features. Take the derivatives of the two terms respectively to get: , . Update the discriminator parameters by gradient ascent according to the derivative to maximize the objective function, which is used to improve the ability to distinguish real samples and generated samples; S34. Optimization of Generator G: Take the derivative of the objective function with respect to G, and the calculation formula is: , where is the generator According to the ship engineering drawings , text features and random noise generate a result, and then input the output result of the generator together with the text features into the discriminator . The discriminator makes a judgment based on the two inputs and outputs a probability indicating that the input is a pair of real ship engineering drawings and text features; let u = G(I, T, z), and we get: , update the generator parameters by gradient descent according to the derivative to minimize the objective function, so as to make the generated samples more realistic and be able to deceive the discriminator; S35. By continuously iteratively optimizing the discriminator and the generator, the generator fuses the image features and text features of the ship engineering drawings to generate high-quality design drawings or design schemes; S4. Predict the performance of the ship based on the neural network; The specific process of step S4 is as follows: S41. Build a ship design scheme database, which includes the dimensions, structural parameters, power system parameters, loading conditions and performance indicators of the ship. The dimensions of the ship include the length between perpendiculars, beam and draft, and the performance indicators of the ship include speed, stability and maneuverability; then select the features related to performance prediction and realize feature transformation based on the multi-layer perceptron model. The features related to performance prediction include main engine power, ship form coefficient, displacement and resistance; S42. Assume that there are q neurons in the input layer, corresponding to the q features selected in the ship performance prediction. Denote the input feature vector as x = ; there are r neurons in the hidden layer and p neurons in the output layer, corresponding to the p performance indicators to be predicted; S43. From the input layer to the hidden layer: The input weighted sum of the hidden layer neurons is: , where is the input weighted sum of the th neuron in the hidden layer; is the connection weight from the th neuron in the input layer to the th neuron in the hidden layer; is the bias of the th neuron in the hidden layer; Based on the activation function σ(·), the output of the hidden layer neurons is non-linearly transformed: S44. Hidden layer to output layer: The weighted sum of inputs to the output layer neurons is: , where is the weighted sum of inputs to the -th neuron in the output layer; is the connection weight from the -th neuron in the hidden layer to the -th neuron in the output layer; , where is the output of the -th neuron in the output layer; σ(·) is the activation function; The overall multi-layer perceptron model is represented as a function mapping: , where is the performance index vector of the predicted ship; W is the matrix of all weights; b is the vector of all biases; is a multi-layer perceptron model function that describes the entire mapping relationship from the input feature vector , through the action of the weight matrix and the bias vector , to the output predicted ship performance index vector ; S5. Cost prediction of ship construction cost based on decision tree; The specific process of step S5 is as follows: S51. Construct a ship cost database, which includes the steel plate size, structural materials, equipment configuration, construction technology, welding materials, man-hours and corresponding cost data of the ship; S52. Construct a decision tree-based ship construction cost model: For each decision tree, the input feature vector is , where t is the number of ship cost-related features; The decision tree recursively selects the best split feature and split point to divide the data set into different subsets until the stopping condition is met; Suppose the decision tree finally divides the samples into the leaf node set { }, for the input feature vector arriving at the leaf node denoted as L( ), in the regression problem of ship cost budget, the predicted value of the decision tree for the input feature vector is the average value of all cost data within the leaf node, that is: , where is the predicted value of the decision tree for the input feature vector ; is the number of samples within the leaf node L( ); is the true cost value of each single sample ; It is a single sample, that is, the single sample is the steel plate size, structural material, equipment configuration, construction technology, welding material or man-hour of the ship; S53. The random forest consists of M decision trees. According to the total cost budget requirement of the newly built ship, the steel plate size, structural material, equipment configuration, construction technology, welding material, and man-hour characteristics of the ship are composed into a feature vector ; The feature vector is input into the random forest model to obtain the predicted value of the ship cost for each decision tree in the random forest. The calculation formula is: , where is the total predicted value of the ship cost by the random forest model for the newly input feature vector ; is the predicted value of the a-th decision tree for the input feature vector ; a is the serial number of the decision tree; M is the number of decision trees; A ship and ocean engineering auxiliary design method based on a multimodal large model further includes step S6. The specific process of step S6 is as follows: S61. Let S be the set of feature vectors of the ship design scheme, S = { }, where is the feature vector of each ship design scheme. The feature vector of the ship design scheme includes the ship length, ship width, draft and deadweight; S62. Let R be the set of requirements of the regulations and specifications, R = { }, where is the specific requirement of each regulation and specification. The specific requirement of the regulation and specification includes the minimum safety distance and the maximum load limit; S63. Let M c be a derivative model variant of the GPT series or a large language model, which receives the set S of feature vectors of the ship design scheme and the set R of requirements of the regulations and specifications as input, and outputs an adaptability evaluation result A. The adaptability evaluation result A is expressed as: A = M c (S, R), where A is a numerical value representing the adaptability degree of the ship design scheme to the regulations and specifications.

[0020] As described above, it is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A ship and ocean engineering auxiliary design method based on a multi-modal large model, characterized in that: The following steps are involved: S1, taking the image features and text features of ship engineering drawings as input, and constructing a ship engineering drawing and text fusion model; The specific process of step S1 is: S11. The image features of ship engineering drawings are , where n is the number of image features, is the dimension of image features; the text features of ship engineering drawings are , where m is the number of text features, is the dimension of text features; S12. Image-to-text attention: Calculate the attention weight of the image features of the ship engineering drawings to the text features. The calculation formula is: , where The attention weight of the image features of the ship engineering drawings to the text features; is the sum index used to calculate the image representation of text features; is the sum index used to calculate the text representation of image features; Image features of ship engineering drawings The feature vectors, ; is the linear transformation matrix from image to text; Text features for ship engineering drawings The The transpose of the eigenvectors, ; is the sum index used to calculate the attention weight; Text features for ship engineering drawings The The transpose of the eigenvectors; Calculate the text representation of image features, the calculation formula is: , where is the text representation of image features; Text features for ship engineering drawings The feature vectors; S13. Text-to-image attention: Calculate the attention weight of the text features of the ship engineering drawings to the image features. The calculation formula is: , where The attention weight of the text features to the image features of the ship engineering drawings; is the linear transformation matrix from text to image; Image features of ship engineering drawings The The transpose of the eigenvectors; Image features of ship engineering drawings The The transpose of the eigenvectors; Calculate the image representation of text features, the calculation formula is: , where is an image representation of text features; S14, fusing the text representation of the image and the image representation of the text with the original features respectively, wherein the calculation formula of the fused image features is: , where To fuse image features; It is a vector concatenation or addition operation; The calculation formula for fusion text features is: , where To integrate text features; S15, input the image features and text features as query and key-value pairs into the multi-head attention mechanism; image features as query , text features as keys and values ; S2. Use the multi-head attention mechanism to calculate the attention weights between image features and text features, and obtain fusion features; S3. Optimize ship design drawings or design plans based on generative adversarial networks; S4. Predicting the performance of the ship based on neural network; S5. Make cost prediction for shipbuilding cost based on decision tree.

2. The ship and ocean engineering auxiliary design method based on multi-modal large model as claimed in claim 1, characterized in that: The specific process of step S2 is: S21. Clarify the multi-head attention mechanism parameters for ship engineering images. The multi-head attention mechanism parameters include the number of heads h and the query dimension. , the dimension of the key The dimension of the sum value ; S22, construct the linear transformation matrix of the randomly initialized multi-head attention mechanism and the output linear transformation matrix; S23. Based on the formula of the multi-head attention mechanism, the following calculation is performed: , , , , where For the The query vector of the attention head; For the The linear transformation matrix used to map image features to query vectors in the attention heads, , is the dimension of the query; For the The key vector of the attention heads; For the The linear transformation matrix used to map text features to key vectors in the attention heads, , is the dimension of the key; For the The value vector of the attention heads; For the The linear transformation matrix used to map text features into value vectors in the attention heads, , is the dimension of the value; For the The output of the attention head; is the activation function; For the The key vector of the attention head The transpose of The multi-head splicing formula is: , where Output features that fuse image features and text features; is a function used to concatenate strings; for The output of the attention head is: is the number of heads in the multi-head attention mechanism; is the matrix used to linearly transform the output of the multi-head attention head, The output features of the fusion of image features and text features are used as input for subsequent tasks, including classification, retrieval, equipment description, and equipment component query of ship engineering drawings.

3. The ship and ocean engineering auxiliary design method based on multi-modal large model as claimed in claim 1, characterized in that: The specific process of step S3 is: S31, generator input: the image feature of the ship engineering drawing is represented as I, and the text feature is represented as T; the image feature I and the text feature T are fused as the input of the generator, and the generator receives the fused features and random noise z to generate a new ship engineering drawing or design scheme G(I, T, z); S32, discriminator input: the discriminator receives the real ship engineering drawings and the corresponding text feature pairs Or the pair generated by the generator ; The discriminator outputs a probability value, which represents the probability of the real data distribution. For the GAN model of ship engineering drawings and text fusion, the objective function is adjusted to: , where is the objective function of the generative adversarial network; is the joint distribution of the real ship engineering drawings and the corresponding text feature pairs Expected operation of ; is the joint distribution of real ship engineering drawings and corresponding text feature pairs; It is the discriminator function, which inputs real data or generated data and outputs a probability value, indicating the probability that the input data belongs to the real data distribution; For the prior distribution and random noise Expected operation of ; is the prior distribution; For the generator According to the input ship engineering drawings , text features and random noise , generate a ship engineering drawing or related design plan, and then input the generated result into the discriminator In the example, the discriminator outputs a probability that the discriminator believes that the generated result is a pair of real ship engineering drawings and text features; S33, optimization of the discriminator D: derive the objective function with respect to D, and the calculation formula is: , where For the generator Random noise based on input Generate a result and then input the generated result into the discriminator , the discriminator judges the result generated only by noise and outputs a probability value, which is used to indicate the probability that the judgment result is a real ship engineering drawing and text feature pair; the derivatives of the two items are obtained respectively: , , update the discriminator parameters according to the derivatives by gradient ascent to maximize the objective function, which is used to improve the ability to distinguish between real samples and generated samples; S34. Optimization of generator G: Derivation of the objective function with respect to G, the calculation formula is: , where For the generator According to ship engineering drawings , text features and random noise Generate a result, and then the output result of the generator Input to the discriminator together with text features In , the discriminator makes a judgment based on two inputs and outputs a probability indicating that the input is a pair of real ship engineering drawings and text features; Let u = G(I, T, z), we get: , update the generator parameters according to the derivatives by gradient descent to minimize the objective function, which is used to make the generated samples more realistic and able to deceive the discriminator; S35. Through continuous iteration and optimization of the discriminator and the generator, the generator is made to integrate the image features and text features of the ship engineering drawings to generate high-quality design drawings or design plans.

4. The ship and ocean engineering auxiliary design method based on multi-modal large model as claimed in claim 1, characterized in that: The specific process of step S4 is: S41. Construct a ship design database, which includes the ship's dimensions, structural parameters, power system parameters, loading conditions and performance indicators. The ship's dimensions include length, width and draft, and the ship's performance indicators include speed, stability and maneuverability. Then, select features related to performance prediction, and implement feature transformation based on a multi-layer perceptron model. The features related to performance prediction include main engine power, ship type coefficient, displacement and resistance. S42. Assume that the input layer has q neurons, corresponding to the q features selected in the ship performance prediction, and the input feature vector is x= ; The hidden layer has r neurons and the output layer has p neurons, corresponding to the p performance indicators to be predicted; S43, input layer to hidden layer: The weighted sum of the inputs of the neurons in the hidden layer is: , where For the hidden layer The weighted sum of the inputs to the neurons; is the connection weight from the i-th neuron in the input layer to the j-th neuron in the hidden layer; is the bias of the jth neuron in the hidden layer; Based on the activation function σ(·), the output of the hidden layer neurons is obtained by nonlinear transformation: , where For the hidden layer The output of a neuron; S44, hidden layer to output layer: The weighted sum of the inputs of the output layer neurons is: , where The output layer The weighted sum of the inputs to the neurons; is the connection weight from the jth neuron in the hidden layer to the kth neuron in the output layer; is the bias of the kth neuron in the output layer; The results of the output layer represent the performance indicators: , where The output layer The output of a neuron; σ(·) is the activation function; The overall multi-layer perceptron model is represented as a function mapping: ,in, is the predicted performance index vector of the ship; W is the matrix of all weights; b is the bias vector; is a multi-layer perceptron model function, describing the input feature vector , after the weight matrix and the bias vector The role of the output predicted ship performance index vector The entire mapping relationship.

5. The ship and ocean engineering auxiliary design method based on multi-modal large model as claimed in claim 1, characterized in that: The specific process of step S5 is: S51, constructing a ship cost database, which includes the ship's steel plate size, structural materials, equipment configuration, construction process, welding materials, labor hours and corresponding cost data; S52. Construct a ship cost model based on decision tree: For each decision tree, the input feature vector is , where t is the number of ship cost related features; the decision tree divides the data set into different subsets by recursively selecting the best split features and split points until the stopping condition is met; assuming that the decision tree eventually divides the samples into the leaf node set { }, for the input feature vector The leaf node reached is denoted as L( ), in the regression problem of ship cost budget, the decision tree is used to evaluate the input feature vector The predicted value of is the average value of all cost data in the leaf node, that is: , where Input feature vector for decision tree The predicted value of For leaf node L( ) For each single sample The true cost value of Single sample, i.e., the single sample is the ship's steel plate size, structural material, equipment configuration, construction process, welding material or labor hours; S53, random forest is composed of M decision trees. According to the total cost budget requirement of the new ship, the steel plate size, structural material, equipment configuration, construction process, welding material, and labor time characteristics of the ship are combined into a feature vector ; The feature vector Input into the random forest model to obtain the predicted value of ship cost for each decision tree in the random forest. The calculation formula is: , where For the random forest model, the new input feature vector Total estimated ship cost; is the a-th decision tree for the input feature vector The predicted value of ; a is the sequence number of the decision tree; M is the number of decision trees.

6. The ship and ocean engineering auxiliary design method based on multi-modal large model as claimed in claim 1, characterized in that: The process further includes step S6, wherein the specific process of step S6 is as follows: S61. Let S be the set of characteristic vectors of ship design schemes, S={ },in, is the characteristic vector of each ship design scheme, and the characteristic vector of the ship design scheme includes the length, width, draft and deadweight; S62, let R be the set of regulatory requirements, R={ },in, The specific requirements of various regulations and specifications include minimum safety distance and maximum load limit; S63, set M c It is a derivative model variant or large language model of the GPT series. It receives the input feature vector set S of the ship design scheme and the requirement set R of the regulatory specifications, and outputs the adaptability evaluation result A. The adaptability evaluation result A is expressed as: A=M c (S, R), where A is a numerical value indicating the degree of adaptability of the ship design scheme to regulatory standards.

Images