Method for generating a multimodal large model offshore oil tanker hull safety inspection report

Abstract

The application discloses a method for generating a safety inspection report of an offshore oil production ship shape of a multimodal large model, relates to the technical field of intelligent generation of ship safety inspection, and first collects an offshore oil production ship image, matches a corresponding safety inspection report, and constructs a data set; subsequently, a multimodal report generation model is constructed, the offshore oil production ship image and the corresponding safety inspection report are input into the model to generate a safety inspection report of an offshore oil production ship shape; subsequently, the model is trained, and after multiple rounds of training, a final multimodal report generation model is obtained; finally, the offshore oil production ship image is input into the final model, and a safety inspection report of an offshore oil production ship shape is output. The application enhances the controllability and pertinence of the content generated by the large language model, improves the accuracy and compliance of the shape safety inspection report, and can be widely applied to scenes such as daily operation management of offshore oil production ships, safety guarantee of ocean navigation, compliance review of ports and ship classification societies, and accident prevention and subsequent maintenance disposal.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent generation of ship safety inspection, and in particular to a method for generating a safety inspection report for the hull of an offshore oil production ship using a multi-modal large model. BACKGROUND

[0002] As the core equipment for deep-sea oil and gas development, offshore oil production ships must undergo systematic safety inspections during navigation, anchoring, and maintenance. By detecting key aspects such as the hull shape and structure, power systems, deck machinery, and oil and gas processing equipment, potential defects or safety hazards can be identified in a timely manner, and the results can be organized into structured and standardized documents. Such safety inspection reports are not only important for ensuring the safe operation of offshore oil production ships, but also serve as key references for ship classification society inspections, port supervision, and insurance claims, playing an irreplaceable role in reducing operational risks and maintaining environmental safety.

[0003] However, current safety inspection reports for the hull of offshore oil production ships are mostly completed manually. Due to the complexity of the inspection objects, large task volume, and differences in personnel experience and subjective judgments, manual inspections often have low efficiency, high omission rates, and incomplete records, making it difficult to ensure the accuracy, consistency, and compliance with industry standards of the report content. Existing partially automated methods lack specificity, with issues such as insufficient alignment of image and text semantics, weak controllability of generated results, and inconsistencies with industry standards. Therefore, there is an urgent need for a new technical approach that can improve efficiency while reducing human subjectivity, ensuring the professionalism, standardization, and traceability of the output report. SUMMARY

[0004] In view of the above, the present application provides a method for generating a safety inspection report for the hull of an offshore oil production ship using a multi-modal large model, to address the issues of low manual efficiency, strong subjectivity of inspection results, and insufficient standardization in the prior art.

[0005] The method for generating a safety inspection report for the hull of an offshore oil production ship using a multi-modal large model provided by the present application includes the following steps:

[0006] S1, collecting images of offshore oil production ships, matching corresponding safety inspection reports, and constructing a data set;

[0007] S2, constructing a multi-modal report generation model; the multi-modal report generation model includes a ConvNeXt V2 network, a prompt pool module, a large language module tokenizer, a multi-modal fusion module, and a large language model, and the large language model is LLaMA2; inputting the images of offshore oil production ships into the ConvNeXt V2 network to obtain a feature sequence of the offshore oil production ship , and then inputting the feature sequence of the offshore oil production ship Input into the prompt pool module to obtain the prompt sequence ; then input the safety inspection report corresponding to the offshore oil production ship image into the large language module tokenizer to obtain the report semantic sequence ; then input the offshore oil production ship feature sequence , prompt sequence and report semantic sequence into the multi-modal fusion module to fuse into a multi-modal sequence ; finally, input the multi-modal sequence into LLaMA2 to generate an offshore oil production ship shape safety inspection report;

[0008] S3, train the model; input the offshore oil production ship image into the multi-modal report generation model, calculate the sequence cross-entropy loss, perform back propagation, and after multiple rounds of training, obtain the final multi-modal report generation model;

[0009] S4, input the offshore oil production ship image into the final multi-modal report generation model, and output the offshore oil production ship shape safety inspection report.

[0010] Further, the specific implementation steps of S1 are:

[0011] Sa1, collect offshore oil production ship images and their corresponding safety inspection reports into an original sample library;

[0012] Sa2, perform quality screening on the safety inspection reports in the original sample library. If the safety inspection report content is complete and the format is standard, it is included in the standardized sample library. If the safety inspection report content is incomplete or the format is not standard, it is standardized and then included in the standardized sample library;

[0013] Sa3, construct the offshore oil production ship images and their corresponding safety inspection reports in the standardized sample library into the data set.

[0014] Further, in S2, the specific steps for obtaining the offshore oil production ship feature sequence are:

[0015] Sb1, in the first stage Stem layer, use a convolution operation with a convolution kernel size of and a stride of 4 to downsample the offshore oil production ship image to obtain primary features;

[0016] Sb2, the second stage Stage1 uses a convolution kernel with a size of and a stride of 2 to further downsample the primary features, and stacks multiple ConvNeXt V2 blocks to obtain local structural features;

[0017] Sb3, the third stage Stage2 uses The convolutional kernels are downsampled, and then ConvNeXtV2 blocks are stacked to extract mid-level semantic features and model larger receptive field information;

[0018] Sb4 and Stage 3, after downsampling in Stage 3, extract global high-level features through residualized ConvNeXt V2 blocks, ultimately obtaining the feature sequence of the offshore oil production vessel. .

[0019] Further, in step S2, the prompt sequence is obtained. The specific steps are as follows:

[0020] Sc1, Characteristic Sequence of Offshore Oil Production Vessels Average pooling is performed to obtain the global characteristics of offshore oil production vessels. The characteristic sequence of the offshore oil production vessel Expressed as:

[0021]

[0022] In the formula, Represented as a matrix composed of real numbers; The number of samples is represented by images of an offshore oil production vessel. Represented as a characteristic sequence of offshore oil production vessels The number of feature vectors contained in each sample. Represented as a characteristic sequence of offshore oil production vessels The feature dimension of a single feature vector;

[0023] Global characteristics of offshore oil production vessels Expressed as:

[0024] ;

[0025] Sc2, The prompt pool module includes a prompt pool. , expressed as:

[0026]

[0027] In the formula, Represented as a hint pool Size, This represents how many cue vectors make up a single cue; Represented as a vector dimension and feature sequence of an offshore oil production vessel The feature dimensions are equal;

[0028] That is, the dimension of each prompt vector is D, which is consistent with the dimension of the feature vector of the offshore oil production ship feature sequence, facilitating similarity calculation and fusion with the image global feature in the prompt pool module; meanwhile, the key vector matrix is initialized , wherein the th row represents the key vector corresponding to the th prompt in the prompt pool

[0029] . Sc3, the global feature of the offshore oil production ship and the initialized key vector matrix are normalized to obtain the normalized global feature of the offshore oil production ship and the normalized key vector matrix , respectively.

[0030] Sc4, cosine similarity calculation is performed on the normalized global feature of the offshore oil production ship and the normalized key vector matrix to obtain a similarity matrix .

[0031] Sc5, the highest similarity prompt is selected from each row in the similarity matrix to form a candidate set, and then a prompt subset corresponding to the candidate set is extracted from the prompt pool to obtain , expressed as:

[0032]

[0033] In the formula, represents the prompt subset selected from the prompt pool according to the similarity; is the prompt, and represents the number of prompts selected from the prompt pool .

[0034] Sc6, finally, the prompts of each sample are flattened and spliced in the D-dimensional space to obtain the associated prompt sequence of the offshore oil production ship feature sequence , expressed as:

[0035] .

[0036] Further, in the S2, the report semantic sequence is expressed as:​​​​

[0037] .

[0038] Furthermore, in S2, during the fusion into a multimodal sequence... First, we need to define the characteristic sequence of offshore oil production vessels. Hint sequence Both are unified to the same dimension as the report's semantic sequence through linear projection; then, while keeping the first dimension unchanged, the feature sequence of the offshore oil production vessel is... Hint sequence and report semantic sequence The multimodal sequence is obtained by concatenating the sequences in the second dimension. Multimodal sequences In fact, it is by Hint sequence and the characteristic sequence of offshore oil production vessels Adding matrices along the second dimension is expressed as:

[0039] .

[0040] Furthermore, multimodal sequences The specific steps for inputting into the large language model LLaMA2 are as follows:

[0041] The multimodal sequence First, the sequence undergoes embedding mapping and positional encoding, followed by multimodal sequence... Semantic modeling is performed by sequentially inputting multi-layer Transformer blocks into the large language model LLaMA2 (using multi-layer Transformer blocks to perform semantic processing on the input multimodal sequences); in the decoding stage, the large language model LLaMA2 generates output tokens step by step in an autoregressive manner to obtain the aforementioned marine oil production vessel shape safety inspection report.

[0042] On the other hand, the present invention also provides a system for generating a safety inspection report on the shape of a multimodal large-scale offshore oil production vessel, including a memory, a processor, and program instructions stored in the memory for the processor to run. The processor executes the program instructions to implement the steps in the above-described method for generating a safety inspection report on the shape of a multimodal large-scale offshore oil production vessel.

[0043] On the other hand, the present invention also provides a computer-readable storage medium storing a computer program that implements the above-described method for generating a safety inspection report on the shape of a multimodal large model of an offshore oil production vessel.

[0044] The beneficial effects of the present application are: the present application enhances the controllability and pertinence of large language model content generation by introducing a prompt driving mechanism, improves the accuracy and compliance of the external shape safety inspection report, and can be widely applied to daily operation management of offshore oil production ships, safety guarantee of ocean navigation, compliance review of ports and ship classification societies, and scenes such as accident prevention and subsequent maintenance disposal. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 Flowchart of the present application;

[0046] Figure 2 Flowchart of the present application for constructing a data set;

[0047] Figure 3 Structure diagram of the multi-modal report generation model of the present application;

[0048] Figure 4 ConvNeXt V2 block structure diagram of the present application;

[0049] Figure 5 Prompt pool module structure diagram of the present application;

[0050] Figure 6 Flowchart of the model training process of the present application. DETAILED DESCRIPTION

[0051] The embodiments of the present application will be described in detail below with reference to the accompanying drawings, but they do not constitute a limitation on the present application, but are only examples. Through the description, the advantages of the present application will be more clearly understood. All modifications that can be directly derived or inferred by those skilled in the art from the content disclosed by the present application should be considered as falling within the scope of protection of the present application. Other parts not described in detail in the embodiments are prior art.

[0052] First, the technical terms involved in the present application are explained and described.

[0053] Multi-modal large model: the multi-modal large model is the core architecture used in the present application to generate a safety inspection report for an offshore oil production ship. Its purpose is to simultaneously receive and process heterogeneous data such as images and text, and to realize information fusion in a unified framework, thereby improving the understanding and generation ability of complex scenes. The basic principle is: first, a special encoder is used to extract features from different modal data; then, through a feature alignment and fusion mechanism, the features of each modality are mapped to a shared semantic representation space; finally, a unified large language model is used as a decoder to realize cross-modal reasoning and natural language report generation. The multi-modal large model in the present application covers all modules, including: a visual feature extraction model, a prompt pool module, a large language model LLaMA2, and a word segmenter.

[0054] Average pooling: Average pooling is introduced in the prompt pool, which aims to extract the global features of the offshore oil production ship image. Average pooling compresses the feature representation by taking the mean value of the local region in the spatial dimension, thereby gradually reducing the resolution of the feature map, reducing the parameter size and improving the computational efficiency, and also alleviating the overfitting problem to some extent.

[0055] Normalization: This is a common vector normalization method, the core idea of which is to scale the input vector according to its Euclidean length, so that the length of the normalized vector is fixed at 1. In the processing of the global features of the offshore oil production ship and the prompt key vector, normalization is introduced The purpose of normalization is to eliminate the difference in numerical scale of different vectors, so as to avoid the deviation of similarity calculation caused by different vector lengths.

[0056] Word segmentation: Word segmentation is a pre-processing module of text input, which converts the original natural language sentence into a discrete symbol sequence that the model can recognize. In the safety inspection report generation method of the present application, the purpose of introducing word segmentation is to divide the safety report text corresponding to the offshore oil production ship image into basic words or sub-word units, so as to obtain semantic sequences that can be further coded by the model. The principle of word segmentation is to map the text into a series of integer identifiers according to the predefined word table, each identifier corresponds to a word or sub-word, while ensuring that the model can cover a wide range of language expressions within a limited word table.

[0057] Embodiment one:

[0058] The present application proposes a multi-modal large model offshore oil production ship shape safety inspection report generation method, as shown in Figure 1 The method comprises the following steps:

[0059] S1, collect offshore oil production ship images, match corresponding safety inspection reports, and construct a data set;

[0060] The specific implementation steps of S1 are:

[0061] Sa1, collect offshore oil production ship images and their corresponding safety inspection reports into an original sample library;

[0062] Sa2, quality screening is performed on the safety inspection reports in the original sample library, if the safety inspection report content is complete and the format is standard, it is included in the standardized sample library; if the safety inspection report content is incomplete or the format is not standard, it is standardized by DeepSeek, and then manually reviewed, modified and confirmed by professional personnel, and included in the standardized sample library;

[0063] Sa3, the offshore oil production ship images and their corresponding safety inspection reports in the standardized sample library are constructed into the data set. As shown inFigure 2 as shown.

[0064] S2, a multi-modal report generation model is constructed, and a structural diagram of the multi-modal report generation model is as shown. Figure 3 The multi-modal report generation model includes a ConvNeXt V2 network, a prompt pool module, a large language module tokenizer, a multi-modal fusion module, and a large language model LLaMA2; an offshore oil production ship image is input into the ConvNeXt V2 network to obtain an offshore oil production ship feature sequence , and then the offshore oil production ship feature sequence is input into the prompt pool module to obtain a prompt sequence associated with the offshore oil production ship feature sequence .

[0065] In the S2, the ConvNeXt V2 network is processed in four stages, and the processing steps are as follows:

[0066] Sb1, in the first stage Stem layer, a convolution operation with a convolution kernel size of and a stride of 4 is adopted to down-sample the offshore oil production ship image to obtain a primary feature; the first stage Stem layer is composed of an initial convolution and a down-sampling operation, and is mainly responsible for converting the input image into a primary feature;

[0067] Sb2, the second stage (Stage 1) adopts a convolution kernel with a size of and a stride of 2 to further down-sample the primary feature, and stacks multiple ConvNeXt V2 blocks to obtain a local structural feature; the second stage (Stage 1) is composed of a down-sampling layer and a plurality of ConvNeXt V2 blocks stacked, each block includes a depth separable convolution, a pointwise convolution, a nonlinear activation function, and a layer normalization, which is used to extract a local structural feature;

[0068] Sb3, the third stage (Stage 2) adopts a convolution kernel for down-sampling, and then continues to stack ConvNeXt V2 blocks to extract middle-level semantic features and model larger receptive field information; the third stage (Stage 2) is also composed of a down-sampling layer and a plurality of ConvNeXt V2 blocks, which emphasizes the extraction and modeling of middle-level semantic features at a lower resolution;

[0069] Sb4, after the third stage (Stage 2) is down-sampled, a residual ConvNeXt V2 block is used to extract global high-level features, and finally the offshore oil production ship feature sequence The fourth stage, Stage 3, consists of a downsampling layer and a residual ConvNeXt V2 block, which is responsible for extracting global high-level features and forming the final feature sequence of the offshore oil production vessel.

[0070] ConvNeXt V2 block structure as follows Figure 4 As shown, in the ConvNeXt V2 block, the input features are first normalized through an LN layer. This normalization process stabilizes training and mitigates the vanishing or exploding gradient problem. The input then undergoes convolution to extract local spatial features and establish a preliminary feature representation. Next, the GELU (Gaussian Error Linear Unit) activation function is used, which smoothly suppresses noise with small inputs and maintains near-linearity with large inputs, thus improving the continuity and stability of gradient propagation. Afterward, GRN (Global Response Normalization) normalizes and reweights the entire feature map, resulting in a more balanced response across different channels and spatial locations, reducing the over-dominance of certain features. Finally, a linear projection and residual connection layer fuses the processed features with the input features, achieving a balance between information preservation and deep feature enhancement.

[0071] In step S2, the prompt sequence is obtained. The specific steps are as follows:

[0072] Sc1, Characteristic Sequence of Offshore Oil Production Vessels Average pooling is performed to obtain the global characteristics of offshore oil production vessels. The characteristic sequence of the offshore oil production vessel Expressed as:

[0073]

[0074] In the formula, Represented as a matrix composed of real numbers; B represents the number of samples, where the samples are images of offshore oil production vessels. Each image of an offshore oil production vessel input into the visual feature extraction model is considered as one sample. If eight images are input at once, then B=8; if only one image is input at once, then B=1. Represented as a characteristic sequence of offshore oil production vessels The number of feature vectors contained in each sample. Represented as a characteristic sequence of offshore oil production vessels The feature dimension of a single feature vector;

[0075] Global characteristics of offshore oil production vessels Expressed as:

[0076] ;

[0077] Sc2, The prompt pool module includes a prompt pool. , expressed as:

[0078]

[0079] In the formula, Represented as a hint pool Size, This represents how many cue vectors make up a single cue; Represented as a vector dimension and feature sequence of an offshore oil production vessel The feature dimensions are equal; that is, the dimension of each cue vector is D, which is consistent with the feature vector dimension of the feature sequence of an offshore oil production vessel, facilitating similarity calculation and fusion with global image features in the cue pool module; at the same time, the key vector matrix is ​​initialized. , of which OK Representation and Hint Pool The Middle A tip The key vector corresponds to the key vector; initialize the key vector matrix. The cue pool module is a set of key vectors set for each cue, used to calculate similarity with global image features, thereby selecting the most relevant cue; the cue pool module structure is as follows: Figure 5 As shown;

[0080] Sc3, Overall characteristics of offshore oil production vessels and initializing the key vector matrix conduct Normalization yields the normalized global characteristics of offshore oil production vessels. and normalized bond vector matrix ;

[0081] Sc4. Global characteristics of normalized offshore oil production vessels and normalized bond vector matrix Perform cosine similarity calculation to obtain the similarity matrix. The ;

[0082] Sc5, Similarity Matrix Select each row independently The most similar prompts form a candidate set, and then the prompt pool is used to select the prompts from the candidate set. Extract the corresponding subset of hints to obtain , expressed as:

[0083]

[0084] In the formula, Indicated from the suggestion pool The prompt subset selected according to the similarity; For prompting, expressed as a prompt pool The number of prompts selected from the prompt pool;

[0085] Sc6, finally The feature sequence of the offshore oil production ship is obtained by flattening and splicing each sample in the prompt sequence dimension associated prompt sequence , expressed as:

[0086] .

[0087] S3, input the safety inspection report corresponding to the offshore oil production ship image into the large language module tokenizer, to obtain the report semantic sequence ; input the safety report corresponding to the offshore oil production ship image into the tokenizer of the large language model LLaMA2, that is, convert the natural language text into a string of token IDs, thereby forming a report semantic sequence that can be processed by the model;

[0088] The report semantic sequence is expressed as:

[0089] .

[0090] S4, input the offshore oil production ship feature sequence , prompt sequence and report semantic sequence to the multi-modal fusion module to fuse into a multi-modal sequence ; then input the multi-modal sequence into the large language model LLaMA2 to generate an offshore oil production ship shape safety inspection report;

[0091] Before being fused into a multi-modal sequence , first unify the offshore oil production ship feature sequence and the prompt sequence to the same dimension as the report semantic sequence through linear projection; then under the premise of keeping the first dimension unchanged, splice the offshore oil production ship feature sequence , prompt sequence and report semantic sequence in the second dimension, to obtain the multi-modal sequence , the multi-modal sequence is actually a matrix addition of , prompt sequence and offshore oil production ship feature sequence in the second dimension, expressed as:

[0092] .

[0093] The multimodal sequence is input to the large language model LLaMA2. The specific steps are as follows:

[0094] The multimodal sequence is first mapped and encoded by embedding, and then the multimodal sequence is sequentially input into the multi-layer Transformer block of the large language model LLaMA2 for semantic modeling (the multi-layer Transformer block is used to process the input multimodal sequence); in the decoding stage, the large language model LLaMA2 generates output tokens step by step in a self-recurrent manner to obtain the offshore oil production ship shape safety inspection report.

[0095] S5, training the model; inputting the offshore oil production ship image into the multimodal report generation model, performing forward propagation, calculating sequence cross-entropy loss, performing back propagation, and training multiple rounds to obtain the final multimodal report generation model;

[0096] In step S5, the specific implementation steps of training the model are as follows: as shown in Figure 6 , first, the ConvNeXt V2 network and the large language model LLaMA2 are initialized with their respective pre-training parameters, and then each sample pair of the offshore oil production ship image and its corresponding safety inspection report in the data set is forward and backward propagated; during the forward propagation process, the offshore oil production ship image is first extracted by the ConvNeXt V2 network to obtain the offshore oil production ship feature sequence , and then the offshore oil production ship global feature is obtained by average pooling, and the cosine similarity is calculated with the key vectors of each prompt in the prompt pool , and the prompt with the highest similarity is selected, and then the prompt is flattened into a prompt sequence ; at the same time, the safety inspection report corresponding to the offshore oil production ship image is input into the large language module tokenizer to obtain the report semantic sequence ; then, the offshore oil production ship feature sequence and the prompt sequence are mapped to the same dimension as the report semantic sequence by linear projection; the offshore oil production ship feature sequence , the prompt sequence and the report semantic sequence are fused into a multimodal sequence The input is input to a large language model LLaMA2, and LLaMA2 is prompted to predict the next token of the report in a self-recurrent manner; the target function of the training adopts a standard sequence cross-entropy loss, the sequence cross-entropy loss loss is back-propagated to the whole "ConvNeXt V2 network + prompt pool parameters + linear projection layer", and these parameters are constantly updated through gradient descent, so that the difference between the report generated by the model and the real safety inspection report at the token level is minimized.

[0097] S6, input the offshore oil production ship image into the final multi-modal report generation model, and output the offshore oil production ship shape safety inspection report.

[0098] The application also provides a multi-modal large model offshore oil production ship shape safety inspection report generation system, comprising a memory, a processor and program instructions stored in the memory for the processor to run, and the processor executes the program instructions to realize the steps in the multi-modal large model offshore oil production ship shape safety inspection report generation method.

[0099] In another aspect, the application also provides a computer readable storage medium, which stores a computer program, and the computer program realizes the multi-modal large model offshore oil production ship shape safety inspection report generation method.

[0100] Although the preferred embodiments of the present application have been described, those skilled in the art can make further changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including all changes and modifications falling within the scope of the present application.

[0101] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A method for generating a safety inspection report on the external shape of an offshore oil production vessel based on a multimodal large model, characterized in that, Includes the following steps: S1. Collect images of offshore oil production vessels, match them with corresponding safety inspection reports, and construct a dataset; S2. Construct a multimodal report generation model; the multimodal report generation model includes a ConvNeXt V2 network, a cue pool module, a large language module segmenter, a multimodal fusion module, and a large language model, wherein the large language model is LLaMA2; input the image of the offshore oil production vessel into the ConvNeXt V2 network to obtain the feature sequence of the offshore oil production vessel. Subsequently, the characteristic sequence of the offshore oil production vessel was... The input is fed into the prompt pool module to obtain the prompt sequence. ; The safety inspection report corresponding to the image of the offshore oil production vessel is then input into the word segmenter of the large language module to obtain the semantic sequence of the report. ; Subsequently, the characteristic sequence of the offshore oil production vessel was... Hint sequence and report semantic sequence The input is fused into a multimodal sequence by the multimodal fusion module. Finally, the multimodal sequence Input the data into LLaMA2 to generate a safety inspection report for the external shape of an offshore oil production vessel; S3, Training the model; The images of the offshore oil production vessel are input into the multimodal report generation model, the sequence cross-entropy loss is calculated, backpropagation is performed, and the final multimodal report generation model is obtained after multiple rounds of training. S4. Input the images of the offshore oil production vessel into the final multimodal report generation model and output the offshore oil production vessel shape safety inspection report.

2. The method for generating a safety inspection report for the shape of an offshore oil production vessel based on a multimodal large model according to claim 1, characterized in that, The specific implementation steps of S1 are as follows: Sa1. Collect images of offshore oil production vessels and their corresponding safety inspection reports to compile a raw sample library; Sa2. Conduct quality screening on the security inspection reports in the original sample library. If the security inspection reports are complete and formatted correctly, they are included in the standardized sample library. If the security inspection reports are incomplete or formatted incorrectly, they are standardized and then included in the standardized sample library. Sa3. The dataset is constructed by combining images of marine oil production vessels and their corresponding safety inspection reports from the standardized sample library.

3. The method for generating a safety inspection report for the shape of an offshore oil production vessel based on a multimodal large model according to claim 1, characterized in that, In step S2, the characteristic sequence of the offshore oil production vessel is obtained. The specific steps are as follows: Sb1, In the first stage Stem layer, a convolution kernel size of [size missing] is used. A convolution operation with a stride of 4 is used to downsample the image of an offshore oil production vessel to obtain primary features. Sb2, the second stage Stage 1 uses a convolutional kernel size of... A convolutional kernel with a stride of 2 further downsamples the primary features and stacks multiple ConvNeXt V2 blocks to obtain local structural features; Sb3, Stage 2 of the third phase adopts The convolutional kernels are downsampled, and then ConvNeXt V2 blocks are stacked to extract mid-level semantic features and model larger receptive field information; Sb4 and Stage 3, after downsampling in Stage 3, extract global high-level features through residualized ConvNeXt V2 blocks, ultimately obtaining the feature sequence of the offshore oil production vessel. .

4. The method for generating a safety inspection report for the shape of an offshore oil production vessel based on a multimodal large model according to claim 1, characterized in that, In step S2, the prompt sequence is obtained. The specific steps are as follows: Sc1, Characteristic Sequence of Offshore Oil Production Vessels Average pooling is performed to obtain the global characteristics of offshore oil production vessels. The characteristic sequence of the offshore oil production vessel Expressed as: ; In the formula, Represented as a matrix composed of real numbers; The number of samples is represented by images of an offshore oil production vessel. Represented as a characteristic sequence of offshore oil production vessels The number of feature vectors contained in each sample. Represented as a characteristic sequence of offshore oil production vessels The feature dimension of a single feature vector; Global characteristics of offshore oil production vessels Expressed as: ; Sc2, The prompt pool module includes a prompt pool. , expressed as: ； In the formula, Represented as a hint pool Size, This represents how many cue vectors make up a single cue; Represented as a vector dimension and feature sequence of an offshore oil production vessel The feature dimensions are equal; That is, each cue vector has a dimension of D, which is consistent with the dimension of the feature vectors in the feature sequence of an offshore oil production vessel, facilitating similarity calculation and fusion with global image features in the cue pool module; at the same time, the key vector matrix is ​​initialized. , of which OK Representation and Hint Pool The Middle A hint The key vector corresponds to; Sc3, Overall characteristics of offshore oil production vessels and initializing the key vector matrix conduct Normalization yields the normalized global characteristics of offshore oil production vessels. and normalized bond vector matrix ; Sc4. Global characteristics of normalized offshore oil production vessels and normalized bond vector matrix Perform cosine similarity calculation to obtain the similarity matrix. The ; Sc5, Similarity Matrix Select each row independently The most similar prompts form a candidate set, and then the prompt pool is used to select the prompts from the candidate set. Extract the corresponding subset of hints to obtain , expressed as: ; In the formula, Indicated from the suggestion pool The subset of hints selected based on similarity; This is a prompt, indicating that it is from the prompt pool. The number of prompts selected from the list; Sc6, Finally Each sample A hint Flattening and stitching the data in different dimensions yields the aforementioned feature sequence of the offshore oil production vessel. Associated cue sequences , expressed as: 。 5. The method for generating a safety inspection report for the shape of an offshore oil production vessel based on a multimodal large model according to claim 1, characterized in that, In S2, the report semantic sequence Expressed as: 。 6. The method for generating a safety inspection report for the shape of an offshore oil production vessel based on a multimodal large model according to claim 1, characterized in that, In S2, the sequences are fused into a multimodal sequence. First, we need to define the characteristic sequence of offshore oil production vessels. Hint sequence Both are unified to the same dimension as the report's semantic sequence through linear projection; then, while keeping the first dimension unchanged, the feature sequence of the offshore oil production vessel is... Hint sequence and report semantic sequence The multimodal sequence is obtained by concatenating the sequences in the second dimension. , expressed as: 。 7. The method for generating a safety inspection report for the shape of an offshore oil production vessel based on a multimodal large model according to claim 1, characterized in that, Multimodal sequences The specific steps for inputting into LLaMA2 are as follows: The multimodal sequence First, the multimodal sequence undergoes embedding mapping and positional encoding, followed by... Semantic modeling is performed by sequentially inputting multi-layer Transformer blocks into LLaMA2; during the decoding phase, LLaMA2 generates output tokens step by step in an autoregressive manner to obtain the aforementioned offshore oil production vessel shape safety inspection report.

8. A system for generating a safety inspection report on the shape of a multimodal large-scale offshore oil production vessel, comprising a memory, a processor, and program instructions stored in the memory for execution by the processor, characterized in that, The processor performs the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that implements the method of any one of claims 1 to 6.

Images