Multi-modal large-model ocean oil production ship appearance safety inspection report generation method

Abstract

The invention discloses a multi-modal large-model ocean oil production ship appearance safety inspection report generation method, and relates to the technical field of intelligent generation of ship safety inspection, and the method comprises the steps: firstly collecting an ocean oil production ship image, matching a corresponding safety inspection report, and constructing a data set; then, constructing a multi-modal report generation model, and inputting the ocean oil production ship image and the corresponding safety inspection report into the model to generate an ocean oil production ship appearance safety inspection report; then training the model, and training for multiple rounds to obtain a final multi-modal report generation model; and finally, inputting an ocean oil production ship image into the final model, and outputting an ocean oil production ship appearance safety inspection report. According to the method, the controllability and pertinence of the generated content of the large language model are enhanced, the accuracy and compliance of the appearance safety inspection report are improved, and the method can be widely applied to daily operation management of ocean oil production ships, ocean navigation safety guarantee, compliance inspection of ports and classification societies, accident prevention, follow-up maintenance and disposal and other scenes.

Description

Technical Field

[0001] This invention relates to the field of intelligent generation technology for ship safety inspections, and in particular to a method for generating a safety inspection report on the shape of a multimodal large-scale offshore oil production vessel. Background Technology

[0002] As core equipment for deep-sea oil and gas development, offshore oil production vessels must undergo systematic safety inspections during navigation, berthing, and maintenance. By inspecting key aspects such as the hull's shape and structure, power system, deck machinery, and oil and gas processing equipment, potential defects or safety hazards can be identified in a timely manner, and the results can be compiled into structured and standardized documents. These safety inspection reports are not only crucial for ensuring the safe operation of offshore oil production vessels but also key references for classification society inspections, port supervision, and insurance claims, playing an irreplaceable role in reducing operational risks and maintaining environmental safety.

[0003] However, current safety inspection reports for offshore oil production vessels largely rely on manual labor. Due to the complexity of the inspection targets, the large workload, and the influence of varying experience and subjective judgment among personnel, manual inspections often suffer from low efficiency, high omission rates, and incomplete records, making it difficult to guarantee the accuracy, consistency, and compliance with industry standards. Existing automated methods lack specificity, exhibiting problems such as insufficient alignment between image and text semantics, weak controllability of generated results, and inconsistencies with industry standards. Therefore, there is an urgent need for a new technological approach that can improve efficiency while reducing human subjectivity, ensuring the professionalism, standardization, and traceability of the output reports. Summary of the Invention

[0004] In view of this, the present invention provides a method for generating a safety inspection report on the shape of an offshore oil production vessel using a multimodal large model, in order to solve the problems of low manual efficiency, strong subjectivity of inspection results, and insufficient standardization in the prior art.

[0005] The method for generating a safety inspection report for the external shape of an offshore oil production vessel based on a multimodal large model provided by this invention includes the following steps: S1. Collect images of offshore oil production vessels, match them with corresponding safety inspection reports, and construct a dataset; S2. Construct a multimodal report generation model; the multimodal report generation model includes a ConvNeXt V2 network, a cue pool module, a large language module segmenter, a multimodal fusion module, and a large language model, wherein the large language model is LLaMA2; input the image of the offshore oil production vessel into the ConvNeXt V2 network to obtain the feature sequence of the offshore oil production vessel. Subsequently, the characteristic sequence of the offshore oil production vessel was... The input is fed into the prompt pool module to obtain the prompt sequence. Subsequently, the safety inspection report corresponding to the image of the offshore oil production vessel is input into the large language module word segmenter to obtain the semantic sequence of the report. Subsequently, the characteristic sequence of the offshore oil production vessel was... Hint sequence and report semantic sequence The input is fused into a multimodal sequence by the multimodal fusion module. Finally, the multimodal sequence Input the data into LLaMA2 to generate a safety inspection report for the external shape of an offshore oil production vessel; S3. Training the model: Input the images of the offshore oil production vessel into the multimodal report generation model, calculate the sequence cross-entropy loss, perform backpropagation, and train for multiple rounds to obtain the final multimodal report generation model; S4. Input the images of the offshore oil production vessel into the final multimodal report generation model and output the offshore oil production vessel shape safety inspection report.

[0006] Furthermore, the specific implementation steps of S1 are as follows: Sa1. Collect images of offshore oil production vessels and their corresponding safety inspection reports to compile a raw sample library; Sa2. Conduct quality screening on the security inspection reports in the original sample library. If the security inspection reports are complete and in a standardized format, they are included in the standardized sample library. If the security inspection reports are incomplete or in a non-standard format, they are standardized and then included in the standardized sample library. Sa3. Construct the dataset by combining images of offshore oil production vessels and their corresponding safety inspection reports from the standardized sample library.

[0007] Further, in step S2, the characteristic sequence of the offshore oil production vessel is obtained. The specific steps are as follows: Sb1, In the first stage Stem layer, a convolution kernel size of [size missing] is used. A convolution operation with a stride of 4 is used to downsample the image of an offshore oil production vessel to obtain primary features. Sb2, the second stage Stage 1 uses a convolutional kernel size of... A convolutional kernel with a stride of 2 further downsamples the primary features and stacks multiple ConvNeXt V2 blocks to obtain local structural features; Sb3, Stage 2 of the third phase adopts The convolutions are downsampled, and then ConvNeXt V2 blocks are stacked to extract mid-level semantic features and model larger receptive field information; Sb4 and Stage 3, after downsampling in Stage 3, extract global high-level features through residualized ConvNeXt V2 blocks, ultimately obtaining the feature sequence of the offshore oil production vessel. .

[0008] Further, in step S2, the prompt sequence is obtained. The specific steps are as follows: Sc1, Characteristic Sequence of Offshore Oil Production Vessels Average pooling is performed to obtain the global characteristics of offshore oil production vessels. The characteristic sequence of the offshore oil production vessel The formula is expressed as:

[0009] In the formula, Represented as a matrix composed of real numbers; The number of samples is represented by images of an offshore oil production vessel. Represented as a characteristic sequence of offshore oil production vessels The number of feature vectors contained in each sample. Represented as a characteristic sequence of offshore oil production vessels The feature dimension of a single feature vector; Global characteristics of offshore oil production vessels The formula is expressed as: ; Sc2, The prompt pool module includes a prompt pool. The formula is expressed as:

[0010] In the formula, Represented as a hint pool Size, This represents how many cue vectors make up a single cue; Represented as a vector dimension and feature sequence of an offshore oil production vessel The feature dimensions are equal; that is, the dimension of each cue vector is D, which is consistent with the feature vector dimension of the feature sequence of an offshore oil production vessel, facilitating similarity calculation and fusion with global image features in the cue pool module; at the same time, the key vector matrix is ​​initialized. , of which OK Representation and Hint Pool The Middle A tip The key vector corresponds to; Sc3, Overall characteristics of offshore oil production vessels and initializing the key vector matrix conduct Normalization yields the normalized global characteristics of offshore oil production vessels. and normalized bond vector matrix ; Sc4. Global characteristics of normalized offshore oil production vessels and normalized bond vector matrix Perform cosine similarity calculation to obtain the similarity matrix. The ; Sc5, Similarity Matrix Select each row independently The most similar prompts form a candidate set, and then prompts are selected from the prompt pool based on the candidate set. Extract the corresponding subset of hints to obtain The formula is expressed as:

[0011] In the formula, Indicated from the suggestion pool The subset of hints selected based on similarity; This is a prompt, indicating that it is from the prompt pool. The number of prompts selected from the list; Sc6, Finally Each sample A hint Flattening and stitching the data in different dimensions yields the aforementioned feature sequence of the offshore oil production vessel. Associated cue sequences The formula is expressed as: .

[0012] Further, in step S2, the report semantic sequence The formula is expressed as: .

[0013] Furthermore, in step S2, after fusing into a multimodal sequence... First, we need to define the characteristic sequence of offshore oil production vessels. Hint sequence Both are unified to the same dimension as the report's semantic sequence through linear projection; then, while keeping the first dimension unchanged, the feature sequence of the offshore oil production vessel is... Hint sequence and report semantic sequence The multimodal sequence is obtained by concatenating the sequences in the second dimension. Multimodal sequences In fact, it is by Hint sequence and the characteristic sequence of offshore oil production vessels The matrix addition along the second dimension is expressed by the following formula: .

[0014] Furthermore, multimodal sequences The specific steps for inputting into the large language model LLaMA2 are as follows: The multimodal sequence First, the sequence undergoes embedding mapping and positional encoding, followed by multimodal sequence... Semantic modeling is performed by sequentially inputting multi-layer Transformer blocks into the large language model LLaMA2 (using multi-layer Transformer blocks to perform semantic processing on the input multimodal sequences); in the decoding stage, the large language model LLaMA2 generates output tokens step by step in an autoregressive manner to obtain the aforementioned marine oil production vessel shape safety inspection report.

[0015] On the other hand, the present invention also provides a system for generating a safety inspection report on the shape of a multimodal large-scale offshore oil production vessel, including a memory, a processor, and program instructions stored in the memory for the processor to run. The processor executes the program instructions to implement the steps in the above-described method for generating a safety inspection report on the shape of a multimodal large-scale offshore oil production vessel.

[0016] On the other hand, the present invention also provides a computer-readable storage medium storing a computer program that implements the above-described method for generating a safety inspection report on the shape of a multimodal large model of an offshore oil production vessel.

[0017] The beneficial effects of this invention are as follows: By introducing a prompt-driven mechanism, this invention enhances the controllability and relevance of the content generated by the large language model, improves the accuracy and compliance of the external safety inspection report, and can be widely applied to scenarios such as daily operation and management of offshore oil production vessels, safety assurance of ocean navigation, compliance review of ports and classification societies, as well as accident prevention and subsequent maintenance and disposal. Attached Figure Description

[0018] Figure 1 This is a flowchart of the present invention; Figure 2 Flowchart for constructing the dataset for this invention; Figure 3 This is a structural diagram of the multimodal report generation model of the present invention; Figure 4 This is a structural diagram of the ConvNeXt V2 block of the present invention; Figure 5 This is a structural diagram of the prompt pool module of the present invention; Figure 6This is a flowchart of the model training process of the present invention. Detailed Implementation

[0019] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. However, these descriptions do not constitute a limitation of the present invention and are merely illustrative. Through these descriptions, the advantages of the present invention will become clearer. All modifications that can be directly derived or conceived by those skilled in the art from the disclosure of the present invention should be considered within the scope of protection of the present invention. Other parts not described in detail in the embodiments are prior art.

[0020] First, the technical terms used in this invention will be explained and described.

[0021] Multimodal Large Model: The multimodal large model is the core architecture used in this invention to generate safety inspection reports for offshore oil production vessels. Its purpose is to simultaneously receive and process heterogeneous data such as images and text, achieving information fusion within a unified framework, thereby improving the understanding and generation capabilities for complex scenarios. Its basic principle is as follows: First, a dedicated encoder is used to extract features from different modal data; then, through feature alignment and fusion mechanisms, the features of each modality are mapped to a shared semantic representation space; finally, a unified large language model is used as a decoder to achieve cross-modal reasoning and natural language report generation. In this invention, the multimodal large model encompasses all modules, including: a visual feature extraction model, a cue pool module, the large language model LLaMA2, and a word segmenter.

[0022] Average pooling: Average pooling is introduced into cue pooling to extract global features from images of offshore oil production vessels. Average pooling compresses feature representations by averaging local regions in the spatial dimension, thereby gradually reducing the resolution of the feature map. This reduces parameter size and improves computational efficiency while also mitigating overfitting to some extent.

[0023] Normalization: This is a common vector standardization method. Its core idea is to scale the input vector according to its Euclidean length, so that the normalized vector has a fixed length of 1. In the processing of global features and cue key vectors for offshore oil production vessels, this method is introduced... The purpose of normalization is to eliminate the differences in numerical scale between different vectors, thereby avoiding similarity calculation errors caused by different vector magnitudes.

[0024] Tokenizer: A tokenizer is a preprocessing module for text input, its function being to transform the original natural language sentence into a discrete sequence of symbols that the model can recognize. In the safety inspection report generation method of this invention, the purpose of introducing a tokenizer is to segment the safety report text corresponding to the image of an offshore oil tanker into basic word or sub-word units, thereby obtaining a semantic sequence that can be further encoded by the model. The principle of the tokenizer is to map the text into a series of integer identifiers according to a predefined vocabulary, with each identifier corresponding to a word or sub-word, while ensuring that the model can cover a wide range of language expressions within a finite vocabulary.

[0025] Example 1: This invention proposes a method for generating a safety inspection report on the external shape of an offshore oil production vessel using a multimodal large model, such as... Figure 1 As shown, it includes the following steps: S1. Collect images of offshore oil production vessels, match them with corresponding safety inspection reports, and construct a dataset; The specific implementation steps of S1 are as follows: Sa1. Collect images of offshore oil production vessels and their corresponding safety inspection reports to compile a raw sample library; Sa2. Conduct quality screening on the security inspection reports in the original sample library. If the security inspection reports are complete and in a standardized format, they will be included in the standardized sample library. If the security inspection reports are incomplete or in a non-standard format, they will be standardized using DeepSeek and manually reviewed, modified, and confirmed by professionals before being included in the standardized sample library. Sa3. Construct the dataset by combining images of offshore oil production vessels and their corresponding safety inspection reports from the standardized sample library. For example... Figure 2 As shown.

[0026] S2. Construct a multimodal report generation model. The structure diagram of the multimodal report generation model is shown below. Figure 3 As shown; the multimodal report generation model includes a ConvNeXt V2 network, a cue pool module, a large language module segmenter, a multimodal fusion module, and a large language model LLaMA2; the marine oil production vessel image is input into the ConvNeXt V2 network to obtain the marine oil production vessel feature sequence. Subsequently, the characteristic sequence of the offshore oil production vessel was... The input is entered into the prompt pool module to obtain the characteristic sequence of the offshore oil production vessel. Associated cue sequences ; In step S2, the ConvNeXt V2 network is processed in four stages, and the processing steps are as follows: Sb1, In the first stage Stem layer, a convolution kernel size of [size missing] is used. A convolution operation with a stride of 4 is used to downsample the image of the offshore oil production vessel to obtain primary features; the first stage Stem layer consists of initial convolution and downsampling operations, and is mainly responsible for converting the input image into primary features; Sb2, the second stage (Stage 1) uses a convolution kernel size of The first stage (Stage 1) uses a convolutional kernel with a stride of 2 to further downsample the primary features and stacks multiple ConvNeXt V2 blocks to obtain local structural features. The second stage (Stage 1) consists of a downsampled layer and several stacked ConvNeXt V2 blocks. Each block contains depthwise separable convolution, pointwise convolution, non-linear activation function and layer normalization to extract local structural features. Sb3, Stage 2 adopts The convolutional layers are downsampled, and then ConvNeXtV2 blocks are stacked to extract mid-level semantic features and model larger receptive field information; the third stage (Stage 2) also consists of downsampling layers and multiple ConvNeXt V2 blocks, emphasizing the extraction and modeling of mid-level semantic features at a lower resolution. Sb4, Stage 3: After downsampling in Stage 3 (Stage 2), global high-level features are extracted through residualized ConvNeXtV2 blocks, ultimately yielding the feature sequence of the offshore oil production vessel. The fourth stage, Stage 3, consists of a downsampling layer and a residual ConvNeXt V2 block, which is responsible for extracting global high-level features and forming the final feature sequence of the offshore oil production vessel.

[0027] ConvNeXt V2 block structure as follows Figure 4 As shown, in the ConvNeXt V2 block, the input features are first normalized through an LN layer. This normalization process stabilizes training and mitigates the vanishing or exploding gradient problem. The input then undergoes convolution to extract local spatial features and establish a preliminary feature representation. Next, the GELU (Gaussian Error Linear Unit) activation function is used, which smoothly suppresses noise with small inputs and maintains near-linearity with large inputs, thus improving the continuity and stability of gradient propagation. Afterward, GRN (Global Response Normalization) normalizes and reweights the entire feature map, resulting in a more balanced response across different channels and spatial locations, reducing the over-dominance of certain features. Finally, a linear projection and residual connection layer fuses the processed features with the input features, achieving a balance between information preservation and deep feature enhancement.

[0028] In step S2, the prompt sequence is obtained. The specific steps are as follows: Sc1, Characteristic Sequence of Offshore Oil Production Vessels Average pooling is performed to obtain the global characteristics of offshore oil production vessels. The characteristic sequence of the offshore oil production vessel The formula is expressed as:

[0029] In the formula, Represented as a matrix composed of real numbers; B represents the number of samples, where the samples are images of offshore oil production vessels. Each image of an offshore oil production vessel input into the visual feature extraction model is considered as one sample. If eight images are input at once, then B=8; if only one image is input at once, then B=1. Represented as a characteristic sequence of offshore oil production vessels The number of feature vectors contained in each sample. Represented as a characteristic sequence of offshore oil production vessels The feature dimension of a single feature vector; Global characteristics of offshore oil production vessels The formula is expressed as: ; Sc2, The prompt pool module includes a prompt pool. The formula is expressed as:

[0030] In the formula, Represented as a hint pool Size, This represents how many cue vectors make up a single cue; Represented as a vector dimension and feature sequence of an offshore oil production vessel The feature dimensions are equal; that is, the dimension of each cue vector is D, which is consistent with the feature vector dimension of the feature sequence of an offshore oil production vessel, facilitating similarity calculation and fusion with global image features in the cue pool module; at the same time, the key vector matrix is ​​initialized. , of which OK Representation and Hint Pool The Middle A tip The key vector corresponds to the key vector; initialize the key vector matrix. The cue pool module is a set of key vectors set for each cue, used to calculate similarity with global image features, thereby selecting the most relevant cue; the cue pool module structure is as follows: Figure 5 As shown; Sc3, Overall characteristics of offshore oil production vessels and initializing the key vector matrix conduct Normalization yields the normalized global characteristics of offshore oil production vessels. and normalized bond vector matrix ; Sc4. Global characteristics of normalized offshore oil production vessels and normalized bond vector matrix Perform cosine similarity calculation to obtain the similarity matrix. The ; Sc5, Similarity Matrix Select each row independently The most similar prompts form a candidate set, and then prompts are selected from the prompt pool based on the candidate set. Extract the corresponding subset of hints to obtain The formula is expressed as:

[0031] In the formula, Indicated from the suggestion pool The subset of hints selected based on similarity; This is a prompt, indicating that it is from the prompt pool. The number of prompts selected from the list; Sc6, Finally Each sample A hint Flattening and stitching the data in different dimensions yields the aforementioned feature sequence of the offshore oil production vessel. Associated cue sequences The formula is expressed as: .

[0032] S3. Input the safety inspection report corresponding to the image of the offshore oil production vessel into the word segmenter of the large language module to obtain the semantic sequence of the report. Inputting the safety report corresponding to the image of the offshore oil production vessel into the word segmenter of the large language model LLaMA2 converts the natural language text into a string of token IDs, thereby forming a semantic sequence of the report that can be processed by the model. The report semantic sequence The formula is expressed as: .

[0033] S4. The feature sequence of the offshore oil production vessel Hint sequence and report semantic sequence The input is fused into a multimodal sequence by the multimodal fusion module. The multimodal sequence is then... The data is input into the large language model LLaMA2 to generate a safety inspection report on the external shape of an offshore oil production vessel. In fusion into multimodal sequences First, we need to define the characteristic sequence of offshore oil production vessels. Hint sequence Both are unified to the same dimension as the report's semantic sequence through linear projection; then, while keeping the first dimension unchanged, the feature sequence of the offshore oil production vessel is... Hint sequence and report semantic sequence The multimodal sequence is obtained by concatenating the sequences in the second dimension. Multimodal sequences In fact, it is by Hint sequence and the characteristic sequence of offshore oil production vessels The matrix addition along the second dimension is expressed by the following formula: .

[0034] Multimodal sequences The specific steps for inputting into the large language model LLaMA2 are as follows: The multimodal sequence First, the sequence undergoes embedding mapping and positional encoding, followed by multimodal sequence... Semantic modeling is performed by sequentially inputting multi-layer Transformer blocks into the large language model LLaMA2 (using multi-layer Transformer blocks to perform semantic processing on the input multimodal sequences); in the decoding stage, the large language model LLaMA2 generates output tokens step by step in an autoregressive manner to obtain the aforementioned marine oil production vessel shape safety inspection report.

[0035] S5. Training the model: Input the images of the offshore oil production vessel into the multimodal report generation model, perform forward propagation, calculate the sequence cross-entropy loss, perform backpropagation, and train for multiple rounds to obtain the final multimodal report generation model; In step S5, the specific implementation steps for training the model are as follows: Figure 6 As shown, the ConvNeXt V2 network and the large language model LLaMA2 are first initialized with their respective pre-trained parameters. Then, forward and backward propagation are performed on each sample pair of marine oil production vessel images and their corresponding safety inspection reports in the dataset. During the forward propagation, the marine oil production vessel images are first processed by the ConvNeXt V2 network to extract the marine oil production vessel feature sequences. Subsequently, the global characteristics of offshore oil production vessels were obtained through average pooling. , and the hint pool Calculate the cosine similarity of the key vectors in each prompt, and select the one with the highest similarity. A prompt, followed by the aforementioned Each prompt is flattened into a prompt sequence. Simultaneously, the safety inspection report corresponding to the image of the offshore oil production vessel is input into the large language module word segmenter to obtain the semantic sequence of the report. Subsequently, the characteristic sequence of offshore oil production vessels was determined. and prompt sequence Mapped to the semantic sequence of the report via linear projection Same dimensions; feature sequence of offshore oil production vessels Hint sequence and report semantic sequence Fusion into multimodal sequences The input is fed into the large language model LLaMA2, which predicts the next token of the report in an autoregressive manner. The objective function of the training adopts the standard sequence cross-entropy loss. The sequence cross-entropy loss is backpropagated on the entire "ConvNeXt V2 network + cue pool parameters + linear projection layer". These parameters are continuously updated through gradient descent to minimize the difference between the report generated by the model and the real security check report at the token level.

[0036] S6. Input the images of the offshore oil production vessel into the final multimodal report generation model and output the offshore oil production vessel shape safety inspection report.

[0037] The present invention also provides a system for generating a safety inspection report on the shape of a multimodal large-scale offshore oil production vessel, including a memory, a processor, and program instructions stored in the memory for the processor to run. The processor executes the program instructions to implement the steps in the above-described method for generating a safety inspection report on the shape of a multimodal large-scale offshore oil production vessel.

[0038] On the other hand, the present invention also provides a computer-readable storage medium storing a computer program that implements the above-described method for generating a safety inspection report on the shape of a multimodal large model of an offshore oil production vessel.

[0039] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0040] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for generating a safety inspection report for the hull of a multi-modal large model offshore oil production vessel, the method comprising: The method comprises the following steps: S1, collecting images of offshore oil production vessels, matching corresponding safety inspection reports, and constructing a data set; S2, a multi-modal report generation model is constructed; the multi-modal report generation model comprises a ConvNeXt V2 network, a prompt pool module, a large language module tokenizer, a multi-modal fusion module and a large language model, the large language model is LLaMA2; an offshore oil production ship image is input into the ConvNeXt V2 network to obtain an offshore oil production ship feature sequence , and then the offshore oil production ship feature sequence is input into the prompt pool module to obtain a prompt sequence ; Subsequently, the safety inspection report corresponding to the offshore oil production ship image is input into a large language module word segmenter to obtain a report semantic sequence ; Subsequently, the characteristic sequence of the offshore oil production vessel was... Hint sequence and report semantic sequence The input is fused into a multimodal sequence by the multimodal fusion module. Finally, the multimodal sequence Input the data into LLaMA2 to generate a safety inspection report for the external shape of an offshore oil production vessel; S3, training the model; The image of the offshore oil production vessel is input into the multi-modal report generation model, the sequence cross-entropy loss is calculated, the back propagation is performed, and the final multi-modal report generation model is obtained after multiple rounds of training; S4, inputting the image of the offshore oil production vessel into the final multi-modal report generation model, and outputting the offshore oil production vessel shape safety inspection report.

2. The method of claim 1, wherein the method is a method of generating a report for a safety inspection of an FPSO hull shape using a multi-modal large model, the method comprising: The specific implementation steps of S1 are: Sa1, collecting images of offshore oil production vessels and their corresponding safety inspection reports into an original sample library; Sa2, performing quality screening on the safety inspection reports in the original sample library, if the safety inspection report content is complete and the format is standard, it is included in the standardized sample library, if the safety inspection report content is incomplete or the format is not standard, it is standardized and then included in the standardized sample library; Sa3, constructing the images of offshore oil production vessels and their corresponding safety inspection reports in the standardized sample library into the data set.

3. The method for generating a safety inspection report for the shape of an offshore oil production vessel based on a multimodal large model according to claim 1, characterized in that, In step S2, the characteristic sequence of the offshore oil production vessel is obtained The specific steps are as follows. Sb1, in the first stage Stem layer, a convolution operation with a convolution kernel size of , and a stride of 4 is adopted to down-sample the offshore oil drilling ship image to obtain a primary feature; Sb2, the second stage Stage1 uses a convolution kernel with a size of , further down-samples the primary features using a convolution kernel with a stride of 2, and stacks multiple ConvNeXt V2 blocks to obtain local structural features; Sb3, the third stage Stage2 adopts down-sampling, and then continues to stack ConvNeXt V2 blocks to extract mid-level semantic features and model larger receptive field information; Sb4, the fourth stage Stage3 extracts global high-level features through a residualized ConvNeXt V2 block after downsampling in the third stage Stage2, and finally obtains the feature sequence of the offshore oil production ship .

4. The method of claim 1, wherein the method further comprises: In the step S2, the prompt sequence is obtained The specific steps are as follows. Sc1, a sequence of features of the offshore oil production vessel performing average pooling to obtain global features of the offshore oil production vessel , the sequence of features of the offshore oil production vessel is expressed by the formula: ； wherein is represented as a matrix consisting of real numbers; is represented as the number of samples, the samples being images of offshore oil rigs; is represented as a sequence of features of offshore oil rigs is represented as the number of feature vectors contained in each sample, is represented as a sequence of features of offshore oil rigs is represented as the feature dimension of a single feature vector in The global features of the offshore oil production vessel The formula is expressed as: ； Sc2, the prompt pool module comprises a prompt pool The formula is expressed as: ; In the formula, Represented as a hint pool Size, This represents how many cue vectors make up a single cue; Represented as a vector dimension and feature sequence of an offshore oil production vessel The feature dimensions are equal; That is, the dimension of each prompt vector is D, which is consistent with the dimension of the feature vector of the offshore oil production ship feature sequence, facilitating similarity calculation and fusion with the image global feature in the prompt pool module; meanwhile, the key vector matrix is initialized , wherein the th row represents the correspondence with the key vector of the th prompt in the prompt pool ; Sc3, overall features of the offshore oil production vessel and initializing the key vector matrix performing normalizing, respectively, to obtain normalized overall features of the offshore oil production vessel and the normalized key vector matrix ; Sc4, normalizing global features of an offshore oil production vessel and normalized key vector matrix performing cosine similarity computation to obtain a similarity matrix , the ; Sc5, selecting the top similarities from each row of the similarity matrix independently ​​​ ； In the formula, is a prompt subset selected from the prompt pool according to the similarity; is a prompt, and is the number of prompts selected from the prompt pool according to the similarity. Sc6, finally each sample of each prompt is flattened and concatenated across dimensions to obtain the offshore oil production vessel feature sequence associated prompt sequence , which is expressed by the formula: 。 5. The method for generating a safety inspection report for the shape of an offshore oil production vessel based on a multimodal large model according to claim 1, characterized in that, In the step S2, the report semantic sequence The formula is expressed as: 。 6. The method of claim 1, wherein the method further comprises: In step S2, the sequences are fused into a multimodal sequence. First, the characteristic sequence of offshore oil production vessels. Hint sequence Both are unified to the same dimension as the report's semantic sequence through linear projection; then, while keeping the first dimension unchanged, the feature sequence of the offshore oil production vessel is... Hint sequence and report semantic sequence The multimodal sequence is obtained by concatenating the sequences in the second dimension. The formula is expressed as: 。 7. The method for generating a safety inspection report for the shape of an offshore oil production vessel based on a multimodal large model according to claim 1, characterized in that, The multimodal sequence The specific steps for input to LLaMA2 are: The multimodal sequence The multimodal sequence is first subjected to embedding mapping and position encoding, and then The multimodal sequence is sequentially input into the multi-layer Transformer block of LLaMA2 for semantic modeling; in the decoding stage, LLaMA2 generates output tokens step by step in a self-recurrent manner to obtain the offshore oil production ship shape safety inspection report.

8. A multi-modal large model FPSO hull safety inspection report generation system comprising a memory, a processor and program instructions stored in the memory for execution by the processor, characterized in that, The processor executes the steps in the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program implements the method of any one of claims 1 to 6.

Images