Progressive bimodal fusion ocean hydrographic design report generation method and system

Abstract

The present application belongs to the technical field of marine hydrological data processing, and provides a marine hydrological design report generation method and system based on progressive bimodal fusion, which extracts global visual feature vectors and local feature vectors of standard images, overall feature vectors of data tables and cell enhanced feature vectors, and establishes an association index of the image local feature vectors and the cell enhanced feature vectors; executes the constructed progressive decision fusion controller instruction, combines the decision vector output by the controller, extracts information from the image local feature vectors and the cell enhanced feature vectors and updates the fusion memory at the current time; meanwhile, a complete sentence is generated by combining the decision vector and the fusion memory at the current time, and the process is iteratively executed until the controller outputs a termination instruction or the change of the fusion memory is less than a threshold value; and all generated complete sentences are combined to obtain the final report paragraph. The marine engineering design report meets the precision.

Description

Technical Field

[0001] This invention belongs to the field of marine hydrological data processing technology, and particularly relates to a method and system for generating marine hydrological design reports using progressive dual-modal fusion. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Marine hydrological parameter design reports are crucial technical bases for the preliminary demonstration and detailed design of marine engineering projects, including marine engineering, port construction, offshore wind power development, channel dredging, and disaster prevention and mitigation. These reports aim to provide fundamental data support for structural safety, construction and operation, and disaster prevention planning by comprehensively analyzing the historical statistical characteristics, extreme value recurrence periods, joint probability distributions, and design benchmark values ​​of multiple factors such as wind, waves, tides, and currents. The report preparation must strictly adhere to national and industry standards, with a rigorous content system covering multiple structured modules such as data source explanations, analysis methods, calculation processes, charts and graphs, and conclusions and recommendations.

[0004] Currently, the preparation of such professional design reports mainly relies on manual operation by professional technicians throughout the entire process. This process typically involves multiple stages, including data processing, chart creation, and text writing, and these stages are often fragmented, involving a large amount of repetitive work. This results in a complete report taking several days or even weeks to prepare. Specifically, existing technical solutions have the following significant problems: Inefficient and error-prone: When manually processing large amounts of raw hydrological observation or reanalysis data, subjective or objective errors can easily be introduced during data transcription, formula application, unit conversion, and chart drawing, affecting the accuracy of the reported data.

[0005] Insufficient standardization: Due to the lack of unified automated auxiliary tools, reports written by different technical personnel often differ in logical organization, terminology, chart format and expression style, making it difficult to guarantee the professionalism and standardization requirements of the results.

[0006] The report is highly dependent on knowledge and experience: its logical structure, extraction of core parameters, and conclusion summarization heavily rely on the depth of technical personnel's understanding of hydrodynamics and their experience in report writing. This model results in a long training cycle for newcomers, and the tacit knowledge of experts is difficult to translate into reusable standardized processes.

[0007] Dynamic updates and maintenance are difficult: When basic hydrological data (such as observation data for new years) or national / industry design standards are revised, the entire report needs to be manually checked, recalculated, and the relevant textual descriptions modified one by one, resulting in extremely high maintenance and iteration costs.

[0008] In recent years, with the rapid development of natural language generation and multimodal understanding technologies, automated report generation has become possible and has been initially applied in standardized report generation in fields such as finance and meteorology. However, existing general text generation technologies are difficult to directly apply to the field of marine hydrological design reports, which are highly structured, logically rigorous, and require collaborative argumentation using multiple charts and graphs. The core challenge lies in how to systematically integrate multidimensional numerical tables and derived statistical charts, and based on this, generate argumentative paragraphs that conform to professional standards, are logically coherent, and have a strong connection between text and graphics. Current technical solutions generally lack the progressive analysis and writing ability of domain experts who "interpret charts and discuss based on tables," and cannot transform discrete data analysis results into conclusive texts with engineering guidance significance, thus failing to meet the stringent requirements of accuracy, rigor, and professionalism for marine engineering design reports. Summary of the Invention

[0009] To address at least one of the technical problems mentioned above, this invention provides a method and system for generating marine hydrological design reports using a progressive bimodal fusion approach. This method constructs an intelligent fusion engine with progressive decision-making capabilities, alternately examines data tables and related charts, gradually extracts key information, dynamically fuses bimodal content, and organizes it into logically rigorous and standardized paragraphs. Ultimately, it achieves automated and intelligent generation of standardized report paragraphs from structured data.

[0010] To achieve the above objectives, the present invention adopts the following technical solution: A first aspect of the present invention provides a method for generating marine hydrological design reports using progressive dual-modal fusion, comprising the following steps: The acquired marine hydrological data files are preprocessed to generate standard images and corresponding data tables; Extract the global and local visual feature vectors of standard images, the overall feature vectors and cell-enhanced feature vectors of data tables, and establish an association index between the local feature vectors and cell-enhanced feature vectors of images; The progressive decision fusion controller instructions are executed, and information is extracted from the local feature vector and cell enhancement feature vector of the image and updated to the fusion memory at the current moment by combining the decision vector and the fusion memory at the current moment. At the same time, a complete sentence is generated by combining the decision vector and the fusion memory at the current moment. This process is executed iteratively until the controller outputs a termination instruction or the change in the fusion memory is less than a threshold. Combine all the generated complete sentences to obtain the final report paragraph.

[0011] Furthermore, the extraction of the overall feature vector and enhanced feature vectors of the data table includes: Parse the table structure to obtain the text and numerical information of the header, row and column cells, and convert the content of each cell into a cell vector through a word embedding layer; A Transformer encoder is used as the table encoder. The cell vector and its row and column position encoding are input together. The semantic relationship between cells is modeled through a self-attention mechanism, and the overall feature vector of the table and the enhanced feature vector of each cell are output.

[0012] Furthermore, the progressive decision fusion controller is implemented using a recurrent neural network based on gated recurrent units, and its hidden state... As the core memory for decision-making, in each generation step The controller receives the current state vector. According to the current state vector Update its hidden state and output a decision vector. , where the current state vector Including semantic encoding of the generated text The fusion memory vector of the previous time step And the contextual summary features extracted from the bimodal features that are most relevant to the text content output up to the previous generation step. .

[0013] Furthermore, the decision vector output by the controller is parsed into modality selection probability distribution, information focusing weight, rhetorical action type, and gating signal through different fully connected layers.

[0014] Furthermore, combining the decision vector output by the controller, information is extracted from the local feature vector and cell-enhanced feature vector of the image and the fusion memory at the current moment is updated, including: Using the output information to focus weights, attention is calculated for the local feature set of the image and the enhanced feature set of the table cells respectively, to obtain the image content vector and the table content vector focused in the current step. Based on the modality selection probability, the two content vectors are weighted and fused to obtain the fused content vector; The fused content vector is input together with the fused memory from the previous time step into the fused memory updater to obtain the fused memory at the current time step.

[0015] Furthermore, the process of generating a complete sentence by combining the decision vector and the fused memory at the current moment includes: The current fused memory Rhetorical Action Types The embedding vector and the hidden states of the last few words of the generated text are concatenated to form a conditional context vector. ; Will As the initial hidden state or additional attention input, it drives the text generation model to predict words one by one in an autoregressive manner. The generation process is influenced by... Strong conditional constraints.

[0016] Furthermore, during the training of the progressive decision fusion controller and the text generation model, a teacher-mandated strategy is used to make the controller refer to real data to focus on trajectories and rhetorical actions, while the text generation model is trained by maximizing the likelihood probability of the target sentence.

[0017] A second aspect of the present invention provides a progressive dual-modal fusion marine hydrological design report generation system, comprising: The data preprocessing module is used to preprocess the acquired marine hydrological data files to generate standard images and corresponding data tables; the feature extraction module is used to extract the global visual feature vectors and local feature vectors of the standard images, the overall feature vectors and cell-enhanced feature vectors of the data tables, and to establish an association index between the image local feature vectors and cell-enhanced feature vectors. The progressive decision fusion process is used to execute the instructions of the constructed progressive decision fusion controller. It extracts information from the local feature vector and cell enhancement feature vector of the image and updates the fusion memory at the current moment by combining the decision vector and the fusion memory at the current moment. At the same time, it generates a complete sentence by combining the decision vector and the fusion memory at the current moment. The process is iteratively executed until the controller outputs a termination instruction or the change in the fusion memory is less than a threshold. The report paragraph generation module combines all the generated complete sentences to obtain the final report paragraph.

[0018] A third aspect of the present invention provides a computer-readable storage medium.

[0019] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the progressive bimodal fusion method for generating marine hydrological design reports as described above.

[0020] A fourth aspect of the present invention provides a computer device.

[0021] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the progressive bimodal fusion method for generating marine hydrological design reports as described above.

[0022] Compared with the prior art, the beneficial effects of the present invention are: This invention constructs an intelligent fusion engine with progressive decision-making capabilities: it alternately examines data tables and related charts, gradually extracts key information, dynamically merges bimodal content, and organizes it into logically rigorous and standardized paragraphs, ultimately achieving automated and intelligent generation of structured data into standardized report paragraphs.

[0023] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0025] Figure 1 This is a flowchart of the method for generating marine hydrological design reports using progressive dual-modal fusion, provided in an embodiment of the present invention. Detailed Implementation

[0026] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0027] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0028] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0029] Example 1 This embodiment provides a method for generating marine hydrological design reports using progressive dual-modal fusion, including the following steps: Step 1: Preprocess the acquired marine hydrological data files or access the database to generate standard images and corresponding data tables; In this embodiment, the marine hydrological data comes from two sources: numerical simulation reanalysis data and field measurement data.

[0030] First, for the numerical simulation data: download ERA5 atmospheric reanalysis data and HYCOM ocean reanalysis data, using these as driving fields or boundary conditions, run WRF (Weather Research and Forecasting Model), SWAN (Nearshore Wave Model), and ROMS (Regional Ocean Model System) respectively to simulate the hydrodynamic processes of the target sea area. After the simulation is completed, extract relevant physical quantities from the NetCDF format (Network Common Data Format) files output by each model, and generate structured wind field data files, wave data files, tide data files, and ocean current data files after format conversion.

[0031] Secondly, regarding the on-site measured data: collect full-tidal observation data of the target sea area, as well as annual observation data on wind field, wave, tide level, and ocean current, and organize them into a structured file format.

[0032] The specific elements contained in the above-mentioned data files are as follows: Wind field data: including timestamp, wind speed, and wind direction; Wave data includes timestamps, significant wave heights, average wave direction, average period, and peak period. Tide data: including timestamp and water level; Ocean current data includes timestamps, surface current velocity, surface current direction, 0.2H current velocity, and 0.2H current direction.

[0033] After data processing is completed, a data processing script is written to perform statistical analysis and calculations on the aforementioned wind field, wave, tide, and current files based on marine engineering hydrological calculation formulas, generating a series of standardized charts. These standardized charts include: wind rose diagrams and wind speed-direction joint distribution maps to characterize wind conditions; wave rose diagrams to characterize wave conditions; and wind speed-wave height joint distribution tables to characterize the wind-wave relationship, etc.

[0034] Step 2: Extract the global visual feature vector and local feature vector of the standard image, and the corresponding overall feature vector and cell-enhanced feature vector of the data table; establish an association index between the local feature vector and cell-enhanced feature vector of the chart; This step aims to transform unstructured charts and images, and structured tabular data, into unified numerical features that are understandable to machines. Specifically, it includes the following steps: Step 201: Extract the global visual feature vector and local feature vector of the standardized image; In this embodiment, the acquired standardized image data includes wave rose diagrams, significant wave height time series diagrams, etc. The feature extraction network used is a pre-trained deep convolutional neural network model, such as ResNet-50. The extracted global visual feature vector of the image is denoted as... ; Simultaneously, for specific image types, region interest networks are used to locate key visual elements (such as sectors in rose diagrams and peak points in distribution maps) and extract their local feature sets. .

[0035] Step 202: Extract the overall feature vector and cell-enhanced feature vector of the data table corresponding to the standardized image; The data tables corresponding to the standardized images are such as "Statistical Table of Wave Height and Period in Each Direction"; First, the table structure is parsed to obtain the text and numerical information of the header, row and column cells. The content of each cell is then converted into a cell vector through a word embedding layer. Subsequently, a Transformer encoder is used as the table encoder. The cell vector and its row and column position encoding are input together. The semantic relationships between cells are modeled through a self-attention mechanism, and the overall feature vector of the table is output. And the enhanced feature vector of each cell ; Step 203: Establish an association index between local image feature vectors and cell-enhanced feature vectors; Based on the homology mapping between standardized images and tabular data, local image features are established. Enhanced features with table cells These relationships are indexed together; for example, the sector features representing the "ENE direction" in the rose diagram are associated with the data cell features in the "ENE" row of the table. These associations serve as prior knowledge to guide attention computation in the subsequent fusion process.

[0036] Step 3: Construct a progressive decision fusion controller and initialize the generated state; In this embodiment, the controller is implemented using a recurrent neural network based on a gated recurrent unit (GRU), and its hidden state... As the core memory for decision-making, in each generation step The controller receives the current state vector. According to the current state vector Update its hidden state and output a decision vector. ; Wherein, the current state vector Including semantic encoding of the generated text The fusion memory vector of the previous time step And the contextual summary features extracted from the bimodal features that are most relevant to the text content output up to the previous generation step. ; Specifically, the semantic encoding of the generated text It is obtained by encoding the generated word sequence through a lightweight text encoder, such as the first few layers of BERT; Decision vector After being analyzed into mode selection probability distributions through different fully connected layers. Information focus weight, rhetorical action type Gating signals ; Among them, the mode selection probability distribution This indicates whether the current step should focus on the chart or the table.

[0037] Information focus weights are used to calculate the attention distribution for local feature sets of charts and feature sets of table cells.

[0038] Rhetorical Action Types This represents a category label that indicates the function of the sentence to be generated, such as "describe a trend", "compare data", "state a fact", "give a conclusion", etc.

[0039] Gating signal It represents a 0 / 1 signal or a continuous probability, which determines whether to trigger text generation in the current step.

[0040] Step 4: Execute the controller instructions, combine the output decision vector to extract information from the local feature vector of the chart and the cell augmented feature vector, and update the fusion memory at the current moment; Specifically, it includes: Focusing weights on the output information, we apply them to the local feature sets of the charts. Enhanced feature set with table cell Calculate attention to obtain the vector of the chart content that is focused in the current step. and table content vector ; Based on the modality selection probability, the two content vectors are weighted and fused to obtain the fused content vector, which is represented as: , The merged content vector Merging memories with the previous moment The fused memory is fed into the fusion memory updater (using another GRU unit) to update the fused memory at the current moment. ,Right now: ,this It encapsulates all the key information that has been merged up to the current step.

[0041] Step 5: When the gating signal is generated Upon activation, a standard text sentence is generated using the fused memory of the current moment; Conditional input construction: Constructing the current fused memory Rhetorical Action Types The embedding vector and the hidden states of the last few words of the generated text are concatenated to form a conditional context vector. .

[0042] Domain-specific text generation: As the initial hidden state or additional attention input, it drives a text generation model based on a Transformer decoder architecture (e.g., using the decoder part of BART).

[0043] The generative model predicts each word in an autoregressive manner to generate a complete sentence. The generation process is affected by Strong conditional constraints to ensure In terms of content and Maintain consistency and conform to the requirements of marine hydrological professional reports in terms of language style (such as using terms such as "significantly dominant", "concentrated distribution", "recurrence period").

[0044] Step 6: The system repeats steps 3 to 5, forming a closed-loop iterative process. This loop continues until the controller outputs a specific "termination" action, or the change in fused memory is less than a threshold, indicating that the main information has been expressed. Finally, all generated sentences are... Combine them in order to form the final report paragraphs.

[0045] As a further implementation, the system's learnable components (the decision controller GRU and the text generator) are trained using the following hybrid paradigm: Supervised pre-training: Utilizing a massive amount of labeled triplet data (charts, tables, standard report paragraphs). During training, a teacher-mandated strategy is employed, guiding the controller to refer to real data to focus on trajectories and rhetorical actions, while the text generator is trained by maximizing the likelihood probability of the target sentence. This stage primarily learns basic language generation and modal correspondence capabilities, implementing a standard strategy widely used in text sequence generation model training.

[0046] Reinforcement learning fine-tuning: Building upon a pre-trained model, reinforcement learning is introduced to optimize the overall quality of decision sequences. A reward function is defined. : , in, It generates paragraphs. This is a reference paragraph. Measure the consistency between the generated content and the input charts and tables. The penalty is repeated, and the parameters of the decision controller are updated using the Proximal Policy Optimization (PPO) algorithm to maximize the expected cumulative reward, thereby learning a better progressive decision strategy.

[0047] Step 7: Combine all the generated complete sentences to obtain the final report paragraphs. Fill the generated paragraphs into the corresponding positions in the predefined Word report template to generate a structurally complete first draft of the report.

[0048] Example 2 This embodiment provides a progressive dual-modal fusion marine hydrological design report generation system, including: The data preprocessing module is used to preprocess the acquired marine hydrological data files to generate standard images and corresponding data tables; The feature extraction module is used to extract global and local visual feature vectors of standard images, overall feature vectors and cell-enhanced feature vectors of data tables, and to establish an association index between local feature vectors and cell-enhanced feature vectors of images. The progressive decision fusion process is used to execute the instructions of the constructed progressive decision fusion controller. It extracts information from the local feature vector and cell enhancement feature vector of the image and updates the fusion memory at the current moment by combining the decision vector and the fusion memory at the current moment. At the same time, it generates a complete sentence by combining the decision vector and the fusion memory at the current moment. The process is iteratively executed until the controller outputs a termination instruction or the change in the fusion memory is less than a threshold. The report paragraph generation module combines all the generated complete sentences to obtain the final report paragraph.

[0049] It should be noted that the specific implementation of the progressive dual-modal fusion marine hydrological design report generation system in this embodiment of the invention is similar to the specific implementation of the progressive dual-modal fusion marine hydrological design report generation method in this embodiment of the invention. For details, please refer to the description in the method section. To reduce redundancy, it will not be repeated here.

[0050] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the progressive dual-modal fusion method for generating marine hydrological design reports as described above.

[0051] Example 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the progressive dual-modal fusion marine hydrological design report generation method described above.

[0052] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0053] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0054] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0055] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0056] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0057] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating marine hydrological design reports based on progressive dual-modal fusion, characterized in that, Includes the following steps: The acquired marine hydrological data files are preprocessed to generate standard images and corresponding data tables; Extract the global and local visual feature vectors of standard images, the overall feature vectors and cell-enhanced feature vectors of data tables, and establish an association index between the local feature vectors and cell-enhanced feature vectors of images; The progressive decision fusion controller instructions are executed, and information is extracted from the local feature vector and cell enhancement feature vector of the image and updated to the fusion memory at the current moment by combining the decision vector and the fusion memory at the current moment. At the same time, a complete sentence is generated by combining the decision vector and the fusion memory at the current moment. This process is executed iteratively until the controller outputs a termination instruction or the change in the fusion memory is less than a threshold. Combine all the generated complete sentences to obtain the final report paragraph.

2. The method for generating marine hydrological design reports using progressive dual-modal fusion as described in claim 1, characterized in that, When extracting the overall feature vector and enhanced feature vectors of a data table, the following steps are included: Parse the table structure to obtain the text and numerical information of the header, row and column cells, and convert the content of each cell into a cell vector through a word embedding layer; A Transformer encoder is used as the table encoder. The cell vector and its row and column position encoding are input together. The semantic relationship between cells is modeled through a self-attention mechanism, and the overall feature vector of the table and the enhanced feature vector of each cell are output.

3. The method for generating marine hydrological design reports using progressive dual-modal fusion as described in claim 1, characterized in that, The progressive decision fusion controller is implemented using a recurrent neural network based on gated recurrent units, and its hidden state... As the core memory for decision-making, in each generation step The controller receives the current state vector. According to the current state vector Update its hidden state and output a decision vector. , where the current state vector Including semantic encoding of the generated text The fusion memory vector of the previous time step And the contextual summary features extracted from the bimodal features that are most relevant to the text content output up to the previous generation step. .

4. The method for generating marine hydrological design reports using progressive dual-modal fusion as described in claim 1, characterized in that, The decision vector output by the controller is parsed into modality selection probability distribution, information focusing weight, rhetorical action type and gating signal through different fully connected layers.

5. The method for generating marine hydrological design reports using progressive dual-modal fusion as described in claim 1, characterized in that, Combining the decision vector output by the controller, information is extracted from the local feature vector and cell-enhanced feature vector of the image and the fusion memory at the current time step is updated, including: Using the output information to focus weights, attention is calculated for the local feature set of the image and the enhanced feature set of the table cells respectively, to obtain the image content vector and the table content vector focused in the current step. Based on the modality selection probability, the two content vectors are weighted and fused to obtain the fused content vector; The fused content vector is input together with the fused memory from the previous time step into the fused memory updater to obtain the fused memory at the current time step.

6. The method for generating marine hydrological design reports using progressive dual-modal fusion as described in claim 1, characterized in that, The process of generating a complete sentence by combining the decision vector and the fused memory at the current moment includes: The current fused memory Rhetorical Action Types The embedding vector and the hidden states of the last few words of the generated text are concatenated to form a conditional context vector. ; Will As the initial hidden state or additional attention input, it drives the text generation model to predict words one by one in an autoregressive manner. The generation process is influenced by... Strong conditional constraints.

7. The method for generating marine hydrological design reports using progressive dual-modal fusion as described in claim 1, characterized in that, During training, the progressive decision fusion controller and text generation model employ a teacher-mandated strategy, which guides the controller to refer to real data to focus on trajectories and rhetorical actions, while the text generation model is trained by maximizing the likelihood probability of the target sentence.

8. A progressive dual-modal fusion marine hydrological design report generation system, characterized in that, include: The data preprocessing module is used to preprocess the acquired marine hydrological data files to generate standard images and corresponding data tables; the feature extraction module is used to extract the global visual feature vectors and local feature vectors of the standard images, the overall feature vectors and cell-enhanced feature vectors of the data tables, and to establish an association index between the image local feature vectors and cell-enhanced feature vectors. The progressive decision fusion process is used to execute the instructions of the constructed progressive decision fusion controller. It extracts information from the local feature vector and cell enhancement feature vector of the image and updates the fusion memory at the current moment by combining the decision vector and the fusion memory at the current moment. At the same time, it generates a complete sentence by combining the decision vector and the fusion memory at the current moment. The process is iteratively executed until the controller outputs a termination instruction or the change in the fusion memory is less than a threshold. The report paragraph generation module combines all the generated complete sentences to obtain the final report paragraph.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the steps in the method for generating marine hydrological design reports by progressive dual-modal fusion as described in any one of claims 1-7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the method for generating marine hydrological design reports by progressive dual-modal fusion as described in any one of claims 1-7.

Images