A port carbon emission prediction method and system based on a large model

Abstract

This invention proposes a port carbon emission prediction method and system based on a large model, relating to the fields of port environmental monitoring and carbon emission prediction technology. The method generates a multimodal data stream by simultaneously collecting port video, ship dynamics, and meteorological data. Visual feature vectors and temporal feature vectors are extracted using the visual encoding and temporal encoding modules in a pre-trained multimodal spatiotemporal prediction model, respectively. These are then fused across modalities through the attention mechanism of the fusion prediction module to generate port carbon emission intensity prediction data. The prediction data is transformed into an evolutionary simulation report containing trend curves and decomposition of key influencing factors. The system supports receiving scenario adjustment commands in natural language through a user interface, driving the model to dynamically extrapolate and generate updated reports, thus achieving high-precision, interpretable prediction and interactive scenario analysis of port carbon emissions.

Description

Technical Field

[0001] This invention relates to the field of port environmental monitoring and carbon emission prediction technology, and in particular to a port carbon emission prediction method and system based on a large model. Background Technology

[0002] As a key hub for global trade, ports generate substantial energy consumption and carbon emissions during their operations. Accurate and dynamic forecasting of port carbon emissions is a crucial prerequisite for developing effective emission reduction strategies and optimizing operational scheduling. Currently, port carbon emission forecasting methods largely rely on historical energy consumption statistics or simple models based on single data sources (such as ship AIS data), which have significant limitations: First, port carbon emissions are the result of complex coupling of multiple factors, including port machinery operations, ship traffic, and meteorological conditions. Relying on only a single or few data dimensions cannot fully reflect the driving mechanisms, thus limiting forecast accuracy. Second, existing methods typically process or simply stitch together data from different modalities (such as images and time series), lacking the ability to deeply fuse heterogeneous time-series and spatial information across modalities, and failing to fully explore the inherent correlations between data. Third, forecast results are often presented as single numerical values ​​or simple charts, resulting in poor interpretability and difficulty in supporting managers in making rapid dynamic deductions and decision optimizations based on different scenarios.

[0003] Therefore, there is an urgent need for a port carbon emission prediction technology that can integrate multi-source heterogeneous data, achieve high accuracy, be interpretable, and support interactive scenario analysis. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, the first aspect of this invention proposes a port carbon emission prediction method based on a large model, comprising: S1: Based on the raw signals synchronously collected from multiple data sources set in the port environment, the raw signals are timestamped and format standardized to generate a synchronous multimodal data stream; S2: Based on multimodal data streams, a pre-trained multimodal spatiotemporal prediction model is used to perform cross-modal fusion and feature extraction on heterogeneous temporal and spatial information to generate port carbon emission intensity prediction data for future periods. The multimodal spatiotemporal prediction model includes a visual encoding module, a temporal encoding module, and a fusion prediction module. The visual encoding module and the temporal encoding module are connected to the fusion prediction module. S2 includes: S21: Based on the video image sequence portion contained in the multimodal data stream, spatiotemporal features are extracted through the convolutional neural network in the visual coding module to generate a visual feature vector; S22: Based on the ship dynamic sequence and meteorological parameter sequence contained in the multimodal data stream, time-series dependency modeling is performed through the recurrent neural network or converter encoder in the time-series coding module to generate time-series feature vectors; S23: Based on visual feature vectors and temporal feature vectors, cross-modal correlations are calculated through the attention mechanism in the fusion prediction module, and the correlated features are mapped and regressed to generate port carbon emission intensity prediction data. S3: Based on port carbon emission intensity prediction data, the data is transformed into charts containing time series curves and key influencing factors through a visualization engine, generating a port carbon emission evolution simulation report. S4: Based on the scenario adjustment instructions received from the user interface and the initial data state corresponding to the port carbon emission evolution simulation report, drive the multimodal spatiotemporal prediction model to perform a new round of prediction calculations and generate an updated port carbon emission evolution simulation report.

[0005] Compared with the prior art, the beneficial effects of the present invention are as follows: Step S1 generates a synchronous multimodal data stream, improving the spatiotemporal consistency of data from different sources and modalities. Step S2 uses a pre-trained multimodal spatiotemporal prediction model to process the multimodal data stream, achieving deep and adaptive fusion of heterogeneous temporal and spatial information, thus improving prediction accuracy. Step S3 transforms the prediction data into a port carbon emission evolution simulation report containing time series curves and decomposition of key influencing factors, improving the interpretability of the prediction results. Step S4, based on the scenario adjustment instructions received from the user interface and the initial state of the report, drives the model to perform a new round of predictions, generating an updated report. This enables managers to make scenario assumptions and dynamic deductions, assess the impact of different scheduling or environmental changes on carbon emissions, and achieve comprehensive, accurate, interpretable, and dynamically adjustable intelligent prediction of port carbon emissions. Attached Figure Description

[0006] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0007] Figure 1 The diagram shown is a flowchart of a port carbon emission prediction method based on a large model provided in an embodiment of the present invention. Figure 2 The figure shown is a comparison chart of port carbon emission intensity prediction data and actual data provided in an embodiment of the present invention; Figure 3 The diagram shown is a schematic representation of a port carbon emission prediction system based on a large model, according to an embodiment of the present invention. Detailed Implementation

[0008] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0009] The specific embodiments of the present invention will be described below.

[0010] Example 1 like Figure 1 As shown, the first aspect of this invention proposes a port carbon emission prediction method based on a large model, comprising: S1: Based on the raw signals synchronously collected from multiple data sources set in the port environment, the raw signals are timestamped and format standardized to generate a synchronous multimodal data stream; S2: Based on multimodal data streams, a pre-trained multimodal spatiotemporal prediction model is used to perform cross-modal fusion and feature extraction on heterogeneous temporal and spatial information to generate port carbon emission intensity prediction data for future periods. The multimodal spatiotemporal prediction model includes a visual encoding module, a temporal encoding module, and a fusion prediction module. The visual encoding module and the temporal encoding module are connected to the fusion prediction module. S2 includes: S21: Based on the video image sequence portion contained in the multimodal data stream, spatiotemporal features are extracted through the convolutional neural network in the visual coding module to generate a visual feature vector; S22: Based on the ship dynamic sequence and meteorological parameter sequence contained in the multimodal data stream, time-series dependency modeling is performed through the recurrent neural network or converter encoder in the time-series coding module to generate time-series feature vectors; S23: Based on visual feature vectors and temporal feature vectors, cross-modal correlations are calculated through the attention mechanism in the fusion prediction module, and the correlated features are mapped and regressed to generate port carbon emission intensity prediction data. S3: Based on port carbon emission intensity prediction data, the data is transformed into charts containing time series curves and key influencing factors through a visualization engine, generating a port carbon emission evolution simulation report. S4: Based on the scenario adjustment instructions received from the user interface and the initial data state corresponding to the port carbon emission evolution simulation report, drive the multimodal spatiotemporal prediction model to perform a new round of prediction calculations and generate an updated port carbon emission evolution simulation report.

[0011] The method relies on various data acquisition devices deployed in different functional areas of the port (such as the quay front, container yard, main road, and near the waterway), which constitute multiple data sources. For example, these data sources may include high-definition network cameras for monitoring operational scenarios, Automatic Identification System (AIS) base station receivers for receiving ship broadcast information, and temperature, humidity, and wind speed sensors deployed in port weather stations. These devices generate raw signals, such as video streams from cameras, text messages output by AIS receivers, and electrical signals output by sensors. Format standardization decodes the video stream into a fixed frame rate, fixed resolution image sequence tensor (e.g., a four-dimensional array of shape [T1, H, W, C], where T1 is the number of time frames, H and W are the image height and width, and C is the number of channels), parses and converts AIS text messages into a structured data sequence containing fields such as ship ID, latitude and longitude, and speed, and converts sensor signals into floating-point arrays.

[0012] The visual encoding module processes the video image sequence portion of a multimodal data stream. Port operation videos contain not only spatial information (such as the static appearance of machinery, vehicles, and containers) but also dynamic information about the movement of these objects over time. To capture both spatial and temporal features simultaneously, the visual encoding module employs a 3D convolutional neural network-based architecture. 3D convolutional kernels perform convolution operations on the video data cube (height, width, and time), extracting low-level joint temporal and spatial features, such as edge movement across consecutive frames. These low-level features are then progressively abstracted and compressed through multiple subsequent 3D convolutional and 3D pooling layers, ultimately allowing the network to learn high-level semantic information and output a visual feature vector.

[0013] The visual encoding module takes a preprocessed video image sequence tensor of shape T×H×W×3 as input, where T is the number of consecutive frames (e.g., 16 frames), H=224, and W=224. The network structure is as follows: the first layer is a 3D convolution with a kernel size of 3×3×3, a stride of 1, and 64 output channels, followed by batch normalization and ReLU activation; the second layer is a 3D pooling layer with a pooling window of 1×2×2 and a stride of 2, used to reduce the spatial dimension; the third layer is a 3D convolution with a kernel size of 3×3×3 and 128 output channels; the fourth layer is a 3D pooling layer with a pooling window of 2×2×2 and a stride of 2, compressing both temporal and spatial dimensions; the fifth to eighth layers are four residual 3D convolutional blocks, each containing two convolutional layers with a kernel size of 3×3×3, and output channels of 256, 256, 512, and 512 respectively; the ninth layer is a global average pooling layer, compressing the spatiotemporal dimension to one dimension to obtain a visual feature vector with a dimension of 512.

[0014] The temporal coding module is used to process temporal sequence data other than video in multimodal data streams, mainly including ship dynamic sequences (such as changes in ship position and speed over time) and meteorological parameter sequences (such as changes in temperature and wind speed over time). The temporal coding module can be constructed using a recurrent neural network or the encoder part of a transformer. The temporal coding module employs a two-layer bidirectional Long Short-Term Memory (LSTM) network. Input features include: number of ships in port, average speed, average wind speed, temperature, and humidity. The input dimension for each time step is 5. The sequence length is taken from the past 60 minutes, with one sampling point per minute, i.e., L=60. The first LSTM layer has 128 hidden units and outputs the hidden states of all time steps; the second LSTM layer has 256 hidden units and takes the output of the last time step as the temporal feature vector, with a dimension of 256.

[0015] The fusion prediction module employs an attention mechanism to achieve cross-modal correlation calculations. Specifically, the system calculates a set of visual attention weights, which quantifies the importance of each component in the temporal feature vector within the current visual scene. For example, when visual features indicate dense activity in a refrigerated container yard, the attention weights favor the temperature-related dimension of the temporal features. Then, these weights are used to perform a weighted summation of the temporal feature vectors, generating a new temporal context vector. Finally, the focused temporal context vector is fused with the original visual feature vector through concatenation or element-wise addition to obtain the correlated features. A series of fully connected layer mappings and regression analyses (such as linear regression layers) are then applied to the correlated features, ultimately outputting predicted data for the port's carbon emission intensity within a specific future time period.

[0016] For example, the fusion prediction module first performs linear mapping on the visual feature vector and the temporal feature vector respectively: the visual features are mapped to a query vector Q with dimension 128 through a fully connected layer; the temporal features are mapped to a key vector K and a value vector V, both with dimension 128 through a fully connected layer. Attention weights are then calculated. This yields a 128×128 weight matrix. Then, the context vector C=AV, with a dimension of 128, is calculated. The visual feature vector is then mapped to a 128-dimensional V′ through another fully connected layer, and element-wise added to C to obtain the fused features. Finally, the predicted carbon emission intensity (in tons per hour) for the next hour is output through two fully connected layers (the first layer is 128-dimensional, ReLU; the second layer is 1-dimensional).

[0017] The pre-training of the multimodal spatiotemporal prediction model consists of three stages. In the first stage, a publicly available port monitoring video dataset (e.g., approximately 500,000 frames of continuous video collected from multiple ports over six months) is used for self-supervised pre-training of the visual encoding module. A temporal consistency task is employed: two sequences with a few frames interval are randomly sampled from the same video segment, and the training module determines whether they originate from the same original segment, using a contrastive loss function. After 100,000 iterations, the module is able to extract stable motion and appearance features. In the second stage, the temporal encoding module is pre-trained using the port's Automatic Identification System (AIS) and historical meteorological data (sampled at a frequency of 1 minute for three consecutive months). The task is to predict changes in ship density and wind speed over the next 10 minutes, using a mean squared error loss function. The LSTM module learns the dynamic temporal patterns of the port through this task. In the third stage, the outputs of the visual encoding module and the temporal encoding module are fixed, and only the fusion prediction module is trained. Multimodal data from the target port over the past three months were collected, and carbon emission intensity was manually labeled (based on actual statistical values ​​from energy consumption monitoring instruments, with one labeling point every 15 minutes), totaling approximately 8640 samples. The Adam optimizer was used with a learning rate of 0.0001, a batch size of 32, and a loss function combining mean absolute error (MAE) and mean absolute percentage error (MAPE). Training stopped when the MAPE on the validation set fell below 8%. Then, all modules were involved in training, and the model was fine-tuned overall for 100 epochs at a lower learning rate (0.00001). The final model achieved an MAE of 0.12 tons / hour and a MAPE of 6.5% on the test set.

[0018] Cross-modal fusion is achieved through an attention mechanism in the fusion prediction module, and its specific computational process is as follows. Let the visual feature vector output by the visual encoding module be V∈R. 512 The time-series feature vector output by the time-series coding module is T∈R 256 The first step is to construct a learnable linear projection matrix W. Q ∈R 128×512 W K ∈R128×256 W V ∈R 128×256 Calculate the query, key, and value respectively: Q=W Q V, K=W K T, V T =W V T. The second step is to calculate the attention score matrix. The shape of S is 128×128. The third step is to apply the softmax function to each row of S (corresponding to one dimension of the query vector) to obtain the attention weight matrix A, where the sum of each row is 1. The fourth step is to calculate the temporal context vector C=AV. T The shape of C is 128×128. Then, the rows of C are summed, compressing it into a 128-dimensional vector c, i.e. The fifth step is to pass the visual feature vector V through another linear layer W. P ∈R 128×512 Mapped as V′=W P V is then added element-wise to c to obtain the fused feature F = V′ + c. In the sixth step, the fused feature F is input into a two-layer fully connected network to output the predicted value. This attention mechanism allows the model to dynamically weight temporal features (such as current wind speed and ship density) based on visual scene content (such as yard operation density), thereby capturing the non-linear coupling relationship between the two.

[0019] The visualization engine can integrate open-source charting libraries (such as Matplotlib and ECharts). First, it reads the prediction data and plots a curve showing the change in carbon emission intensity over time, displaying the predicted trend, peaks, and troughs of emissions over a future period. The system uses a fusion prediction module to generate intermediate data generated during the prediction process, such as attention weights and feature activation values. Using feature importance ranking algorithms (such as SHAP, LIME, or gradient weight-based algorithms), the system reverse-engineers which input features (or combinations thereof) contribute the most to the final prediction results. These identified high-contribution features are compiled into a list of key influencing factors. The system also needs to map the abstract features within the model back to specific business meanings based on predefined mapping relationships. For example, a high-contribution visual feature might correspond to crane operation intensity, and a high-contribution time-series feature might correspond to southeasterly wind speed. The system queries the knowledge base or metadata to generate feature explanations for each element in the list.

[0020] Port managers may want to assess the potential impact of different operational strategies or changes in external conditions on carbon emissions; for example, how would the carbon emission curve change if 20% of operations were shifted to nighttime next week? Users input such scenario adjustment instructions through a user interface. Upon receiving the instructions, the system processes them in conjunction with the initial data state corresponding to the current port carbon emission evolution simulation report (i.e., the multimodal data stream used to generate the previous report). The system first converts the user's instructions into actionable, structured parameter modification instructions. Then, based on these instructions, it modifies the corresponding data segments in the multimodal data stream (e.g., simulating increased visual feature data for nighttime operations). Using the adjusted multimodal spatiotemporal prediction model as input, S2 and S3 are re-executed.

[0021] Taking the measured data of a coastal container port from 9:00 AM to 10:00 AM on March 15, 2025 as an example. The input data includes: continuous video (16 frames / second, 960 frames in total) covering the quay crane operation area of ​​berth No. 3 by a camera; 21 vessels entering and leaving the port during that hour as recorded by an AIS receiver, with the vessel density varying between 4 and 7 vessels per minute; and an average wind speed of 3.2 m / s, southeast wind, temperature of 15℃, and humidity of 68% recorded by a weather station. After standardization, this data is input into a pre-trained multimodal spatiotemporal prediction model.

[0022] like Figure 2As shown, the model outputs the predicted carbon emission intensity curves for the next hour (10:00 to 11:00): 2.34 tons / hour from 10:00 to 10:15, 2.67 tons / hour from 10:15 to 10:30, 2.91 tons / hour from 10:30 to 10:45, and 2.85 tons / hour from 10:45 to 11:00. Meanwhile, the feature importance ranking shows that the three features contributing most to the prediction results are the 42nd dimension of the visual feature vector (corresponding to the continuous lifting and lowering motion mode of the quay crane, contributing 38%), the 17th dimension of the temporal feature vector (corresponding to the 15-minute slope of the number of ships in port, contributing 25%), and the 89th dimension of the temporal feature vector (corresponding to the current value of wind speed, contributing 18%). The visualization engine generates a carbon emission trend curve based on this data and lists key influencing factors in the report: high continuous operation intensity of the quay crane at Berth 3, peak ship arrival frequency around 10:30, and persistent southeast wind speeds below 3.5 m / s, which are unfavorable for dispersion. After viewing the report, the user enters a natural language command on the interactive interface: "Assuming 20% ​​of the operations at Berth 3 between 10:30 and 11:00 are shifted to the afternoon period, re-predict carbon emissions." The system parses the command, modifies the visual feature encoding of the corresponding time period in the multimodal data stream (simulating a reduction in operations by adjusting attention weights), drives the model to recalculate, and obtains an updated carbon emission intensity of 2.43 tons / hour, a decrease of approximately 16.5%. The system overlays the old and new curves and updates the list of key influencing factors, indicating that the decrease in operation intensity is the main emission reduction factor. This example verifies that the model can make high-precision predictions based on real multimodal data and supports interactive scenario simulation.

[0023] Figure 2 This paper demonstrates a comparison between port carbon emission intensity prediction data generated by a multimodal spatiotemporal prediction model and actual carbon emission intensity data collected during the same period, according to one embodiment of the present invention. The prediction curve shows the emission intensity change trend within the next hour, for example, reaching a peak of 2.91 tons / hour between 10:30 and 10:45. By comparing the trends of the two curves, it can be intuitively seen that the model prediction value and the actual value have a high consistency in the time series, verifying the prediction accuracy and effectiveness of the cross-modal fusion prediction method proposed in this invention in actual port scenarios.

[0024] In some implementations, S1 includes: S11: Based on the video image sequence acquired by the visual acquisition equipment deployed in the port loading and unloading area and the storage yard, extract the activity trajectory of port machinery and the changes in the storage yard area to generate visual feature data of port operations. S12: Based on the received Automatic Identification System (AIS) broadcast messages, analyze the ship's identity, location, and speed information and associate them with port geographic information to generate ship traffic situation data; S13: Based on meteorological observation and forecast data obtained from connected meteorological monitoring equipment or service interfaces, extract temperature, humidity, wind speed, and wind direction parameters of the port area to generate climate data for the port area. S14: Based on port operation visual feature data, ship traffic status data and port area climate data, time synchronization and data packaging are performed to generate a multimodal data stream.

[0025] Computer vision algorithms are applied to video sequences to track the movement trajectories of port machinery (such as quay cranes, yard cranes, and reach stackers) and detect changes in the yard area (such as container stacking and retrieval, and the occupancy status of cargo spaces). Area change detection can be achieved by comparing the differences between consecutive frames or the background model.

[0026] AIS base stations continuously receive radio frequency signals transmitted from surrounding vessels, which include vessel identification information (such as vessel name), dynamic information (latitude and longitude, speed, heading, and bow direction), and voyage information (destination and estimated time of arrival). Key dynamic fields are extracted from the raw messages: vessel identity (unique identifier), real-time position (latitude and longitude coordinates), and speed. Spatial calculations determine whether each vessel is currently at its berth, anchorage, or navigating in a channel, and calculate its distance to the target berth. Further, the number of vessels within a specific area (such as the harbor basin) is aggregated, i.e., the vessel density in port. Through parsing, correlation, and aggregation operations, the raw AIS data stream is transformed into vessel traffic situation data.

[0027] In some implementations, before S14, the following is also included: Based on the real-time current and voltage signals collected by power sensors deployed on major energy-consuming equipment such as port cranes, lighting systems, and refrigerated container sockets, instantaneous power is calculated and accumulated to generate real-time energy consumption data at the equipment level. S14 includes: based on real-time energy consumption data at the equipment level, visual feature data of port operations, ship traffic situation data, and climate data of the port area, time synchronization and data packaging are performed to generate a multimodal data stream.

[0028] In some implementations, S21 includes: S211: Based on video image sequences, low-level motion and texture features of images in the temporal and spatial dimensions are extracted through three-dimensional convolutional layers to generate an initial spatiotemporal feature map; S212: Based on the initial spatiotemporal feature map, visual feature vectors representing the types of port machinery actions, motion patterns, and regional activity intensity are obtained through subsequent convolutional and pooling layers.

[0029] The preprocessed video image sequence can be a four-dimensional tensor [time frame number T1, height H, width W, number of channels C]. A three-dimensional convolutional kernel slides simultaneously across all three dimensions (time frame number T1, height H, width W) and performs convolution calculations. As the kernel slides over a video block, it can simultaneously perceive pixel changes in a local spatial region within a small segment of consecutive frames. The three-dimensional convolutional layer captures low-level spatiotemporal patterns from the raw pixel data. For example, a three-dimensional convolutional filter detects edge movement from left to right (corresponding to lateral motion of an object) or texture changes spreading outward from the center. After the first layer of three-dimensional convolution processing, an initial spatiotemporal feature map is output.

[0030] The second 3D convolutional layer performs convolution on the initial spatiotemporal feature map. This second convolutional layer learns more complex feature combinations. 3D pooling layers are interspersed between the convolutional layers, primarily functioning to downsample in both spatial and temporal dimensions, progressively reducing the size of the feature map (H, W, T1). The visual feature vector comprehensively encodes the information contained in the input video sequence: representing the specific action categories of port machinery (e.g., loading / unloading, shifting, waiting) and the motion patterns of the actions (e.g., uniform speed, acceleration, periodicity).

[0031] In some implementations, in S23, calculating cross-modal associations through the attention mechanism in the fusion prediction module includes: S231: Based on visual feature vectors and temporal feature vectors, the weight distribution of each element in the visual feature vector relative to the temporal feature vector is calculated through an attention mechanism to generate visual attention weights; S232: Based on visual attention weights, the temporal feature vectors are weighted and aggregated to generate a temporal context vector related to the visual context. S233: Based on visual feature vectors and temporal context vectors, the associated features are generated by concatenation or element-wise addition.

[0032] In step S231, assuming the visual feature vector V has dimension Dv and the temporal feature vector T3 has dimension Dt, for each element in the visual feature vector (which can be understood as each visual semantic unit, totaling Dv elements), its correlation with the overall temporal feature vector is calculated. First, the visual feature vector V is mapped to a query vector Q through a linear transformation layer (weight matrix Wq), and the temporal feature vector T3 is mapped to a key vector K through another linear transformation layer (weight matrix Wk). Then, the similarity between query Q and key K is calculated (e.g., using dot product attention: similarity score = Q × K). T3After scaling and normalization, we obtain the visual attention weight matrix A (with dimension [Dv, a dimension related to T3]). Each row in the weight matrix A corresponds to an element in the visual feature V, and its numerical distribution indicates the importance ratio of each part in the temporal feature vector T3 when focusing on that visual element.

[0033] T3 is mapped to a value vector Vt through a third linear transformation layer (weight matrix Wv). Then, the attention weights A calculated in the previous step are used to perform a weighted sum on Vt. Specifically, for the i-th element in the visual feature vector V, its corresponding attention weight row Ai is used to weight all elements of Vt to generate the corresponding aggregate vector. This operation is performed on all visual elements, or the weight matrix is ​​multiplied as a whole (such as matrix multiplication A×Vt) to generate the temporal context vector C.

[0034] In some implementations, S3 includes: S31: Based on port carbon emission intensity prediction data, use a chart library to plot the curve of carbon emission intensity changing over time and generate a carbon emission trend chart; S32: Based on the intermediate features and weights generated by the fusion prediction module in the process of generating port carbon emission intensity prediction data, the feature importance ranking algorithm is used to identify the most contributing features and generate a list of key influencing factors. S33: Query the original data or business meaning corresponding to the list of key influencing factors, and generate explanations of the factors; S34: Based on carbon emission trend charts, a list of key influencing factors, and explanations of these factors, combine and format these elements to generate a port carbon emission evolution simulation report.

[0035] Step S31 utilizes the chart library functionality within the visualization engine to visualize the data. The chart library provides standardized functions for drawing line charts, bar charts, and other graphs. The system calls the APIs of these libraries, taking a data sequence containing timestamps and corresponding carbon emission intensity predictions as input, and drives the chart library to draw a smooth curve with time as the horizontal axis and carbon emission intensity as the vertical axis, thereby generating a carbon emission trend chart.

[0036] Intermediate data can include the output tensor of the feature mapping layer, the weight matrix of the attention layer, or the activation values ​​of the fully connected layer. The feature importance ranking algorithm works by systematically perturbing the model's input or intermediate features and observing the changes in the model's final output (i.e., the predicted carbon emission intensity). Based on this, it calculates the contribution of each feature to the output. The algorithm processes all relevant intermediate features, calculates their contribution scores, and ranks them. Finally, the algorithm selects the features with the highest contribution scores to generate a list of key influencing factors.

[0037] After obtaining the list of key impact factors, each feature identifier in the list is traversed, and its predefined business meaning description is queried in the data dictionary. An explanation is then generated for each entry. This explanation translates the technical feature identifiers into language familiar to business personnel. For example, key feature A: the number of operation cycles of the quay crane at berth No. 3, a value directly related to the power consumption of the loading and unloading equipment.

[0038] In some implementations, in S32, the elements in the list of key influencing factors include specific mechanical operation status represented by visual feature vectors, and ship density in port or weather conditions represented by time-series feature vectors.

[0039] For example, in the feature dimension corresponding to the pattern learning of the reciprocating handling operation of the rubber-tired container gantry crane in the container yard, the contribution of the specific mechanical operation state is more prominent. In the list of key influencing factors, this factor is presented as a description such as the intensity of the handling operation in the container yard.

[0040] On the other hand, the elements originate from the vessel density in port or meteorological conditions represented by the time-series feature vector. Vessel density in port is obtained by interpreting the relevant dimensions in the time-series features that characterize the number of vessels, berth occupancy status, and anchorage waiting time. For example, feature importance algorithms reveal that feature patterns reflecting the total number of vessels in the port access channel and anchorage exceeding conventional thresholds are core factors in predicting emission peaks. Correspondingly, anchorage and channel vessel congestion indices or similar entries appear in the list of key influencing factors. Meteorological conditions correspond to the dimensions in the time-series features that encode temperature, wind speed, wind direction, etc. Feature combinations representing sustained low wind speeds (less than 2 meters per second) and high humidity significantly contribute to the cumulative effect of predicted emission concentrations, resulting in the element of calm, stable, and humid weather conditions being generated in the list.

[0041] The analysis report generated in this embodiment is linked to familiar and monitorable scenario factors in port operations, such as ship traffic congestion or adverse weather conditions, enabling managers to take measures from the perspective of macro scheduling and emergency response: if the density of ships in port is a key factor, port operation plans and ship arrival sequences can be optimized to reduce waiting time; if specific weather conditions are the main cause, heavy pollution weather operation plans can be activated in advance.

[0042] In some implementations, context-adjusting instructions include natural language instructions, and S4 includes: S41: Based on natural language instructions input at the user interface that modify port operation or environmental parameters, convert them into structured parameter modification instructions through semantic parsing. S42: Based on parameter modification instructions and multimodal data stream, adjust the corresponding data segments in the multimodal data stream or the input parameters of the multimodal spatiotemporal prediction model to generate the adjusted multimodal spatiotemporal prediction model input; S43: Based on the adjusted multimodal spatiotemporal prediction model input, re-execute the prediction and report generation processes of S2 and S3 to generate an updated port carbon emission evolution simulation report.

[0043] Natural language commands allow users to pose hypothetical questions in a way that aligns with human habits. For example, inputting: "If 10% of daytime bulk cargo handling operations are shifted to the evening next month, and the average evening wind speed is expected to increase by one level, please simulate the change in carbon emissions." This interactive approach lowers the barrier to entry for specialized tools, allowing users to perform complex simulations without needing in-depth software training.

[0044] Semantic parsing performs word segmentation, part-of-speech tagging, syntactic analysis, and semantic role labeling on natural language instructions to understand the user's intent and the entities, attributes, and operations involved in the instructions. For example, from the above instructions, the parser needs to identify that the intent is to request a simulation, the entities include bulk cargo loading and unloading operations and evening, the attributes include next month, 10%, average wind speed, and level 1, and the operations include transfer and increase.

[0045] The system, based on the specified modification object (e.g., bulk cargo handling operations) and modification content (e.g., transferring 10% from daytime to evening) in the instruction, locates the corresponding data segment in the multimodal data stream for simulation adjustment. For example, modifying the port operation visual feature data for the next month's time period simulates the redistribution of operation volume between daytime and evening. Simultaneously, for instructions to increase the average evening wind speed by one level, the system modifies the corresponding time period and wind speed values ​​in the port area's climate data.

[0046] In some implementations, S41 includes: S411: Identify entities, attributes, and expected modification actions mentioned in natural language instructions through a pre-trained natural language understanding model, and generate semantic understanding results; S412: Based on the semantic understanding results, query the port operation knowledge graph to determine the data fields or model parameters corresponding to entities and attributes, and generate parameter mapping relationships; S413: Based on parameter mapping relationships and modification actions, assemble into structured parameter modification instructions.

[0047] Natural language understanding models, through their deep neural network structures, identify and extract key semantic components from instructions. This includes identifying the mentioned entities (e.g., bulk cargo loading / unloading, evening, level 1 wind), determining the attribute categories of the entities (e.g., bulk cargo loading / unloading belongs to the operation type, evening belongs to the time attribute, and level 1 wind belongs to the meteorological intensity attribute), and parsing the user's expected modification actions for these entities (e.g., transfer is a spatial or temporal reallocation action, and improvement is a numerical increase or decrease action). Ultimately, the model outputs a structured semantic understanding result, which can be a set containing an intent classification and a series of (entity, relation, attribute value) triples.

[0048] The port operations knowledge graph is a structured semantic knowledge base organized in a graph structure. Nodes represent entities in the port domain (such as equipment, operational processes, areas, and parameters), and edges represent relationships between entities (such as belonging to, occurring in, and affecting). Furthermore, nodes or attributes in the knowledge graph are linked to the data structure of the backend system. For example, the bulk cargo loading and unloading node is primarily linked to the grab unloader node through equipment relationships, while the activity intensity feature attribute of the grab unloader node is linked to a specific feature index of the port operations visual feature data in the multimodal data stream.

[0049] Step S413, based on the modification actions (such as setting to a value, increasing an amount, or shifting time) parsed from the semantic understanding results, and combined with the target identifier provided by the parameter mapping relationship, assembles all the information into a structured parameter modification instruction according to the structured instruction format defined by the system.

[0050] Steps S411 to S413 utilize the general semantic understanding capabilities of the pre-trained model and ensure the accuracy of the mapping by leveraging the domain knowledge graph, thereby effectively handling the diversity of language expressions and implicit domain common sense.

[0051] Example 2 like Figure 3 As shown, in a second aspect, the present invention proposes a port carbon emission prediction system based on a large model. The system employs a port carbon emission prediction method based on a large model proposed in any of the above embodiments. The system includes: The multimodal data synchronization and preprocessing module is used to perform step S1: based on the raw signals synchronously collected from multiple data sources set in the port environment, the raw signals are processed by timestamp alignment and format standardization to generate a synchronized multimodal data stream; A multimodal spatiotemporal prediction model is used to execute step S2: Based on the multimodal data stream, the pre-trained multimodal spatiotemporal prediction model performs cross-modal fusion and feature extraction on the heterogeneous temporal and spatial information to generate port carbon emission intensity prediction data for future periods; wherein, the multimodal spatiotemporal prediction model includes a visual encoding module, a temporal encoding module, and a fusion prediction module, and the visual encoding module and the temporal encoding module are respectively connected to the fusion prediction module. S2 includes: S21: Based on the video image sequence portion contained in the multimodal data stream, spatiotemporal features are extracted through the convolutional neural network in the visual coding module to generate a visual feature vector; S22: Based on the ship dynamic sequence and meteorological parameter sequence contained in the multimodal data stream, time-series dependency modeling is performed through the recurrent neural network or converter encoder in the time-series coding module to generate time-series feature vectors; S23: Based on visual feature vectors and temporal feature vectors, cross-modal correlations are calculated through the attention mechanism in the fusion prediction module, and the correlated features are mapped and regressed to generate port carbon emission intensity prediction data. The carbon emission report generation module is used to perform step S3: based on the port carbon emission intensity prediction data, it transforms the data into a chart containing time series curves and key influencing factors through a visualization engine, and generates a port carbon emission evolution simulation report. An interactive scenario simulation engine is used to execute step S4: based on the scenario adjustment instructions received from the user interface and the initial data state corresponding to the port carbon emission evolution simulation report, the multimodal spatiotemporal prediction model is driven to perform a new round of prediction calculations to generate an updated port carbon emission evolution simulation report.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A port carbon emission prediction method based on a large model, characterized in that, include: S1: Based on the raw signals synchronously collected from multiple data sources set in the port environment, the raw signals are timestamped and format standardized to generate a synchronous multimodal data stream; S2: Based on multimodal data streams, a pre-trained multimodal spatiotemporal prediction model is used to perform cross-modal fusion and feature extraction on heterogeneous temporal and spatial information to generate port carbon emission intensity prediction data for future periods. The multimodal spatiotemporal prediction model includes a visual encoding module, a temporal encoding module, and a fusion prediction module. The visual encoding module and the temporal encoding module are connected to the fusion prediction module. S2 includes: S21: Based on the video image sequence portion contained in the multimodal data stream, spatiotemporal features are extracted through the convolutional neural network in the visual coding module to generate a visual feature vector; S22: Based on the ship dynamic sequence and meteorological parameter sequence contained in the multimodal data stream, time-series dependency modeling is performed through the recurrent neural network or converter encoder in the time-series coding module to generate time-series feature vectors; S23: Based on visual feature vectors and temporal feature vectors, cross-modal correlations are calculated through the attention mechanism in the fusion prediction module, and the correlated features are mapped and regressed to generate port carbon emission intensity prediction data. S3: Based on port carbon emission intensity prediction data, the data is transformed into charts containing time series curves and key influencing factors through a visualization engine, generating a port carbon emission evolution simulation report. S4: Based on the scenario adjustment instructions received from the user interface and the initial data state corresponding to the port carbon emission evolution simulation report, drive the multimodal spatiotemporal prediction model to perform a new round of prediction calculations and generate an updated port carbon emission evolution simulation report.

2. The port carbon emission prediction method based on a large model according to claim 1, characterized in that, S1 includes: S11: Based on the video image sequence acquired by the visual acquisition equipment deployed in the port loading and unloading area and the storage yard, extract the activity trajectory of port machinery and the changes in the storage yard area to generate visual feature data of port operations. S12: Based on the received Automatic Identification System (AIS) broadcast messages, analyze the ship's identity, location, and speed information and associate them with port geographic information to generate ship traffic situation data; S13: Based on meteorological observation and forecast data obtained from connected meteorological monitoring equipment or service interfaces, extract temperature, humidity, wind speed, and wind direction parameters of the port area to generate climate data for the port area. S14: Based on port operation visual feature data, ship traffic status data and port area climate data, time synchronization and data packaging are performed to generate a multimodal data stream.

3. The port carbon emission prediction method based on a large model according to claim 2, characterized in that, Before S14, it also includes: Based on the real-time current and voltage signals collected by power sensors deployed on major energy-consuming equipment such as port cranes, lighting systems, and refrigerated container sockets, instantaneous power is calculated and accumulated to generate real-time energy consumption data at the equipment level. S14 includes: based on real-time energy consumption data at the equipment level, visual feature data of port operations, ship traffic situation data, and climate data of the port area, time synchronization and data packaging are performed to generate a multimodal data stream.

4. The port carbon emission prediction method based on a large model according to claim 1, characterized in that, S21 includes: S211: Based on video image sequences, low-level motion and texture features of images in the temporal and spatial dimensions are extracted through three-dimensional convolutional layers to generate an initial spatiotemporal feature map; S212: Based on the initial spatiotemporal feature map, visual feature vectors representing the types of port machinery actions, motion patterns, and regional activity intensity are obtained through subsequent convolutional and pooling layers.

5. The port carbon emission prediction method based on a large model according to claim 1, characterized in that, In S23, calculating cross-modal associations through the attention mechanism in the fusion prediction module includes: S231: Based on visual feature vectors and temporal feature vectors, the weight distribution of each element in the visual feature vector relative to the temporal feature vector is calculated through an attention mechanism to generate visual attention weights; S232: Based on visual attention weights, the temporal feature vectors are weighted and aggregated to generate a temporal context vector related to the visual context. S233: Based on visual feature vectors and temporal context vectors, the associated features are generated by concatenation or element-wise addition.

6. The port carbon emission prediction method based on a large model according to claim 1, characterized in that, S3 include: S31: Based on port carbon emission intensity prediction data, use a chart library to plot the curve of carbon emission intensity changing over time and generate a carbon emission trend chart; S32: Based on the intermediate features and weights generated by the fusion prediction module in the process of generating port carbon emission intensity prediction data, the feature importance ranking algorithm is used to identify the most contributing features and generate a list of key influencing factors. S33: Query the original data or business meaning corresponding to the list of key influencing factors, and generate explanations of the factors; S34: Based on carbon emission trend charts, a list of key influencing factors, and explanations of these factors, combine and format these elements to generate a port carbon emission evolution simulation report.

7. The port carbon emission prediction method based on a large model according to claim 6, characterized in that, In S32, the key influencing factors list includes specific mechanical operation status represented by visual feature vectors, and ship density in port or weather conditions represented by time-series feature vectors.

8. The port carbon emission prediction method based on a large model according to claim 1, characterized in that, Context adjustment instructions include natural language instructions, and S4 includes: S41: Based on natural language instructions input at the user interface that modify port operation or environmental parameters, convert them into structured parameter modification instructions through semantic parsing. S42: Based on parameter modification instructions and multimodal data stream, adjust the corresponding data segments in the multimodal data stream or the input parameters of the multimodal spatiotemporal prediction model to generate the adjusted multimodal spatiotemporal prediction model input; S43: Based on the adjusted multimodal spatiotemporal prediction model input, re-execute the prediction and report generation processes of S2 and S3 to generate an updated port carbon emission evolution simulation report.

9. The port carbon emission prediction method based on a large model according to claim 8, characterized in that, S41 includes: S411: Identify entities, attributes, and expected modification actions mentioned in natural language instructions through a pre-trained natural language understanding model, and generate semantic understanding results; S412: Based on the semantic understanding results, query the port operation knowledge graph to determine the data fields or model parameters corresponding to entities and attributes, and generate parameter mapping relationships; S413: Based on parameter mapping relationships and modification actions, assemble into structured parameter modification instructions.

10. A port carbon emission prediction system based on a large model, characterized in that, The system employs a port carbon emission prediction method based on a large model as described in any one of claims 1 to 9, and the system comprises: The multimodal data synchronization and preprocessing module is used to perform step S1: based on the raw signals synchronously collected from multiple data sources set in the port environment, the raw signals are processed by timestamp alignment and format standardization to generate a synchronized multimodal data stream; A multimodal spatiotemporal prediction model is used to execute step S2: Based on the multimodal data stream, the pre-trained multimodal spatiotemporal prediction model performs cross-modal fusion and feature extraction on the heterogeneous temporal and spatial information to generate port carbon emission intensity prediction data for future periods; wherein, the multimodal spatiotemporal prediction model includes a visual encoding module, a temporal encoding module, and a fusion prediction module, and the visual encoding module and the temporal encoding module are respectively connected to the fusion prediction module. S2 includes: S21: Based on the video image sequence portion contained in the multimodal data stream, spatiotemporal features are extracted through the convolutional neural network in the visual coding module to generate a visual feature vector; S22: Based on the ship dynamic sequence and meteorological parameter sequence contained in the multimodal data stream, time-series dependency modeling is performed through the recurrent neural network or converter encoder in the time-series coding module to generate time-series feature vectors; S23: Based on visual feature vectors and temporal feature vectors, cross-modal correlations are calculated through the attention mechanism in the fusion prediction module, and the correlated features are mapped and regressed to generate port carbon emission intensity prediction data. The carbon emission report generation module is used to perform step S3: based on the port carbon emission intensity prediction data, it transforms the data into a chart containing time series curves and key influencing factors through a visualization engine, and generates a port carbon emission evolution simulation report. An interactive scenario simulation engine is used to execute step S4: based on the scenario adjustment instructions received from the user interface and the initial data state corresponding to the port carbon emission evolution simulation report, the multimodal spatiotemporal prediction model is driven to perform a new round of prediction calculations to generate an updated port carbon emission evolution simulation report.

Images