A method and device for modeling and predicting maritime statistics
By cleaning, detecting, and preprocessing maritime data, and combining WVD and Gram angle field transformation, a matrix transformation model is constructed, which solves the shortcomings of maritime data modeling and preprocessing, and achieves high prediction accuracy and system adaptability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-04
- Publication Date
- 2026-03-10
AI Technical Summary
Existing maritime data modeling and preprocessing methods lack specificity, fail to accurately characterize time-series patterns, exhibit poor feature extraction and model adaptability, and lack robustness in prediction models, thus failing to meet the high-reliability prediction requirements of maritime systems.
A three-step preprocessing method of data cleaning, category detection and pattern reduction is adopted, combined with series modeling and interval verification. The time-frequency features of maritime data are extracted by WVD and Gram angle field transformation, matrix transformation models of signal transformation matrix and state transformation matrix are constructed, data consistency is enforced, and a maritime prediction model is constructed.
It improves the effectiveness of preprocessed data, enhances prediction accuracy, adapts to the dynamic changes of the maritime system, and meets the high reliability requirements of maritime operation scheduling and safety early warning.
Smart Images
Figure CN121031903B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of industrial data processing, text data processing, and big data mining, specifically to a method and apparatus for modeling and predicting maritime statistical data. Background Technology
[0002] With the rapid development of the global shipping industry, the operational efficiency and safety management of maritime systems (covering the three core links of crew, ports, and ships) increasingly rely on the accurate analysis and prediction of massive amounts of statistical data. Currently, maritime statistical data exhibits characteristics of being "multi-sourced, time-seriesd, and high-dimensional": crew operation data (such as watchkeeping hours and operation records), port operation data (such as berth occupancy rate and cargo throughput), and ship operation data (such as speed and equipment condition) are all continuously generated in time-series form. Furthermore, each type of data is further subdivided into operational signals (such as real-time operation commands) and status data (such as equipment static parameters), resulting in a complex data structure and significant dynamic changes. Existing maritime data modeling and prediction technologies face the following key technical challenges:
[0003] Insufficient targeting in data preprocessing: Maritime statistical data contains noise (such as instantaneous sensor errors) and outliers (such as parameter jumps caused by sudden equipment failures), and the time-series patterns of different data types (signal data / status data) vary greatly. Existing preprocessing methods mostly use general filtering or interpolation techniques, without designing adaptation schemes for the "temporal continuity" and "type differences" of maritime data. This results in biases in the preprocessed data, affecting the accuracy of subsequent modeling.
[0004] Inaccurate time series pattern description: The time series of maritime parameters (such as daily port throughput and hourly ship speed) often imply specific distribution patterns (such as periodic fluctuations under normal distribution). However, existing technologies lack correlation analysis of the "statistical distribution characteristics-pattern boundaries" of time series data, making it difficult to effectively define the normal range of data values, filter invalid data, or identify potential abnormal trends.
[0005] Poor feature extraction and model adaptability: Operational signal data (such as pulsed operation commands) have prominent time-frequency characteristics, and state data (such as equipment steady-state parameters) have significant spatial correlation characteristics. Existing technologies mostly use single feature extraction methods (such as time domain analysis only or frequency domain analysis only), which cannot capture the core features of the two types of data at the same time. Moreover, the inherent consistency of "signal-state" data is not considered in the modeling process, resulting in insufficient adaptability of the prediction model to the dynamic changes of the maritime system.
[0006] Insufficient robustness of prediction models: Existing prediction models are mostly based on the extrapolation of time-series trends of a single parameter, ignoring the correlation between "crew-port-ship" data and the coupling relationship between parameters in the maritime system. When the data in a certain link fluctuates, it is easy to cause the prediction results to deviate, which cannot meet the needs of high-reliability prediction in scenarios such as maritime operation scheduling and risk warning. Summary of the Invention
[0007] This invention primarily addresses the aforementioned problems in maritime data modeling and prediction, specifically how to accurately perform multi-dimensional prediction and modeling of maritime data. This invention discloses a method and apparatus for modeling and predicting maritime statistical data.
[0008] In a first aspect, this invention discloses a method for modeling and predicting maritime statistical data, comprising:
[0009] S1, collect a set of maritime statistical data; the set of maritime statistical data includes a subset of crew work operation data, a subset of port operation data, and a subset of ship operation data; each subset of operation data includes a subset of operation signals and a subset of status data; each subset includes a sequence of collection time data for several parameters; the sequence of collection time data is obtained by collecting parameters at preset time intervals;
[0010] S2, preprocess the maritime statistical data set to obtain a preprocessed dataset;
[0011] S3, perform prediction model construction processing on the preprocessed dataset to obtain maritime prediction model information; the maritime prediction model information is used to predict the future values of each data subset.
[0012] The preprocessing of the maritime statistical data set to obtain a preprocessed dataset includes:
[0013] S21, perform data cleaning on the maritime statistics dataset to obtain the first dataset;
[0014] S22, Perform category detection processing on the first dataset to obtain the second dataset;
[0015] S23, perform pattern reduction on the second dataset to obtain a preprocessed dataset.
[0016] The process of performing pattern reduction on the second dataset to obtain a preprocessed dataset includes:
[0017] The acquisition time data sequences of each type of parameter in the second dataset are subjected to series modeling to obtain the pattern information of each sequence;
[0018] Using the pattern information of each sequence, the corresponding acquisition time data sequence is verified to obtain the verified acquisition time data sequence.
[0019] A preprocessed dataset was constructed using all the validated time-series data collected.
[0020] The series modeling process includes:
[0021] Statistical distribution modeling was performed on the data sequence collected over time to obtain the mean, variance, probability distribution function, and corresponding test power value;
[0022] Based on the mean, variance, and power value, the quantile proportion factor α is calculated.
[0023] Based on the quantile ratio factor and probability distribution function, the upper quantile and lower quantile are obtained;
[0024] The upper and lower quantiles are determined as pattern information for the sequence.
[0025] The step of performing prediction model construction on the preprocessed dataset to obtain maritime prediction model information includes:
[0026] S31, Perform feature analysis on each parameter of the preprocessed dataset to obtain a set of time-frequency parameters;
[0027] S32, Perform statistical consistency analysis on the time-frequency parameter set to obtain the state prediction model for each parameter;
[0028] S33, using the state prediction model of all parameters, constructs the maritime prediction model information.
[0029] The feature analysis is performed on each parameter of the preprocessed dataset to obtain a set of time-frequency parameters, including:
[0030] S311, For the acquisition time data sequence of each parameter of each running signal subset of each type of running data subset, perform WVD transformation respectively to obtain the signal transformation matrix of each parameter;
[0031] S312, for the acquisition time data sequence of each parameter of each state data subset of each type of running data subset, perform Gram angle field transformation respectively to obtain the state transformation matrix of each parameter;
[0032] S313, using the signal transformation matrix and state transformation matrix of all parameters, constructs the time-frequency parameter set.
[0033] The step of performing statistical consistency analysis on the time-frequency parameter set to obtain a state prediction model for each parameter includes:
[0034] S321, using the signal transformation matrix and state transformation matrix with the same parameters, construct the matrix transformation model of the parameters;
[0035] S322, Solve the matrix transformation model for each parameter to obtain the corresponding prediction transformation matrix;
[0036] S323, based on the prediction transformation matrix of each parameter, constructs the state prediction model for each parameter.
[0037] A second aspect of the present invention discloses a modeling and prediction apparatus for maritime statistical data, the apparatus comprising:
[0038] Memory containing executable program code;
[0039] A processor coupled to the memory;
[0040] The processor calls the executable program code stored in the memory to execute the maritime statistical data modeling and prediction method.
[0041] In a third aspect of this invention, a computer-storable medium is disclosed, the computer-storable medium storing computer instructions, which, when invoked by a computer, are used to execute the aforementioned modeling and prediction method for maritime statistical data.
[0042] In a fourth aspect of this invention, an information data processing terminal is disclosed, which is used to implement the aforementioned method for modeling and predicting maritime statistical data.
[0043] The beneficial effects of this invention are as follows:
[0044] This invention breaks down preprocessing into three steps: data cleaning, category detection, and pattern reduction. In particular, the pattern reduction step introduces a "series modeling + interval verification" mechanism: statistical distribution modeling is used to determine the mean, variance, and confidence level of time-series data, and quantile proportion factors are calculated to define normal value ranges, accurately filtering noise and outliers. This approach is adapted to the time-series distribution characteristics of maritime data (such as parameter fluctuations under a normal distribution). Compared to general preprocessing methods, it improves the effectiveness of preprocessed data by more than 30%, providing a high-quality data foundation for subsequent modeling.
[0045] To address the type differences between "operational signals" and "state data" in maritime data, this invention extracts the "upper quantile - lower quantile" pattern information for each time series through series modeling, thereby quantitatively defining the normal fluctuation range of the data. For example, for a time series of ship main engine speeds, outliers with instantaneous jumps can be filtered out by quantile intervals, retaining steady-state data that conforms to a normal distribution. This process not only verifies the data but also uncovers the inherent patterns in the time series data, providing a pattern basis for feature extraction.
[0046] This invention employs WVD (Wigner-Variety Transform) to accurately extract the time-frequency coupling features of operational signal data (such as the pulse frequency and duration of operational commands); and Gram angle field transform to convert time-series data into a graphical matrix to capture the correlation between parameters (such as the spatiotemporal correlation between port berth occupancy and cargo throughput). The time-frequency parameter set constructed by combining these two types of transforms comprehensively covers the core features of maritime data, providing multi-dimensional information support for subsequent modeling.
[0047] By constructing a matrix transformation model of "signal transformation matrix - state transformation matrix", this invention enforces the statistical consistency of the two types of data (such as the matching relationship between ship navigation signals and equipment status) during the modeling process, avoiding prediction errors caused by single data deviations. Simultaneously, the model enhances its ability to fit parameter coupling relationships by minimizing the nonlinear objective function of matrix element differences, ensuring that the prediction results not only conform to the trend of a single parameter but also satisfy the overall operational logic of the maritime system, effectively improving prediction accuracy compared to existing models. Attached Figure Description
[0048] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention. Detailed Implementation
[0049] To better understand the content of this invention, an embodiment is provided here.
[0050] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention.
[0051] In a first aspect, this invention discloses a method for modeling and predicting maritime statistical data, comprising:
[0052] S1, collect a set of maritime statistical data; the set of maritime statistical data includes a subset of crew work operation data, a subset of port operation data, and a subset of ship operation data; each subset of operation data includes a subset of operation signals and a subset of status data; each subset includes a sequence of collection time data for several parameters; the sequence of collection time data is obtained by collecting parameters at preset time intervals;
[0053] S2, preprocess the maritime statistical data set to obtain a preprocessed dataset;
[0054] S3, perform prediction model construction processing on the preprocessed dataset to obtain maritime prediction model information; the maritime prediction model information is used to predict the future values of each data subset.
[0055] From data acquisition (subdividing into three subsets: crew, port, and vessel), preprocessing (pattern reduction to adapt to temporal distribution), feature extraction (transformation methods to match data types), to modeling (consistency constraints to fit system logic), each step of this invention is optimized for the specificities of maritime scenarios. It can be directly applied to practical scenarios such as maritime operation scheduling (e.g., port throughput forecasting), safety early warning (e.g., ship equipment failure trend forecasting), and crew scheduling optimization (e.g., watch load forecasting), providing precise decision support for maritime management.
[0056] The preprocessing of the maritime statistical data set to obtain a preprocessed dataset includes:
[0057] S21, perform data cleaning on the maritime statistics dataset to obtain the first dataset;
[0058] S22, Perform category detection processing on the first dataset to obtain the second dataset;
[0059] S23, Perform pattern reduction on the second dataset to obtain a preprocessed dataset;
[0060] The data cleaning process includes data smoothing, outlier removal, etc.
[0061] The category detection process involves determining whether the data attributes of each category in the first dataset are consistent with preset data attributes, and deleting inconsistent data from the first dataset.
[0062] The process of performing pattern reduction on the second dataset to obtain a preprocessed dataset includes:
[0063] The acquisition time data sequences of each type of parameter in the second dataset are subjected to series modeling to obtain the pattern information of each sequence;
[0064] Using the pattern information of each sequence, the corresponding acquisition time data sequence is verified to obtain the verified acquisition time data sequence.
[0065] A preprocessed dataset was constructed using all the validated time-series data collected.
[0066] The series modeling process includes:
[0067] Statistical distribution modeling was performed on the data sequence collected over time to obtain the mean, variance, probability distribution function, and corresponding test power value;
[0068] Based on the mean, variance, and power value, the quantile proportion factor α is calculated.
[0069] Based on the quantile ratio factor and probability distribution function, the upper quantile and lower quantile are obtained;
[0070] The upper and lower quantiles are determined as pattern information for the sequence;
[0071] The statistical distribution modeling involves assuming that the data sequence collected over time follows a normal distribution, performing statistical distribution modeling on it to obtain a probability distribution function, and then performing hypothesis testing on the probability distribution function to obtain the test power value.
[0072] The formula for calculating the quantile scaling factor α is as follows:
[0073] α = ε × sin(β / γ),
[0074] Where β and γ are the mean and variance, respectively, and ε is the power value of the test.
[0075] The step of obtaining the upper quantile and lower quantile based on the quantile proportion factor and the probability distribution function involves, specifically, determining the upper quantile and lower quantile of the corresponding probability distribution function within the quantile proportion factor, using the quantile proportion factor as α, μ(α) as the α quantile of the normal distribution, μ(1-α) as the 1-α quantile of the normal distribution, and μ as the probability distribution function, determining the smaller of μ(α) and μ(1-α) as the lower quantile, and determining the larger of μ(α) and μ(1-α) as the upper quantile.
[0076] The calculation of the quantile scaling factor incorporates the mean (β), reflecting the central tendency of the data, the variance (γ), reflecting the dispersion, and the confidence level (ε), reflecting the reliability, into the same calculation framework. Through the nonlinear mapping of the sine function, the quantile scaling factor is dynamically adjusted with the ratio of the mean to the variance (e.g., when the mean is much larger than the variance, the scaling factor shrinks, and the interval becomes more compact), and is also constrained by the confidence level (e.g., when ε increases at high confidence levels, the interval is appropriately widened), adapting to the characteristics of maritime data: "small fluctuations in steady state and large fluctuations in dynamic state." The range of the sine function is limited to [-1, 1]. Even if the ratio of the mean to the variance is too large (e.g., a sudden anomaly causes a sharp increase in variance), the sine function's suppressive effect can prevent excessive expansion of the quantile interval, ensuring the interval's coverage of normal data, while effectively excluding extreme outliers. This is particularly suitable for data such as port throughput and ship speed, which are easily affected by sudden factors (e.g., weather, congestion).
[0077] The ratio of mean to variance reflects the "stability" of the data (the larger the ratio, the more stable the data), while the confidence level reflects the "reliability" of the data. The combination of the two makes the calculation logic of the quantile ratio factor consistent with the intuitive understanding of the data by maritime personnel (e.g., for stable and reliable data, the interval should be more precise), which facilitates parameter debugging and result verification when the project is implemented.
[0078] The step of using the pattern information of each sequence to verify the corresponding acquisition time data sequence to obtain the verified acquisition time data sequence includes:
[0079] Determine whether each element of the acquired time data sequence is within the value range constructed by the upper and lower quantiles of the pattern information. Data that is not within the first discrimination interval is deleted from the acquired time data sequence to obtain the verified acquired time data sequence.
[0080] The step of performing prediction model construction on the preprocessed dataset to obtain maritime prediction model information includes:
[0081] S31, Perform feature analysis on each parameter of the preprocessed dataset to obtain a set of time-frequency parameters;
[0082] S32, Perform statistical consistency analysis on the time-frequency parameter set to obtain the state prediction model for each parameter;
[0083] S33, using the state prediction model of all parameters, constructs the maritime prediction model information.
[0084] The feature analysis is performed on each parameter of the preprocessed dataset to obtain a set of time-frequency parameters, including:
[0085] S311, For the acquisition time data sequence of each parameter of each running signal subset of each type of running data subset, perform WVD transformation respectively to obtain the signal transformation matrix of each parameter;
[0086] S312, for the acquisition time data sequence of each parameter of each state data subset of each type of running data subset, perform Gram angle field transformation respectively to obtain the state transformation matrix of each parameter;
[0087] S313, using the signal transformation matrix and state transformation matrix of all parameters, constructs the time-frequency parameter set.
[0088] The step of performing statistical consistency analysis on the time-frequency parameter set to obtain a state prediction model for each parameter includes:
[0089] S321, using the signal transformation matrix and state transformation matrix with the same parameters, construct the matrix transformation model of the parameters;
[0090] S322, Solve the matrix transformation model for each parameter to obtain the corresponding prediction transformation matrix;
[0091] S323, based on the prediction transformation matrix of each parameter, constructs the state prediction model for each parameter.
[0092] The expression for the matrix transformation model is:
[0093]
[0094] D = AB, BB T =E,
[0095] Where A is the signal transformation matrix, D is the intermediate matrix, F is the state transformation matrix, B is the prediction transformation matrix to be solved, E is the identity matrix, and D... ij and F ij are the elements in the i-th row and j-th column of the intermediate matrix and the state transformation matrix, respectively, and N and M are the row dimension and column dimension of the state transformation matrix, respectively.
[0096] The matrix transformation model calculates the average difference of each row of matrix elements using the Nth square root (geometric mean), avoiding the extreme deviation of a single element from dominating the objective function while retaining the difference contribution of all elements. Then, by using a sine function and scaling with 1 / π, the difference values are mapped to the [-1,1] interval, making the optimization direction of the objective function clearer (minimizing the sine value is equivalent to minimizing the overall difference), which is suitable for the system characteristics of "multi-parameter synergistic influence" of maritime data (such as the difference between ship signals and status needing to be evaluated as a whole, rather than a single parameter).
[0097] The constraints ensure that the prediction transformation matrix is invertible and numerically stable, avoiding singular values during matrix inversion (such as ill-conditioned matrices caused by the high dimensionality of maritime data). At the same time, the orthogonal transformation can preserve the inner product structure of the original data (i.e., the inherent correlation between signal and state data), so that the prediction model can fit the data differences without destroying the parameter coupling relationship of the maritime system.
[0098] The nonlinear (sine function + geometric mean) nature of the objective function makes it more sensitive to subtle differences in the data, and can capture "slowly accumulating small deviations" in maritime systems (such as the gradual decline in signal-state matching due to crew fatigue). Compared with linear objective functions (such as mean square error), it is easier to discover potential trend anomalies and provide support for early warning (such as precursors to equipment failure).
[0099] The state prediction model includes:
[0100] For the acquisition time data sequence of each parameter of each running signal subset of a class of running data subsets, WVD transformation is performed respectively to obtain the signal transformation matrix of each parameter;
[0101] The state transformation matrix of each parameter is multiplied by the prediction transformation matrix of the corresponding parameter to obtain the state prediction matrix of the corresponding parameter.
[0102] The WVD transformation is a transformation of the Wigner-Willi distribution;
[0103] The step of performing WVD transformation to obtain the signal transformation matrix for each parameter includes obtaining a two-dimensional signal through WVD transformation, and performing discrete sampling on the two-dimensional signal to obtain the signal transformation matrix for each parameter.
[0104] A second aspect of the present invention discloses a modeling and prediction apparatus for maritime statistical data, the apparatus comprising:
[0105] Memory containing executable program code;
[0106] A processor coupled to the memory;
[0107] The processor calls the executable program code stored in the memory to execute the maritime statistical data modeling and prediction method.
[0108] In a third aspect of this invention, a computer-storable medium is disclosed, the computer-storable medium storing computer instructions, which, when invoked by a computer, are used to execute the aforementioned modeling and prediction method for maritime statistical data.
[0109] In a fourth aspect of this invention, an information data processing terminal is disclosed, which is used to implement the aforementioned method for modeling and predicting maritime statistical data.
[0110] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.
Claims
1. A method of modeling and forecasting maritime statistics data, characterized in that, The method comprises the following steps: S1, collecting maritime statistical data sets; The maritime statistical data sets include crew operation data subsets, port operation data subsets, and ship operation data subsets; Each operation data subset includes operation signal subsets and state data subsets; each subset includes a plurality of parameter acquisition time data sequences; The acquisition time data sequences are obtained by collecting parameters at preset time intervals; S2, preprocessing the maritime statistical data sets to obtain a preprocessed data set, comprising: S21, performing data cleaning processing on the maritime statistical data sets to obtain a first data set; S22, performing category detection processing on the first data set to obtain a second data set; S23, performing mode reduction processing on the second data set to obtain a preprocessed data set, comprising: Performing series modeling processing on the acquisition time data sequence of each parameter of the second data set respectively to obtain mode information of each sequence; Using the mode information of each sequence, performing verification processing on the corresponding acquisition time data sequence to obtain a verified acquisition time data sequence; Using all verified acquisition time data sequences, a preprocessed data set is constructed; The series modeling processing comprises: Statistical distribution modeling is performed on the acquisition time data sequence to obtain mean, variance, probability distribution function, and corresponding test efficacy value; According to the mean, variance, and test efficacy value, a quantile proportion factor a is calculated; the calculation expression of the quantile proportion factor a is: a = ε × sin(β / γ), where β and γ are the mean and variance, and ε is the test efficacy value; According to the quantile proportion factor and the probability distribution function, upper and lower quantiles are obtained; The upper and lower quantiles are determined as the mode information of the sequence; S3, performing prediction model construction processing on the preprocessed data set to obtain maritime prediction model information; the maritime prediction model information is used to predict the future values of each data subset.
2. The method of claim 1, wherein, The prediction model construction processing on the preprocessed data set to obtain maritime prediction model information comprises: S31, performing feature analysis on each parameter of the preprocessed data set respectively to obtain a time-frequency parameter set; S32, performing statistical consistency analysis on the time-frequency parameter set to obtain a state prediction model for each parameter; S33, using the state prediction models of all parameters to construct maritime prediction model information.
3. The method of claim 2, wherein the statistical data is maritime statistical data. The feature analysis on each parameter of the preprocessed data set respectively to obtain a time-frequency parameter set comprises: S311, performing WVD transformation on the acquisition time data sequence of each parameter of each operation signal subset of each operation data subset to obtain a signal transformation matrix of each parameter; S312, performing Gram angle field transformation on the acquisition time data sequence of each parameter of each state data subset of each operation data subset to obtain a state transformation matrix of each parameter; S313, using all signal transformation matrices and state transformation matrices of parameters to construct a time-frequency parameter set.
4. The method of claim 2, wherein the statistical data is maritime statistical data. The statistical consistency analysis on the time-frequency parameter set is performed to obtain a state prediction model of each parameter, including: S321, a matrix transformation model of the parameter is constructed by using the signal transformation matrix and the state transformation matrix of the same parameter; S322, the matrix transformation model of each parameter is solved to obtain a corresponding prediction transformation matrix; S323, a state prediction model of each parameter is constructed based on the prediction transformation matrix of each parameter.
5. A device for modeling and predicting maritime statistics data, characterized by, The device comprises: a memory storing executable program codes; a processor coupled with the memory; the processor invokes the executable program codes stored in the memory to execute the modeling prediction method of maritime statistical data according to any one of claims 1 to 4.
6. A computer storable medium, characterized by The computer storage medium stores computer instructions, which are invoked by a computer to execute the modeling prediction method of maritime statistical data according to any one of claims 1 to 4.
7. An information data processing terminal, characterized by The information data processing terminal is used to implement the modeling prediction method of maritime statistical data according to any one of claims 1 to 4.