Logistics anomaly monitoring analysis method and system based on big data

Abstract

The invention provides a logistics abnormity monitoring analysis method and system based on big data, and the method comprises the steps: collecting multi-dimensional feature data in a logistics transportation process, and carrying out the standardization processing of the multi-dimensional feature data; obtaining a time window for judging logistics abnormity based on the multi-dimensional feature data; calculating a dynamic threshold value of the multi-dimensional feature data according to the time window, and if the numerical value of the multi-dimensional feature data is greater than or equal to the dynamic threshold value, judging that the data is abnormal; and training a logistics anomaly prediction model based on the abnormal logistics data, obtaining real-time multi-dimensional feature data, inputting the real-time multi-dimensional feature data into the logistics anomaly prediction model, and predicting whether logistics is abnormal or not. Through the method and the corresponding system, the accuracy and robustness of logistics anomaly detection are improved, the false alarm rate of logistics anomaly monitoring is reduced, and logistics anomaly monitoring is more prospective.

Description

Technical Field

[0001] The present invention proposes a logistics abnormality monitoring and analysis method and system based on big data, which relates to the technical field of logistics transportation monitoring and early warning. Background Art

[0002] Existing logistics anomaly monitoring and analysis methods based on big data usually use fixed time windows to calculate thresholds for anomaly detection. However, this method has several defects and limitations: insufficient flexibility, fixed time windows cannot adapt to dynamically changing logistics environments. For example, when logistics business volume fluctuates greatly, fixed windows may not be able to capture rapidly changing patterns and trends, resulting in inaccurate anomaly detection; a static threshold may not be suitable for different seasons, market conditions or sudden business changes. This method is prone to high false positives or missed positives due to improper threshold setting; or the inability to predict the occurrence of logistics problems in advance, resulting in obvious anomalies in logistics transportation being discovered and solved, and the problem cannot be suppressed in the cradle. Summary of the invention

[0003] The present invention provides a logistics abnormality monitoring and analysis method and system based on big data to solve the above-mentioned problems:

[0004] The present invention proposes a logistics abnormality monitoring and analysis method based on big data, the method comprising:

[0005] Collect multi-dimensional feature data in the process of logistics transportation, and perform standardization processing on the multi-dimensional feature data;

[0006] Obtain the time window for judging logistics anomalies based on multi-dimensional feature data;

[0007] Calculate the dynamic threshold of the multi-dimensional feature data according to the time window. If the value of the multi-dimensional feature data is greater than or equal to the dynamic threshold, it is judged as abnormal.

[0008] A logistics anomaly prediction model is trained based on abnormal logistics data to obtain real-time multi-dimensional feature data, and the real-time multi-dimensional feature data is input into the logistics anomaly prediction model to predict whether there will be abnormalities in the logistics.

[0009] Furthermore, multi-dimensional feature data in the logistics transportation process is collected, and the multi-dimensional feature data is standardized to obtain a comprehensive evaluation value of the multi-dimensional feature data in the logistics transportation process, including:

[0010] Collect multi-dimensional feature data in the logistics transportation process, wherein the multi-dimensional feature data includes: logistics route transportation time, logistics route transportation distance, logistics route transportation temperature, logistics route transportation cargo flow and logistics route transportation fuel consumption;

[0011] Using filling, deleting and interpolation methods to detect and process missing values ​​in the multidimensional feature data, and using the Z-score method to identify and delete outliers in the multidimensional feature data;

[0012] The format of the multidimensional feature data is unified, and the category features of the multidimensional feature data are converted into numerical form, including using one-hot encoding.

[0013] Furthermore, the time window for determining abnormalities is calculated based on the multi-dimensional feature data, including:

[0014] Determine the basic time window for judging whether there are abnormalities in logistics data;

[0015] The dynamic time window is calculated by a dynamic time window model. Specifically, the dynamic time window model is:

[0016]

[0017] Among them, t represents the time window, N represents the basic time window, β k represents the data change rate sensitivity coefficient, k represents the data category of multidimensional feature data, ΔR t-i,k represents the rate of change between time ti and ti-1 for data category k, n represents the number of data points of multidimensional feature data, W i represents the time frequency weight, γ represents the time frequency weight coefficient, λ represents the abnormal proportion coefficient, T represents the total number of data points of the multidimensional feature data, and A n Indicates the number of outliers in multidimensional feature data.

[0018] Furthermore, a dynamic threshold of the multi-dimensional feature data is calculated according to the time window. If the value of the multi-dimensional feature data is greater than or equal to the dynamic threshold, it is determined to be abnormal, including:

[0019] According to the time window, Threshold(t) = μ t +b·σ t Calculate the dynamic threshold of each data category in the multidimensional feature data, where Threshold(t) represents the dynamic threshold in the time window t, μ t The mean value, v, calculated within the time window t t represents the standard deviation calculated in the time window t, and b represents the sensitivity adjustment coefficient;

[0020] If there are data points in the corresponding data category that are greater than or equal to the corresponding dynamic threshold, it is determined that the logistics data is abnormal.

[0021] Furthermore, a logistics anomaly prediction model is trained based on the abnormal logistics data to obtain real-time multi-dimensional feature data, and the real-time multi-dimensional feature data is input into the logistics anomaly prediction model to predict whether there will be anomalies in the logistics, including:

[0022] Obtain logistics data sets containing historical anomalies;

[0023] Handle missing values, outliers, and other inconsistent data in logistics datasets;

[0024] Label abnormal and normal samples based on historical records to provide a basis for supervised learning for the model;

[0025] Perform feature conversion and extraction. The extracted features include: mean and standard deviation of transportation time, threshold overrun ratio, mean and standard deviation of transportation distance, deviation distance, temperature fluctuation, temperature deviation, mean and standard deviation of cargo flow, cargo flow peak, mean and standard deviation of fuel consumption, fuel consumption efficiency fluctuation, transportation efficiency index, and ratio of cargo transportation volume to fuel consumption per unit transportation time. The extracted features constitute a feature data set, and the feature data set is divided into a training set and a validation set.

[0026] Training an LSTM model using the training set containing the annotations, and adjusting hyperparameters of the LSTM model to optimize performance;

[0027] The performance of the LSTM model is evaluated by cross-validation, and the accuracy, recall and F1 score indicators are used to evaluate the LSTM model to obtain a trained LSTM model;

[0028] Input the real-time multi-dimensional feature data into the trained LSTM model, and the output result of the LSTM model identifies the properties of the logistics data, which includes abnormality and normality;

[0029] Trigger an alarm for data predicted to be abnormal, notify relevant personnel to take measures, and collect anomaly detection feedback information to improve the model. Anomaly detection feedback information includes case analysis of correct and incorrect identification.

[0030] The present invention proposes a logistics abnormality monitoring and analysis system based on big data, the system comprising:

[0031] A data collection module is used to collect multi-dimensional feature data in the logistics transportation process and perform standardization processing on the multi-dimensional feature data;

[0032] A time window acquisition module is used to acquire a time window for judging logistics anomalies based on multi-dimensional feature data;

[0033] A determination module is used to calculate a dynamic threshold of the multi-dimensional feature data according to a time window, and if the value of the multi-dimensional feature data is greater than or equal to the dynamic threshold, it is determined to be abnormal;

[0034] The anomaly prediction module is used to train a logistics anomaly prediction model based on abnormal logistics data, obtain real-time multi-dimensional feature data, input the real-time multi-dimensional feature data into the logistics anomaly prediction model, and predict whether there will be anomalies in the logistics.

[0035] Furthermore, the data collection module includes:

[0036] A multi-dimensional feature data collection module is used to collect multi-dimensional feature data in the logistics transportation process, wherein the multi-dimensional feature data includes: logistics line transportation time, logistics line transportation distance, logistics line transportation temperature, logistics line transportation cargo flow and logistics line transportation fuel consumption;

[0037] A data preprocessing module, used to detect and process missing values ​​in the multidimensional feature data by using filling, deletion and interpolation methods, and to identify and delete abnormal values ​​in the multidimensional feature data by using a Z-score method;

[0038] The unified format module is used to unify the format of the multidimensional feature data and convert the category features of the multidimensional feature data into a numerical form, including using one-hot encoding.

[0039] Furthermore, the acquisition time window module includes:

[0040] Determine the basic time window module, which is used to determine the basic time window for judging whether there is an abnormality in the logistics data;

[0041] The dynamic time window calculation module is used to calculate the dynamic time window through the dynamic time window model. Specifically, the dynamic time window model is:

[0042]

[0043] Among them, t represents the time window, N represents the basic time window, β k represents the data change rate sensitivity coefficient, k represents the data category of multidimensional feature data, ΔR t-i,k represents the rate of change between time ti and ti-1 for data category k, n represents the number of data points of multidimensional feature data, W i represents the time frequency weight, γ represents the time frequency weight coefficient, λ represents the abnormal proportion coefficient, T represents the total number of data points of the multidimensional feature data, and A n Indicates the number of outliers in multidimensional feature data.

[0044] Furthermore, the determination module includes:

[0045] The dynamic threshold calculation module is used to calculate the dynamic threshold according to the time window through Threshold(t)=μ t+b·σ t Calculate the dynamic threshold of each data category in the multidimensional feature data, where Threshold(t) represents the dynamic threshold in the time window t, μ t The mean value calculated within the time window t, ·σ t represents the standard deviation calculated in the time window t, and b represents the sensitivity adjustment coefficient;

[0046] The comparison module is used to determine that there is an anomaly in the logistics data if there are data points in the data of the corresponding data category that are greater than or equal to the corresponding dynamic threshold.

[0047] Furthermore, the abnormality prediction module includes:

[0048] The module for obtaining the initial data set is used to obtain the logistics data set containing historical anomalies;

[0049] Data cleaning module, used to process missing values, outliers and other inconsistent data in logistics data sets;

[0050] The labeling module is used to label abnormal and normal samples based on historical records, providing a basis for supervised learning for the model;

[0051] A feature extraction module is used for feature conversion and extraction. The extracted features include: mean and standard deviation of transportation time, threshold overlimit ratio, mean and standard deviation of transportation distance, deviation distance, temperature fluctuation, temperature deviation, mean and standard deviation of cargo flow, cargo flow peak, mean and standard deviation of fuel consumption, fuel consumption efficiency fluctuation, transportation efficiency index and ratio of cargo transportation volume to fuel consumption per unit transportation time. The extracted features constitute a feature data set, and the feature data set is divided into a training set and a validation set.

[0052] A training module, for training an LSTM model using a training set containing annotations, and adjusting hyperparameters of the LSTM model to optimize performance;

[0053] A tuning module is used to evaluate the performance of the LSTM model through cross-validation, and to evaluate the LSTM model using accuracy, recall and F1 score indicators to obtain a trained LSTM model;

[0054] A prediction module is used to input real-time multi-dimensional feature data into a trained LSTM model, and the output result of the LSTM model identifies the nature of the logistics data, which includes abnormality and normality;

[0055] The feedback adjustment module is used to trigger alarms for data predicted to be abnormal, notify relevant personnel to take measures, and collect anomaly detection feedback information to improve the model. The anomaly detection feedback information includes case analysis of correct identification and incorrect identification.

[0056] The beneficial effects of the present invention are as follows: by dynamically calculating thresholds and flexibly selecting time windows, the system can adapt to changes in the logistics environment and different business needs, thereby improving the accuracy and robustness of anomaly detection; the technical solution allows for instant analysis and processing of real-time logistics data, so as to detect potential problems early and take corrective measures before the problems worsen, thereby improving logistics operation efficiency; by training anomaly prediction models, the system can not only detect current anomalies, but also predict impending anomalies in advance, making logistics management more forward-looking and helping enterprises optimize resource allocation and risk management; reducing false alarm rates, and combining standardization with dynamic adjustment reduces unnecessary false alarms, optimizes the alarm mechanism, and saves time and cost for manual intervention; using data-driven methods to achieve efficient monitoring and prediction of logistics anomalies, providing technical support for improving the safety, reliability and efficiency of logistics transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a schematic diagram of a logistics anomaly monitoring and analysis method based on big data described in the present invention. DETAILED DESCRIPTION

[0058] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0059] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. The embodiments described are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0061] One embodiment of the present invention provides a logistics anomaly monitoring and analysis method based on big data, the method comprising:

[0062] Collect multi-dimensional feature data in the process of logistics transportation, and perform standardization processing on the multi-dimensional feature data;

[0063] Obtain the time window for judging logistics anomalies based on multi-dimensional feature data;

[0064] Calculate the dynamic threshold of the multi-dimensional feature data according to the time window. If the value of the multi-dimensional feature data is greater than or equal to the dynamic threshold, it is judged as abnormal.

[0065] A logistics anomaly prediction model is trained based on abnormal logistics data to obtain real-time multi-dimensional feature data, and the real-time multi-dimensional feature data is input into the logistics anomaly prediction model to predict whether there will be abnormalities in the logistics.

[0066] The working principle and effect of the above technical solution are as follows: during the logistics transportation process, the system will collect various relevant data in real time, including but not limited to multi-dimensional information such as transportation time, transportation distance, temperature, humidity, load, vehicle status, etc.; standardize the multi-dimensional feature data; intelligently select appropriate time windows according to the change pattern and historical trend of the multi-dimensional feature data to capture the abnormal situations that may occur in the short term in the logistics process; dynamically calculate the threshold value according to the statistical properties of the multi-dimensional feature data (such as mean, standard deviation, etc.) within the selected time window, and this threshold value can be adjusted with the change of time and data pattern; compare the multi-dimensional feature data collected in real time with the calculated dynamic threshold value, and if the feature value exceeds or equals the threshold value, the current logistics state is judged to be abnormal; use the previously detected abnormal data to train the logistics abnormality prediction model. The model can be trained using machine learning or deep learning algorithms (such as random forests, neural networks) to learn feature patterns from historical abnormal data; in practical applications, the multi-dimensional feature data collected in real time is input into the trained abnormality prediction model, and the model outputs the results to predict whether there is an abnormality in the current logistics process. Adaptability: By dynamically calculating thresholds and flexibly selecting time windows, the system can adapt to changes in the logistics environment and different business needs, improving the accuracy and robustness of anomaly detection; this technical solution allows instant analysis and processing of real-time logistics data to detect potential problems early and take corrective measures before the problems worsen, thereby improving logistics operation efficiency; by training anomaly prediction models, the system can not only detect current anomalies, but also predict upcoming anomalies in advance, making logistics management more forward-looking and helping enterprises optimize resource allocation and risk management; reducing false alarm rates: the combination of standardization and dynamic adjustment reduces unnecessary false alarms, optimizes the alarm mechanism, and saves time and cost for manual intervention; using data-driven methods to achieve efficient monitoring and prediction of logistics anomalies, providing technical support for improving the safety, reliability and efficiency of logistics transportation.

[0067] In one embodiment of the present invention, multidimensional feature data in a logistics transportation process is collected, and the multidimensional feature data is standardized to obtain a comprehensive evaluation value of the multidimensional feature data in the logistics transportation process, including:

[0068] Collect multi-dimensional feature data in the logistics transportation process, wherein the multi-dimensional feature data includes: logistics route transportation time, logistics route transportation distance, logistics route transportation temperature, logistics route transportation cargo flow and logistics route transportation fuel consumption;

[0069] Using filling, deleting and interpolation methods to detect and process missing values ​​in the multidimensional feature data, and using the Z-score method to identify and delete outliers in the multidimensional feature data;

[0070] The format of the multidimensional feature data is unified, and the category features of the multidimensional feature data are converted into numerical form, including using one-hot encoding.

[0071] The working principle and effect of the above technical solution are as follows: Collect multi-dimensional feature data in the logistics transportation process, including but not limited to the transportation time, transportation distance, transportation temperature, transportation cargo flow and fuel consumption of the logistics route. These data are derived from sensors, GPS and other data collection equipment in the logistics system; fill, delete and interpolate. For the missing values ​​that may exist in the collected multi-dimensional feature data, a variety of strategies are used to process: fill in the missing values ​​using mean filling, median filling or using the most recent valid data; delete: For some extremely missing records, you can choose to delete them to avoid affecting the overall data quality; interpolate: Apply linear interpolation or other interpolation methods to smooth the time series data and fill in the missing parts of the data; use Z-score to identify outliers: By calculating the Z-score (i.e., standard score) of each feature, identify those outliers that deviate from the mean by multiple standard deviations. Once the outliers are identified, they can be processed by deletion, replacement or truncation to reduce their interference with the data analysis results; ensure that all feature data use a unified numerical format for easy storage and analysis; categorical features are digitized, using One-Hot Encoding (One-Hot Encoding) converts any discrete categorical features into numerical form, which makes the categorical features suitable for numerical processing algorithms and facilitates subsequent analysis and model training. Improve data quality. Through systematic missing value filling, outlier processing and format unification, the integrity and consistency of data are significantly improved, thereby enhancing the reliability of analysis results and the accuracy of model prediction; accurately identify anomalies. The Z-score method is used to accurately identify outliers in logistics and transportation data, eliminating the negative impact of outliers on analysis and model training, and improving the robustness of the model; data parseability. By converting categorical features into numerical form, especially using unique hot encoding, further processing and modeling of multidimensional feature data becomes more direct, so that more types of analysis models can be used; enhance processing efficiency. For subsequent data analysis and modeling, the standardized and complete data provided by this solution can be used more efficiently by various machine learning algorithms, reducing the workload in the data preprocessing step; lay a solid foundation for logistics and transportation data analysis and prediction model training, and propose an automated and standardized data preprocessing pipeline to reduce the complexity of preprocessing and improve the effect and efficiency of subsequent analysis.

[0072] In one embodiment of the present invention, a time window for determining abnormality is calculated based on multi-dimensional feature data, including:

[0073] Determine the basic time window for judging whether there are abnormalities in logistics data;

[0074] The dynamic time window is calculated by a dynamic time window model. Specifically, the dynamic time window model is:

[0075]

[0076] Among them, t represents the time window, N represents the basic time window, β k represents the data change rate sensitivity coefficient, k represents the data category of multidimensional feature data, ΔR t-i,k represents the rate of change between time ti and ti-1 for data category k, n represents the number of data points of multidimensional feature data, W i represents the time frequency weight, γ represents the time frequency weight coefficient, λ represents the abnormal proportion coefficient, T represents the total number of data points of the multidimensional feature data, and A n Indicates the number of outliers in multidimensional feature data.

[0077]

[0078] Among them, t i and t i-1 Indicates the timestamp of two consecutive data sampling points, t i Indicates the timestamp of the i-th data sampling point, t i-1 represents the timestamp of the i-1th data sampling point, and α represents the control parameter.

[0079] The working principle and effect of the above technical solution are: According to the average change rate of each feature, the corresponding sensitivity coefficient is used for weighting. Determine how the dynamic changes of data on the horizontal plane affect the window size. Adjust the window size according to the weighted average of the data sampling time interval. This part is about the data frequency adjustment in the time dimension. Adjust the window size according to the proportion of abnormal data points to better reflect the impact of abnormal data; determine a basic time window, which is preset based on historical data or experience; modify the basic time window through the following factors to cope with the dynamic changes of the actual logistics environment. The data change rate sensitivity measures the rate of change of the multidimensional characteristic data of logistics between two consecutive time points. It affects the expansion or contraction of the time window through the accumulation and average of the change rate to reflect the abnormal degree of data fluctuation; the time frequency weight and coefficient are adjusted according to the frequency weight of data in different time periods. For example, high-frequency data may lead to a smaller time window to capture changes in time; the abnormal proportion coefficient is used to adjust the time window to reflect the proportion of outliers in the data set, and the ratio of the number of abnormal points to the total number of data points is used as the basis for adjustment; the dynamic time window (t) is calculated by a formula that takes into account the rate of change, time frequency and abnormal proportion, and each part provides a different weighted correction for the basic time window; the calculated dynamic time window determines which data will be used to determine whether there is an anomaly within this time period, thereby improving the accuracy and real-time performance of detection; to enhance adaptability, the dynamic time window model is based on the actual observed data fluctuations and In case of abnormal situations, the detection time window is adaptively adjusted to make the system more adaptable to different business scenarios and environments; to improve detection accuracy, through a dynamic adjustment mechanism that takes into account the influence of multiple factors, the solution effectively reduces the false alarm rate while ensuring that no abnormalities are missed, especially in the ever-changing and complex and diverse logistics environment, it can more accurately reflect abnormal situations; when the characteristics of logistics data fluctuate or a large number of abnormalities occur, the time window can be quickly adjusted to facilitate the timely capture and processing of logistics abnormal events, thereby improving the response efficiency of logistics management; more efficient anomaly detection avoids unnecessary calculations within a time range, thereby saving computing resources and improving the efficiency of the overall system; it provides an intelligent time window dynamic adjustment method for logistics abnormality monitoring, and improves the anomaly detection capability through an adaptive mechanism, so that it can more efficiently serve the complex logistics operation environment.

[0080] In one embodiment of the present invention, a dynamic threshold of multidimensional feature data is calculated according to a time window, and if the value of the multidimensional feature data is greater than or equal to the dynamic threshold, it is determined to be abnormal, including:

[0081] According to the time window, Threshold(t) = μ t +b·σ t Calculate the dynamic threshold of each data category in the multidimensional feature data, where Threshold(t) represents the dynamic threshold in the time window t, μ t The mean value calculated within the time window t, σ t represents the standard deviation calculated in the time window t, and b represents the sensitivity adjustment coefficient;

[0082] If there are data points in the corresponding data category that are greater than or equal to the corresponding dynamic threshold, it is determined that the logistics data is abnormal.

[0083] The working principle and effect of the above technical solution are as follows: within the set time window, collect multi-dimensional feature data, which may include transportation time, temperature, logistics flow, fuel consumption, etc.; within the selected time window, calculate the mean and standard deviation of each data category, and these statistical parameters are the basis for setting dynamic thresholds; for time window t, calculate the dynamic threshold of each feature in the corresponding time window according to the formula, where b is the sensitivity adjustment coefficient, which is used to control the sensitivity of the threshold. Appropriate selection of b can affect the degree of judgment of anomalies. A larger b will make the threshold higher, reduce false positives, but may increase missed reports; comparative detection, for each calculated dynamic threshold, check the multi-dimensional feature data within the time window. If the data point of a feature is equal to or exceeds its corresponding dynamic threshold, it is considered that the data category is abnormal; once an anomaly is detected, the system will mark the data category as abnormal and may trigger corresponding subsequent processing and response mechanisms. Real-time and dynamic adaptation: by dynamically calculating the threshold, the system can adjust itself according to the current logistics situation, improve the sensitivity to data fluctuations, and respond to abnormal situations in real time; improve detection accuracy: the dynamic threshold utilizes the statistical characteristics of the data in the time window (mean and standard deviation), which is more closely related to the actual data than the fixed threshold, thus improving the accuracy of reflecting the actual abnormal situation; by adjusting the sensitivity coefficient b, the system can flexibly strike a balance between false alarms (false positives) and missed alarms (false negatives), which is suitable for application in diverse business needs and environments; in different logistics conditions and environments, dynamically adjusting the threshold can effectively reduce misjudgments due to short-term fluctuations or anomalies and optimize the alarm mechanism; it has stronger adaptability in a multi-dimensional data environment, including responses to seasonal changes and special logistics tasks, without the need to re-adjust parameters and make large-scale changes to the system; by calculating and using dynamic thresholds to enhance the ability of logistics anomaly detection, the system's adaptability to environmental changes and sensitivity to abnormal situations are improved, integrating real-time, accuracy and flexibility, and providing a more powerful decision support tool for logistics management.

[0084] In one embodiment of the present invention, a logistics anomaly prediction model is trained based on abnormal logistics data to obtain real-time multi-dimensional feature data, and the real-time multi-dimensional feature data is input into the logistics anomaly prediction model to predict whether there will be anomalies in logistics, including:

[0085] Obtain logistics data sets containing historical anomalies;

[0086] Handle missing values, outliers, and other inconsistent data in logistics datasets;

[0087] Label abnormal and normal samples based on historical records to provide a basis for supervised learning for the model;

[0088] Perform feature conversion and extraction. The extracted features include: mean and standard deviation of transportation time, threshold overrun ratio, mean and standard deviation of transportation distance, deviation distance, temperature fluctuation, temperature deviation, mean and standard deviation of cargo flow, cargo flow peak, mean and standard deviation of fuel consumption, fuel consumption efficiency fluctuation, transportation efficiency index, and ratio of cargo transportation volume to fuel consumption per unit transportation time. The extracted features constitute a feature data set, and the feature data set is divided into a training set and a validation set.

[0089] Training an LSTM model using the training set containing the annotations, and adjusting hyperparameters of the LSTM model to optimize performance;

[0090] The performance of the LSTM model is evaluated by cross-validation, and the accuracy, recall and F1 score indicators are used to evaluate the LSTM model to obtain a trained LSTM model;

[0091] Input the real-time multi-dimensional feature data into the trained LSTM model, and the output result of the LSTM model identifies the properties of the logistics data, which includes abnormality and normality;

[0092] Trigger an alarm for data predicted to be abnormal, notify relevant personnel to take measures, and collect anomaly detection feedback information to improve the model. Anomaly detection feedback information includes case analysis of correct and incorrect identification.

[0093] The working principle and effect of the above technical solution are as follows: obtain a logistics data set containing historical anomalies, which may come from a logistics tracking system to record various indicators during the transportation process; process missing values, outliers and other data inconsistencies, and make the quality of the data set meet the modeling requirements through filling, deletion or interpolation strategies; mark abnormal and normal samples in the data according to historical records to provide a basis for subsequent supervised learning; calculate and extract features such as the mean and standard deviation of transportation time, threshold overlimit ratio, mean and standard deviation of transportation distance, deviation distance, temperature fluctuation and deviation, mean and peak value of cargo flow, mean fuel consumption, fluctuation of fuel consumption efficiency, and transportation efficiency index, which help to reveal abnormal patterns in the logistics process; construct a feature data set with the extracted features and divide it into a training set and a validation set for model training and evaluation; use the labeled training set to train long and short The Long-Term Memory Network (LSTM) model is good at processing time series data and is suitable for situations where logistics characteristics change over time. The model performance is optimized by adjusting the hyperparameters of the LSTM model (such as learning rate, number of hidden layers, time step, etc.). The model performance is evaluated through cross-validation technology, and the model's anomaly detection ability is comprehensively evaluated using indicators such as accuracy, recall, and F1 score. The training parameters are repeatedly adjusted according to the evaluation indicators to improve the model's ability to identify anomalies and distinguish between normal situations. Real-time multi-dimensional feature data is input into the trained LSTM model, and the model output results are used to identify whether the logistics data is normal. For data predicted to be abnormal, the model triggers an early warning mechanism to notify relevant personnel for rapid disposal. By analyzing actual feedback information and cases, the model is improved and improved, especially the false positives and missed negatives, to improve the accuracy of future detection. Enhance detection accuracy, use the LSTM model to detect anomalies in time series data, and ensure higher detection accuracy and timeliness; improve response efficiency, the system provides real-time warnings, can promptly detect and respond to anomalies, and help logistics operators take quick measures to reduce potential losses; dynamic adaptability, by acquiring and processing input data in real time, the LSTM model can dynamically adapt to changes in different transportation environments, improving the flexibility of the system; feedback-driven optimization, by analyzing and applying the feedback information of the alarm, continuously improve the model performance, form a closed-loop improvement process, and enable the detection system to continuously learn and optimize; resource optimization allocation, through automated anomaly detection and warning, reduces the need for manual monitoring, improves resource utilization efficiency, and also improves the accuracy and reliability of decision-making; use machine learning and data analysis technology to achieve efficient and accurate detection of logistics anomalies, and improve the operational efficiency and risk management capabilities of logistics companies.

[0094] One embodiment of the present invention is a logistics anomaly monitoring and analysis system based on big data, the system comprising:

[0095] A data collection module is used to collect multi-dimensional feature data in the logistics transportation process and perform standardization processing on the multi-dimensional feature data;

[0096] A time window acquisition module is used to acquire a time window for judging logistics anomalies based on multi-dimensional feature data;

[0097] A determination module is used to calculate a dynamic threshold of the multi-dimensional feature data according to a time window, and if the value of the multi-dimensional feature data is greater than or equal to the dynamic threshold, it is determined to be abnormal;

[0098] The anomaly prediction module is used to train a logistics anomaly prediction model based on abnormal logistics data, obtain real-time multi-dimensional feature data, input the real-time multi-dimensional feature data into the logistics anomaly prediction model, and predict whether there will be anomalies in the logistics.

[0099] The working principle and effect of the above technical solution are as follows: during the logistics transportation process, the system will collect various relevant data in real time, including but not limited to multi-dimensional information such as transportation time, transportation distance, temperature, humidity, load, vehicle status, etc.; standardize the multi-dimensional feature data; intelligently select appropriate time windows according to the change pattern and historical trend of the multi-dimensional feature data to capture the abnormal situations that may occur in the short term in the logistics process; dynamically calculate the threshold value according to the statistical properties of the multi-dimensional feature data (such as mean, standard deviation, etc.) within the selected time window, and this threshold value can be adjusted with the change of time and data pattern; compare the multi-dimensional feature data collected in real time with the calculated dynamic threshold value, and if the feature value exceeds or equals the threshold value, the current logistics state is judged to be abnormal; use the previously detected abnormal data to train the logistics abnormality prediction model. The model can be trained using machine learning or deep learning algorithms (such as random forests, neural networks) to learn feature patterns from historical abnormal data; in practical applications, the multi-dimensional feature data collected in real time is input into the trained abnormality prediction model, and the model outputs the results to predict whether there is an abnormality in the current logistics process. Adaptability: By dynamically calculating thresholds and flexibly selecting time windows, the system can adapt to changes in the logistics environment and different business needs, improving the accuracy and robustness of anomaly detection; this technical solution allows instant analysis and processing of real-time logistics data to detect potential problems early and take corrective measures before the problems worsen, thereby improving logistics operation efficiency; by training anomaly prediction models, the system can not only detect current anomalies, but also predict upcoming anomalies in advance, making logistics management more forward-looking and helping enterprises optimize resource allocation and risk management; reducing false alarm rates: the combination of standardization and dynamic adjustment reduces unnecessary false alarms, optimizes the alarm mechanism, and saves time and cost for manual intervention; using data-driven methods to achieve efficient monitoring and prediction of logistics anomalies, providing technical support for improving the safety, reliability and efficiency of logistics transportation.

[0100] In one embodiment of the present invention, the data collection module includes:

[0101] A multi-dimensional feature data collection module is used to collect multi-dimensional feature data in the logistics transportation process, wherein the multi-dimensional feature data includes: logistics line transportation time, logistics line transportation distance, logistics line transportation temperature, logistics line transportation cargo flow and logistics line transportation fuel consumption;

[0102] A data preprocessing module, used to detect and process missing values ​​in the multidimensional feature data by using filling, deletion and interpolation methods, and to identify and delete abnormal values ​​in the multidimensional feature data by using a Z-score method;

[0103] The unified format module is used to unify the format of the multidimensional feature data and convert the category features of the multidimensional feature data into a numerical form, including using one-hot encoding.

[0104] The working principle and effect of the above technical solution are as follows: Collect multi-dimensional feature data in the logistics transportation process, including but not limited to the transportation time, transportation distance, transportation temperature, transportation cargo flow and fuel consumption of the logistics route. These data are derived from sensors, GPS and other data collection equipment in the logistics system; fill, delete and interpolate. For the missing values ​​that may exist in the collected multi-dimensional feature data, a variety of strategies are used to process: fill in the missing values ​​using mean filling, median filling or using the most recent valid data; delete: For some extremely missing records, you can choose to delete them to avoid affecting the overall data quality; interpolate: Apply linear interpolation or other interpolation methods to smooth the time series data and fill in the missing parts of the data; use Z-score to identify outliers: By calculating the Z-score (i.e., standard score) of each feature, identify those outliers that deviate from the mean by multiple standard deviations. Once the outliers are identified, they can be processed by deletion, replacement or truncation to reduce their interference with the data analysis results; ensure that all feature data use a unified numerical format for easy storage and analysis; categorical features are digitized, using One-Hot Encoding (One-Hot Encoding) converts any discrete categorical features into numerical form, which makes the categorical features suitable for numerical processing algorithms and facilitates subsequent analysis and model training. Improve data quality. Through systematic missing value filling, outlier processing and format unification, the integrity and consistency of data are significantly improved, thereby enhancing the reliability of analysis results and the accuracy of model prediction; accurately identify anomalies. The Z-score method is used to accurately identify outliers in logistics and transportation data, eliminating the negative impact of outliers on analysis and model training, and improving the robustness of the model; data parseability. By converting categorical features into numerical form, especially using unique hot encoding, further processing and modeling of multidimensional feature data becomes more direct, so that more types of analysis models can be used; enhance processing efficiency. For subsequent data analysis and modeling, the standardized and complete data provided by this solution can be used more efficiently by various machine learning algorithms, reducing the workload in the data preprocessing step; lay a solid foundation for logistics and transportation data analysis and prediction model training, and propose an automated and standardized data preprocessing pipeline to reduce the complexity of preprocessing and improve the effect and efficiency of subsequent analysis.

[0105] In one embodiment of the present invention, the module for obtaining a time window includes:

[0106] Determine the basic time window module, which is used to determine the basic time window for judging whether there is an abnormality in the logistics data;

[0107] The dynamic time window calculation module is used to calculate the dynamic time window through the dynamic time window model. Specifically, the dynamic time window model is:

[0108]

[0109] Among them, t represents the time window, N represents the basic time window, β k represents the data change rate sensitivity coefficient, k represents the data category of multidimensional feature data, ΔR t-i,k represents the rate of change between time ti and ti-1 for data category k, n represents the number of data points of multidimensional feature data, W i represents the time frequency weight, γ represents the time frequency weight coefficient, λ represents the abnormal proportion coefficient, T represents the total number of data points of the multidimensional feature data, and A n Indicates the number of outliers in multidimensional feature data.

[0110]

[0111] Among them, t i and t i-1 Indicates the timestamp of two consecutive data sampling points, t i Indicates the timestamp of the i-th data sampling point, t i-1 represents the timestamp of the i-1th data sampling point, and α represents the control parameter.

[0112] The working principle and effect of the above technical solution are: According to the average change rate of each feature, the corresponding sensitivity coefficient is used for weighting. Determine how the dynamic changes of data on the horizontal plane affect the window size. Adjust the window size according to the weighted average of the data sampling time interval. This part is about the data frequency adjustment in the time dimension. Adjust the window size according to the proportion of abnormal data points to better reflect the impact of abnormal data; determine a basic time window, which is preset based on historical data or experience; modify the basic time window through the following factors to cope with the dynamic changes of the actual logistics environment. The data change rate sensitivity measures the rate of change of the multidimensional characteristic data of logistics between two consecutive time points. It affects the expansion or contraction of the time window through the accumulation and average of the change rate to reflect the abnormal degree of data fluctuation; the time frequency weight and coefficient are adjusted according to the frequency weight of data in different time periods. For example, high-frequency data may lead to a smaller time window to capture changes in time; the abnormal proportion coefficient is used to adjust the time window to reflect the proportion of outliers in the data set, and the ratio of the number of abnormal points to the total number of data points is used as the basis for adjustment; the dynamic time window (t) is calculated by a formula that takes into account the rate of change, time frequency and abnormal proportion, and each part provides a different weighted correction for the basic time window; the calculated dynamic time window determines which data will be used to determine whether there is an anomaly within this time period, thereby improving the accuracy and real-time performance of detection; to enhance adaptability, the dynamic time window model is based on the actual observed data fluctuations and In case of abnormal situations, the detection time window is adaptively adjusted to make the system more adaptable to different business scenarios and environments; to improve detection accuracy, through a dynamic adjustment mechanism that takes into account the influence of multiple factors, the solution effectively reduces the false alarm rate while ensuring that no abnormalities are missed, especially in the ever-changing and complex and diverse logistics environment, it can more accurately reflect abnormal situations; when the characteristics of logistics data fluctuate or a large number of abnormalities occur, the time window can be quickly adjusted to facilitate the timely capture and processing of logistics abnormal events, thereby improving the response efficiency of logistics management; more efficient anomaly detection avoids unnecessary calculations within a time range, thereby saving computing resources and improving the efficiency of the overall system; it provides an intelligent time window dynamic adjustment method for logistics abnormality monitoring, and improves the anomaly detection capability through an adaptive mechanism, so that it can more efficiently serve the complex logistics operation environment.

[0113] In one embodiment of the present invention, the determination module includes:

[0114] The dynamic threshold calculation module is used to calculate the dynamic threshold according to the time window through Threshold(t)=μ t +b·σ t Calculate the dynamic threshold of each data category in the multidimensional feature data, where Threshold(t) represents the dynamic threshold in the time window t, μ t The mean value calculated within the time window t, ·σ t represents the standard deviation calculated in the time window t, and b represents the sensitivity adjustment coefficient;

[0115] The comparison module is used to determine that there is an anomaly in the logistics data if there are data points in the data of the corresponding data category that are greater than or equal to the corresponding dynamic threshold.

[0116] The working principle and effect of the above technical solution are as follows: within the set time window, collect multi-dimensional feature data, which may include transportation time, temperature, logistics flow, fuel consumption, etc.; within the selected time window, calculate the mean and standard deviation of each data category, and these statistical parameters are the basis for setting dynamic thresholds; for time window t, calculate the dynamic threshold of each feature in the corresponding time window according to the formula, where b is the sensitivity adjustment coefficient, which is used to control the sensitivity of the threshold. Appropriate selection of b can affect the degree of judgment of anomalies. A larger b will make the threshold higher, reduce false positives, but may increase missed reports; comparative detection, for each calculated dynamic threshold, check the multi-dimensional feature data within the time window. If the data point of a feature is equal to or exceeds its corresponding dynamic threshold, it is considered that the data category is abnormal; once an anomaly is detected, the system will mark the data category as abnormal and may trigger corresponding subsequent processing and response mechanisms. Real-time and dynamic adaptation: by dynamically calculating the threshold, the system can adjust itself according to the current logistics situation, improve the sensitivity to data fluctuations, and respond to abnormal situations in real time; improve detection accuracy: the dynamic threshold utilizes the statistical characteristics of the data in the time window (mean and standard deviation), which is more closely related to the actual data than the fixed threshold, thus improving the accuracy of reflecting the actual abnormal situation; by adjusting the sensitivity coefficient b, the system can flexibly strike a balance between false alarms (false positives) and missed alarms (false negatives), which is suitable for application in diverse business needs and environments; in different logistics conditions and environments, dynamically adjusting the threshold can effectively reduce misjudgments due to short-term fluctuations or anomalies and optimize the alarm mechanism; it has stronger adaptability in a multi-dimensional data environment, including responses to seasonal changes and special logistics tasks, without the need to re-adjust parameters and make large-scale changes to the system; by calculating and using dynamic thresholds to enhance the ability of logistics anomaly detection, the system's adaptability to environmental changes and sensitivity to abnormal situations are improved, integrating real-time, accuracy and flexibility, and providing a more powerful decision support tool for logistics management.

[0117] In one embodiment of the present invention, the abnormality prediction module includes:

[0118] The module for obtaining the initial data set is used to obtain the logistics data set containing historical anomalies;

[0119] Data cleaning module, used to process missing values, outliers and other inconsistent data in logistics data sets;

[0120] The labeling module is used to label abnormal and normal samples based on historical records, providing a basis for supervised learning for the model;

[0121] A feature extraction module is used for feature conversion and extraction. The extracted features include: mean and standard deviation of transportation time, threshold overlimit ratio, mean and standard deviation of transportation distance, deviation distance, temperature fluctuation, temperature deviation, mean and standard deviation of cargo flow, cargo flow peak, mean and standard deviation of fuel consumption, fuel consumption efficiency fluctuation, transportation efficiency index and ratio of cargo transportation volume to fuel consumption per unit transportation time. The extracted features constitute a feature data set, and the feature data set is divided into a training set and a validation set.

[0122] A training module, for training an LSTM model using a training set containing annotations, and adjusting hyperparameters of the LSTM model to optimize performance;

[0123] A tuning module is used to evaluate the performance of the LSTM model through cross-validation, and to evaluate the LSTM model using accuracy, recall and F1 score indicators to obtain a trained LSTM model;

[0124] A prediction module is used to input real-time multi-dimensional feature data into a trained LSTM model, and the output result of the LSTM model identifies the nature of the logistics data, which includes abnormality and normality;

[0125] The feedback adjustment module is used to trigger alarms for data predicted to be abnormal, notify relevant personnel to take measures, and collect anomaly detection feedback information to improve the model. The anomaly detection feedback information includes case analysis of correct identification and incorrect identification.

[0126] The working principle and effect of the above technical solution are as follows: obtain a logistics data set containing historical anomalies, which may come from a logistics tracking system to record various indicators during the transportation process; process missing values, outliers and other data inconsistencies, and make the quality of the data set meet the modeling requirements through filling, deletion or interpolation strategies; mark abnormal and normal samples in the data according to historical records to provide a basis for subsequent supervised learning; calculate and extract features such as the mean and standard deviation of transportation time, threshold overlimit ratio, mean and standard deviation of transportation distance, deviation distance, temperature fluctuation and deviation, mean and peak value of cargo flow, mean fuel consumption, fluctuation of fuel consumption efficiency, and transportation efficiency index, which help to reveal abnormal patterns in the logistics process; construct a feature data set with the extracted features and divide it into a training set and a validation set for model training and evaluation; use the labeled training set to train long and short The Long-Term Memory Network (LSTM) model is good at processing time series data and is suitable for situations where logistics characteristics change over time. The model performance is optimized by adjusting the hyperparameters of the LSTM model (such as learning rate, number of hidden layers, time step, etc.). The model performance is evaluated through cross-validation technology, and the model's anomaly detection ability is comprehensively evaluated using indicators such as accuracy, recall, and F1 score. The training parameters are repeatedly adjusted according to the evaluation indicators to improve the model's ability to identify anomalies and distinguish between normal situations. Real-time multi-dimensional feature data is input into the trained LSTM model, and the model output results are used to identify whether the logistics data is normal. For data predicted to be abnormal, the model triggers an early warning mechanism to notify relevant personnel for rapid disposal. By analyzing actual feedback information and cases, the model is improved and improved, especially the false positives and missed negatives, to improve the accuracy of future detection. Enhance detection accuracy, use the LSTM model to detect anomalies in time series data, and ensure higher detection accuracy and timeliness; improve response efficiency, the system provides real-time warnings, can promptly detect and respond to anomalies, and help logistics operators take quick measures to reduce potential losses; dynamic adaptability, by acquiring and processing input data in real time, the LSTM model can dynamically adapt to changes in different transportation environments, improving the flexibility of the system; feedback-driven optimization, by analyzing and applying the feedback information of the alarm, continuously improve the model performance, form a closed-loop improvement process, and enable the detection system to continuously learn and optimize; resource optimization allocation, through automated anomaly detection and warning, reduces the need for manual monitoring, improves resource utilization efficiency, and also improves the accuracy and reliability of decision-making; use machine learning and data analysis technology to achieve efficient and accurate detection of logistics anomalies, and improve the operational efficiency and risk management capabilities of logistics companies.

[0127] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A logistics abnormality monitoring and analysis method based on big data, characterized in that: The method comprises: Collect multi-dimensional feature data in the process of logistics transportation, and perform standardization processing on the multi-dimensional feature data; Obtain the time window for judging logistics anomalies based on multi-dimensional feature data; Calculate the dynamic threshold of the multi-dimensional feature data according to the time window. If the value of the multi-dimensional feature data is greater than or equal to the dynamic threshold, it is judged as abnormal. A logistics anomaly prediction model is trained based on abnormal logistics data to obtain real-time multi-dimensional feature data, and the real-time multi-dimensional feature data is input into the logistics anomaly prediction model to predict whether there will be abnormalities in the logistics.

2. According to the big data-based logistics abnormality monitoring and analysis method of claim 1, it is characterized in that: Collect multi-dimensional feature data in the logistics transportation process, standardize the multi-dimensional feature data, and obtain a comprehensive evaluation value of the multi-dimensional feature data in the logistics transportation process, including: Collect multi-dimensional feature data in the logistics transportation process, wherein the multi-dimensional feature data includes: logistics route transportation time, logistics route transportation distance, logistics route transportation temperature, logistics route transportation cargo flow and logistics route transportation fuel consumption; Using filling, deleting and interpolation methods to detect and process missing values ​​in the multidimensional feature data, and using the Z-score method to identify and delete outliers in the multidimensional feature data; The format of the multidimensional feature data is unified, and the category features of the multidimensional feature data are converted into numerical form, including using one-hot encoding.

3. According to the big data-based logistics abnormality monitoring and analysis method of claim 1, it is characterized in that: The time window for determining abnormalities is calculated based on multi-dimensional feature data, including: Determine the basic time window for judging whether there are abnormalities in logistics data; The dynamic time window is calculated by a dynamic time window model. Specifically, the dynamic time window model is: Among them, t represents the time window, N represents the basic time window, β k represents the data change rate sensitivity coefficient, k represents the data category of multidimensional feature data, ΔR t-i,k represents the rate of change between time ti and ti-1 for data category k, n represents the number of data points of multidimensional feature data, W i represents the time frequency weight, γ represents the time frequency weight coefficient, λ represents the abnormal proportion coefficient, T represents the total number of data points of the multidimensional feature data, and A n Indicates the number of outliers in multidimensional feature data.

4. According to the big data-based logistics abnormality monitoring and analysis method of claim 1, it is characterized in that: The dynamic threshold of the multi-dimensional feature data is calculated according to the time window. If the value of the multi-dimensional feature data is greater than or equal to the dynamic threshold, it is judged as abnormal, including: According to the time window, Threshold(t) = μ t +b·σ t Calculate the dynamic threshold of each data category in the multidimensional feature data, where Threshold(t) represents the dynamic threshold in the time window t, μ t The mean value calculated within the time window t, σ t represents the standard deviation calculated in the time window t, and b represents the sensitivity adjustment coefficient; If there are data points in the corresponding data category that are greater than or equal to the corresponding dynamic threshold, it is determined that the logistics data is abnormal.

5. According to the big data-based logistics abnormality monitoring and analysis method of claim 1, it is characterized in that: Training a logistics anomaly prediction model based on abnormal logistics data, obtaining real-time multi-dimensional feature data, inputting the real-time multi-dimensional feature data into the logistics anomaly prediction model, and predicting whether there will be anomalies in logistics, including: Obtain logistics data sets containing historical anomalies; Handle missing values, outliers, and other inconsistent data in logistics datasets; Label abnormal and normal samples based on historical records to provide a basis for supervised learning for the model; Perform feature conversion and extraction. The extracted features include: mean and standard deviation of transportation time, threshold overrun ratio, mean and standard deviation of transportation distance, deviation distance, temperature fluctuation, temperature deviation, mean and standard deviation of cargo flow, cargo flow peak, mean and standard deviation of fuel consumption, fuel consumption efficiency fluctuation, transportation efficiency index, and ratio of cargo transportation volume to fuel consumption per unit transportation time. The extracted features constitute a feature data set, and the feature data set is divided into a training set and a validation set. Training an LSTM model using the training set containing the annotations, and adjusting hyperparameters of the LSTM model to optimize performance; The performance of the LSTM model is evaluated by cross-validation, and the accuracy, recall and F1 score indicators are used to evaluate the LSTM model to obtain a trained LSTM model; Input the real-time multi-dimensional feature data into the trained LSTM model, and the output result of the LSTM model identifies the properties of the logistics data, which includes abnormality and normality; Trigger an alarm for data predicted to be abnormal, notify relevant personnel to take measures, and collect anomaly detection feedback information to improve the model. Anomaly detection feedback information includes case analysis of correct and incorrect identification.

6. A logistics abnormality monitoring and analysis system based on big data, characterized in that: The system comprises: A data collection module is used to collect multi-dimensional feature data in the logistics transportation process and perform standardization processing on the multi-dimensional feature data; A time window acquisition module is used to acquire a time window for determining logistics anomalies based on multi-dimensional feature data; A determination module is used to calculate a dynamic threshold of the multi-dimensional feature data according to a time window, and if the value of the multi-dimensional feature data is greater than or equal to the dynamic threshold, it is determined to be abnormal; The anomaly prediction module is used to train a logistics anomaly prediction model based on abnormal logistics data, obtain real-time multi-dimensional feature data, input the real-time multi-dimensional feature data into the logistics anomaly prediction model, and predict whether there will be anomalies in the logistics.

7. According to the big data-based logistics abnormality monitoring and analysis system of claim 6, it is characterized in that: The data collection module includes: A multi-dimensional feature data collection module is used to collect multi-dimensional feature data in the logistics transportation process, wherein the multi-dimensional feature data includes: logistics line transportation time, logistics line transportation distance, logistics line transportation temperature, logistics line transportation cargo flow and logistics line transportation fuel consumption; A data preprocessing module, used to detect and process missing values ​​in the multidimensional feature data by using filling, deletion and interpolation methods, and to identify and delete abnormal values ​​in the multidimensional feature data by using a Z-score method; The unified format module is used to unify the format of the multidimensional feature data and convert the category features of the multidimensional feature data into a numerical form, including using one-hot encoding.

8. The logistics abnormality monitoring and analysis system based on big data according to claim 6 is characterized in that: The acquisition time window module includes: Determine the basic time window module, which is used to determine the basic time window for judging whether there is an abnormality in the logistics data; The dynamic time window calculation module is used to calculate the dynamic time window through the dynamic time window model. Specifically, the dynamic time window model is: Among them, t represents the time window, N represents the basic time window, β k represents the data change rate sensitivity coefficient, k represents the data category of multidimensional feature data, ΔR t-i,k represents the rate of change between time ti and ti-1 for data category k, n represents the number of data points of multidimensional feature data, W i represents the time frequency weight, γ represents the time frequency weight coefficient, λ represents the abnormal proportion coefficient, T represents the total number of data points of the multidimensional feature data, and A n Indicates the number of outliers in multidimensional feature data.

9. The logistics abnormality monitoring and analysis system based on big data according to claim 6 is characterized in that: The determination module comprises: The dynamic threshold calculation module is used to calculate the dynamic threshold according to the time window through Threshold(t)=μ t +b·σ t Calculate the dynamic threshold of each data category in the multidimensional feature data, where Threshold(t) represents the dynamic threshold in the time window t, μ t The mean value calculated within the time window t, σ t represents the standard deviation calculated in the time window t, and b represents the sensitivity adjustment coefficient; The comparison module is used to determine that there is an anomaly in the logistics data if there are data points in the data of the corresponding data category that are greater than or equal to the corresponding dynamic threshold.

10. The logistics abnormality monitoring and analysis system based on big data according to claim 6 is characterized in that: The abnormality prediction module includes: The module for obtaining the initial data set is used to obtain the logistics data set containing historical anomalies; Data cleaning module, used to process missing values, outliers and other inconsistent data in logistics data sets; The labeling module is used to label abnormal and normal samples based on historical records, providing a basis for supervised learning for the model; A feature extraction module is used for feature conversion and extraction. The extracted features include: mean and standard deviation of transportation time, threshold overlimit ratio, mean and standard deviation of transportation distance, deviation distance, temperature fluctuation, temperature deviation, mean and standard deviation of cargo flow, cargo flow peak, mean and standard deviation of fuel consumption, fuel consumption efficiency fluctuation, transportation efficiency index and ratio of cargo transportation volume to fuel consumption per unit transportation time. The extracted features constitute a feature data set, and the feature data set is divided into a training set and a validation set. A training module, for training an LSTM model using a training set containing annotations, and adjusting hyperparameters of the LSTM model to optimize performance; A tuning module is used to evaluate the performance of the LSTM model through cross-validation, and to evaluate the LSTM model using accuracy, recall and F1 score indicators to obtain a trained LSTM model; A prediction module is used to input real-time multi-dimensional feature data into a trained LSTM model, and the output result of the LSTM model identifies the nature of the logistics data, which includes abnormality and normality; The feedback adjustment module is used to trigger alarms for data predicted to be abnormal, notify relevant personnel to take measures, and collect anomaly detection feedback information to improve the model. The anomaly detection feedback information includes case analysis of correct identification and incorrect identification.

Images