A Logistics Abnormality Process Identification System and Method Based on Data Analysis

Abstract

The present invention discloses a system and method for identifying abnormal processes in logistics based on data analysis, which relates to the technical field of data analysis. The method includes the following steps: Step 1, determine the deployment positions of sensors and image acquisition devices, collect sensor and image data at a fixed frequency, and perform manual annotation; Step 2, analyze and process the sensor data, construct a classification model in combination with the annotation situation; select important features to locate the abnormal space part; Step 3, obtain corresponding image data according to the abnormal space part, analyze and process the image data, and construct an image classification model in combination with the annotation situation; Step 4, perform predictive classification on the sensor data and image data of new data, and give feedback according to different situations. The present invention can effectively improve the situation in the prior art that it is difficult to effectively identify abnormalities in the items at the bottom of logistics containers.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and specifically to a logistics anomaly process recognition system and method based on data analysis. Background Technique

[0002] At present, with the booming development of the logistics industry, the efficiency and safety of the logistics process are of crucial importance. With the deep integration of Internet of Things, big data and artificial intelligence technologies, obtaining data in the logistics process through sensors and image acquisition devices and analyzing these data to identify abnormal situations have become the key means to ensure the smooth operation of logistics. Sensors can monitor various physical parameters of logistics containers in real time, and image acquisition devices can visually present the logistics scenarios, providing rich information for logistics status analysis.

[0003] However, the existing logistics anomaly recognition technologies have exposed many shortcomings when dealing with the complex conditions of logistics containers. In the monitoring of abnormal situations of logistics containers, most of the existing solutions only focus on the easily detectable parts on the surface of the containers. For areas such as the bottom of the containers that are difficult to directly observe, there are often no effective monitoring means, making these areas become blind spots for anomaly monitoring. When potential safety hazards such as wear and deformation occur to the items at the bottom of the containers, the existing technologies cannot capture and predict these abnormal situations in a timely manner, thus bringing serious safety risks to logistics transportation. Summary of the Invention

[0004] The purpose of the present invention is to provide a logistics anomaly process recognition system and method based on data analysis to solve the problems raised in the existing technologies.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A logistics anomaly process recognition method based on data analysis, the method includes the following steps:

[0006] Step 1: Determine the deployment positions of sensors and image acquisition devices, collect sensor and image data at a fixed frequency, and perform manual annotation;

[0007] Step 2: Analyze and process the sensor data, construct a classification model in combination with the annotation situation; select important features to locate the abnormal space part;

[0008] Step 3: Obtain the corresponding image data according to the abnormal space part, analyze and process the image data, and construct an image classification model in combination with the annotation situation;

[0009] Step 4: Perform prediction classification on the sensor data and image data of new data, and give feedback according to different situations.

[0010] In step 1, sensors are arranged on side 1, side 2, side 3, side 4, the bottom surface and the top surface of the logistics container respectively.

[0011] For the opposite sides 1 and 3, they are each divided into a×b parts; where a and b are both positive integers; a means the side where side 1 connects to side 2 or side 4 is evenly divided into a parts; b means the side where side 1 connects to the bottom surface or the top surface is evenly divided into b parts.

[0012] For the opposite sides 2 and 4, they are each divided into a×c parts; where c is a positive integer; c means the side where side 2 connects to the bottom surface or the top surface is evenly divided into c parts.

[0013] For the bottom surface and the top surface, they are each divided into b×c parts.

[0014] The container is divided into a×b×c spatial parts, denoted as K1 to K a·b·c ;

[0015] Composite sensor groups are deployed in 2(a×b + a×c + b×c) parts on the six surfaces of the container respectively.

[0016] The composite sensor group is denoted as: [S1, S2, …, S d ; where d is a positive integer representing the number of sensor types; S1 to S d represent the 1st to dth types of sensors respectively.

[0017] Sensor data is collected at a fixed frequency, and the collection times are denoted as: [T1, T2, …, T e ; where e is a positive integer representing the number of data collection times; T1 to T e represent the 1st to eth sensor data collection times respectively.

[0018] A image acquisition devices are arranged on the top surface of the container. Considering the perspectives of the image acquisition devices, the entire container is covered, and each spatial part is covered by at least two image acquisition devices.

[0019] Record the spatial parts covered by each image acquisition device.

[0020] The image acquisition devices acquire images at the same frequency as the sensor data collection.

[0021] Align the sensor data and the image data in time; using manual annotation, perform manual analysis on the sensor data and the image data at each moment, and label the normal state and abnormal classifications; the labeled abnormal classifications are denoted as: [B1, B2, …, B f ; where f is a positive integer representing the number of abnormal classifications; B1 to B f represent the 1st to fth types of abnormal classifications respectively.

[0022] In step 2, for the sensor data, the sensor data at different times is normalized;

[0023] Extract the sensor data normalized at time T i and time T i-1 and calculate the gradient change of the sensor data between time T i and time T i-1 , which is denoted as ΔC i ; calculate the Euclidean distance between the sensor data at the abnormal time and the previous moment, which is denoted as D i ;

[0024] where i is a positive integer representing the data acquisition time sequence; 1 < i ≤ e;

[0025] Using the sensor data gradient change ΔC i and the Euclidean distance D i as features, combined with the annotation at time T i to construct a decision tree classification model; ΔC i is represented by the gradient change of 2(a×b + a×c + b×c)·d data at time T i and time T i-1 ; D i is the Euclidean distance between 2(a×b + a×c + b×c)·d data at time T i and 2(a×b + a×c + b×c)·d data at time T i-1 ;

[0026] Calculate the information gain brought by each feature at each split, and accumulate the information gains of a feature at all split nodes; select the top t features with the highest accumulated values from ΔC i ; where t is a positive integer, t ≤ [2(a×b + a×c + b×c)·d];

[0027] For the time T m labeled as abnormal classification, select the top t features with the highest accumulated information gain values, locate the sensor positions where they are located, and further locate the abnormal space part; where m is a positive integer representing any time sequence labeled as abnormal classification; 1 < m ≤ e.

[0028] In step 3, for each abnormal space part, screen j data acquisition devices covering the abnormal space part. When the images collected by the data acquisition devices corresponding to the abnormal space part are not blocked, segment the abnormal space part based on deep learning segmentation and multi-view 3D reconstruction to form an image set: [I1(T m ), I2(T m ), …, I j (T m ), I1(Tm-1 ), I2(T m-1 ), …, I j (T m-1 )]; where j is a positive integer representing the number of data acquisition devices; 2 ≤ j ≤ A; I1(T m ) ~ I j (T m ) respectively represent the images collected and segmented by 1 to j data acquisition devices covering the abnormal space part at time T m .

[0029] Analyze whether the abnormality is visible based on the structural similarity SSIM: SSIM(I u (T m ), I u (T m-1 )); When any SSIM is lower than the preset threshold, it is considered that the image has changed and belongs to a visible abnormality; where u is a positive integer representing the image sequence; 1 ≤ u ≤ j.

[0030] Select the images at the time corresponding to all visible abnormalities and the previous moment, and construct a ResNet image classification model in combination with the abnormal classification annotation corresponding to this moment.

[0031] In step 4, for the new time T e+1 , calculate the gradient change ΔC e+1 of the sensor data between time T e and time T e+1 and the Euclidean distance D e+1 ; Use the constructed decision tree classification model for classification prediction.

[0032] When the classification prediction is in the normal state, no warning is issued.

[0033] When the classification prediction is an abnormal classification, select the top t features with the highest cumulative value of information gain for this prediction to locate the abnormal space part; for each abnormal space part, screen the data acquisition devices covering this abnormal space part.

[0034] When the image corresponding to the abnormal space part collected by the data acquisition device is blocked, feedback the sensor data prediction result to the staff.

[0035] When the image corresponding to the abnormal space part collected by the data acquisition device is not blocked, segment this part of the abnormal space based on deep learning segmentation and multi-view 3D reconstruction; analyze whether the abnormality is visible based on the structural similarity SSIM.

[0036] When all SSIMs are not lower than the preset threshold, it is considered that the image has not changed and belongs to an invisible abnormality, and feedback the sensor data prediction result to the staff.

[0037] When any SSIM is lower than the preset threshold, it is considered that there is a change in the image, which belongs to visible anomalies; the ResNet image classification model is used to predict the anomaly classification at the new moment, and the prediction results are fed back to the staff.

[0038] A logistics anomaly process identification system based on data analysis, which includes a data acquisition module, a data analysis module, an image analysis module and a prediction feedback module;

[0039] The data acquisition module is used to determine the deployment positions of sensors and image acquisition devices, collect sensor and image data at a fixed frequency, and perform manual annotation;

[0040] The data analysis module is used to analyze and process sensor data, construct a classification model in combination with the annotation situation; select important features to locate the abnormal spatial part;

[0041] The image analysis module is used to obtain corresponding image data according to the abnormal spatial part, analyze and process the image data, and construct an image classification model in combination with the annotation situation;

[0042] The prediction feedback module is used to predict and classify sensor data and image data for new data, and give feedback according to different situations.

[0043] The data acquisition module includes a sensor deployment unit, an image device deployment unit and a data annotation unit;

[0044] The sensor deployment unit is used to deploy a composite sensor group on 6 faces of the container and establish a spatial part division of a×b×c;

[0045] The image device deployment unit is used to arrange A cameras on the top surface to ensure that each spatial part is covered by two cameras;

[0046] The data annotation unit is used to align sensor and image data and manually annotate normal and abnormal states and classifications.

[0047] The data analysis module includes a data preprocessing unit, a feature extraction unit and an anomaly location unit;

[0048] The data preprocessing unit is used to standardize sensor data and calculate the gradient change ΔC and the Euclidean distance D;

[0049] The feature extraction unit is used to calculate the feature information gain with a decision tree model and screen the top t key features;

[0050] The anomaly location unit is used to locate the specific spatial part where the anomaly occurs according to the key features.

[0051] The image analysis module includes an image screening unit, an image processing unit, and a model construction unit;

[0052] The image screening unit is used to screen the camera images covering the area according to the abnormal space part;

[0053] The image processing unit is used for multi-view 3D reconstruction to segment the abnormal area and SSIM analysis for abnormal visibility;

[0054] The model construction unit is used to construct a ResNet classification model based on the visible abnormal images.

[0055] The prediction feedback module includes a prediction classification unit, an occlusion detection unit, and a result feedback unit;

[0056] The prediction classification unit is used to predict whether new data is abnormal using a decision tree model;

[0057] The occlusion detection unit is used to determine whether the abnormal space is occluded and call SSIM to verify the visibility;

[0058] The result feedback unit is used to combine the sensor and image prediction results and output a warning or manual verification instruction.

[0059] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention arranges a composite sensor group on multiple sides, the bottom surface, and the top surface of the logistics container, and arranges multiple image acquisition devices on the top surface, which can comprehensively collect data during the logistics process. Each space part is covered by at least two image acquisition devices to ensure no data acquisition dead angle, providing a rich data basis for accurately identifying abnormalities; The present invention can accurately locate the sensor position and space part where the abnormality is located by calculating the gradient change and Euclidean distance of the sensor data and selecting features with high information gain; The present invention can quickly perform prediction classification on new data. When it is determined to be abnormal, it can quickly feedback to the staff according to different situations. Whether the abnormal space part is occluded, or it is determined to be an invisible abnormality or a visible abnormality based on image analysis, relevant information can be conveyed to the staff in a timely manner, facilitating them to quickly take measures to handle the abnormality and reduce logistics losses. Description of the Drawings

[0060] Figure 1 It is a step schematic diagram of a logistics abnormal process identification method based on data analysis of the present invention;

[0061] Figure 2 It is a flow schematic diagram of a logistics abnormal process identification system based on data analysis of the present invention;

[0062] Figure 3 It is a container division schematic diagram of a logistics abnormal process identification method based on data analysis of the present invention. Detailed implementation manners

[0063] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0064] Embodiment: As Figures 1 - 3 shown, the present invention provides a technical solution, a method for identifying abnormal processes in logistics based on data analysis. The method includes the following steps:

[0065] Step 1: Determine the deployment positions of sensors and image acquisition devices, collect sensor and image data at a fixed frequency, and perform manual annotation;

[0066] Step 2: Analyze and process the sensor data, and construct a classification model in combination with the annotation situation; select important features to locate the abnormal space part;

[0067] Step 3: Obtain the corresponding image data according to the abnormal space part, analyze and process the image data, and construct an image classification model in combination with the annotation situation;

[0068] Step 4: Perform predictive classification on the sensor data and image data of the new data, and give feedback according to different situations.

[0069] In step 1, sensors are respectively arranged on the side 1, side 2, side 3, side 4, bottom surface and top surface of the logistics container;

[0070] For the opposite sides 1 and 3, both are divided into a×b parts; where a and b are both positive integers; a represents that the side where side 1 is connected to side 2 or side 4 is evenly divided into a parts; b represents that the side where side 1 is connected to the bottom surface or the top surface is evenly divided into b parts;

[0071] For the opposite sides 2 and 4, both are divided into a×c parts; where c is a positive integer; c represents that the side where side 2 is connected to the bottom surface or the top surface is evenly divided into c parts;

[0072] For the bottom surface and the top surface, both are divided into b×c parts;

[0073] The container is divided into a×b×c space parts, denoted as K1~K a·b·c ;

[0074] Composite sensor groups are respectively deployed in 2(a×b + a×c + b×c) parts on the 6 surfaces of the container;

[0075] The composite sensor group is represented as: [S1, S2, …, S d ; where d is a positive integer representing the number of sensor types; S1 to S d respectively represent the 1st to dth types of sensors;

[0076] Collect sensor data at a fixed frequency, and the collection times are represented as: [T1, T2, …, T e ; where e is a positive integer representing the number of times of collecting data; T1 to T e respectively represent the 1st to eth sensor data collection times;

[0077] Arrange A image acquisition devices on the top surface of the container. Considering the perspectives of the image acquisition devices, cover the entire container, and each spatial part is covered by at least two image acquisition devices;

[0078] Record the spatial parts covered by each image acquisition device;

[0079] The image acquisition devices acquire images at the same frequency as the sensor data acquisition;

[0080] Align the sensor data and the image data in time; use manual annotation to perform manual analysis on the sensor data and the image data at each moment, and annotate the normal state and abnormal classifications; the annotated abnormal classifications are represented as: [B1, B2, …, B f ; where f is a positive integer representing the number of abnormal classifications; B1 to B f respectively represent the 1st to fth types of abnormal classifications.

[0081] In step 2, for the sensor data, perform normalization processing on the sensor data at different times;

[0082] Extract the normalized sensor data at time T i and time T i-1 , calculate the gradient change of the sensor data between time T i and time T i-1 , which is represented as ΔC i ; calculate the Euclidean distance between the sensor data at the abnormal time and the previous moment, which is represented as D i ;

[0083] where i is a positive integer representing the sequence of data collection times; 1 < i ≤ e;

[0084] Using the gradient change ΔC i of the sensor data and the Euclidean distance D i as features, combine with the annotation at time T i to construct a decision tree classification model; ΔC i is obtained from time T i and time T i-1The gradient change of (a×b + a×c + b×c)·d data at time 2 is represented by; D i is T i The (a×b + a×c + b×c)·d data at time 2 and T i-1 The Euclidean distance of the (a×b + a×c + b×c)·d data at time 2

[0085] Calculate the information gain brought by each feature at each split, and accumulate the information gains of a feature at all split nodes; Select the top t features with the highest accumulated value from ΔC i where t is a positive integer, t ≤ [2(a×b + a×c + b×c)·d];

[0086] For the time T labeled as abnormal classification m , select the top t features with the highest accumulated information gain, locate the position of its sensor, and further locate the abnormal space part; where m is a positive integer, representing any time sequence labeled as abnormal classification; 1 < m ≤ e.

[0087] In step 3, for each abnormal space part, screen j data acquisition devices covering this abnormal space part. When the image collected by the data acquisition device corresponding to the abnormal space part is not blocked, segment the abnormal space part based on deep learning segmentation and multi-view 3D reconstruction to form an image set: [I1(T m ), I2(T m ), …, I j (T m ), I1(T m-1 ), I2(T m-1 ), …, I j (T m-1 )]; where j is a positive integer, representing the number of data acquisition devices; 2 ≤ j ≤ A; I1(T m ) ~ I j (T m ) respectively represent the images collected and segmented by 1 to j data acquisition devices covering this abnormal space part at time T m ;

[0088] Analyze whether the abnormality is visible based on the structural similarity SSIM: SSIM(I u (T m ), I u (T m-1 )); When any SSIM is lower than the preset threshold, it is considered that the image has changed and belongs to a visible abnormality; where u is a positive integer, representing the image sequence; 1 ≤ u ≤ j;

[0089] Filter out the images corresponding to all visible anomalies and the previous moment, and construct a ResNet image classification model by combining the anomaly classification labels corresponding to this moment.

[0090] In step 4, for the new moment T e+1 , calculate the gradient change ΔC e+1 of the sensor data between moment T e and moment T e+1 and the Euclidean distance D e+1 ; use the constructed decision tree classification model for classification prediction;

[0091] When the classification prediction is in the normal state, no warning is issued;

[0092] When the classification prediction is an abnormal classification, select the top t features with the highest cumulative value of information gain for this prediction, and locate the abnormal space part; for each abnormal space part, filter out the data acquisition devices covering this abnormal space part;

[0093] When the image corresponding to the abnormal space part collected by the data acquisition device is blocked, feedback the sensor data prediction result to the staff;

[0094] When the image corresponding to the abnormal space part collected by the data acquisition device is not blocked, segment this part of the abnormal space based on deep learning segmentation and multi-view 3D reconstruction; analyze whether the anomaly is visible based on the structural similarity SSIM;

[0095] When all SSIMs are not lower than the preset threshold, it is considered that the image has no change and belongs to an invisible anomaly, and the sensor data prediction result is feedback to the staff;

[0096] When any SSIM is lower than the preset threshold, it is considered that the image has a change and belongs to a visible anomaly; use the ResNet image classification model to predict the abnormal classification at the new moment and feedback the prediction result to the staff.

[0097] A logistics anomaly process recognition system based on data analysis, which includes a data acquisition module, a data analysis module, an image analysis module, and a prediction feedback module;

[0098] The data acquisition module is used to determine the deployment positions of sensors and image acquisition devices, collect sensor and image data at a fixed frequency, and perform manual annotation;

[0099] The data analysis module is used to analyze and process sensor data, construct a classification model in combination with the annotation situation; select important features to locate the abnormal space part;

[0100] The image analysis module is used to obtain corresponding image data according to the abnormal space part, analyze and process the image data, and construct an image classification model in combination with the annotation situation;

[0101] The prediction feedback module is used to perform prediction classification on sensor data and image data of new data, and give feedback according to different situations.

[0102] The data acquisition module includes a sensor deployment unit, an image device deployment unit, and a data annotation unit;

[0103] The sensor deployment unit is used to deploy a composite sensor group on six faces of the container to establish a partial division of the a×b×c space;

[0104] The image device deployment unit is used to arrange A cameras on the top surface to ensure that each space part is covered by two cameras;

[0105] The data annotation unit is used to align sensor and image data, and manually annotate normal and abnormal states and classifications.

[0106] The data analysis module includes a data preprocessing unit, a feature extraction unit, and an anomaly localization unit;

[0107] The data preprocessing unit is used to standardize sensor data and calculate the gradient change ΔC and the Euclidean distance D;

[0108] The feature extraction unit is used to calculate the feature information gain using a decision tree model and screen the top t key features;

[0109] The anomaly localization unit is used to locate the specific space part where the anomaly occurs according to the key features.

[0110] The image analysis module includes an image screening unit, an image processing unit, and a model construction unit;

[0111] The image screening unit is used to screen the camera images covering the area according to the abnormal space part;

[0112] The image processing unit is used to perform multi-view 3D reconstruction to segment the abnormal area and perform SSIM analysis on the abnormal visibility;

[0113] The model construction unit is used to construct a ResNet classification model based on the visible abnormal images.

[0114] The prediction feedback module includes a prediction classification unit, an occlusion detection unit, and a result feedback unit;

[0115] The prediction classification unit is used to predict whether new data is abnormal using a decision tree model;

[0116] The occlusion detection unit is used to determine whether the abnormal space is occluded and call SSIM to verify the visibility;

[0117] The result feedback unit is used to combine the sensor and image prediction results and output a warning or manual verification instruction.

[0118] In this embodiment, the temperature of the cold chain logistics container is monitored for abnormalities;

[0119] Container specifications: length 12 meters (sides 1 and 3), width 2.4 meters (sides 2 and 4), height 2.4 meters (bottom and top);

[0120] Space division parameters: a = 6 (the long side of sides 1 / 3 is evenly divided into 6 segments, each segment is 2 meters); b = 3 (the height of sides 1 / 3 is evenly divided into 3 segments, each segment is 0.8 meters); c = 3 (the height of sides 2 / 4 is evenly divided into 3 segments, each segment is 0.8 meters);

[0121] The container is divided into 6×3×3 = 54 space parts (K1~K54).

[0122] Sensor deployment: A composite sensor group (including temperature, humidity, and acceleration sensors, d = 3) is deployed on each side;

[0123] Total number of sensor groups: 2(6×3 + 6×3 + 3×3) = 2×45 = 90 groups;

[0124] Image acquisition device: A = 6 wide-angle cameras are deployed on the top surface to cover all space parts;

[0125] Step 1: Data collection and annotation;

[0126] Sensor data collection: Data is collected every 5 minutes for 24 hours, a total of e = 288 moments (T1~T288);

[0127] Data volume at a single moment: 90 groups × 3 types of sensors = 270 pieces of data;

[0128] Image data collection: The camera takes pictures synchronously every 5 minutes, and the image resolution is 1920×1080;

[0129] Manual annotation:

[0130] Annotate the normal state;

[0131] Annotate abnormal events: B1 (temperature abnormality: the temperature in a certain area > 8°C for more than 10 minutes), B2 (goods tilt: the acceleration sensor detects continuous vibration + the image shows the displacement of the cargo box);

[0132] Example: At the T150 moment, the temperature in the K27 space rises to 10°C and is annotated as B1; at the T210 moment, the acceleration in the K18 space is abnormal and the image shows the goods tilted, and it is annotated as B2;

[0133] Step 2: Sensor data analysis and anomaly localization;

[0134] Data preprocessing: Standardization: Perform Z-score standardization on temperature, humidity, and acceleration data respectively;

[0135] Calculate the gradient change ΔC and Euclidean distance D:

[0136] Gradient ΔC (change rate at adjacent times): Subtract the corresponding 270 data at the previous time from the 270 data at a certain time to obtain 270 gradient changes;

[0137] Euclidean distance D: The Euclidean distance between a set of 270 data at a certain time and the corresponding 270 data at the previous time;

[0138] Construct a decision tree model: Input features: ΔC and D at 288 times (a total of 270 + 1 = 271 features);

[0139] Target label: Normal / B1 / B2;

[0140] Feature selection: Calculate the cumulative information gain value and select the top t = 20 key features;

[0141] Anomaly localization: At T150, the key features are: sensor group S45 (temperature), S32 (temperature), S18 (acceleration); Locate that sensor group S45 is in the 2nd row and 1st column of side 1, corresponding to space K27.

[0142] Step 3: Image analysis and model construction;

[0143] Image screening and processing: Abnormal space K27 is covered by camera 3 and camera 5;

[0144] Multi-view 3D reconstruction: Use U-Net to segment the contour of area K27; Generate a point cloud from the images of camera 3 and 5 and reconstruct the 3D structure;

[0145] SSIM analysis: SSIM(I3T150},I3T149}) = 0.65 < 0.8; Determined as visible anomaly (temperature anomaly causes frosting on the surface of the goods);

[0146] Construct a ResNet model: Input: Segmented images of area K27 at T150 and T149 (a total of 4 images, 2 cameras × 2 times);

[0147] Output: Classified as B1 (temperature anomaly);

[0148] Step 4: Prediction and feedback of new data;

[0149] Prediction of new time T289 data:

[0150] Sensor data: ΔC289, D289;

[0151] The decision tree model predicts B1 (temperature anomaly);

[0152] Abnormality location and image verification: The key features point to space K32 (sensor group S58); Cameras 2 and 4 cover K32. Check for image occlusion: The image of camera 2 is occluded, and the image of camera 4 is normal;

[0153] SSIM analysis: SSIM(I4T289, I4288) = 0.75 < 0.8, determined to be a visible anomaly;

[0154] Feedback result: The ResNet model classifies the image of camera 4 as B1 (temperature anomaly);

[0155] System warning: The temperature in space K32 is abnormal. It is recommended to immediately check the refrigeration equipment.

[0156] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved.

Claims

1. A method for identifying abnormal logistics processes based on data analysis, characterized in that: The method includes the following steps: Step 1: Determine the deployment positions of sensors and image acquisition devices, collect sensor and image data at a fixed frequency, and perform manual annotation; Step 2: Analyze and process the sensor data, construct a classification model in combination with the annotation situation; select important features to locate the abnormal spatial part; Step 3: Obtain the corresponding image data according to the abnormal spatial part, analyze and process the image data, and construct an image classification model in combination with the annotation situation; Step 4: Perform predictive classification on the sensor data and image data of new data, and give feedback according to different situations; In step 1, sensor data is collected at a fixed frequency, and the collection times are represented as: [T1, T2, …, T e ; where e is a positive integer representing the number of times sensor data is collected; T1, T2, …, T e respectively represent the 1st, 2nd, …, e-th sensor data collection times; Use manual annotation to manually analyze the sensor data and image data at each moment, and annotate the normal state and abnormal classification; In Step 2, for the sensor data, perform standardization processing on the sensor data at different moments; Extract T i Time and T i-1 Sensor data after normalization at the time, calculate T i Time and T i-1 Gradient change of sensor data after normalization at the time, expressed as ΔC i ; Calculate the Euclidean distance between the sensor data after normalization at the abnormal time and the previous moment, expressed as D i ; where i is a positive integer, representing the sequence of sensor data acquisition moments; 1 < i ≤ e; With the gradient change ΔC of sensor data i and the Euclidean distance D i as features, combined with the annotation at time T i to construct a decision tree classification model; Calculate the information gain brought by each feature at each split, and accumulate the information gains of a feature at all split nodes; select the top t features with the highest accumulated values from ΔC i ; For the moment T marked as abnormal classification m , select the top t features with the highest cumulative information gain, locate their sensor positions, and further locate the abnormal space part; where m is a positive integer representing any time sequence marked as abnormal classification; 1 < m ≤ e.

2. The method for identifying abnormal logistics processes based on data analysis according to claim 1, wherein: In Step 1, a rectangular parallelepiped container is selected as the logistics container; Sensors are arranged on each surface inside the logistics container; Each surface inside the logistics container includes four side surfaces, a bottom surface, and a top surface; among the four side surfaces, two opposite side surfaces are divided into a×b parts, and the other two opposite side surfaces are divided into a×c parts; the bottom surface and the top surface are both divided into b×c parts; where a, b, and c are positive integers; The logistics container is divided into a×b×c spatial parts, denoted as K1~K a·b·c ; Composite sensor groups are deployed in each part of the 6 surfaces of the logistics container respectively; The composite sensor group is represented as: [S1, S2, …, S d ; where d is a positive integer representing the number of sensor types; S1, S2, …, S d respectively represent the 1st, 2nd, …, d-th types of sensors; A image acquisition devices are arranged on the top surface of the logistics container. Combining the perspectives of the image acquisition devices, the entire logistics container is covered, and each spatial part is covered by at least two image acquisition devices; Record the spatial parts covered by each image acquisition device; The image acquisition device acquires images at the same frequency as the sensor data acquisition; Align the sensor data with the image data in time; label the abnormal classifications as: [B1, B2, …, B f ; where f is a positive integer representing the number of abnormal classifications; B1, B2, …, B f represent the first, second, …, f-th abnormal classifications, respectively.

3. The method for identifying abnormal logistics processes based on data analysis according to claim 2, wherein: In step 3, for each abnormal space part, j data acquisition devices covering the abnormal space part are screened. When the image corresponding to the abnormal space part collected by the data acquisition device is not blocked, the abnormal space part is segmented based on deep learning segmentation and multi-view 3D reconstruction to form an image set: [I1(T m ), I2(T m ), …, I j (T m ), I1(T m-1 ), I2(T m-1 ), …, I j (T m-1 )]; Among them, j is a positive integer representing the number of data acquisition devices, where 2 ≤ j ≤ A; I1(T m ), I2(T m ), …, I j (T m ) respectively represent the images obtained by the 1st, 2nd, …, jth data acquisition devices that cover the abnormal space part at time T m and have been segmented; Analyze whether the anomaly is visible based on the structural similarity SSIM: SSIM(I u (T m ), I u (T m-1 )); When any SSIM is lower than the preset threshold, it is considered that the image has changed and belongs to a visible anomaly; where u is a positive integer representing the image sequence; 1 ≤ u ≤ j; I u (T m ), I u (T m-1 ) respectively represent the images collected by the u-th image acquisition device at time T m and time T m-1 ​ Screen out the images corresponding to all visible abnormal moments and the previous moment, and construct a ResNet image classification model in combination with the abnormal classification annotation corresponding to this moment.

4. The method for identifying abnormal logistics processes based on data analysis according to claim 3, characterized in that: In step 4, for the new time T e+1 , calculate T e+1 Moment and T e The gradient change of sensor data at the moment ΔC e+1 and Euclidean distance D e+1 ; Use the constructed decision tree classification model for classification prediction; When the classification prediction is in the normal state, no warning is given; When the classification prediction is an abnormal classification, select the top t features with the highest cumulative value of information gain for this prediction from the gradient change ΔC of the sensor data e+1 to locate the abnormal space part; For each abnormal spatial part, screen out the data acquisition devices covering this abnormal spatial part; When the image corresponding to the abnormal spatial part acquired by the data acquisition device is blocked, feedback the sensor data prediction result to the staff; When the image corresponding to the abnormal spatial part acquired by the data acquisition device is not blocked, segment this part of the abnormal space based on deep learning segmentation and multi-view 3D reconstruction; analyze whether the abnormality is visible based on the structural similarity SSIM; When all SSIMs are not lower than the preset threshold, it is considered that the image has no change and belongs to an invisible abnormality, and feedback the sensor data prediction result to the staff; When any SSIM is lower than the preset threshold, it is considered that the image has a change and belongs to a visible abnormality; Use the ResNet image classification model to predict the abnormal classification at the new moment, and feedback the prediction result to the staff.

5. A logistics anomaly process recognition system based on data analysis, which is applied to the logistics anomaly process recognition method according to any one of claims 1-4, and is characterized in that: The system includes a data acquisition module, a data analysis module, an image analysis module, and a prediction feedback module; The data acquisition module is used to determine the deployment positions of sensors and image acquisition devices, collect sensor and image data at a fixed frequency, and perform manual annotation; The data analysis module is used to analyze and process sensor data, construct a classification model in combination with the annotation situation, and select important features to locate the abnormal spatial part; The image analysis module is used to obtain corresponding image data according to the abnormal spatial part, analyze and process the image data, and construct an image classification model in combination with the annotation situation; The prediction and feedback module is used to perform prediction classification on new data for sensor data and image data, and give feedback according to different situations.

6. The logistics anomaly process recognition system based on data analysis according to claim 5, characterized in that: The data acquisition module includes a sensor deployment unit, an image device deployment unit, and a data annotation unit; The sensor deployment unit is used to deploy a composite sensor group on six faces of the container to establish a spatial division of a×b×c; The image device deployment unit is used to arrange A cameras on the top surface to ensure that each spatial part is covered by two cameras; The data annotation unit is used to align the sensor and image data, and manually annotate the normal and abnormal states and classifications.

7. The logistics anomaly process identification system based on data analysis according to claim 6, characterized in that: The data analysis module includes a data preprocessing unit, a feature extraction unit, and an anomaly localization unit; The data preprocessing unit is used to standardize the sensor data and calculate the gradient change ΔC and the Euclidean distance D; The feature extraction unit is used to calculate the feature information gain using a decision tree model and screen the top t key features; The anomaly localization unit is used to locate the specific spatial part where the anomaly occurs according to the key features.

8. The logistics anomaly process recognition system based on data analysis according to claim 7, characterized in that: The image analysis module includes an image screening unit, an image processing unit, and a model construction unit; The image screening unit is used to screen the camera images covering the area according to the abnormal spatial part; The image processing unit is used to perform multi-view 3D reconstruction to segment the abnormal area and analyze the anomaly visibility by SSIM; The model construction unit is used to construct a ResNet classification model based on the visible abnormal images.

9. The logistics anomaly process identification system based on data analysis according to claim 8, wherein: The prediction and feedback module includes a prediction classification unit, an occlusion detection unit, and a result feedback unit; The prediction classification unit is used to predict whether the new data is abnormal using a decision tree model; The occlusion detection unit is used to judge whether the abnormal space is occluded and call SSIM to verify the visibility; The result feedback unit is used to combine the sensor and image prediction results and output a warning or manual verification instruction.

Images