Logistics abnormal process identification system and method based on data analysis

Abstract

The invention discloses a logistics abnormal process identification system and method based on data analysis, and relates to the technical field of data analysis, and the method comprises the following steps: 1, determining the deployment positions of a sensor and an image collection device, collecting the sensor and image data at a fixed frequency, and carrying out the manual marking; 2, analyzing and processing the sensor data, and constructing a classification model in combination with a labeling condition; important features are selected to locate an abnormal space part; 3, acquiring corresponding image data according to the abnormal space part, analyzing and processing the image data, and constructing an image classification model in combination with the labeling condition; and step 4, performing prediction classification of sensor data and image data on the new data, and performing feedback according to different conditions. The method can effectively improve the condition that effective anomaly recognition is difficult to carry out on articles at the bottom of the logistics container in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and specifically to a logistics anomaly process recognition system and method based on data analysis. Background Art

[0002] At present, with the booming development of the logistics industry, the efficiency and safety of the logistics process are of crucial importance. With the deep integration of Internet of Things, big data and artificial intelligence technologies, obtaining data in the logistics process through sensors and image acquisition devices and analyzing these data to identify abnormal situations have become the key means to ensure the smooth operation of logistics. Sensors can monitor various physical parameters of logistics containers in real time, and image acquisition devices can visually present the logistics scenarios, providing rich information for logistics status analysis.

[0003] However, the existing logistics anomaly recognition technologies have exposed many shortcomings when dealing with the complex conditions of logistics containers. In the monitoring of abnormal situations of logistics containers, most existing solutions only focus on the easily noticeable parts on the surface of the containers. For areas such as the bottom of the containers that are difficult to directly observe, there are often lack of effective monitoring means, making these areas become blind spots for anomaly monitoring. When potential safety hazards such as wear and deformation occur to the items at the bottom of the containers, the existing technologies cannot capture and predict these abnormal situations in time, thus bringing serious safety risks to logistics transportation. Summary of the Invention

[0004] The purpose of the present invention is to provide a logistics anomaly process recognition system and method based on data analysis to solve the problems raised in the existing technologies.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A logistics anomaly process recognition method based on data analysis, the method includes the following steps: Step 1, determine the deployment positions of sensors and image acquisition devices, collect sensor and image data at a fixed frequency, and perform manual annotation; Step 2, analyze and process the sensor data, construct a classification model in combination with the annotation situation; select important features to locate the abnormal space part; Step 3, obtain the corresponding image data according to the abnormal space part, analyze and process the image data, and construct an image classification model in combination with the annotation situation; Step 4, perform prediction classification on the sensor data and image data of new data, and give feedback according to different situations.

[0006] In step 1, sensors are respectively arranged on the side 1, side 2, side 3, side 4, bottom surface and top surface of the logistics container; For the opposite sides 1 and 3, each is divided into a×b parts; where a and b are both positive integers; a represents that the side of side 1 connected to side 2 or side 4 is evenly divided into a parts; b represents that the side of side 1 connected to the bottom or the top is evenly divided into b parts; For the opposite sides 2 and 4, each is divided into a×c parts; where c is a positive integer; c represents that the side of side 2 connected to the bottom or the top is evenly divided into c parts; For the bottom and the top, each is divided into b×c parts; The container is divided into a×b×c spatial parts, denoted as K 1 ~K a·b·c ; Deploy composite sensor groups on 2(a×b + a×c + b×c) parts of the six faces of the container respectively; The composite sensor group is denoted as: [S 1 ,S 2 ,…,S d ; where d is a positive integer, representing the number of sensor types; S 1 ~S d respectively represent the 1st to dth types of sensors; Collect sensor data at a fixed frequency, and the collection times are denoted as: [T 1 ,T 2 ,…,T e ; where e is a positive integer, representing the number of times of collecting data; T 1 ~T e respectively represent the 1st to eth sensor data collection times; Arrange A image acquisition devices on the top surface of the container. Combining with the perspectives of the image acquisition devices, cover the entire container, and each spatial part is covered by at least two image acquisition devices; Record the spatial parts covered by each image acquisition device; The image acquisition devices acquire images at the same frequency as the sensor data collection; Align the sensor data and the image data in time; Use manual annotation to perform manual analysis on the sensor data and the image data at each moment, and annotate the normal state and abnormal classifications; The annotated abnormal classifications are denoted as: [B 1 ,B 2 ,…,B f ; where f is a positive integer, representing the number of abnormal classifications; B 1 ~B f respectively represent the 1st to fth types of abnormal classifications.

[0007] In step 2, for the sensor data, perform normalization processing on the sensor data at different times; Extract the data at time T i and at time Ti-1 Calculate T from the sensor data after moment normalization i The moment and T i-1 The gradient change of the sensor data at the moment, denoted as ΔC i ; Calculate the Euclidean distance between the sensor data at the abnormal moment and the previous moment, denoted as D i ; Where i is a positive integer, representing the data acquisition moment sequence; 1 < i ≤ e; Using the sensor data gradient change ΔC i and the Euclidean distance D i as features, combined with the annotation at the moment of T i to construct a decision tree classification model; ΔC i is represented by the gradient change of 2(a×b + a×c + b×c)·d data at the moment of T i and the moment of T i-1 ; D i is the Euclidean distance between 2(a×b + a×c + b×c)·d data at the moment of T i and 2(a×b + a×c + b×c)·d data at the moment of T i-1 ; Calculate the information gain brought by each feature at each split, and accumulate the information gains of a feature at all split nodes; Select the top t features with the highest accumulated value from ΔC i ; Where t is a positive integer, t ≤ [2(a×b + a×c + b×c)·d]; For the moment T m labeled as abnormal classification, select the top t features with the highest accumulated information gain, locate the sensor position where it is located, and further locate the abnormal space part; Where m is a positive integer, representing any moment sequence labeled as abnormal classification; 1 < m ≤ e.

[0008] In step 3, for each abnormal space part, screen j data acquisition devices covering this abnormal space part. When the image corresponding to the abnormal space part collected by the data acquisition device is not blocked, segment the abnormal space part based on deep learning segmentation and multi-view 3D reconstruction to form an image set: [I 1 (T m ), I 2 (T m ), …, I j (T m ), I 1 (T m-1 ), I 2 (T m-1 ), …, I j (T m-1 )]; Where j is a positive integer, representing the number of data acquisition devices; 2 ≤ j ≤ A; I1 (T m ) to I j (T m ) respectively represent the images collected by 1 to j data acquisition devices covering the abnormal space part at time T m and segmented; Analyze whether the abnormality is visible based on the structural similarity SSIM: SSIM(I u (T m ), I u (T m-1 )); When any SSIM is lower than the preset threshold, it is considered that the image has changed and belongs to a visible abnormality; where u is a positive integer representing the image sequence; 1 ≤ u ≤ j; Screen out the images at the corresponding time and the previous time of all visible abnormalities, and construct a ResNet image classification model in combination with the abnormal classification annotation corresponding to this time.

[0009] In step 4, for the new time T e+1 , calculate the gradient change ΔC e+1 of the sensor data between time T e and time T e+1 and the Euclidean distance D e+1 ; Use the constructed decision tree classification model for classification prediction; When the classification prediction is in the normal state, no warning is issued; When the classification prediction is an abnormal classification, select the top t features with the highest cumulative value of information gain for this prediction, and locate the abnormal space part; for each abnormal space part, screen out the data acquisition devices covering this abnormal space part; When the image corresponding to the abnormal space part collected by the data acquisition device is blocked, feedback the sensor data prediction result to the staff; When the image corresponding to the abnormal space part collected by the data acquisition device is not blocked, segment this part of the abnormal space based on deep learning segmentation and multi-view 3D reconstruction; Analyze whether the abnormality is visible based on the structural similarity SSIM; When all SSIMs are not lower than the preset threshold, it is considered that the image has not changed and belongs to an invisible abnormality, and feedback the sensor data prediction result to the staff; When any SSIM is lower than the preset threshold, it is considered that the image has changed and belongs to a visible abnormality; Use the ResNet image classification model to predict the abnormal classification at the new time and feedback the prediction result to the staff.

[0010] A logistics abnormal process recognition system based on data analysis, the system includes a data acquisition module, a data analysis module, an image analysis module and a prediction feedback module; The data acquisition module is used to determine the deployment positions of sensors and image acquisition devices, collect sensor and image data at a fixed frequency, and perform manual annotation; The data analysis module is used to analyze and process sensor data, construct a classification model in combination with the annotation situation; select important features to locate the abnormal space part; The image analysis module is used to obtain corresponding image data according to the abnormal space part, analyze and process the image data, and construct an image classification model in combination with the annotation situation; The prediction and feedback module is used to perform prediction classification on sensor data and image data of new data, and give feedback according to different situations.

[0011] The data acquisition module includes a sensor deployment unit, an image device deployment unit, and a data annotation unit; The sensor deployment unit is used to deploy a composite sensor group on six sides of the container, and establish a spatial division of a×b×c; The image device deployment unit is used to arrange A cameras on the top surface to ensure that each spatial part is covered by two cameras; The data annotation unit is used to align sensor and image data, and manually annotate normal and abnormal states and classifications.

[0012] The data analysis module includes a data preprocessing unit, a feature extraction unit, and an anomaly localization unit; The data preprocessing unit is used to standardize sensor data and calculate the gradient change ΔC and the Euclidean distance D; The feature extraction unit is used to calculate the feature information gain with a decision tree model and screen the top t key features; The anomaly localization unit is used to locate the specific spatial part where the anomaly occurs according to the key features.

[0013] The image analysis module includes an image screening unit, an image processing unit, and a model construction unit; The image screening unit is used to screen the camera images covering the area according to the abnormal space part; The image processing unit is used to perform multi-view 3D reconstruction to segment the abnormal area and analyze the anomaly visibility by SSIM; The model construction unit is used to construct a ResNet classification model based on the visible abnormal images.

[0014] The prediction and feedback module includes a prediction classification unit, an occlusion detection unit, and a result feedback unit; The prediction classification unit is used to predict whether new data is abnormal with a decision tree model; The occlusion detection unit is used to judge whether the abnormal space is occluded and call SSIM to verify the visibility; The result feedback unit is used to combine the sensor and image prediction results and output a warning or manual verification instruction.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention arranges a composite sensor group on multiple sides, the bottom surface and the top surface of the logistics container, and arranges a plurality of image acquisition devices on the top surface, which can comprehensively collect data in the logistics process. Each spatial part is covered by at least two image acquisition devices to ensure that there is no dead angle in data acquisition, providing a rich data basis for accurately identifying abnormalities; The present invention calculates the gradient change and Euclidean distance of sensor data, selects features with high information gain, and can accurately locate the sensor position and spatial part where the abnormality is located; The present invention can quickly perform prediction classification on new data. When it is judged as an abnormality, it can quickly feedback to the staff according to different situations. Whether the abnormal spatial part is blocked, or it is determined as an invisible abnormality or a visible abnormality based on image analysis, relevant information can be conveyed to the staff in a timely manner, facilitating them to quickly take measures to handle the abnormality and reduce logistics losses. Description of the Drawings

[0016] Figure 1 It is a step schematic diagram of a logistics anomaly process identification method based on data analysis according to the present invention; Figure 2 It is a process schematic diagram of a logistics anomaly process identification system based on data analysis according to the present invention; Figure 3 It is a container division schematic diagram of a logistics anomaly process identification method based on data analysis according to the present invention. Detailed Embodiments

[0017] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0018] Embodiment: As Figures 1-3 shown, the present invention provides a technical solution, a logistics anomaly process identification method based on data analysis, and the method includes the following steps: Step 1: Determine the deployment positions of sensors and image acquisition devices, collect sensor and image data at a fixed frequency, and perform manual annotation; Step 2: Analyze and process the sensor data, construct a classification model in combination with the annotation situation; select important features to locate the abnormal spatial part; Step 3: Obtain corresponding image data according to the abnormal space part, analyze and process the image data, and construct an image classification model in combination with the annotation situation; Step 4: Perform predictive classification on the sensor data and image data of the new data, and give feedback according to different situations.

[0019] In Step 1, sensors are arranged on the side 1, side 2, side 3, side 4, bottom surface and top surface of the logistics container respectively; For the opposite sides 1 and 3, both are divided into a×b parts; where a and b are both positive integers; a represents that the side where side 1 is connected to side 2 or side 4 is evenly divided into a parts; b represents that the side where side 1 is connected to the bottom surface or the top surface is evenly divided into b parts; For the opposite sides 2 and 4, both are divided into a×c parts; where c is a positive integer; c represents that the side where side 2 is connected to the bottom surface or the top surface is evenly divided into c parts; For the bottom surface and the top surface, both are divided into b×c parts; The container is divided into a×b×c space parts, denoted as K 1 ~K a·b·c ; Composite sensor groups are deployed in 2(a×b + a×c + b×c) parts on the six surfaces of the container respectively; The composite sensor group is expressed as: [S 1 , S 2 , …, S d ; where d is a positive integer, representing the number of sensor types; S 1 ~S d represent the 1st to dth sensors respectively; Collect sensor data at a fixed frequency, and the collection time is expressed as: [T 1 , T 2 , …, T e ; where e is a positive integer, representing the number of data collection times; T 1 ~T e represent the 1st to eth sensor data collection times respectively; Arrange A image acquisition devices on the top surface of the container, covering the entire container in combination with the perspectives of the image acquisition devices, and each space part is covered by at least two image acquisition devices; Record the space parts covered by each image acquisition device; The image acquisition devices acquire images at the same frequency as the sensor data acquisition; Align the sensor data and the image data in time; use manual annotation to manually analyze the sensor data and the image data at each moment, and annotate the normal state and abnormal classification; the annotation of abnormal classification is expressed as: [B 1 , B2 , …, B f ; where f is a positive integer representing the number of anomaly classifications; B 1 ~B f respectively represent the 1st to fth anomaly classifications.

[0020] In step 2, for the sensor data, the sensor data at different times is normalized; Extract the sensor data normalized at time T i and time T i-1 , and calculate the gradient change of the sensor data between time T i and time T i-1 , denoted as ΔC i ; calculate the Euclidean distance between the sensor data at the anomaly time and the previous moment, denoted as D i ; where i is a positive integer representing the data acquisition time sequence; 1 < i ≤ e; Using the sensor data gradient change ΔC i and the Euclidean distance D i as features, combined with the annotation at time T i , construct a decision tree classification model; ΔC i is represented by the gradient change of 2(a×b + a×c + b×c)·d data at time T i and time T i-1 ; D i is the Euclidean distance between 2(a×b + a×c + b×c)·d data at time T i and 2(a×b + a×c + b×c)·d data at time T i-1 ; Calculate the information gain brought by each feature at each split, and accumulate the information gains of a feature at all split nodes; select the top t features with the highest accumulated value from ΔC i ; where t is a positive integer, t ≤ [2(a×b + a×c + b×c)·d]; For the time T m labeled as an anomaly classification, select the top t features with the highest accumulated information gain, locate its sensor position, and further locate the abnormal space part; where m is a positive integer representing any time sequence labeled as an anomaly classification; 1 < m ≤ e.

[0021] In step 3, for each abnormal space part, screen j data acquisition devices covering this abnormal space part. When the image collected by the data acquisition device corresponding to the abnormal space part is not blocked, segment the abnormal space part based on deep learning segmentation and multi-view 3D reconstruction to form an image set: [I 1 (T m ), I2 (T m ), …, I j (T m ), I 1 (T m-1 ), I 2 (T m-1 ), …, I j (T m-1 )); where j is a positive integer representing the number of data acquisition devices; 2 ≤ j ≤ A; I 1 (T m ) to I j (T m ) respectively represent the images collected and segmented by 1 to j data acquisition devices covering this part of the abnormal space at time T m . Analyze whether the anomaly is visible based on the structural similarity SSIM: SSIM(I u (T m ), I u (T m-1 )); When any SSIM is lower than the preset threshold, it is considered that the image has changed and belongs to a visible anomaly; where u is a positive integer representing the image sequence; 1 ≤ u ≤ j; Filter out the images at the corresponding time and the previous time of all visible anomalies, and construct a ResNet image classification model in combination with the anomaly classification annotation corresponding to this time.

[0022] In step 4, for the new time T e+1 , calculate the gradient change ΔC e+1 of the sensor data between time T e and time T e+1 and the Euclidean distance D e+1 ; Use the constructed decision tree classification model for classification prediction; When the classification prediction is in the normal state, no warning is issued; When the classification prediction is an abnormal classification, select the top t features with the highest cumulative value of the information gain of this prediction, and locate the abnormal space part; For each abnormal space part, filter out the data acquisition devices covering this abnormal space part; When the image corresponding to the abnormal space part collected by the data acquisition device is blocked, feedback the sensor data prediction result to the staff; When the image corresponding to the abnormal space part collected by the data acquisition device is not blocked, segment this part of the abnormal space based on deep learning segmentation and multi-view 3D reconstruction; Analyze whether the anomaly is visible based on the structural similarity SSIM; When all SSIMs are not lower than the preset threshold, it is considered that the image has not changed and belongs to an invisible anomaly, and feedback the sensor data prediction result to the staff; When any SSIM is lower than the preset threshold, it is considered that there is a change in the image, which belongs to visible anomalies; the ResNet image classification model is used to predict the anomaly classification at the new moment, and the prediction results are fed back to the staff.

[0023] A logistics anomaly process identification system based on data analysis, which includes a data acquisition module, a data analysis module, an image analysis module, and a prediction feedback module; The data acquisition module is used to determine the deployment positions of sensors and image acquisition devices, collect sensor and image data at a fixed frequency, and perform manual annotation; The data analysis module is used to analyze and process sensor data, construct a classification model in combination with the annotation situation; select important features to locate the abnormal spatial part; The image analysis module is used to obtain corresponding image data according to the abnormal spatial part, analyze and process the image data, and construct an image classification model in combination with the annotation situation; The prediction feedback module is used to predict and classify sensor data and image data for new data, and give feedback according to different situations.

[0024] The data acquisition module includes a sensor deployment unit, an image device deployment unit, and a data annotation unit; The sensor deployment unit is used to deploy a composite sensor group on 6 faces of the container, and establish a spatial part division of a×b×c; The image device deployment unit is used to arrange A cameras on the top surface to ensure that each spatial part is covered by two cameras; The data annotation unit is used to align sensor and image data, and manually annotate normal and abnormal states and classifications.

[0025] The data analysis module includes a data preprocessing unit, a feature extraction unit, and an anomaly localization unit; The data preprocessing unit is used to standardize sensor data and calculate the gradient change ΔC and Euclidean distance D; The feature extraction unit is used to calculate the feature information gain with a decision tree model and screen the top t key features; The anomaly localization unit is used to locate the specific spatial part where the anomaly occurs according to the key features.

[0026] The image analysis module includes an image screening unit, an image processing unit, and a model construction unit; The image screening unit is used to screen the camera images covering this area according to the abnormal spatial part; The image processing unit is used to perform multi-view 3D reconstruction to segment the abnormal area and analyze the anomaly visibility by SSIM; The model construction unit is used to construct a ResNet classification model based on visible abnormal images.

[0027] The prediction and feedback module includes a prediction classification unit, an occlusion detection unit, and a result feedback unit; The prediction classification unit is used to predict whether new data is abnormal using a decision tree model; The occlusion detection unit is used to determine whether the abnormal space is occluded and call SSIM to verify visibility; The result feedback unit is used to combine sensor and image prediction results and output a warning or manual verification instruction.

[0028] In this embodiment, temperature anomaly monitoring of cold chain logistics containers; Container specifications: length 12 meters (sides 1 and 3), width 2.4 meters (sides 2 and 4), height 2.4 meters (bottom and top); Spatial division parameters: a = 6 (the long side of side 1 / 3 is evenly divided into 6 segments, each segment is 2 meters); b = 3 (the height of side 1 / 3 is evenly divided into 3 segments, each segment is 0.8 meters); c = 3 (the height of side 2 / 4 is evenly divided into 3 segments, each segment is 0.8 meters); The container is divided into 6×3×3 = 54 spatial parts (K1~K54).

[0029] Sensor deployment: A composite sensor group (including temperature, humidity, and acceleration sensors, d = 3) is deployed on each side; Total number of sensor groups: 2(6×3 + 6×3 + 3×3)=2×45 = 90 groups; Image acquisition device: A = 6 wide-angle cameras are deployed on the top surface to cover all spatial parts; Step 1: Data collection and annotation; Sensor data collection: Data is collected once every 5 minutes for 24 hours, a total of e = 288 moments (T1~T288); Data volume at a single moment: 90 groups × 3 types of sensors = 270 pieces of data; Image data collection: The camera takes pictures synchronously every 5 minutes, and the image resolution is 1920×1080; Manual annotation: Annotate the normal state; Annotate abnormal events: B1 (temperature anomaly: the temperature in a certain area > 8°C for more than 10 minutes), B2 (cargo tilt: continuous vibration detected by the acceleration sensor + image shows cargo displacement in the container); Example: At the moment of T150, the temperature in the K27 space rises to 10°C and is annotated as B1; at the moment of T210, the acceleration in the K18 space is abnormal and the image shows cargo tilt, which is annotated as B2; Step 2: Sensor data analysis and anomaly location; Data preprocessing: Standardization: Perform Z-score standardization on temperature, humidity, and acceleration data respectively; Calculate the gradient change ΔC and Euclidean distance D: Gradient ΔC (change rate at adjacent times): Subtract 270 data at the previous time from 270 data at a certain time to obtain 270 gradient changes; Euclidean distance D: The Euclidean distance between a set of 270 data at a certain time and the corresponding 270 data at the previous time; Construct a decision tree model: Input features: ΔC and D at 288 times (a total of 270 + 1 = 271 features); Target labels: Normal / B1 / B2; Feature selection: Calculate the cumulative information gain value and select the top t = 20 key features; Anomaly localization: At time T150, the key features are: sensor group S45 (temperature), S32 (temperature), S18 (acceleration); Locate that sensor group S45 is in the 2nd row and 1st column on side 1, corresponding to space K27.

[0030] Step 3: Image analysis and model construction; Image screening and processing: Abnormal space K27 is covered by camera 3 and camera 5; Multi-view 3D reconstruction: Use U-Net to segment the contour of area K27; Generate a point cloud from the images of camera 3 and 5 and reconstruct the 3D structure; SSIM analysis: SSIM(I3T150},I3T149}) = 0.65 < 0.8; Determined as a visible anomaly (temperature anomaly causes frosting on the surface of the goods); Construct a ResNet model: Input: Segmented images of area K27 at times T150 and T149 (a total of 4 images, 2 cameras × 2 times); Output: Classified as B1 (temperature anomaly); Step 4: New data prediction and feedback; Prediction of data at new time T289: Sensor data: ΔC289, D289; The decision tree model predicts B1 (temperature anomaly); Anomaly localization and image verification: The key features point to space K32 (sensor group S58); Camera 2 and 4 cover K32, check for image occlusion: The image of camera 2 is occluded, and the image of camera 4 is normal; SSIM analysis: SSIM(I4T289,I4288) = 0.75 < 0.8, determined as a visible anomaly; Feedback result: The ResNet model classifies the image of camera 4 as B1 (temperature anomaly); System warning: The temperature in space K32 is abnormally high. It is recommended to immediately check the refrigeration equipment.

[0031] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claims involved.

Claims

1. A method for identifying abnormal logistics processes based on data analysis, characterized in that: The method comprises the following steps: Step 1: Determine the deployment location of sensors and image acquisition equipment, collect sensor and image data at a fixed frequency, and perform manual annotation; Step 2: Analyze and process the sensor data, build a classification model based on the annotation situation, select important features to locate the abnormal space part; Step 3: Obtain corresponding image data according to the abnormal space part, analyze and process the image data, and build an image classification model based on the annotation situation; Step 4: Predict and classify the new data into sensor data and image data, and provide feedback according to different situations.

2. The method for identifying abnormal logistics processes based on data analysis according to claim 1 is characterized in that: In step 1, a rectangular container is selected as a logistics container; Arrange sensors on each surface of the logistics container; The surfaces inside the logistics container include four side surfaces, a bottom surface and a top surface; among the four side surfaces, two opposite side surfaces are divided into a×b parts, and the other two opposite side surfaces are divided into a×c parts; the bottom surface and the top surface are both divided into b×c parts; a, b, and c are positive integers; The logistics container is divided into a×b×c space parts, denoted as K1~K a·b·c ; Deploy a composite sensor group on each of the six sides of the logistics container; The composite sensor group is represented as: [S1,S2,…,S d ]; where d is a positive integer, indicating the number of sensor types; S1, S2, …, S d They represent the 1st, 2nd, ..., dth sensors respectively; The sensor data is collected at a fixed frequency, and the collection time is expressed as: [T1, T2, …, T e ]; where e is a positive integer, indicating the number of times sensor data is collected; T1, T2, …, T e Respectively represent the 1st, 2nd, ..., eth sensor data collection moments; Arrange A image acquisition devices on the top surface of the logistics container, and combine the image acquisition device viewing angle to cover the entire logistics container. Each space part is covered by at least two image acquisition devices; Record the space portion covered by each image acquisition device; The image acquisition device acquires images at the same frequency as the sensor data acquisition; Align the sensor data and image data in time; use manual annotation to manually analyze the sensor data and image data at each moment, and mark the normal state and abnormal classification; the anomaly classification is expressed as: [B1, B2, …, B f ]; where f is a positive integer, indicating the number of abnormal classifications; B1, B2, …, B f They represent the 1st, 2nd, ..., fth abnormal classifications respectively.

3. The method for identifying abnormal logistics processes based on data analysis according to claim 2 is characterized in that: In step 2, for sensor data, the sensor data at different times are standardized; Extract T i Time and T i-1 The sensor data after time standardization is calculated as T i Moment and T i-1 The gradient change of sensor data after normalization at a certain moment is expressed as ΔC i ; Calculate the Euclidean distance between the abnormal moment and the normalized sensor data at the previous moment, expressed as D i ; Where i is a positive integer, representing the sensor data collection time sequence; 1<i≤e; The sensor data gradient changes ΔC i , Euclidean distance D i As a feature, combined with T i Construct a decision tree classification model based on the annotation of each moment; Calculate the information gain of each feature at each split, and accumulate the information gain of a feature on all split nodes; from ΔC i Select the first t features with the highest cumulative values; where t is a positive integer, t≤[2(a×b+a×c+b×c)·d]; For the moment T marked as an abnormal classification m , select the t features with the highest information gain cumulative value, locate the sensor position where they are located, and further locate the abnormal space part; where m is a positive integer, representing any time sequence marked as an abnormal classification; 1<m≤e.

4. The method for identifying abnormal logistics processes based on data analysis according to claim 3 is characterized in that: In step 3, for each abnormal space part, j data acquisition devices covering the abnormal space part are selected. When the abnormal space part corresponding to the image collected by the data acquisition device is not blocked, the abnormal space part is segmented based on deep learning segmentation and multi-view 3D reconstruction to form an image set: [I1(T m ),I2(T m ),…,I j (T m ),I1(T m-1 ),I2(T m-1 ),…,I j (T m-1 )]; Where j is a positive integer, indicating the number of data acquisition devices; 2≤j≤A; I1(T m ),I2(T m ),…,I j (T m ) respectively represent the m The images collected and segmented by 1,2,...,j data acquisition devices that cover the abnormal space part at all times; Based on the structural similarity SSIM, we analyze whether the anomaly is visible: SSIM (I u (T m ),I u (T m-1 )); When any SSIM is lower than the preset threshold, it is considered that the image has changed and is a visible anomaly; where u is a positive integer representing the image sequence; 1≤u≤j; I u (T m ),I u (T m-1 ) represent T m Moment and T m-1 The image captured by the image acquisition device at time u Filter out all images corresponding to the moment and the previous moment of visible anomalies, and build a ResNet image classification model based on the anomaly classification annotations corresponding to the moment.

5. The method for identifying abnormal logistics processes based on data analysis according to claim 4 is characterized in that: In step 4, for the new time T e+1 , calculate T e+1 Moment and T e The gradient change of sensor data at the moment ΔC e+1 and Euclidean distance D e+1 ; Use the constructed decision tree classification model for classification prediction; When the classification prediction is normal, no warning is issued; When the classification prediction is an abnormal classification, the gradient change ΔC from the sensor data e+1 Select the first t features with the highest gain accumulation value to locate the abnormal space part; For each abnormal space portion, screening data acquisition devices covering the abnormal space portion; When the abnormal space corresponding to the image collected by the data acquisition device is partially blocked, the sensor data prediction results are fed back to the staff; When the part of the abnormal space corresponding to the image collected by the data acquisition device is not blocked, the abnormal space is segmented based on deep learning segmentation and multi-view 3D reconstruction; the abnormality is analyzed based on the structural similarity SSIM to see whether it is visible; When all SSIMs are not lower than the preset threshold, it is considered that there is no change in the image, which is an invisible anomaly, and the sensor data prediction results are fed back to the staff; When any SSIM is lower than the preset threshold, it is considered that the image has changed and is a visible anomaly; Use the ResNet image classification model to predict the abnormal classification of the new moment and feed the prediction results back to the staff.

6. A logistics abnormal process identification system based on data analysis, applied to a logistics abnormal process identification method based on data analysis as described in any one of claims 1 to 5, characterized in that: The system includes a data acquisition module, a data analysis module, an image analysis module and a prediction feedback module; The data acquisition module is used to determine the deployment location of sensors and image acquisition equipment, collect sensor and image data at a fixed frequency, and perform manual annotation; The data analysis module is used to analyze and process the sensor data, build a classification model based on the annotation situation, and select important features to locate the abnormal space part; The image analysis module is used to obtain corresponding image data according to the abnormal space part, analyze and process the image data, and build an image classification model in combination with the annotation situation; The prediction feedback module is used to predict and classify new data into sensor data and image data, and provide feedback according to different situations.

7. The system for identifying abnormal logistics processes based on data analysis according to claim 6 is characterized in that: The data acquisition module includes a sensor deployment unit, an image device deployment unit and a data annotation unit; The sensor deployment unit is used to deploy a composite sensor group on the six sides of the container to establish an a×b×c spatial partition; The image device deployment unit is used to arrange A cameras on the top surface to ensure that each space portion is covered by two cameras; The data annotation unit is used to align sensor and image data, and manually annotate normal and abnormal states and classifications.

8. The logistics abnormal process identification system based on data analysis according to claim 7 is characterized by: The data analysis module includes a data preprocessing unit, a feature extraction unit and an anomaly location unit; The data preprocessing unit is used to standardize the sensor data and calculate the gradient change ΔC and the Euclidean distance D; The feature extraction unit is used to calculate feature information gain using a decision tree model and screen the first t key features; The anomaly locating unit is used to locate the specific spatial part where the anomaly occurs according to the key features.

9. The logistics abnormal process identification system based on data analysis according to claim 8 is characterized by: The image analysis module includes an image screening unit, an image processing unit and a model building unit; The image screening unit is used to screen the camera images covering the space according to the abnormal space part; The image processing unit is used for multi-view 3D reconstruction and segmentation of abnormal space, and SSIM analysis of abnormal visibility; The model building unit is used to build a ResNet classification model based on the visible abnormal image.

10. The logistics abnormal process identification system based on data analysis according to claim 9 is characterized in that: The prediction feedback module includes a prediction classification unit, an occlusion detection unit and a result feedback unit; The prediction classification unit is used to predict whether new data is abnormal using a decision tree model; The occlusion detection unit is used to determine whether the abnormal space is occluded and call SSIM to verify visibility; The result feedback unit is used to combine the sensor and image prediction results to output warnings or manual verification instructions.

Images