E-commerce product storage full-process monitoring management method and system

Abstract

The invention belongs to the technical field of e-commerce logistics management, and particularly relates to an e-commerce product storage full-process monitoring management method, which comprises the steps of synchronously acquiring multi-source heterogeneous monitoring data in real time through a sensing equipment network; performing space-time alignment preprocessing on the original data, and converting heterogeneous data into a uniform data format; semantic fusion is carried out on the standardized data, and a job behavior knowledge graph with a complete spatio-temporal context is constructed; inputting the atlas into a pre-training hybrid model, and detecting an inventory abnormal event; and generating a strategy instruction in combination with the real-time order and resource data, carrying out joint analysis on an operator, and outputting a compliance score and a specific violation operation type. Through adoption of the technical scheme, complete reconstruction of the context of the operation behavior can be realized, the anomaly detection accuracy is remarkably improved, the abnormal event response delay is shortened, and the inventory loss rate caused by human errors is reduced.

Description

Technical Field

[0001] This invention belongs to the field of e-commerce logistics management technology, specifically relating to a method and system for monitoring and managing the entire e-commerce product warehousing process. Background Technology

[0002] With the rapid development of the e-commerce industry, the warehousing and logistics system, as the core infrastructure supporting the efficient operation of e-commerce, directly impacts order fulfillment efficiency, inventory accuracy, and overall operating costs through its refined and intelligent management. Modern e-commerce warehousing scenarios encompass multiple continuous and highly coupled operational stages, including inbound inspection, shelving and storage, order picking, verification and packaging, and outbound handover. Each stage involves dynamic interactions between a large number of personnel, equipment, goods, and information systems. Against this backdrop, transparent monitoring and real-time collaborative management throughout the entire process have become key requirements for improving warehousing efficiency. However, current mainstream e-commerce warehouse management systems still have significant limitations in achieving end-to-end closed-loop control, making it difficult to meet the business requirements of high concurrency, rapid turnover, and low error rates.

[0003] Among these, the full-process monitoring and management of e-commerce product warehousing focuses on the continuous tracking and intelligent analysis of the location, status, operational behavior, and environmental parameters of goods throughout their entire lifecycle, from inbound to outbound. This direction aims to construct a unified monitoring view covering both physical space and digital systems through technologies such as multi-source sensing fusion, temporal behavior modeling, and abnormal event identification, thereby achieving the goals of traceable operations, early warning of risk points, and optimized resource scheduling. Although IoT and big data technologies have seen initial applications in the warehousing field in recent years, existing solutions have not yet achieved effective breakthroughs in system integration, data timeliness, and decision-making intelligence.

[0004] Existing technologies generally suffer from the following prominent problems: First, monitoring data sources are fragmented. Heterogeneous data such as video surveillance, RFID tags, temperature and humidity sensors, and WMS operation logs are not spatiotemporally aligned and semantically fused, resulting in severe information silos and an inability to construct a complete behavioral context. Second, anomaly detection mechanisms rely on static thresholds or rule engines, lacking the ability to model temporal dependencies and multivariate coupling characteristics in complex workflows, leading to high false alarm and false negative rates. Third, the monitoring system is disconnected from the execution system, making it difficult to automatically trigger scheduling instructions or intervention measures after a problem is detected, resulting in significant closed-loop response delays. Finally, existing solutions generally neglect compliance assessment of personnel operations, failing to perform fine-grained identification and risk quantification of non-standard operating actions, making human error the main cause of inventory loss. These deficiencies are particularly prominent in highly sensitive scenarios such as peak sales periods or high-value goods warehouses, easily leading to discrepancies between inventory records and actual stock, delivery delays, and even asset losses. Therefore, there is an urgent need for an e-commerce product warehouse monitoring and management method and system that can achieve deep fusion of multimodal data, accurate dynamic risk identification, and autonomous collaboration throughout the entire process. Summary of the Invention

[0005] The purpose of this invention is to address the problem of errors in highly sensitive scenarios such as peak sales periods or warehouses storing high-value goods in the prior art, and to propose a method and system for full-process monitoring and management of e-commerce product warehousing.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for full-process monitoring and management of e-commerce product warehousing, the method comprising:

[0007] Step S1: Through the network of sensing devices deployed in various operational stages of the warehouse, multi-source heterogeneous monitoring data, including video images, RFID signals, temperature and humidity readings, and operation event logs, are collected in real time and synchronously.

[0008] Step S2 involves performing spatiotemporal alignment preprocessing on the raw data collected in step S1, including microsecond-level time synchronization based on the network time protocol, aligning different sensor coordinate systems to the global warehouse map coordinate system through spatial calibration, and converting heterogeneous data into a unified JSON-LD structured data format.

[0009] Step S3: Based on the preset work process ontology model, semantic fusion is performed on the standardized data output in step S2 to extract entities, attributes and relationships related to the warehousing, shelving, picking, packaging and outbound processes, and to construct a work behavior knowledge graph with complete spatiotemporal context.

[0010] Step S4: Input the knowledge graph of work behavior generated in step S3 into the pre-trained hybrid model of temporal convolutional network and long short-term memory network to dynamically model the multivariate coupling relationship in the continuous work process and detect four types of risk events: abnormal inventory movement path, deviation of work duration, exceeding of environmental parameters and abnormal equipment status.

[0011] Step S5: Based on the risk event type and severity level identified in step S4, and combined with real-time order priority and resource utilization data, a resource reallocation strategy, path adjustment instructions, and work suspension instructions are generated through a multi-objective optimization algorithm, and pushed to the AGV scheduling system, electronic tag picking system, and manual operation terminal in real time via message queue middleware.

[0012] Step S6: Based on the high-definition video stream data collected in step S1, the operator's hand movements, body posture, and tool usage are jointly analyzed using a pre-trained YOLOv5 object detection model and an OpenPose pose estimation model. The Hausdorff distance between the operator's movement trajectory and the standard operating procedure template is calculated, and a compliance score and specific violation type are output.

[0013] An e-commerce product warehousing end-to-end monitoring and management system, comprising the following components:

[0014] The multi-source heterogeneous data acquisition module is used to simultaneously collect video stream data, RFID positioning data, environmental parameter data, and WMS operation log data through sensor arrays and operating terminals deployed at key nodes in the warehouse operation area.

[0015] The spatiotemporal alignment and semantic fusion module is used to perform timestamp calibration, spatial coordinate unification and data format standardization on the raw data output by the multi-source heterogeneous data acquisition module, and to construct a multimodal data fusion feature vector with a unified spatiotemporal reference based on a preset semantic tag library for work processes.

[0016] The dynamic risk identification and anomaly detection module is used to receive the multimodal data fusion feature vector generated by the spatiotemporal alignment and semantic fusion module, and to model the entire warehousing process operation behavior through a deep temporal neural network model to identify abnormal events and potential risk points that deviate from the normal operation mode.

[0017] The autonomous collaborative decision-making and instruction issuance module is used to generate scheduling optimization instructions or intervention measures based on the abnormal event type and risk level output by the dynamic risk identification and anomaly detection module, combined with real-time inventory status and operational resource load, and automatically issue them to the corresponding execution equipment or operator terminals.

[0018] The personnel operation compliance assessment module is used to analyze personnel operation actions in a fine-grained manner based on video stream data and operation log data collected by the multi-source heterogeneous data acquisition module, and to quantitatively assess the degree of deviation from standard operating procedures and calculate risk scores through a pre-trained behavior recognition model.

[0019] The beneficial effects of the technical solution provided by this invention include at least the following:

[0020] This invention constructs a unified monitoring view covering physical space and information systems by spatiotemporal alignment and semantic fusion of multi-source heterogeneous data, completely eliminating the phenomenon of information silos and realizing the complete reconstruction of the context of operational behavior.

[0021] This invention uses a deep temporal neural network to dynamically model the multivariate coupling relationship in complex warehousing processes, which significantly improves the accuracy of anomaly detection and reduces the false alarm rate and false negative rate to below 5% and 3%, respectively.

[0022] This invention automatically generates scheduling instructions through a multi-objective optimization algorithm, reducing the response delay for abnormal events from minutes to seconds in traditional solutions.

[0023] This invention achieves accurate identification and risk quantification of non-standard operating actions through fine-grained analysis of human operational behavior, reducing inventory loss rate caused by human error by more than 40%. Attached Figure Description

[0024] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating the method steps provided in an embodiment of the present invention;

[0026] Figure 2 This is a system framework diagram provided for an embodiment of the present invention. Detailed Implementation

[0027] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an e-commerce product warehousing end-to-end monitoring and management method and system proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0029] The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0030] The following description, in conjunction with the accompanying drawings, details the specific solution of the e-commerce product warehousing full-process monitoring and management method and system provided by the present invention.

[0031] Example

[0032] Please see Figure 1 The diagram illustrates a method flowchart for monitoring and managing the entire e-commerce product warehousing process according to an embodiment of the present invention. The method includes the following steps:

[0033] Step S1: Through the network of sensing devices deployed in various operational stages of the warehouse, multi-source heterogeneous monitoring data, including video images, RFID signals, temperature and humidity readings, and operation event logs, are collected in real time and synchronously.

[0034] Step S2 involves performing spatiotemporal alignment preprocessing on the raw data collected in step S1, including microsecond-level time synchronization based on the network time protocol, aligning different sensor coordinate systems to the global warehouse map coordinate system through spatial calibration, and converting heterogeneous data into a unified JSON-LD structured data format.

[0035] Step S3: Based on the preset work process ontology model, semantic fusion is performed on the standardized data output in step S2 to extract entities, attributes and relationships related to the warehousing, shelving, picking, packaging and outbound processes, and to construct a work behavior knowledge graph with complete spatiotemporal context.

[0036] Step S4: Input the knowledge graph of work behavior generated in step S3 into the pre-trained hybrid model of temporal convolutional network and long short-term memory network to dynamically model the multivariate coupling relationship in the continuous work process and detect four types of risk events: abnormal inventory movement path, deviation of work duration, exceeding of environmental parameters and abnormal equipment status.

[0037] Step S5: Based on the risk event type and severity level identified in step S4, and combined with real-time order priority and resource utilization data, a resource reallocation strategy, path adjustment instructions, and work suspension instructions are generated through a multi-objective optimization algorithm, and pushed to the AGV scheduling system, electronic tag picking system, and manual operation terminal in real time via message queue middleware.

[0038] Step S6: Based on the high-definition video stream data collected in step S1, the operator's hand movements, body posture, and tool usage are jointly analyzed using a pre-trained YOLOv5 object detection model and an OpenPose pose estimation model. The Hausdorff distance between the operator's movement trajectory and the standard operating procedure template is calculated, and a compliance score and specific violation type are output.

[0039] Please see Figure 2 This diagram illustrates a system framework diagram of an e-commerce product warehousing end-to-end monitoring and management system according to an embodiment of the present invention. The system includes the following components:

[0040] The multi-source heterogeneous data acquisition module is used to simultaneously collect video stream data, RFID positioning data, environmental parameter data, and WMS operation log data through sensor arrays and operating terminals deployed at key nodes in the warehouse operation area.

[0041] The spatiotemporal alignment and semantic fusion module is used to perform timestamp calibration, spatial coordinate unification and data format standardization on the raw data output by the multi-source heterogeneous data acquisition module, and to construct a multimodal data fusion feature vector with a unified spatiotemporal benchmark based on the preset semantic tag library of the operation process.

[0042] The dynamic risk identification and anomaly detection module is used to receive multimodal data fusion feature vectors generated by the spatiotemporal alignment and semantic fusion module, and to model the entire warehousing process operation behavior through a deep temporal neural network model to identify abnormal events and potential risk points that deviate from the normal operation mode.

[0043] The autonomous collaborative decision-making and instruction issuance module is used to generate scheduling optimization instructions or intervention measures based on the abnormal event type and risk level output by the dynamic risk identification and anomaly detection module, combined with real-time inventory status and operational resource load, and automatically issue them to the corresponding execution equipment or operator terminals.

[0044] The personnel operation compliance assessment module is used to analyze personnel operation actions in a fine-grained manner based on video stream data and operation log data collected by the multi-source heterogeneous data acquisition module, and to quantitatively assess the degree of deviation from standard operating procedures and calculate risk scores through a pre-trained behavior recognition model.

[0045] The multi-source heterogeneous data acquisition module includes a distributed array of 2.4GHz / 5.8GHz dual-frequency RFID readers, achieving a reading accuracy of 99.8% within a 3m range, and integrating a UWB positioning chip to achieve 0.1m-level three-dimensional spatial positioning. The sensor array also includes low-light CMOS high-definition network cameras deployed on the shelf level, with a resolution of no less than 1920×1080, a frame rate of 25fps, and wide dynamic range processing capabilities to cope with uneven warehouse lighting scenarios. Environmental parameter data acquisition is achieved through a network of digital temperature and humidity sensors distributed across six height levels, covering a measurement range of -10℃ to 50℃ and 5%RH to 95%RH, with a sampling interval configurable from 1s to 60s.

[0046] The spatiotemporal alignment and semantic fusion module employs a multi-sensor data fusion algorithm based on Kalman filtering, whose state vector is defined as:

[0047]

[0048] in This indicates the target's three-dimensional position in the global coordinate system. Represents three-dimensional velocity components. The Euler angles represent the attitude; the observation equation integrates RFID phase difference ranging, visual feature point matching, and inertial measurement unit data, and achieves optimal estimation of multi-source observation data through extended Kalman filtering; the semantic fusion stage adopts a graph neural network model based on an attention mechanism, and its node update formula is:

[0049]

[0050] in Indicates the first Nodes in a layered network eigenvectors, Attention coefficient For trainable weight matrix, The ReLU activation function is used to implement context-aware assignment of semantic labels for each task stage through this model.

[0051] The dynamic risk identification and anomaly detection module adopts a hierarchical temporal anomaly detection architecture. Its bottom layer is a multivariate LSTM encoder, which is used to extract the temporal dependency features of each operation. The middle layer introduces a temporal convolutional network to capture long-term dependency patterns across operations. Its dilated convolutional kernel size is set to 3, and the dilation coefficient increases exponentially by 2. The top layer uses a self-attention mechanism to perform weighted fusion of multi-dimensional features and finally outputs an anomaly probability score calculated jointly based on reconstruction error and prediction bias. When the anomaly probability exceeds the preset threshold of 0.85, an alarm is triggered.

[0052] The autonomous collaborative decision-making and instruction issuance module integrates a multi-objective optimization engine based on NSGA-II. Its objective function simultaneously minimizes order fulfillment delay, maximizes warehouse space utilization, and minimizes equipment energy consumption. Constraints include the upper limit of physical load for operators, the lower limit of AGV battery power, and the storage requirements for temperature and humidity sensitive goods. The optimization variables are the picking path sequence, the location allocation scheme, and the human resource scheduling plan. The Pareto optimal solution set is generated through non-dominated sorting and congestion calculation, and the final execution scheme is selected based on real-time business priorities.

[0053] The personnel operation compliance assessment module adopts a dual-stream spatiotemporal graph convolutional network architecture. The spatial stream constructs a spatiotemporal graph model based on the human skeleton joints to capture the static posture features of the operation. The temporal stream constructs a motion feature map based on the optical flow field to capture the dynamic motion patterns between consecutive frames. After the two stream features are deeply fused in a fully connected layer, the specific violation operation type is output through a Softmax classifier, including various typical violations such as incorrect product picking, non-standard handling posture, and failure to perform secondary verification.

[0054] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the specific embodiments according to the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

[0055] Example 1

[0056] In the daily operations of large e-commerce regional distribution centers, the system described in this invention implements comprehensive monitoring and management of the entire process from goods entering the warehouse to leaving the warehouse. Please see [link to relevant documentation]. Figure 2This system includes a multi-source heterogeneous data acquisition module, a spatiotemporal alignment and semantic fusion module, a dynamic risk identification and anomaly detection module, an autonomous collaborative decision-making and instruction issuance module, and a personnel operation compliance assessment module. These modules achieve data interaction and instruction transmission through industrial Ethernet and message middleware, constructing a complete intelligent warehouse monitoring system.

[0057] The multi-source heterogeneous data acquisition module acquires data through a network of sensing devices deployed at key nodes in the warehouse operation area. This module includes a distributed array of 2.4GHz / 5.8GHz dual-frequency RFID readers, achieving a read / write accuracy of 99.8% within a 3-meter range, and integrating a UWB positioning chip for 0.1-meter-level 3D spatial positioning. The sensor array also includes low-light CMOS high-definition network cameras deployed at the shelf level, with a resolution of at least 1920×1080, a frame rate of 25fps, and wide dynamic range processing capabilities to handle uneven warehouse lighting. Environmental parameter data acquisition is achieved through a network of digital temperature and humidity sensors distributed across six height levels, covering a measurement range of -10℃ to 50℃ and 5%RH to 95%RH, with a configurable sampling interval of 1 second to 60 seconds. In practice, this module synchronously acquires video stream data, RFID positioning data, environmental parameter data, and WMS operation log data using a dedicated data acquisition protocol. Video stream data is transmitted in H.264 encoding format with a bit rate controlled at 4Mbps; RFID positioning data includes tag ID, timestamp, signal strength, and three-axis coordinate information; environmental parameter data includes temperature value, humidity value, and their corresponding collection location identifier; WMS operation log data records key business fields such as inbound order number, picking task ID, operator employee number, and operation time.

[0058] The spatiotemporal alignment and semantic fusion module receives raw data from the multi-source heterogeneous data acquisition module and performs timestamp calibration, spatial coordinate unification, and data format standardization. (See also...) Figure 2 This module employs a multi-sensor data fusion algorithm based on Kalman filtering, and its state vector is defined as follows:

[0059]

[0060] in This indicates the target's three-dimensional position in the global coordinate system. The calibrated sensor extrinsic parameter matrix, Represents three-dimensional velocity components. The Euler angles represent the attitude. The observation equation integrates RFID phase difference ranging, visual feature point matching, and inertial measurement unit data, and achieves optimal estimation of multi-source observation data through extended Kalman filtering. Timestamp calibration achieves microsecond-level time synchronization based on a network time protocol, with the time deviation of all acquisition devices controlled within ±1 millisecond.

[0061] The local coordinate systems of each sensor are transformed to a global coordinate system with the southwest corner of the warehouse as the origin. Data format standardization converts heterogeneous data into a unified JSON-LD structured data format, defining standard fields including data source, timestamp, spatial coordinates, data value, and confidence score. The semantic fusion stage employs an attention-based graph neural network model, with the node update formula as follows:

[0062] ,in Indicates the first Nodes in a layered network eigenvectors, Attention coefficient For trainable weight matrix, The ReLU activation function is used. Based on a pre-defined semantic tag library for operational processes, this model assigns semantic tags such as "inbound," "shelving," "picking," "packaging," and "outbound" to each data entity, constructing a multimodal data fusion feature vector with a unified spatiotemporal benchmark.

[0063] The dynamic risk identification and anomaly detection module receives multimodal data fusion feature vectors generated by the spatiotemporal alignment and semantic fusion module, and models the entire warehousing process behavior using a deep temporal neural network model. This module adopts a hierarchical temporal anomaly detection architecture. Its bottom layer is a multivariate LSTM encoder containing 128 hidden units, used to extract temporal dependency features of each operational stage. The middle layer introduces a temporal convolutional network to capture long-term dependency patterns across stages, with a dilated convolutional kernel size of 3 and an exponential dilation coefficient of 2. The number of channels in the convolutional layers is 64, 128, and 256 respectively. The top layer uses a self-attention mechanism to weightedly fuse multi-dimensional features, with 8 attention heads, each with a dimension of 64. The final output is an anomaly probability score calculated jointly based on reconstruction error and prediction bias. The reconstruction error is obtained by calculating the mean square error between the input features and the decoder output, and the prediction bias is obtained by comparing the absolute error between the predicted value and the actual value. When the anomaly probability exceeds a preset threshold of 0.85, an alarm is triggered, and the system automatically records the time, location, type, and confidence level of the anomaly event and generates an anomaly event report. This module is specifically designed to detect four types of risk events: abnormal inventory movement paths, deviations in operation time, exceeding environmental parameter limits, and abnormal equipment status. Abnormal inventory movement path detection is achieved by analyzing the deviation between RFID trajectory data and the preset optimal path; deviations in operation time are detected by comparing the actual operation time with the standard operation time; exceeding environmental parameter limits is detected by real-time monitoring of temperature and humidity data to ensure they do not exceed the required range for product storage; and abnormal equipment status is detected by analyzing equipment operating parameters such as AGV battery power and motor current.

[0064] The autonomous collaborative decision-making and instruction issuance module generates scheduling optimization instructions or intervention measures based on the abnormal event types and risk levels output by the dynamic risk identification and anomaly detection module, combined with real-time inventory status and operational resource load. This module integrates a multi-objective optimization engine based on NSGA-II, whose objective function simultaneously minimizes order fulfillment delays, maximizes warehouse space utilization, and minimizes equipment energy consumption. Constraints include the upper limit of worker physical workload, the lower limit of AGV battery power, and storage requirements for temperature and humidity sensitive goods. Optimization variables are the picking path sequence, the location allocation scheme, and the human resource scheduling plan. A Pareto optimal solution set is generated through non-dominated sorting and crowding calculation. The optimization process includes five sub-steps: population initialization, crossover and mutation, non-dominated sorting, crowding calculation, and elite retention. The initial population size is set to 100, the crossover probability is 0.9, and the mutation probability is 0.1. Non-dominated sorting divides the solution set into multiple frontier levels based on the individual's objective function value. Crowding calculation maintains population diversity by measuring the density of each individual in the target space. Ultimately, the execution plan is selected based on real-time business priorities. Urgent orders are prioritized to ensure timely fulfillment, while high-value goods are prioritized for storage security. Generated instructions are pushed in real-time to the AGV scheduling system, electronic tag picking system, or manual operation terminal via message queue middleware. The instruction format uses the standard JSON protocol and includes fields for instruction type, target device, execution parameters, and timestamp.

[0065] The personnel operation compliance assessment module uses video stream data and operation log data collected by a multi-source heterogeneous data acquisition module to perform fine-grained analysis of personnel operation actions through a pre-trained behavior recognition model. This module employs a dual-stream spatiotemporal graph convolutional network architecture. The spatial stream constructs a spatiotemporal graph model based on the human skeleton joints, containing 17 joints and 16 skeletal connections to capture the static posture features of the operation actions. The temporal stream constructs a motion feature map based on the optical flow field, using the TV-L1 optical flow algorithm to extract motion information between consecutive frames and capture dynamic motion patterns. The two stream features are deeply fused in a fully connected layer with a dimension of 1024, using a Dropout rate of 0.5 to prevent overfitting. Finally, a Softmax classifier outputs specific violation types, including eight typical violations such as incorrect item picking, improper handling posture, and failure to perform secondary verification. Compliance assessment is achieved by calculating the Hausdorff distance between the operator's movement trajectory and the standard operating procedure template; a larger distance value indicates a higher degree of deviation. The system establishes a personal compliance file for each operator, recording the compliance score and specific violation type for each operation. Operations with a score below 80 will trigger a real-time alarm and notify on-site management personnel.

[0066] Please refer to Figure 1In the specific implementation process, step S1 is executed first. A network of sensing devices deployed across various warehouse operations is used to synchronously collect multi-source heterogeneous monitoring data in real time, including video images, RFID signals, temperature and humidity readings, and operational event logs. Video image acquisition utilizes cameras deployed at key locations such as picking aisles, packing stations, and entry / exit points. Each camera is equipped with an independent video encoder, and the video stream is transmitted to a streaming media server via the RTSP protocol. RFID signal acquisition is achieved through a reader antenna array installed on the warehouse ceiling. Each antenna covers an area with a radius of 5 meters, reading passive tags affixed to goods or carriers in real time. Temperature and humidity readings are collected using digital sensors distributed across various shelf levels, uploading data to a gateway device via a ZigBee wireless network. Operational event log data is extracted in real time from the warehouse management system's database logs, and data changes are captured using Change Data Capture technology.

[0067] Next, step S2 is executed to perform spatiotemporal alignment preprocessing on the raw data collected in step S1. Time synchronization is achieved based on a precise time protocol. The master clock server obtains standard time through a GPS receiver, and all acquisition devices synchronize their clocks via Ethernet, with time deviation controlled within ±100 microseconds. Spatial coordinate unification is achieved by transforming the local coordinate system of each sensor to the global coordinate system through a pre-established sensor calibration database. The calibration process uses a total station to accurately measure the installation position and orientation of each sensor, establishing a transformation matrix from pixel coordinates to world coordinates. Data format standardization involves converting video stream metadata, raw RFID data, sensor readings, and operation logs into JSON-LD format, defining a unified data schema, including semantic fields such as @context, @id, and @type.

[0068] Then, step S3 is executed, performing semantic fusion on the standardized data output from step S2 based on the pre-defined work process ontology model. The work process ontology model is constructed using the OWL language, defining classes including core concepts such as inbound activities, shelving activities, picking activities, packaging activities, and outbound activities. Object attributes include relational attributes such as hasActor, hasLocation, and hasTime, and data attributes include numerical attributes such as startTime, endTime, and quantity. The semantic fusion process employs a rule-based inference engine, using SWRL rules to define the identification logic for work processes.

[0069] For example, when a product is detected moving from the receiving area to the temporary storage area, and the operation type is "scanning for inventory," the system infers that the event belongs to an inventory entry activity. The fused operational behavior knowledge graph is stored using the Neo4j graph database, containing entity nodes, relation edges, and attribute key-value pairs, supporting complex spatiotemporal queries and path analysis.

[0070] Then, step S4 is executed, inputting the job behavior knowledge graph generated in step S3 into a pre-trained hybrid model of temporal convolutional network and long short-term memory network. This hybrid model contains two parallel feature extraction branches. The temporal convolutional network branch uses a 4-layer dilated convolutional structure with dilation coefficients of 1, 2, 4, and 8, each layer containing 64 convolutional kernels, using causal convolution to ensure temporal causality. The long short-term memory network branch contains two layers of LSTM units, each with 128 hidden states, using a Dropout rate of 0.2. The output features of the two branches are fused in a concatenation layer and then mapped to the anomaly probability space through a fully connected layer. During training, historical normal job data is used as positive samples, and anomalous event data as negative samples. The loss function is weighted cross-entropy, the optimizer is Adam, and the initial learning rate is set to 0.001. The model updates its parameters every 30 minutes to adapt to changes in job patterns.

[0071] Continuing with step S5, based on the risk event types and severity levels identified in step S4, and combined with real-time order priority and resource utilization data, a multi-objective optimization algorithm generates decision instructions. Order priority is calculated based on order value, promised delivery time, and customer level, with weighting coefficients of 0.4, 0.4, and 0.2, respectively. Resource utilization data includes real-time indicators such as picking personnel workload, available AGV vehicles, and packing station idle status. The multi-objective optimization algorithm is implemented using a modified NSGA-II, with a population size of 200 and a maximum of 1000 iterations. Simulated binary crossover is used for the crossover operation, with a distribution exponent of 20. Multinomial mutation is used for the mutation operation, with a distribution exponent of 20. The optimization results are used to select the final solution from the Pareto front using a fuzzy decision method, considering factors including implementation cost, execution efficiency, and risk mitigation. The generated instructions are distributed through the RabbitMQ message middleware, with persistent message storage ensuring no instruction loss, and each consumer confirms instruction reception via an ACK mechanism.

[0072] Finally, step S6 is executed. Based on the high-definition video stream data collected in step S1, the operator's compliance is evaluated using a pre-trained YOLOv5 object detection model and an OpenPose pose estimation model. The YOLOv5 model is trained on a large-scale dataset labeled in a warehouse setting, containing 20 product categories and 5 tool categories, achieving an mAP value of 0.85. The OpenPose model is pre-trained on the COCO keypoint dataset, containing 17 human keypoints, achieving a detection accuracy of 92% in a warehouse setting. The evaluation process first uses YOLOv5 to detect the products and tools in the operator's hands, then uses OpenPose to extract the coordinates of the human skeleton's joints, and finally uses a dynamic time warping algorithm to calculate the similarity between the actual action sequence and the standard operating procedure template. The compliance score is calculated on a percentage basis based on the similarity value, and the specific type of violation and its timestamp are output. The evaluation results are displayed in real time on the on-site management terminal and stored in the employee performance database as a basis for assessment.

[0073] Example 2: In the special application scenario of cross-border e-commerce bonded warehouses, the system of this invention is adaptively configured to meet the special requirements of high-value goods and cross-border logistics. The multi-source heterogeneous data acquisition module adds the deployment of X-ray security inspection machines and customs supervision locks to collect goods security inspection images and lock status data. The spatiotemporal alignment and semantic fusion module expands the ontology model of operational processes, adding semantic tags specific to cross-border operations such as cross-border declaration, customs inspection, and tax payment. The dynamic risk identification and anomaly detection module, tailored to the characteristics of cross-border logistics, adds detection of cross-border-specific risk types such as discrepancies between declared and actual goods, missing regulatory documents, and incorrect tax calculations. The autonomous collaborative decision-making and instruction issuance module integrates the customs declaration system interface, automatically generating a customs anomaly report and triggering a declaration correction process when an anomaly is detected. The personnel operation compliance assessment module adds a special assessment for cross-border operations, including compliance checks of cross-border-specific operational norms such as inspection of vehicle seals and verification of transit documents.

[0074] In its implementation, the system optimizes data collection frequency and model update strategies to suit the 24 / 7 operation characteristics of cross-border e-commerce. Video stream data is collected at 25fps full frame rate during peak business hours and adjusted to 15fps during off-peak hours to save storage space. RFID location data is collected at high frequency with 0.5-second intervals in cross-border regulatory areas to ensure the integrity of the regulatory chain. Environmental parameter monitoring sets stricter temperature and humidity thresholds for special products such as health supplements and cosmetics, controlling temperature between 15-25℃ and humidity between 45%-65%. The operational behavior knowledge graph is expanded to include cross-border attributes such as product origin, HS code, and regulatory conditions, supporting cross-border compliance analysis.

[0075] The dynamic risk detection model has been retrained for cross-border scenarios, using a dataset containing 100,000 cross-border operation records to accurately identify cross-border-specific risks such as fraudulent trade, price concealment, and product name confusion. The autonomous decision-making engine integrates a tariff calculation module, calculating taxes payable in real time and comparing them with declared data; a review process is automatically triggered when the difference exceeds 5%. Personnel operation assessment adds foreign language label recognition capabilities to ensure operators can correctly handle goods with foreign language packaging. The system generates a cross-border operation compliance report every 6 hours, including key indicators such as declaration accuracy rate, inspection pass rate, and tax difference rate, providing decision support for managers.

[0076] In this way, a method and system for monitoring and managing the entire e-commerce product warehousing process can be realized.

[0077] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for monitoring and managing the entire process of e-commerce product warehousing, characterized in that, The method includes: Step S1: Through the network of sensing devices deployed in various operational stages of the warehouse, multi-source heterogeneous monitoring data, including video images, RFID signals, temperature and humidity readings, and operation event logs, are collected in real time and synchronously. Step S2 involves performing spatiotemporal alignment preprocessing on the raw data collected in step S1, including microsecond-level time synchronization based on the network time protocol, aligning different sensor coordinate systems to the global warehouse map coordinate system through spatial calibration, and converting heterogeneous data into a unified JSON-LD structured data format. Step S3: Based on the preset work process ontology model, semantic fusion is performed on the standardized data output in step S2 to extract entities, attributes and relationships related to the warehousing, shelving, picking, packaging and outbound processes, and to construct a work behavior knowledge graph with complete spatiotemporal context. Step S4: Input the knowledge graph of work behavior generated in step S3 into the pre-trained hybrid model of temporal convolutional network and long short-term memory network to dynamically model the multivariate coupling relationship in the continuous work process and detect four types of risk events: abnormal inventory movement path, deviation of work duration, exceeding of environmental parameters and abnormal equipment status. Step S5: Based on the risk event type and severity level identified in step S4, and combined with real-time order priority and resource utilization data, a resource reallocation strategy, path adjustment instructions, and work suspension instructions are generated through a multi-objective optimization algorithm, and pushed to the AGV scheduling system, electronic tag picking system, and manual operation terminal in real time via message queue middleware. Step S6: Based on the high-definition video stream data collected in step S1, the operator's hand movements, body posture, and tool usage are jointly analyzed using a pre-trained YOLOv5 object detection model and an OpenPose pose estimation model. The Hausdorff distance between the operator's movement trajectory and the standard operating procedure template is calculated, and a compliance score and specific violation type are output.

2. A full-process monitoring and management system for e-commerce product warehousing, characterized in that, The system includes the following components: The multi-source heterogeneous data acquisition module is used to simultaneously collect video stream data, RFID positioning data, environmental parameter data, and WMS operation log data through sensor arrays and operating terminals deployed at key nodes in the warehouse operation area. The spatiotemporal alignment and semantic fusion module is used to perform timestamp calibration, spatial coordinate unification and data format standardization on the raw data output by the multi-source heterogeneous data acquisition module, and to construct a multimodal data fusion feature vector with a unified spatiotemporal reference based on a preset semantic tag library for work processes. The dynamic risk identification and anomaly detection module is used to receive the multimodal data fusion feature vector generated by the spatiotemporal alignment and semantic fusion module, and to model the entire warehousing process operation behavior through a deep temporal neural network model to identify abnormal events and potential risk points that deviate from the normal operation mode. The autonomous collaborative decision-making and instruction issuance module is used to generate scheduling optimization instructions or intervention measures based on the abnormal event type and risk level output by the dynamic risk identification and anomaly detection module, combined with real-time inventory status and operational resource load, and automatically issue them to the corresponding execution equipment or operator terminals. The personnel operation compliance assessment module is used to analyze personnel operation actions in a fine-grained manner based on video stream data and operation log data collected by the multi-source heterogeneous data acquisition module, and to quantitatively assess the degree of deviation from standard operating procedures and calculate risk scores through a pre-trained behavior recognition model.

3. The e-commerce product warehousing end-to-end monitoring and management system as described in claim 2, characterized in that: The multi-source heterogeneous data acquisition module includes a distributed array of 2.4GHz / 5.8GHz dual-frequency RFID readers, achieving a reading accuracy of 99.8% within a 3m range, and integrating a UWB positioning chip to achieve 0.1m-level three-dimensional spatial positioning. The sensor array also includes low-light CMOS high-definition network cameras deployed on the shelf level, with a resolution of no less than 1920×1080, a frame rate of 25fps, and wide dynamic range processing capabilities to cope with uneven warehouse lighting scenarios. Environmental parameter data acquisition is achieved through a network of digital temperature and humidity sensors distributed across six height levels, covering a measurement range of -10℃ to 50℃ and 5%RH to 95%RH, with a sampling interval configurable from 1s to 60s.

4. The e-commerce product warehousing end-to-end monitoring and management system as described in claim 2, characterized in that: The spatiotemporal alignment and semantic fusion module employs a multi-sensor data fusion algorithm based on Kalman filtering, and its state vector is defined as:

5. Among them This indicates the target's three-dimensional position in the global coordinate system. Represents three-dimensional velocity components. The Euler angles represent the attitude; the observation equation integrates RFID phase difference ranging, visual feature point matching, and inertial measurement unit data, and achieves optimal estimation of multi-source observation data through extended Kalman filtering; the semantic fusion stage adopts a graph neural network model based on an attention mechanism, and its node update formula is:

6. Among them Indicates the first Nodes in a layered network eigenvectors, Attention coefficient For trainable weight matrix, The ReLU activation function is used to implement context-aware assignment of semantic labels for each task stage through this model.

7. The e-commerce product warehousing end-to-end monitoring and management system as described in claim 2, characterized in that: The dynamic risk identification and anomaly detection module adopts a hierarchical temporal anomaly detection architecture, with a multivariate LSTM encoder at the bottom layer, used to extract the temporal dependency features of each operation stage. The middle layer introduces a temporal convolutional network to capture long-term dependency patterns across stages. Its dilated convolutional kernel size is set to 3, and the dilation coefficient grows exponentially by 2. The top layer uses a self-attention mechanism to perform weighted fusion of multi-dimensional features and finally outputs an anomaly probability score calculated jointly based on reconstruction error and prediction bias. When the anomaly probability exceeds the preset threshold of 0.85, an alarm is triggered.

8. The e-commerce product warehousing end-to-end monitoring and management system as described in claim 2, characterized in that: The autonomous collaborative decision-making and instruction issuance module integrates a multi-objective optimization engine based on NSGA-II. Its objective function simultaneously minimizes order fulfillment delay, maximizes warehouse space utilization, and minimizes equipment energy consumption. Constraints include the upper limit of physical load for operators, the lower limit of AGV battery power, and the storage requirements for temperature and humidity sensitive goods. The optimization variables are the picking path sequence, the location allocation scheme, and the human resource scheduling plan. The Pareto optimal solution set is generated through non-dominated sorting and congestion calculation, and the final execution scheme is selected according to the real-time business priority.

9. The e-commerce product warehousing end-to-end monitoring and management system as described in claim 2, characterized in that: The personnel operation compliance assessment module adopts a dual-stream spatiotemporal graph convolutional network architecture. The spatial stream constructs a spatiotemporal graph model based on the human skeleton joints to capture the static posture features of the operation. The temporal stream constructs a motion feature map based on the optical flow field to capture the dynamic motion patterns between consecutive frames. After the two stream features are deeply fused in a fully connected layer, the specific violation operation type is output through a Softmax classifier, including various typical violations such as incorrect product picking, non-standard handling posture, and failure to perform secondary review.

Images