A method and system for spatiotemporal prediction of behavior of an edge ai chip

Abstract

The application relates to the technical field of edge AI computing systems of graph neural networks, and discloses a behavior space-time prediction method and system of an edge AI chip. The method comprises the following steps: extracting customers and shelf entities in real time based on a video stream, and constructing a dynamic space-time node set; generating a sparse heterogeneous space-time high-order correlation structure based on the node set; and performing behavior prediction and loss prevention response with topological perception based on the structure through a topological graph neural network after knowledge distillation. The application is aimed at solving the problems of high memory occupation in traditional unstructured data processing, difficulty of traditional graph structure in capturing multi-body interaction, low behavior discrimination precision of an edge model, and lag of a loss prevention response. Through model quantization pruning, dynamic sparse pruning, and multi-dimensional loss prevention evidence fusion, the application adapts to resource constraints of an edge chip, improves the fine behavior discrimination precision and real-time response speed, and realizes accurate loss prevention and shelf layout optimization in an unmanned sales scene.

Description

Technical Field

[0001] This invention relates to the field of edge AI computing system technology of graph neural networks, and specifically to a spatiotemporal prediction method and system for the behavior of edge AI chips. Background Technology

[0002] In unmanned retail scenarios, video analytics is typically employed to achieve customer behavior understanding, intelligent loss prevention, and dynamic shelf management. This technology uses multiple cameras deployed within the store to capture images of customer activity, transmits the image data to a processing unit, and uses algorithms to analyze and predict customer movement trajectories and behavioral patterns. The processing unit can be a cloud server or an edge computing device. When using edge computing devices, the goal is to complete processing closer to the data source to reduce network latency and bandwidth consumption, and improve the real-time performance of data processing. Such systems typically rely on edge AI chips with a specific architecture, containing dedicated neural network processors, general-purpose processors, and storage units, all working together to perform model inference tasks. The algorithm model needs to be able to process spatiotemporal information to capture the temporal continuity and spatial correlation of customer behavior.

[0003] However, in practical applications, existing algorithms based on complex spatiotemporal models face performance bottlenecks when running on edge AI chips with specific architectures. This problem stems from a mismatch between the data access patterns required by tensor operations within the algorithm model and the bandwidth allocation strategy of the chip's memory system. Simultaneously, there is a lack of coordination between the density of different operators during model computation and the chip's peak computational capabilities optimized for specific operators. This hardware-software mismatch leads to inefficient model execution, manifesting as inference speeds failing to meet real-time requirements and high energy consumption per unit of computation. Under continuous operation, excessive chip resource consumption may affect the normal operation of other functional modules, ultimately resulting in sluggish overall system response or abnormal interruptions. For example, during periods of high customer traffic, the system's prediction of customer trajectories may lag, preventing loss prevention mechanisms from responding promptly to sudden behaviors, and delaying data updates required for dynamic shelf management, thus affecting the accuracy of operational decisions. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, the present invention provides a spatiotemporal prediction method for the behavior of edge AI chips, comprising:

[0005] Step S1: Extract customers and shelf entities in real time based on video stream and construct a dynamic spatiotemporal node set;

[0006] Step S2: Based on the dynamic spatiotemporal node set, generate a sparse heterogeneous spatiotemporal high-order correlation structure;

[0007] Step S3: Based on the sparse heterogeneous spatiotemporal high-order correlation structure, perform topology-aware behavior prediction and loss prevention response.

[0008] Furthermore, the step of extracting customers and shelf entities in real time based on video streams and constructing a dynamic spatiotemporal node set includes:

[0009] S11: Deploy and run a lightweight multi-task perception model on an edge AI chip to perform entity detection on the video stream and obtain a set of entity detection results.

[0010] S12: Based on the customer human body bounding box and human skeleton key points in the entity detection result set, perform online pedestrian re-identification operation and generate a customer ID mapping table;

[0011] S13: Based on the 2D entity bounding boxes in the entity detection result set, combined with the assumption of the camera participating in the ground plane, perform a spatial coordinate transformation operation to obtain a set of relative three-dimensional spatial coordinates;

[0012] S14: Based on the entity detection result set, customer ID mapping table, and relative three-dimensional space coordinate set, perform entity node encapsulation operation to generate a node object set; each node object in the node object set contains five core fields, namely semantic category, relative three-dimensional space coordinates, skeleton vector, unique ID, and timestamp;

[0013] S15: Construct a dynamic spatiotemporal node set based on a collection of node objects.

[0014] Furthermore, the operation steps of the lightweight multi-task perception model include:

[0015] S111: Perform quantization processing on the lightweight multi-task perception model, specifically including: using historical customer activity video frames in the store as calibration samples, converting the original high-precision floating-point parameters of the model into low-precision integer parameters through a calibration algorithm that minimizes inference error;

[0016] S112: Perform pruning on the quantized lightweight multi-task perception model, specifically including: evaluating the contribution of each convolutional kernel to the inference result and removing redundant convolutional kernels whose contribution is lower than a preset contribution threshold.

[0017] S113: Input the video stream frame by frame into the lightweight multi-task perception model, output the customer human body bounding box, human skeleton key points, shelf area bounding box, product instance bounding box and product category information for each frame, integrate the detection results of all frames to form an entity detection result set.

[0018] Furthermore, the process of the online pedestrian re-identification operation includes:

[0019] S121: For the bounding box of the customer's body in each frame of the entity detection result set, output the appearance feature vector through the feature extraction branch to obtain the set of appearance feature vectors of the customer in each frame;

[0020] S122: Based on the appearance feature vector of each customer in the current frame, calculate the cosine similarity with the appearance feature vectors of all customers in the previous frame; at the same time, based on the change of the center coordinates of the customer's human body bounding box between the two frames, obtain the customer's motion vector, and calculate the motion consistency score based on the customer's motion vector; and sum the cosine similarity and the motion consistency score by weight to obtain the cross-frame matching score.

[0021] S123: If the cross-frame matching score between the current frame customer and the previous frame customer is higher than the preset matching threshold, then the previous frame customer ID is assigned to the current frame customer; if the current frame customer has no matching previous frame customer ID, then a new unique ID is generated; record the index association relationship between each frame customer ID and the corresponding customer human body bounding box and human skeleton key points to form a customer ID mapping table.

[0022] Furthermore, the step of generating a sparse heterogeneous spatiotemporal higher-order correlation structure based on the dynamic spatiotemporal node set includes:

[0023] S21: Based on the timestamps and node subsets of the dynamic spatiotemporal node set, a set of sub-window node subsets is generated by dividing the time window through a sliding time window;

[0024] S22: Based on the subset of nodes in the window and the semantic category labels of nodes, perform heterogeneous node pair filtering and spatial distance calculation to generate a first-order association set; the semantic category labels of nodes are read from the node objects of the dynamic spatiotemporal node set, with "human" type nodes marked as semantic category 1, and "goods" and "shelf" type nodes uniformly marked as semantic category 0;

[0025] S23: Based on the unique IDs of the 1st-order association set, the window node subset set, and the dynamic spatiotemporal node set, perform 2nd-order association unit construction to generate an initial 2nd-order association unit set;

[0026] S24: Based on the unique ID and skeleton vector of the dynamic spatiotemporal node set, perform temporal trajectory unit modeling and generate a set of temporal trajectory units;

[0027] S25: Based on the initial set of second-order association units and the memory resource threshold of the edge AI chip, perform dynamic sparse pruning to generate a pruned set of higher-order association units;

[0028] S26: Based on the set of higher-order correlation units after pruning, perform boundary operator construction operations to generate a boundary operator matrix;

[0029] S27: Integrate the pruned high-order correlation unit set, the temporal trajectory unit set, and the boundary operator matrix, perform structural validity verification, and generate a sparse heterogeneous spatiotemporal high-order correlation structure; the sparse heterogeneous spatiotemporal high-order correlation structure is a structured data set containing 0-order units, 1-order correlation sets, 2-order correlation units, temporal trajectory units, and boundary operators.

[0030] Furthermore, the process of the dynamic sparse pruning operation includes:

[0031] S251: For each initial second-order association unit, the interaction confidence is obtained by weighted summation of spatial distance score, action consistency score, and node detection confidence.

[0032] S252: Sort the initial 2nd-order unit set in descending order of interaction confidence, and retain units whose number does not exceed the maximum number of 2nd-order associated units;

[0033] S253: Simultaneously retain the first-order association set corresponding to the selected unit, and remove the remaining second-order association units and the first-order association set to form a pruned set of higher-order association units.

[0034] Furthermore, the steps of performing topology-aware behavior prediction and loss prevention response include:

[0035] S31: Based on the sparse heterogeneous spatiotemporal high-order correlation structure, the topological graph neural network compressed by knowledge distillation is called to perform the model initialization operation and generate the initialized topological graph neural network model.

[0036] S32: Based on the initialized topological graph neural network model, combined with the sparse heterogeneous spatiotemporal high-order correlation structure, cross-order message passing operation is performed to generate 1st-order aggregate features and 2nd-order geometric feature vectors;

[0037] S33: Based on the temporal trajectory units contained in the sparse heterogeneous spatiotemporal high-order correlation structure, perform temporal evolution feature extraction operations to generate temporal feature vectors;

[0038] S34: Based on the 1st-order aggregated features, 2nd-order geometric feature vectors, and time-series feature vectors, perform feature fusion operation to generate fused feature vectors;

[0039] S35: Based on the fused feature vector, the prediction head of the initialized topological graph neural network model is used to perform behavior prediction and spatial trajectory offset prediction, and generate the probability distribution of customer behavior categories and spatial trajectory offset.

[0040] S36: Based on the probability distribution of customer behavior categories, spatial trajectory offset, and dynamic spatiotemporal node set associated with sparse heterogeneous spatiotemporal high-order correlation structure, perform multi-dimensional loss prevention evidence collection and generate trajectory departure judgment results, abnormal posture detection results, and lingering judgment results.

[0041] S37: Based on the trajectory departure determination result, abnormal posture detection result and lingering determination result, perform comprehensive risk score calculation and generate comprehensive risk score and loss prevention response instruction;

[0042] S38: Based on loss prevention response instructions and records of retrieval-unsettled behavior, perform spatial clustering of high-risk events to generate high-risk area clustering results and shelf layout optimization schemes.

[0043] Furthermore, the process of the model initialization operation includes:

[0044] S311: Training the teacher model, specifically including: using a historical interaction sample set containing the entire customer shopping process, allowing the teacher model to learn the mapping relationship between high-order correlation features and customer behavior, and outputting the intermediate layer features and prediction probabilities of the teacher model;

[0045] S312: Construct a student model, specifically including: reducing the number of convolutional layers and neurons in the teacher model, retaining only the modules that capture 1st-order aggregate features and 2nd-order geometric feature vectors;

[0046] S313: The student model is jointly trained by combining the KL divergence loss of the student model output and the soft labels of the teacher model with the cross-entropy loss of the student model prediction results and the real customer behavior labels; at the same time, the model parameter scale is adjusted through the computing power simulator of the edge AI chip.

[0047] S314: The trained Topo-GNN model is the initialized Topo-GNN model, which is the topological graph neural network model.

[0048] Furthermore, the process for collecting multi-dimensional evidence for loss prevention includes:

[0049] S361: Based on the store layout map and customer movement trajectory data from the past three months, the area that more than 90% of customers pass through is selected as the regular area and stored as a set of area coordinates.

[0050] S362: If the probability of taking something in the probability distribution of customer behavior categories exceeds the threshold, combine the spatial trajectory offset to predict the position of the next 3 frames, determine whether it continues to move away from the regular shopping process area, and output the trajectory moving away judgment result.

[0051] S363: Perform abnormal posture detection, extract the skeleton vector of the customer in the dynamic spatiotemporal node set, calculate the angle between the arm and the torso, and the rate of change of the angle between the waist and the leg. If any abnormal condition is met, output the abnormal posture detection result.

[0052] S364: Perform a dwell time determination, count the duration of the customer in the non-selection area, and output the dwell time determination result if it exceeds the preset dwell time.

[0053] A spatiotemporal prediction system for the behavior of an edge AI chip, used to implement the aforementioned spatiotemporal prediction method for the behavior of an edge AI chip, the system comprising:

[0054] Dynamic spatiotemporal node set construction module: used to extract customers and shelf entities in real time based on video streams and construct dynamic spatiotemporal node sets;

[0055] Sparse heterogeneous spatiotemporal higher-order correlation structure generation module: used to generate sparse heterogeneous spatiotemporal higher-order correlation structures based on the dynamic spatiotemporal node set;

[0056] Topology-aware behavior prediction and loss prevention response module: used to perform topology-aware behavior prediction and loss prevention response based on sparse heterogeneous spatiotemporal high-order correlation structure.

[0057] Compared to existing technologies, the beneficial effects of this invention are as follows: Addressing the core issues in unmanned sales scenarios, such as the mismatch between edge AI chip hardware and software, and the limitation of traditional graph structures to modeling only binary associations, leading to difficulties in discerning subtle behaviors and delayed loss prevention responses, the hardware-software mismatch manifests as an imbalance between model data access patterns and memory bandwidth, and a mismatch between operator density and chip peak computing power. This invention achieves a targeted breakthrough through layered technology design. First, the quantization and pruning optimization of the lightweight multi-task perception model allows the model data access pattern to adapt to the edge AI chip's memory bandwidth allocation strategy, coordinating operator density with the chip's peak computing power. This fundamentally resolves the hardware-software compatibility contradiction, preventing excessive chip resource consumption from affecting functional module collaboration and reducing unit computing energy consumption, providing a hardware adaptation foundation for the model's continuous and stable operation. Second, the sparse heterogeneous spatiotemporal high-order association structure retains high-value "person-product-shelf" multi-entity interaction units through dynamic pruning, breaking through the limitations of traditional graph structure binary associations in expression, and enabling the identification of subtle customer behaviors such as "touching" and "taking". The system provides key feature support for discrimination and eliminates redundant computing units, further reducing the computing load on edge chips and achieving a balance between "precise correlation modeling" and "efficient resource utilization". In addition, the knowledge-distilled Topo-GNN model, combined with multi-dimensional loss prevention evidence fusion, maintains high-precision behavior discrimination capabilities while ensuring the model's lightweight nature, avoiding misjudgments based on single rules, and solves the problem of delayed loss prevention response with millisecond-level inference speed. Meanwhile, spatial clustering of high-risk events transforms loss prevention data into a basis for shelf layout optimization, enabling the system to upgrade from "passive loss prevention response" to "proactive operation optimization", forming a complete closed loop for unmanned sales scenario services. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart of a spatiotemporal prediction method for the behavior of an edge AI chip according to the present invention;

[0060] Figure 2 This is a flowchart of the online pedestrian re-identification operation in an embodiment of the present invention;

[0061] Figure 3 This is a flowchart of dynamic sparse pruning in an embodiment of the present invention;

[0062] Figure 4 This is a functional block diagram of a behavior spatiotemporal prediction system for an edge AI chip in this invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Example 1:

[0065] Please see Figure 1 As shown, this embodiment provides a spatiotemporal prediction method for the behavior of an edge AI chip, including:

[0066] Step S1: Extract customers and shelf entities in real time based on video stream and construct a dynamic spatiotemporal node set.

[0067] This step addresses the video analytics needs of edge AI chips in unmanned sales scenarios. It tackles the issues of high memory consumption and processing latency caused by traditional methods handling unstructured data (pixel sequences, fixed grid features). By extracting key information about customers and shelves from the edge AI chip, a structured dynamic spatiotemporal node set is constructed, providing lightweight atomic units for subsequent high-order relationship modeling. First, continuous video stream data is received from in-store cameras. This video stream data is in RGB format (and includes depth image data if the camera supports RGB-D), with the frame rate and resolution adapted to the edge AI chip's processing capabilities, and is acquired in real-time through the chip's video input interface.

[0068] Specifically, the steps for extracting customers and shelf entities in real time based on video streams and constructing a dynamic spatiotemporal node set are as follows:

[0069] Step S11: Deploy and run a lightweight multi-task perception model on the edge AI chip to perform entity detection on the video stream data and obtain a set of entity detection results.

[0070] The lightweight multi-task perception model is a neural network model optimized for the computing power and memory constraints of edge AI chips. It can simultaneously output customer body bounding boxes, key points of the human skeleton, shelf area bounding boxes, product instance bounding boxes, and product category information. Its optimization and detection process is as follows:

[0071] Step S111: Perform quantization processing on the model. Using historical customer activity video frames in the store as calibration samples, the original high-precision floating-point parameters of the model are converted into low-precision integer parameters through a calibration algorithm that minimizes inference error, thereby reducing the model's memory usage. The preferred calibration algorithm for minimizing inference error is the EoRA algorithm.

[0072] Step S112: Perform pruning on the quantized model, evaluate the contribution of each convolutional kernel to the inference result, and remove redundant convolutional kernels whose contribution is lower than a preset contribution threshold (the contribution is determined by comparing the change in entity detection accuracy of the model on the validation set before and after removing the convolutional kernel), further compressing the model size. The preset contribution threshold is a critical value set based on the change in entity detection accuracy on the model validation set and statistically analyzing the contribution distribution of redundant and effective convolutional kernels in historical pruning experiments. It is used to determine whether a convolutional kernel is redundant. Convolutional kernels whose change in entity detection accuracy on the validation set is lower than the preset contribution threshold will be considered redundant and removed.

[0073] Step S113: Input the video stream data frame by frame into the optimized model. The model fuses monocular RGB images (or combines depth information if RGB-D function is available) to perform multi-task inference and outputs the customer human body bounding box, human skeleton key points, shelf area bounding box, product instance bounding box and product category information for each frame. Integrate the detection results of all frames to form a set of entity detection results.

[0074] The advantage of this model lies in its ability to adapt to the resource constraints of edge AI chips through quantization and pruning, while directly outputting structured entity information, avoiding redundant transmission and processing of unstructured data, and reducing memory usage and latency.

[0075] Step S12: Based on the customer human body bounding box and human skeleton key points in the entity detection result set, perform online pedestrian re-identification operation to generate a customer ID mapping table.

[0076] Specifically, to achieve customer identity consistency across time steps and ensure continuous tracking of subsequent behaviors, please refer to the process of the online pedestrian re-identification operation. Figure 2 The online pedestrian re-identification process is as follows:

[0077] Step S121: Extract customer appearance features. For the customer body bounding box of each frame in the entity detection result set, output a 512-dimensional appearance feature vector through the feature extraction branch of the model. This vector contains discriminative information such as the color, texture, and body contour of the customer's clothing, and obtain the set of appearance feature vectors of the customer for each frame.

[0078] Step S122: Calculate the cosine similarity between each customer appearance feature vector in the current frame and the appearance feature vectors of all customers in the previous frame; at the same time, obtain the customer motion vector based on the change of the center coordinates of the customer's human body bounding box between the two frames, and calculate the motion consistency score based on the customer motion vector (the more coherent the motion trajectory, the higher the score); and sum the cosine similarity and the motion consistency score by weight to obtain the cross-frame matching score.

[0079] Step S123: Perform ID matching and allocation. If the cross-frame matching score between the current frame customer and the previous frame customer is higher than the preset matching threshold (determined by analyzing the historical re-identification data in the store and taking the intersection point of the distribution of correct and incorrect matching scores), then the previous frame customer ID is assigned to the current frame customer. If the current frame customer does not have a matching previous frame customer ID, then a new unique ID is generated (composed of a combination of timestamp and intra-frame sequence number). Record the index association relationship between each frame customer ID and the corresponding customer body bounding box and human skeleton key points to form a customer ID mapping table.

[0080] The advantage of this operation is that, given the characteristics of slow customer movement and minimal obstruction in unmanned sales scenarios, high-precision ID matching can be achieved by simply integrating appearance and motion features, without the need for complex models that require additional computing power.

[0081] Step S13: Based on the 2D entity bounding boxes in the entity detection result set, and combined with the assumption of the ground plane within the camera, perform a spatial coordinate transformation operation to obtain a set of relative three-dimensional spatial coordinates.

[0082] Specifically, to upgrade 2D detection results into 3D position information and provide a basis for spatial relationship calculation, the spatial coordinate transformation operation process is as follows:

[0083] Step S131: Extract the center coordinates of the 2D bounding boxes. For the bounding boxes of the customer human body, shelf area, and product instance in the entity detection result set, take the geometric center of each bounding box as the 2D pixel coordinates of each entity to form a set of 2D pixel coordinates.

[0084] Step S132: Call the pre-calibrated camera intrinsic parameters (including camera focal length, principal point coordinates, and distortion coefficients, obtained by calibrating the in-store camera using a camera calibration tool), convert the 2D pixel coordinates into a 3D ray in the camera coordinate system, and establish the mapping relationship between pixels and spatial directions.

[0085] Step S133: Determine the three-dimensional coordinates based on the assumption of the ground plane. Take the preset ground reference plane in the store (such as the fixed area in front of the cashier) as the reference and set the Z-axis coordinate of the plane to 0. Combine the prior height of the entities (average height of customers, standard height of shelves, obtained through the statistics of the physical dimensions in the store) to calculate the relative three-dimensional spatial coordinates of each entity in the local coordinate system of the store (with the origin directly below the installation position of the camera, the X-axis along the length of the store, the Y-axis along the width, and the Z-axis perpendicular to the ground and upward). Integrate them to form a set of relative three-dimensional spatial coordinates.

[0086] Step S14: Based on the entity detection result set, customer ID mapping table, and relative three-dimensional spatial coordinate set, perform entity node encapsulation operation to generate a set of node objects.

[0087] Specifically, to integrate scattered entity information into structured units, the entity node encapsulation operation process is as follows:

[0088] Step S141: Define the node object structure. Each node object contains five core fields: semantic category, relative 3D spatial coordinates, skeleton vector, unique ID, and timestamp. The semantic category distinguishes the entity type (customers are marked as "person", shelves are marked as "shelves", and products are marked as corresponding categories such as "snacks" and "beverages"). The relative 3D spatial coordinates are taken from the relative 3D spatial coordinate set. The skeleton vector is only contained in customer nodes and is formed by concatenating the 3D coordinates of key points of the human skeleton in the order of "hand-forearm-upper arm-torso-leg". The unique ID is only contained in customer nodes and is taken from the customer ID mapping table. The timestamp is synchronized with the acquisition time of the corresponding video frame (accurate to the millisecond level).

[0089] Step S142: Encapsulate node objects by frame. For the entity detection results of each frame, match the corresponding field information according to the entity type to generate node objects for all entities in that frame. For shelf and product nodes without skeleton vectors and unique IDs, mark the corresponding fields as none to ensure that the node object structure is consistent.

[0090] Step S143: Integrate the node objects of all frames to form a node object set. Assign a unique index to each node object for subsequent precise positioning.

[0091] Step S15: Based on the set of node objects, perform time-series organization operations to construct a dynamic spatiotemporal node set.

[0092] Specifically, to preserve the temporal continuity of entities and structured storage, the time-series organization operation process is as follows:

[0093] Step S151: Assign time step indexes. Assign a unique time step index to each frame according to the video frame acquisition order. The index value is incremented. Record the start and end times of each time step.

[0094] Step S152: Associate a subset of nodes with a time step. Treat the node objects of each frame as a subset of nodes and bind them with the corresponding time step index to ensure the correspondence between time and nodes.

[0095] Step S153: Construct a time step index table, recording the storage address of the node subset corresponding to each time step index, to facilitate quick querying of the entity node of a certain time step.

[0096] Step S154: Perform node subset validity verification, remove node objects in each node subset whose detection confidence is lower than the preset confidence threshold (the preset confidence threshold is determined by statistically analyzing the entity recognition accuracy in historical detection data), and ensure the authenticity of node objects; integrate the verified node subset with the time step index table to obtain the dynamic spatiotemporal node set.

[0097] This step fundamentally solves the problem of mismatch between traditional unstructured data and edge AI chip memory by constructing a dynamic spatiotemporal node set. Traditional methods for processing pixel sequences or grid features require a large amount of edge memory and necessitate repeated extraction of entity information, leading to memory access latency and computational redundancy. The dynamic spatiotemporal node set achieves deep adaptation in three aspects: First, by quantizing and pruning the lightweight multi-task perception model, the memory consumption during model runtime is reduced, while directly outputting key entity information, avoiding redundant transmission of pixel-level data; second, by encapsulating structured node objects, redundant information unrelated to behavior analysis (such as background pixels) is eliminated, significantly compressing data volume and improving the utilization efficiency of edge memory; third, by online pedestrian re-identification and spatial coordinate transformation, cross-frame ID consistency and 3D position information are supplemented for node objects, ensuring that subsequent high-order relationship modeling can be carried out based on coherent and accurate entity positions. Meanwhile, this node set preserves the semantic category and temporal continuity of entities, providing complete and lightweight atomic units for constructing a sparse heterogeneous spatiotemporal high-order relational structure in step two. This alleviates the memory and computing power pressure on edge AI chips from the data source, laying the foundation for the efficient operation of the entire behavior prediction system.

[0098] For example, if a customer moves from the beverage shelf to the snack shelf in the store: Step S11 detects the customer's human bounding box, skeletal key points, and bounding boxes of the two types of shelves and goods; Step S12 assigns a unique ID "T123_001" to the customer and records it in the customer ID mapping table; Step S13 calculates the relative three-dimensional spatial coordinates of the customer in different frames (e.g., (2.1, 3.5, 1.6), (2.3, 3.2, 1.6)); Step S14 encapsulates this information into customer node objects and integrates them with shelf and goods node objects; Step S15 organizes these node objects according to time steps to form a dynamic spatiotemporal node set for subsequent analysis of the interaction between the customer and the shelves and goods.

[0099] Step S2: Based on the dynamic spatiotemporal node set, generate a sparse heterogeneous spatiotemporal high-order correlation structure;

[0100] This step addresses the association modeling needs of edge AI chips in unmanned sales scenarios. It tackles the issues of lack of association representation in dynamic spatiotemporal node sets, the inability of traditional graph structures to capture multi-body coupling interactions, and the lack of full high-order association support for ultra-edge resources. By calling the dynamic spatiotemporal node set generated in step S1, and controlling the range through a sliding time window, constructing high-order units through strong spatiotemporal constraints, and dynamic sparse pruning, a sparse heterogeneous spatiotemporal high-order association structure is generated, providing a precise association feature foundation for subsequent topology perception prediction.

[0101] Specifically, the steps for generating a sparse heterogeneous spatiotemporal high-order correlation structure based on the dynamic spatiotemporal node set are as follows:

[0102] Step S21: Based on the timestamps and node subsets of the dynamic spatiotemporal node set, perform a sliding time window partitioning operation to generate a set of node subsets for each window.

[0103] The sliding time window is a time segment that extracts a dynamic spatiotemporal set of nodes at a fixed duration. The set of sub-window node subsets is an ordered set composed of multiple sub-window node subsets, each subset corresponding to a window and arranged in ascending order of its starting timestamp. Specifically, to decompose a continuous set of nodes into local analysis units, the sliding time window partitioning operation process is as follows:

[0104] Step S211: Determine the window size and sliding step size. The window size is determined by statistically analyzing the average duration of a single customer interaction in the store. The time distribution of typical interactions such as picking up and putting back goods in the past month is used as a sample, and the median of the distribution is taken as the initial length. Then, it is adjusted according to the real-time customer traffic (decreasing during peak hours and increasing during off-peak hours). The sliding step size is set to 1 time step (1 frame) to ensure time continuity.

[0105] Step S212: Extract a dynamic spatiotemporal node set according to the window size and step size. Each window corresponds to a sub-window node subset, which contains node objects of all time steps within the window. Mark the start and end timestamps of the window on the extracted subset and associate them with the corresponding node objects.

[0106] Step S213: Sort all window node subsets in ascending order of their start timestamps to form a set of window node subsets, thus avoiding disorder of time sequence.

[0107] Step S22: Based on the subset of nodes in the window and the semantic category labels of the nodes, perform heterogeneous node pair filtering and spatial distance calculation to generate a first-order association set.

[0108] The heterogeneous node pairs are combinations of nodes with different semantic categories, namely "person-product" and "person-shelf". The semantic category labels of the nodes are read from the node objects in the dynamic spatiotemporal node set ("person" is 1, "product" and "shelf" are 0). The first-order association set is a structured set containing valid interactive node pairs. Specifically, to capture basic binary interactions, the operation process is as follows:

[0109] Step S221: Filter heterogeneous node pairs, traverse each sub-window node subset, generate all node pairs with "semantic 1-semantic 0", exclude isomorphic node pairs, and obtain the initial node pair list.

[0110] Step S222: Calculate spatial distance. Based on the relative three-dimensional spatial coordinates of the node objects, use the Euclidean distance formula to calculate the distance between each pair of nodes in the initial node pair list, and obtain the node pair distance calculation results.

[0111] Step S223: Filter valid node pairs, set a spatial distance threshold (determined by the 95th percentile of the maximum distance between node pairs in historical valid interaction samples), and retain node pairs with a distance less than the threshold; assign a unique association identifier to each valid node pair, and record the identifier, node index, spatial distance, and the identifier of the sub-window to which it belongs, forming a first-order association set.

[0112] Step S23: Based on the customer's unique ID from the 1-order association set, the window node subset set, and the dynamic spatiotemporal node set, perform the 2-order association unit construction to generate the initial 2-order association unit set.

[0113] The second-order association unit is a triangular faceted structure with three nodes (representing multi-person interaction between "person-product-shelf"), and must satisfy the conditions of "spatial co-occurrence, temporal continuity, and action consistency." The initial set of second-order association units is a set containing units that meet these conditions. Specifically, to model multi-person coupled interaction, the construction process of the initial set of second-order association units is as follows:

[0114] Step S231: Spatial co-occurrence determination. In the same sub-window node subset, filter two first-order associations that share the same "human" class node, and the distance between the corresponding "product" and "shelf" nodes is less than the preset co-occurrence distance threshold (statistics based on the fixed placement distance between shelves and products), to obtain a candidate triplet list.

[0115] Step S232: Determine the continuity of time and consistency of action. Check whether there are similar triplets in the adjacent previous window (matching degree ≥ 80% and distance change rate ≤ 20% are similar). If they are satisfied, the time continuity determination is passed. Extract the skeleton vector of the shared "human" node, calculate the angle (0-180 degrees) of the displacement vector of the hand key point and the rate of change of velocity magnitude. If the angle is less than the angle threshold and the rate of change is less than the rate of change threshold (both are based on the statistics of historical smooth action samples), the action consistency determination is passed.

[0116] Step S233: Generate initial 2nd-order units. For candidate triples that pass the triple judgment, assign a unique 2nd-order unit identifier, record the identifier, node index, 1st-order association identifier, and action consistency score (the smaller the angle and the lower the rate of change, the higher the score), and form an initial 2nd-order association unit set.

[0117] Step S24: Based on the unique customer ID and skeleton vector of the dynamic spatiotemporal node set, perform temporal trajectory unit modeling to generate a set of temporal trajectory units.

[0118] The temporal trajectory unit is a sequence of positions of the same type of key points within a continuous window of the same customer, and the set of temporal trajectory units is a collection containing all trajectory units. Specifically, to supplement the temporal features of action evolution, the modeling process of the temporal trajectory unit is as follows:

[0119] Step S241: Group by customer's unique ID, extract the skeleton vector of each ID in the continuous sub-window, and ensure that the skeleton data of the same customer is consistent.

[0120] Step S242: Split the key point sequence. For the skeleton vector of each ID, split the continuous coordinate sequence of single-type key points according to the hand, forearm, upper arm, torso, and leg.

[0121] Step S243: Generate time-series trajectory units, assign a unique trajectory identifier to each coordinate sequence, and record the identifier, customer ID, key point type, sub-window identifier, and relative three-dimensional spatial coordinates to form a set of time-series trajectory units.

[0122] Step S25: Based on the initial set of second-order association units and the memory resource threshold of the edge AI chip, perform dynamic sparse pruning to generate a pruned set of higher-order association units.

[0123] The memory resource threshold is the maximum number of second-order units (typically ≤5 per frame) calculated from the real-time available memory of the edge chip. The set of higher-order association units after pruning is a set containing both retained first-order and second-order association units. For the specific dynamic sparse pruning process, please refer to [link to relevant documentation]. Figure 3 To control resource consumption, the dynamic sparse pruning process is as follows:

[0124] Step S251: Calculate the interaction confidence score. For each initial second-order association unit, the score is weighted and summed according to the spatial distance score (the smaller the distance, the higher the score), the action consistency score, and the node detection confidence score (the average of the three nodes). The weights are determined according to the historical prediction accuracy.

[0125] Step S252: Sort and filter, sort the initial 2nd-order unit set in descending order of interaction confidence, and retain units whose number does not exceed the maximum number of 2nd-order related units.

[0126] Step S253: Retain associations, simultaneously retain the first-order association set corresponding to the selected unit, and remove the remaining second-order association units and the first-order association set to form a pruned set of higher-order association units.

[0127] Step S26: Based on the set of higher-order correlation units after pruning, perform boundary operator construction operation to generate boundary operator matrix.

[0128] The boundary operator is a topological mapping operator from a 2-order unit to a 1-order association, and the boundary operator matrix is ​​a sparse matrix representing this mapping. Specifically, to maintain the topological relationships, the construction process of the boundary operator is as follows:

[0129] Step S261: Generate an index table to record the unique identifiers of each 1st-order association and 2nd-order unit in the set of higher-order association units after pruning, ensuring accurate index correspondence.

[0130] Step S262: Construct a matrix where rows correspond to 2-order unit identifiers and columns correspond to 1-order association identifiers; if a 2-order unit contains a 1-order association, the corresponding position is set to 1, otherwise it is set to 0, forming a boundary operator matrix.

[0131] Step S263: Verify the matrix dimensions to ensure that the number of rows and columns of the matrix matches the number of 2-order units and 1-order associations, respectively, to avoid dimension errors.

[0132] Step S27: Integrate the pruned high-order correlation unit set, the temporal trajectory unit set, and the boundary operator matrix, perform structural validity verification, and generate a sparse heterogeneous spatiotemporal high-order correlation structure.

[0133] The sparse heterogeneous spatiotemporal high-order correlation structure is a structured data set containing 0-order units (nodes), 1-order correlation sets, 2-order correlation units, temporal trajectory units, and boundary operators. Specifically, to ensure the integrity and effectiveness of the structure, the integration and verification process is as follows:

[0134] Step S271: Integrate the data. The 0-order units are taken from the valid nodes in the subset of window nodes. The 1-order association and 2-order units are taken from the pruned set. The time-series trajectory units are taken from the time-series trajectory set. Boundary operators are included in the form of a matrix.

[0135] Step S272: Validity verification, execute four rules: 1) The node corresponding to the 1-order association must exist in the 0-order unit; 2) The 1-order association contained in the 2-order unit must exist in the pruned set; 3) The dimension of the boundary operator matrix matches the number of 2-order units and 1-order associations; 4) The total data volume does not exceed the real-time available memory of the edge chip.

[0136] Step S273: Storage structure. After verification, the sparse heterogeneous spatiotemporal high-order correlation structure is stored in the edge AI chip storage unit as input data for step S3.

[0137] This step addresses key issues in edge AI chip processing by generating sparse, heterogeneous, spatiotemporal high-order association structures in two ways: First, it overcomes the limitations of traditional binary associations by modeling multi-body coupling interactions such as "person-product-shelf" using second-order association units. This captures subtle action associations, such as "the torso moving closer to the shelf when the hand grasps the product," providing crucial features for distinguishing between "touching" and "picking up" behaviors. Traditional graph structures cannot express such multi-body relationships, easily leading to confusion in behavior prediction. Second, it controls the data scale through sliding time windows and dynamic sparse pruning. Sliding windows limit association analysis to local time segments, avoiding redundant calculations of the entire time series. Dynamic pruning retains key interactions and removes invalid units based on confidence and memory thresholds, maintaining the overall sparsity of the structure above 80%. This significantly reduces the memory footprint and computing power consumption of edge chips, resolving the contradiction between high-order association modeling and edge resource constraints. Meanwhile, the introduction of boundary operators provides clear topological relationships for topological graph neural networks, ensuring the accuracy of cross-order message passing. Temporal trajectory units supplement the temporal features of action evolution, laying a multi-dimensional feature foundation for accurate behavior prediction in step three. This alleviates the bottleneck of software and hardware mismatch in the background technology from the aspects of association expression and resource adaptation.

[0138] For example, when a customer reaches for a beverage on the shelf: S21 captures 3 frames containing the interaction to form a subset of windows; S22 generates two 1st-order associations: hand-beverage and hand-shelf; S23 generates one 2nd-order unit through triple determination; S24 generates a hand temporal trajectory unit; S25 retains the high-confidence 2nd-order unit; S26 constructs the corresponding boundary operator matrix; the final integrated sparse heterogeneous spatiotemporal high-order association structure contains three nodes, two 1st-order associations, one 2nd-order unit, and hand trajectory, which is adapted for edge chip storage.

[0139] Step S3: Based on the sparse heterogeneous spatiotemporal high-order correlation structure, perform topology-aware behavior prediction and loss prevention response.

[0140] This step addresses the needs of behavior prediction and loss prevention response driven by edge AI chips. First, the sparse, heterogeneous spatiotemporal high-order correlation structure generated in step S2 is received in the neural network processing unit (NPU) of the edge AI chip. The core challenges are: existing edge behavior prediction models rely on pairwise relationships or manual rules, making it difficult to distinguish subtle differences in actions such as "touching" and "taking," and the computational density of these models is mismatched with the peak computing power of the edge AI chip, resulting in slow inference speed and delayed loss prevention response; simultaneously, the lack of spatial clustering analysis for high-risk events fails to provide accurate basis for dynamic shelf management. This step plans to extract cross-order spatiotemporal features using a knowledge-distilled and compressed topological graph neural network (Topo-GNN), combine this with multi-dimensional evidence to execute loss prevention responses, and generate shelf optimization suggestions through lightweight spatial clustering, thereby solving the problems of low action discrimination accuracy, poor hardware and software compatibility, and passive shelf management.

[0141] Specifically, the steps for performing topology-aware behavior prediction and loss prevention response are as follows:

[0142] Step S31: Based on the sparse heterogeneous spatiotemporal high-order correlation structure generated in step S2, call the knowledge distilled and compressed topological graph neural network (Topo-GNN) to perform model initialization operations and generate the initialized Topo-GNN model.

[0143] The knowledge distillation and compression mentioned here refers to the optimization process of transferring knowledge from a complex high-order graph neural network (teacher model) to a lightweight model (student model, i.e., Topo-GNN) to adapt to the limited computing power of edge AI chips. Specifically, the process of knowledge distillation and compression and model initialization is as follows:

[0144] Step S311: Train the teacher model by using a historical interaction sample set that includes the entire customer shopping process (browsing, picking up, putting back, and checking out) to allow the teacher model to learn the mapping relationship between high-order association features and customer behavior, and output the intermediate layer features and predicted probabilities (soft labels) of the teacher model.

[0145] Step S312: Construct a student model, reduce the number of convolutional layers and neurons in the teacher model, retain only the core modules that capture 1st-order and 2nd-order related features, and reduce the computational complexity of the model.

[0146] Step S313: Combine the KL divergence loss of the student model output and the soft label of the teacher model with the cross-entropy loss of the student model prediction results and the real customer behavior labels to jointly train the student model; at the same time, adjust the model parameter scale through the computing power simulator of the edge AI chip to ensure that the NPU computing unit utilization rate is ≥75% during model inference, balancing accuracy and computing power consumption.

[0147] Step S314: Quantize the trained Topo-GNN model into 8-bit integer parameters, adapt it to the low-precision computation operator of the edge chip, complete the model initialization, and obtain the initialized Topo-GNN model.

[0148] The advantage of this method is that it can significantly reduce the number of model parameters (by more than 60%) while maintaining the behavioral discrimination accuracy of the teacher model and avoiding excessive consumption of edge chip computing power by complex models.

[0149] Step S32: Based on the initialized Topo-GNN model generated in step S31, and combined with the sparse heterogeneous spatiotemporal high-order correlation structure, perform cross-order message passing operation to generate 1st-order aggregate features and 2nd-order geometric feature vectors.

[0150] Specifically, in order to accurately capture the differences in association features at different levels, the process of the cross-level message passing operation is as follows:

[0151] Step S321: Aggregate neighbor states along the first-order association edges. (1) For each customer node, extract the features (relative three-dimensional spatial coordinates, semantic category) of all neighbor nodes (goods or shelves) connected by the first-order association edges. (2) Assign weights to the neighbor features based on the interaction confidence of the first-order association edges (generated in step S2, reflecting the credibility of the association). The higher the confidence, the greater the weight. (3) Perform a summation operation on the weighted neighbor features to obtain the first-order aggregated features of the customer node.

[0152] Step S322: Capture local action intent through the in-plane geometric features of the 2nd-order associated unit. (1) Calculate the area change rate of the 2nd-order associated unit (triangular face), take the absolute difference between the area of ​​the triangular face in the current frame and the previous frame, and divide it by the time interval between the two frames (consistent with the video frame acquisition frame rate); (2) Calculate the angular curvature of the triangular face, extract the change of the three interior angles of the triangular face in consecutive frames, and take the average of the three changes as the angular curvature; (3) Calculate the dynamic normal vector of the triangular face, calculate the included angle by the dot product of the normal vectors of the triangular faces in adjacent frames, and the larger the included angle, the more intense the action change; (4) Concatenate the area change rate, angular curvature, and dynamic normal vector in sequence to generate the 2nd-order geometric feature vector corresponding to each 2nd-order associated unit, and associate it with the customer node through the node index.

[0153] This operation accurately captures the difference between the binary interaction of "hand-product" and the ternary coupled interaction of "hand-product-shelf" by distinguishing between first-order and second-order association features, providing a feature basis for distinguishing between "touching" (only first-order association changes) and "taking" (first-order and second-order associations change simultaneously).

[0154] Step S33: Based on the temporal trajectory units contained in the sparse heterogeneous spatiotemporal high-order correlation structure generated in step S2, perform temporal evolution feature extraction operation to generate temporal feature vectors.

[0155] The temporal trajectory unit is a sequence of key points (such as hands and torso) modeled for the same customer ID in step S2, used to supplement the action trend in the time dimension. Specifically, the process of extracting the temporal evolution features is as follows:

[0156] Step S331: For the time-series trajectory unit of the customer's hand, calculate the displacement vector of 3 consecutive frames, and judge the continuity of the action by the change in the direction of the displacement vector (such as the displacement direction of the picking action continuously pointing to the product).

[0157] Step S332: Calculate the velocity magnitude of the displacement vector (displacement distance divided by frame interval) and statistically analyze the velocity change trend (e.g., the velocity increases first and then decreases when picking up the object).

[0158] Step S333: Concatenate the direction change feature (encoded as a 0-1 vector, 1 for consistent direction, 0 otherwise) with the velocity change trend (encoded as a numerical feature) to generate a time-series feature vector, and associate it with the first-order aggregated feature of the corresponding customer node.

[0159] This operation can avoid misjudgment of actions caused by relying solely on single-frame associated features, such as distinguishing between a customer's hand accidentally approaching a product (with no coherent trend in temporal features) and actually taking the product (with a coherent trend in temporal features).

[0160] Step S34: Based on the 1st-order aggregated features and 2nd-order geometric feature vectors generated in step S32, and the temporal feature vectors generated in step S33, perform feature fusion operation to generate fused feature vectors.

[0161] Specifically, to integrate multi-dimensional features and avoid a single feature dominating the prediction result, the feature fusion process is as follows:

[0162] Step S341: Perform L2 normalization on the 1st-order aggregated features, 2nd-order geometric feature vectors, and temporal feature vectors respectively to ensure that the scales of the three types of features are consistent.

[0163] Step S342: Concatenate the following features in the order of “1st-order aggregated features (5-dimensional) + 2nd-order geometric feature vector (3-dimensional) + temporal feature vector (4-dimensional)” to form a 12-dimensional feature vector.

[0164] Step S343: The concatenated 12-dimensional feature vector is compressed using a 1×1 convolution kernel (a vector operation operator adapted to the edge NPU) to output an 8-dimensional fused feature vector. During the compression process, the feature dimensions with the highest correlation to behavior prediction are retained (determined through feature importance analysis, such as area change rate and speed trend).

[0165] The fusion feature vector integrates spatial correlation and temporal evolution information, providing comprehensive feature support for accurate prediction.

[0166] Step S35: Based on the fused feature vector generated in step S34, use the prediction head of the initialized Topo-GNN model generated in step S31 to perform behavior prediction and spatial trajectory offset prediction, and generate the customer behavior category probability distribution and spatial trajectory offset.

[0167] The prediction head is a lightweight structure adapted to the edge NPU. Specifically, the prediction process for behavior prediction and spatial trajectory offset prediction is as follows:

[0168] Step S351: Perform behavior category prediction using a fully connected layer (input 8-dimensional fused feature vector, output 3-dimensional probability vector), corresponding to the three behaviors of "browsing", "taking", and "hiding". The weights of the fully connected layer are sparsely initialized (50% sparsity) to adapt to the sparse matrix multiplication operator of the NPU and reduce the amount of computation.

[0169] Step S352: Perform spatial trajectory offset prediction. After the behavior category prediction layer, connect a linear regression layer in parallel. Input the spatial related dimensions (such as relative three-dimensional spatial coordinates and displacement velocity) in the fused feature vector and output the trajectory offset in the X, Y and Z directions in the next time step (1 frame).

[0170] Step S353: Perform a validity check on the prediction results. If the maximum probability in the probability distribution of the behavior category is lower than the preset confidence threshold (obtained through historical prediction accuracy statistics), then mark the customer as "to be observed" and do not output the prediction results for the time being to avoid low confidence predictions interfering with decision-making.

[0171] This prediction process enables millisecond-level inference on edge chips, meeting real-time requirements.

[0172] Step S36: Based on the customer behavior category probability distribution and spatial trajectory offset generated in step S35, and the dynamic spatiotemporal node set associated with the sparse heterogeneous spatiotemporal high-order correlation structure generated in step S2, perform multi-dimensional loss prevention evidence collection and generate trajectory departure judgment results, abnormal posture detection results and lingering judgment results.

[0173] Specifically, in order to comprehensively capture the characteristics of high-risk behaviors, the process for collecting multi-dimensional loss prevention evidence is as follows:

[0174] Step S361: Define the regular shopping process area, which refers to the area where customers frequently move during normal shopping. This area is determined by statistical analysis of customer movement trajectory data over the past three months using the store layout map (coordinate range of the cashier area, exit, and shopping cart parking area). The area that more than 90% of customers will pass through is selected as the regular area and stored as a set of area coordinates.

[0175] Step S362: Perform trajectory distance determination. If the probability of taking something in the probability distribution of customer behavior category exceeds the taking threshold (e.g., 0.7), combine the spatial trajectory offset to predict the position of the next 3 frames, determine whether it continues to move away from the regular shopping process area (the position is not in the area for 3 consecutive frames), and output the trajectory distance determination result (1 for moving away, 0 for not moving away).

[0176] Step S363: Perform abnormal posture detection, extract the skeleton vector of the customer in the dynamic spatiotemporal node set, calculate the angle between the arm and the torso (if less than 30 degrees, it is judged as "arm occluding torso"), and the angle change rate between the waist and the legs (if more than 50% / frame, it is judged as "rapid bending over"). If any condition is met, output the abnormal posture detection result (1 for abnormal, 0 for normal).

[0177] Step S364: Perform a dwell determination and count the duration of the customer in the non-selection area (such as shelf gaps or blind spots of the monitoring) (determined by comparing the customer's position with the set of area coordinates in consecutive frames). If the dwell time exceeds the preset dwell time (obtained by the statistics of historical risk-free dwell samples), output the dwell determination result (1 for overdue dwell, 0 for normal).

[0178] Step S37: Based on the trajectory distance determination result, abnormal posture detection result and lingering determination result generated in step S36, perform comprehensive risk score calculation and generate comprehensive risk score and loss prevention response instruction.

[0179] Specifically, to quantify the level of risk and trigger a precise response, the process for generating integrated risk scores and response instructions is as follows:

[0180] Step S371: Assign weights to evidence in each dimension. The weight of the trajectory distance judgment result is 0.4 (distance after taking is the core risk), the weight of the abnormal posture detection result is 0.3 (abnormal posture is often accompanied by hiding), and the weight of the stay judgment result is 0.3 (staying beyond the time limit may indicate anomalies). The weights are obtained by training 1,000 historical high-risk event samples (such as successfully identified hiding and unsettled taking out).

[0181] Step S372: Convert the results of each dimension into scores (1 is full marks, 0 is 0 marks), perform weighted summation, and obtain the comprehensive risk score.

[0182] Step S373: Set a risk threshold, which is determined by the score distribution of historical risk-free events, and take the 95th percentile as the threshold (to ensure that normal shopping customers are not misjudged).

[0183] Step S374: If the comprehensive risk score exceeds the threshold, a loss prevention response instruction is generated. The instruction includes the trigger location (the customer's current three-dimensional coordinates) and the response type (shelf indicator light flashing, real-time alarm in the background). The instruction is sent to the control module of the edge AI chip, and the control module drives the hardware to execute the response action.

[0184] Step S38: Based on the loss prevention response instructions and retrieval-unsettlement behavior records generated in Step S37 for the past week (obtained by matching the settlement system data with the behavior prediction results in Step S35), perform high-risk event spatial clustering to generate high-risk area clustering results and shelf layout optimization schemes.

[0185] Specifically, to achieve the transformation from passive loss prevention to proactive shelving management, the process for generating spatial clustering and optimization solutions is as follows:

[0186] Step S381: Collect high-risk event data. Each data point includes the three-dimensional coordinates of the event and the associated product category.

[0187] Step S382: Perform lightweight DBSCAN clustering to simplify the calculation of clustering parameters (the cluster radius is determined by the average shelf spacing of the store, and the minimum number of points is set to 3), avoiding complex calculations that occupy edge resources; output the clustering results of high-risk areas (each area includes the coordinates of the cluster center and the frequency of events).

[0188] Step S383: Generate a shelf layout optimization plan. If the shelf products associated with a high-risk area are high-value categories (such as tobacco, alcohol, and cosmetics), it is recommended to move them to a location with better monitoring coverage (determined by calculating the camera's field of view to ensure the products are in the center of the camera's field of view); if the product display height in the area is less than 1.2 meters (easily obstructed), it is recommended to raise it to 1.5 meters (at the customer's eye level); if the products in the area do not have anti-loss tags, it is recommended to add RFID anti-loss tags; the plan should clearly define the shelf ID, adjustment measures, and priorities (areas with high event frequency have higher priorities).

[0189] This step fundamentally solves the core problems of low accuracy, slow response, and hardware / software mismatch in edge behavior prediction in the background technology by using topology-aware reasoning and a multi-dimensional loss prevention mechanism. Compared with existing models that rely on pairwise relationships, this step has advantages in three aspects: First, the cross-order message passing mechanism of Topo-GNN accurately distinguishes subtle actions such as "touching" and "taking" through 1-order and 2-order feature coupling, avoiding information loss in single-relationship modeling; Second, knowledge distillation and sparse operator adaptation enable the model to achieve millisecond-level inference on edge chips, solving the problem of mismatch between computational density and peak computing power, while low-precision quantization reduces unit computing energy consumption; Third, multi-dimensional loss prevention evidence fusion and spatial clustering avoid misjudgment by a single rule (such as the possibility that the customer temporarily changed their route if the trajectory is far away), and the shelf optimization suggestions achieve a closed loop from "passive loss prevention" to "active optimization". In addition, all variables in the entire process are based on the output of the previous steps, without any arbitrary introduction, and all computing modules are adapted to the hardware characteristics of edge chips, meeting the real-time operation requirements of unmanned sales scenarios in terms of inference speed, accuracy, and energy consumption.

[0190] For example, if a customer takes a bottle of red wine from the tobacco and alcohol shelf (the probability of "taking" is 0.85 in the probability distribution of the behavior category output in step S35), the predicted trajectory shows that the customer continues to move towards the back door of the store (an area of ​​non-standard shopping process) (the trajectory distance judgment result 1 is output in step S36), and at the same time, the skeleton vector detects that the customer's arm is close to the torso (the abnormal posture detection result 1 is output in step S36); step S37 calculates the comprehensive risk score to be 0.82 (preset threshold 0.7), immediately generates a loss prevention response command, triggers the indicator light on the tobacco and alcohol shelf to flash, and sends an alarm to the background; if the shelf accumulates 6 "take-not-settled" events within a week (data is collected in step S38), spatial clustering marks it as a high-risk area, and an optimization plan is generated: move the red wine to the shelf layer facing the main camera and affix an RFID loss prevention tag, and subsequent high-risk events are significantly reduced.

[0191] Example 2:

[0192] This embodiment, based on Embodiment 1, provides a behavior spatiotemporal prediction system for an edge AI chip, such as... Figure 4 As shown, it includes:

[0193] Dynamic spatiotemporal node set construction module: used to extract customers and shelf entities in real time based on video streams and construct dynamic spatiotemporal node sets;

[0194] Sparse heterogeneous spatiotemporal higher-order correlation structure generation module: used to generate sparse heterogeneous spatiotemporal higher-order correlation structures based on the dynamic spatiotemporal node set;

[0195] Topology-aware behavior prediction and loss prevention response module: used to perform topology-aware behavior prediction and loss prevention response based on sparse heterogeneous spatiotemporal high-order correlation structure.

Claims

1. A spatiotemporal prediction method for the behavior of an edge AI chip, characterized in that, The method includes: Based on real-time extraction of customers and shelf entities from video streams, a dynamic spatiotemporal node set is constructed. Based on the dynamic spatiotemporal node set, a sparse heterogeneous spatiotemporal high-order correlation structure is generated. Based on a sparse heterogeneous spatiotemporal high-order correlation structure, topology-aware behavior prediction and loss prevention response are performed. The step of generating a sparse heterogeneous spatiotemporal high-order correlation structure based on the dynamic spatiotemporal node set includes: Based on the timestamps and node subsets of the dynamic spatiotemporal node set, a set of sub-window node subsets is generated by dividing the time window; Based on the subset of nodes in the window and the semantic category labels of nodes, heterogeneous node pair filtering and spatial distance calculation are performed to generate a first-order association set; the semantic category labels of nodes are read from the node objects of the dynamic spatiotemporal node set, with "person" type nodes marked as semantic category 1 and "goods" and "shelf" type nodes uniformly marked as semantic category 0; Based on the unique IDs of the 1-order association set, the window node subset set, and the dynamic spatiotemporal node set, the 2-order association unit is constructed to generate the initial 2-order association unit set. Based on the unique ID and skeleton vector of the dynamic spatiotemporal node set, perform temporal trajectory unit modeling to generate a set of temporal trajectory units; Based on the initial set of second-order association units and the memory resource threshold of the edge AI chip, a dynamic sparse pruning operation is performed to generate a pruned set of higher-order association units. Based on the pruned set of higher-order correlation units, a boundary operator construction operation is performed to generate a boundary operator matrix; The pruned high-order correlation unit set, the temporal trajectory unit set, and the boundary operator matrix are integrated, and the structural validity is verified to generate a sparse heterogeneous spatiotemporal high-order correlation structure. The sparse heterogeneous spatiotemporal high-order correlation structure is a structured data set containing 0-order units, 1-order correlation sets, 2-order correlation units, temporal trajectory units, and boundary operators.

2. The spatiotemporal prediction method for the behavior of an edge AI chip according to claim 1, characterized in that, The steps of extracting customers and shelf entities in real time based on video streams and constructing a dynamic spatiotemporal node set include: Deploy and run a lightweight multi-task perception model on an edge AI chip to perform entity detection on a video stream and obtain a set of entity detection results; Based on the customer human body bounding box and human skeleton key points in the entity detection result set, perform online pedestrian re-identification operation to generate a customer ID mapping table; Based on the 2D entity bounding boxes in the entity detection result set, combined with the assumption of in-camera participation on the ground plane, a spatial coordinate transformation operation is performed to obtain a set of relative three-dimensional spatial coordinates; Based on the entity detection result set, customer ID mapping table, and relative three-dimensional space coordinate set, an entity node encapsulation operation is performed to generate a node object set; each node object in the node object set contains five core fields, namely semantic category, relative three-dimensional space coordinates, skeleton vector, unique ID, and timestamp; A dynamic spatiotemporal node set is constructed based on a collection of node objects.

3. The spatiotemporal prediction method for the behavior of an edge AI chip according to claim 2, characterized in that, The operation steps of the lightweight multi-task awareness model include: The lightweight multi-task perception model is subjected to quantization processing, specifically including: using historical customer activity video frames in the store as calibration samples, and converting the original high-precision floating-point parameters of the model into low-precision integer parameters through a calibration algorithm that minimizes inference error; The quantized lightweight multi-task perception model is pruned, specifically including: evaluating the contribution of each convolutional kernel to the inference result and removing redundant convolutional kernels whose contribution is lower than a preset contribution threshold. The video stream is input frame by frame into a lightweight multi-task perception model, which outputs the bounding box of the customer's body, key points of the human skeleton, the bounding box of the shelf area, the bounding box of the product instance, and the product category information for each frame. The detection results of all frames are integrated to form a set of entity detection results.

4. The spatiotemporal prediction method for the behavior of an edge AI chip according to claim 3, characterized in that, The online pedestrian re-identification operation process includes: For the bounding box of the customer's body in each frame of the entity detection result set, the appearance feature vector is output through the feature extraction branch to obtain the set of appearance feature vectors of the customer in each frame; Based on the appearance feature vector of each customer in the current frame, the cosine similarity is calculated with the appearance feature vectors of all customers in the previous frame; at the same time, the customer motion vector is obtained according to the change of the center coordinates of the customer's human body bounding box between the two frames, and the motion consistency score is calculated based on the customer motion vector; the cosine similarity and the motion consistency score are weighted and summed to obtain the cross-frame matching score. If the cross-frame matching score between the current frame customer and the previous frame customer is higher than the preset matching threshold, the previous frame customer ID is assigned to the current frame customer; if the current frame customer has no matching previous frame customer ID, a new unique ID is generated; the index association relationship between each frame customer ID and the corresponding customer human body bounding box and human skeleton key points is recorded to form a customer ID mapping table.

5. The spatiotemporal prediction method for the behavior of an edge AI chip according to claim 4, characterized in that, The process of the dynamic sparse pruning operation includes: For each initial second-order association unit, the interaction confidence is obtained by weighting and summing the spatial distance score, action consistency score, and node detection confidence. The initial 2nd-order unit set is sorted in descending order of interaction confidence, and units whose number does not exceed the maximum number of 2nd-order associated units are retained. Simultaneously retain the first-order association set corresponding to the selected unit, and remove the remaining second-order association units and the first-order association set to form a pruned set of higher-order association units.

6. The spatiotemporal prediction method for the behavior of an edge AI chip according to claim 5, characterized in that, The steps for performing topology-aware behavior prediction and loss prevention response include: Based on the sparse heterogeneous spatiotemporal high-order correlation structure, the topological graph neural network compressed by knowledge distillation is called to perform model initialization operations and generate the initialized topological graph neural network model. Based on the initialized topological graph neural network model, combined with the sparse heterogeneous spatiotemporal high-order correlation structure, cross-order message passing operation is performed to generate 1st-order aggregate features and 2nd-order geometric feature vectors. Based on the temporal trajectory units contained in the sparse heterogeneous spatiotemporal high-order correlation structure, a temporal evolution feature extraction operation is performed to generate a temporal feature vector; Based on 1st-order aggregated features, 2nd-order geometric feature vectors, and time-series feature vectors, a feature fusion operation is performed to generate a fused feature vector; Based on the fused feature vector, the prediction head of the initialized topological graph neural network model is used to perform behavior prediction and spatial trajectory offset prediction, generating the probability distribution of customer behavior categories and spatial trajectory offset. Based on the probability distribution of customer behavior categories, spatial trajectory offset, and dynamic spatiotemporal node set associated with sparse heterogeneous spatiotemporal high-order correlation structure, multi-dimensional loss prevention evidence collection is performed to generate trajectory departure judgment results, abnormal posture detection results, and lingering judgment results. Based on the trajectory departure determination result, abnormal posture detection result, and lingering determination result, a comprehensive risk score is calculated, and a comprehensive risk score and loss prevention response instruction are generated. Based on loss prevention response instructions and records of retrieval and unsettled actions, high-risk event spatial clustering is performed to generate high-risk area clustering results and shelf layout optimization schemes.

7. The spatiotemporal prediction method for the behavior of an edge AI chip according to claim 6, characterized in that, The model initialization process includes: Training the teacher model specifically includes: using a historical interaction sample set that includes the entire customer shopping process, allowing the teacher model to learn the mapping relationship between high-order correlation features and customer behavior, and outputting the intermediate layer features and prediction probabilities of the teacher model; The student model is constructed by reducing the number of convolutional layers and neurons in the teacher model, retaining only the modules that capture first-order aggregate features and second-order geometric feature vectors. The student model is jointly trained by combining the KL divergence loss of the student model output and the soft labels of the teacher model with the cross-entropy loss of the student model prediction results and the real customer behavior labels; at the same time, the model parameter scale is adjusted through the computing power simulator of the edge AI chip. The trained topological graph neural network model is the initialized topological graph neural network model.

8. The spatiotemporal prediction method for the behavior of an edge AI chip according to claim 7, characterized in that, The process of collecting evidence for multi-dimensional loss prevention includes: Based on the store layout map and customer movement trajectory data from the past three months, we determined that areas that more than 90% of customers would pass through were designated as regular areas and stored as a set of area coordinates. If the probability of taking something in the probability distribution of customer behavior categories exceeds the threshold, the position of the next 3 frames is predicted by combining the spatial trajectory offset, and it is determined whether the customer continues to move away from the regular shopping process area. The trajectory moving away judgment result is then output. Perform abnormal posture detection, extract the skeleton vector of the customer in the dynamic spatiotemporal node set, calculate the angle between the arm and the torso, and the rate of change of the angle between the waist and the leg. If any abnormal condition is met, output the abnormal posture detection result. Perform a dwell time determination, count the duration of the customer in the non-selection area, and output the dwell time determination result if it exceeds the preset dwell time.

9. A spatiotemporal behavior prediction system for an edge AI chip, used to implement the spatiotemporal behavior prediction method for an edge AI chip according to any one of claims 1-8, characterized in that, The system includes: Dynamic spatiotemporal node set construction module: used to extract customers and shelf entities in real time based on video streams and construct dynamic spatiotemporal node sets; Sparse heterogeneous spatiotemporal higher-order correlation structure generation module: used to generate sparse heterogeneous spatiotemporal higher-order correlation structures based on the dynamic spatiotemporal node set; Topology-aware behavior prediction and loss prevention response module: used to perform topology-aware behavior prediction and loss prevention response based on sparse heterogeneous spatiotemporal high-order correlation structure.

Images