Tobacco industry method and system for monitoring video analysis

Abstract

This invention relates to a method and system for analyzing surveillance video in the tobacco industry. The method includes: extracting video intervals containing valid human activity from the original surveillance video stream of the tobacco industry; simultaneously detecting human targets and pre-defined tobacco industry object targets within the video intervals; tracking the detected human targets and object targets respectively, generating corresponding human motion trajectories and object motion trajectories; determining the spatiotemporal overlap relationship between the human targets and object targets based on the human motion trajectories and object motion trajectories; identifying interaction behavior tags between the human targets and associated object targets based on the spatiotemporal overlap relationship; and fusing the human target's facial image, appearance attributes, human motion trajectory, and interaction behavior tags to generate corresponding records of tobacco industry-related criminal activities, thereby improving the accuracy and efficiency of surveillance video analysis in the tobacco industry.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a method and system for analyzing surveillance videos in the tobacco industry. Background Technology

[0002] With the tobacco industry's digital and intelligent transformation in business scenarios such as warehousing and logistics management, monopoly inspection, and case analysis, massive amounts of surveillance video have become key data assets for industry governance. How to efficiently and accurately extract valuable personal identities, behavioral patterns, and related clues from this unstructured video data is the core challenge for improving the efficiency of industry supervision and case investigation.

[0003] Currently, video analytics technologies applied to the tobacco industry suffer from the following limitations: First, they generally employ single models based on single frames or short time-series segments (such as face recognition and general action recognition), which lack robustness under common industry-specific monitoring conditions such as complex lighting, occlusion, and long distances. Second, general action recognition models cannot effectively understand industry-specific behaviors such as "carrying cigarette boxes," "pushing trolleys," and "delivering cigarette packs," which rely on long-term spatiotemporal interactions between people and specific objects. The recognition results lack interpretable business semantics. Furthermore, frame-by-frame analysis of surveillance videos containing numerous static segments leads to significant waste of computational resources, reducing the accuracy and efficiency of video analytics in the tobacco industry. Summary of the Invention

[0004] In view of this, it is necessary to provide a method and system for analyzing surveillance videos in the tobacco industry to solve the technical problems of low accuracy and efficiency in the analysis of surveillance videos in the tobacco industry in the existing technology.

[0005] To address the aforementioned problems, in a first aspect, the present invention provides a method for analyzing surveillance video in the tobacco industry, comprising: Extract video segments containing valid human activities from the original surveillance video streams of the tobacco industry; Within the video range, both human targets and preset tobacco industry objects are detected simultaneously. The detected human and object targets are tracked separately to generate their respective motion trajectories. Based on the movement trajectory of the person and the movement trajectory of the object, the spatiotemporal overlap relationship between the person target and the object target is determined; Based on the spatiotemporal overlap relationship, the interaction behavior tags between the human target and the object target are identified; By integrating the facial image, appearance attributes, movement trajectory, and interactive behavior tags of the target person, a corresponding record of tobacco industry-related criminal behavior is generated.

[0006] In one possible implementation, extracting video segments containing valid human activity from the original surveillance video stream of the tobacco industry includes: Adaptive background modeling and motion region detection are performed on the video frame sequence of the original monitoring video stream; Based on the motion area detection results, the continuous video intervals in the original monitoring video stream where the motion area exceeds a preset threshold are determined as the video intervals containing valid human activity.

[0007] In one possible implementation, the simultaneous detection of human targets and preset tobacco industry objects within the video interval includes: Extract the video frames corresponding to the video interval containing valid human activities; A pre-trained target detection model is invoked to simultaneously detect human targets and object targets belonging to the preset tobacco industry in the video frame. The object targets include at least one of cigarette boxes, handcarts, or cigarette packs.

[0008] In one possible implementation, the person target and the object target include corresponding bounding boxes; the step of tracking the detected person target and the object target respectively to generate their respective person motion trajectory and object motion trajectory includes: A multi-target tracking model is used to associate the appearance features and / or motion vectors of the bounding boxes of human targets and object targets in different video frames to form human motion trajectories and object motion trajectories that include trajectory IDs, timestamps and serialized bounding boxes.

[0009] In one possible implementation, determining the spatiotemporal overlap relationship between the person target and the object target based on the person's motion trajectory and the object's motion trajectory includes: Obtain the movement trajectory of the person and the movement trajectory of the object, and the bounding box of the corresponding timestamp within the time series in which the person target is continuously tracked; The spatiotemporal overlap relationship is obtained by calculating the cross-union ratio (CUI) change data between the bounding box of the person target and the bounding box of the object target.

[0010] In one possible implementation, the interactive behavior label includes at least one of carrying, pushing, and delivering; The step of identifying interaction behavior tags between the human target and the object target based on the spatiotemporal overlap relationship includes: When the crossover ratio change data is continuously higher than the first threshold and the duration exceeds the first time threshold, it is determined that a transfer behavior has occurred; When the crossover ratio change data remains within the second threshold range and the synchronous change of the movement trajectories of the human target and the object target continues to exceed the second time threshold, it is determined that a pushing behavior has occurred. When the crossover ratio change data shows a peak change in a short period of time and conforms to the transmission trajectory pattern, it is determined that a delivery behavior has occurred.

[0011] In one possible implementation, after tracking the detected human target and generating the human motion trajectory, the method further includes: Based on the time interval corresponding to the movement trajectory of the person, extract the original video segment from the original monitoring video stream; The original video clip is masked to generate a corresponding masked video clip.

[0012] In one possible implementation, the step of determining the facial image and / or appearance attributes of the target person includes: A pre-trained face recognition model is invoked to extract and cluster the facial features of the target person in the masked video segment, thereby obtaining the face image; A pre-trained visual classification model is invoked to identify the appearance information of the human target in the masked video segment, thereby obtaining the appearance attributes; The steps for determining the spatiotemporal overlap relationship include: The spatiotemporal overlap relationship is determined based on the masked video segments corresponding to the movement trajectories of the person and the object, respectively.

[0013] In one possible implementation, after generating the corresponding record of tobacco industry-related conduct, the following is also included: In response to a received search request, a query is performed based on the records of tobacco industry-related offenses. The search conditions corresponding to the search request include at least one of the following: feature vector similarity search based on the facial image, filtering based on the appearance attributes, and location search based on the interaction behavior tags. Output a list of records of the criminal acts that match the search criteria and the corresponding video clips.

[0014] Secondly, the present invention also provides a video surveillance analysis system for the tobacco industry, comprising: The extraction unit is used to extract video segments containing valid human activities from the original surveillance video stream of the tobacco industry; The detection unit is used to simultaneously detect human targets and preset tobacco industry objects within the video range. The tracking unit is used to track the detected human target and the object target respectively, and generate their respective corresponding human motion trajectory and object motion trajectory; The determining unit is used to determine the spatiotemporal overlap relationship between the human target and the object target based on the human movement trajectory and the object movement trajectory; The identification unit is used to identify the interaction behavior tags between the human target and the object target based on the spatiotemporal overlap relationship; The generation unit is used to fuse the facial image, appearance attributes, movement trajectory, and interactive behavior tags of the target person to generate a corresponding record of tobacco industry-related criminal behavior.

[0015] Thirdly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, which, when executed by a processor, can implement the steps of the tobacco industry monitoring video analysis method described in any of the above implementations.

[0016] The beneficial effects of this invention are: The tobacco industry surveillance video analysis method provided by this invention extracts video intervals containing valid human activities from the original surveillance video stream of the tobacco industry, avoiding redundant analysis of static segments without motion content, which helps reduce the computational load of subsequent video analysis and improves video analysis efficiency. Within the video interval, human targets and preset tobacco industry object targets are detected simultaneously, realizing the acquisition of human targets and associated object targets under a unified spatiotemporal reference, ensuring the spatiotemporal consistency of human target and object target data, which helps improve the accuracy and efficiency of subsequent analysis of the interaction behavior and trajectory association between human and key objects. The detected human targets and object targets are tracked separately to generate their corresponding human movement trajectory and object movement trajectory, thereby integrating the detection results across frames to form a coherent and accurate motion data chain, which helps improve the rationality and accuracy of subsequent interaction behavior recognition. Based on the human movement trajectory and object movement trajectory, the method determines... By defining the spatiotemporal overlap between human and object targets and quantitatively analyzing the continuous changes in this overlap, behavior recognition can be deepened from the perception of instantaneous postures to the logical judgment of long-term interaction processes, which is beneficial to improving the rationality and accuracy of subsequent interactive behavior recognition. Based on the spatiotemporal overlap, interactive behavior tags between human targets and associated object targets are identified, improving the accuracy of behavior recognition results, enhancing the precision of video analysis, and making the behavior recognition results interpretable. This also improves the business semantic clarity and physical interpretability of interactive behavior tags. By integrating human targets' facial images, appearance attributes, movement trajectories, and interactive behavior tags, corresponding records of tobacco industry-related behaviors are generated. By providing effective identification evidence through appearance attributes, trajectory continuity, and other modalities of behavior patterns, technical compensation for weak modal information is achieved, making the recognition results more stable and reliable, and improving the accuracy and efficiency of surveillance video analysis in the tobacco industry. Attached Figure Description

[0017] Figure 1 A schematic flowchart of an embodiment of the tobacco industry surveillance video analysis method provided by the present invention; Figure 2 Provided by the present invention Figure 1 A schematic diagram of an embodiment of S104; Figure 3 This invention provides a schematic diagram of the business process in the analysis of tobacco cases. Figure 4 The overall structural framework diagram of the tobacco industry monitoring video analysis and retrieval system provided by the present invention; Figure 5 This is a schematic diagram of the structure of the tobacco industry monitoring video analysis system provided by the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] In the description of the embodiments of the present invention, unless otherwise stated, "a plurality of" means two or more.

[0020] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] This invention provides a method and system for analyzing surveillance videos in the tobacco industry, which will be described in detail below.

[0023] The execution entity of the tobacco industry surveillance video analysis method in this application embodiment can be the tobacco industry surveillance video analysis system provided in this application embodiment, or different types of electronic devices such as server equipment, physical host, or user equipment (UE) that integrate the tobacco industry surveillance video analysis system. The tobacco industry surveillance video analysis system can be implemented in hardware or software. The UE can be a terminal device such as a smartphone, tablet computer, laptop computer, handheld computer, desktop computer, or personal digital assistant (PDA).

[0024] The core value of the video surveillance analysis method for the tobacco industry in this application lies in its deep adaptation of general video analysis technology to the unique business logic and regulatory needs of the tobacco industry, such as monopoly inspection and case analysis (case investigation scenario), intelligent supervision of warehousing and logistics (internal risk control and efficiency scenario), and supervision of retail terminal market behavior (standardized operation scenario), which can directly produce structured intelligence that can be used for decision-making and action.

[0025] Figure 1 A schematic flowchart of an embodiment of the tobacco industry surveillance video analysis method provided by the present invention is shown below. Figure 1 As shown, the methods for analyzing surveillance videos in the tobacco industry include: S101. Extract video segments containing valid human activities from the original surveillance video stream of the tobacco industry.

[0026] Among them, the original surveillance video stream of the tobacco industry refers to the original surveillance video stream in the tobacco industry scenario, such as the original surveillance video stream of tobacco warehouses.

[0027] Specifically, the original surveillance video stream usually contains a large number of static background segments without human activity. In order to improve processing efficiency, a dynamic pre-screening mechanism can be adopted. That is, by performing motion analysis on the video frame sequence in the original surveillance video stream, the video frame sequence containing only valid human actions in a continuous time period is automatically identified and extracted to obtain video intervals. It can be understood that in this embodiment, by extracting video intervals containing valid human activities, redundant analysis of static segments without motion content is avoided, which helps to reduce the computational load of subsequent video analysis and facilitates the improvement of video analysis efficiency.

[0028] S102. Within the video range, simultaneously detect human targets and preset tobacco industry objects.

[0029] Among them, "person targets" refers to human entities identified and located in the surveillance video footage within the video range using computer vision technology. "Predefined tobacco industry object targets" refers to one or more categories of key objects, such as cigarette boxes, trolleys, or cigarette cartons, pre-defined based on the specific business scenarios and analytical needs of the tobacco industry before the video analysis task begins.

[0030] People and objects can be represented by a bounding box and its associated category label and confidence score. The bounding box defines the instantaneous position and extent of the person or object in the two-dimensional image space.

[0031] Specifically, for each video frame within a video interval, a pre-trained object detection model, such as YOLOv5 or Faster R-CNN, can be called frame by frame. This object detection model is trained to simultaneously identify people and specific object categories in the tobacco industry. For each video frame, the object detection model outputs multiple detection boxes, each including the object category (e.g., person, cigarette box), location coordinates (bounding box), and confidence score. Detection results with confidence scores higher than a preset threshold are retained, thus obtaining the person target and the object target. In essence, this embodiment performs simultaneous detection of both person and object targets within the corresponding video frames of the video interval, thereby simultaneously locating both. Through simultaneous detection, it achieves the acquisition of person targets and associated object targets under a unified spatiotemporal reference, ensuring the spatiotemporal consistency of person and object target data. This establishes an accurate and reliable data foundation for subsequent precise analysis of the interaction behavior and trajectory association between person and object targets, which is beneficial for improving the accuracy and efficiency of subsequent analysis of the interaction behavior and trajectory association between people and key objects.

[0032] S103. Track the detected human target and the object target respectively, and generate their respective human motion trajectory and object motion trajectory.

[0033] In this context, a person's motion trajectory refers to the continuous motion path data formed by linking multiple discrete detection boxes belonging to the same person target in a video sequence corresponding to a video interval along the temporal dimension. Each person's motion trajectory includes a unique trajectory ID, a series of bounding box coordinate sequences arranged by timestamps, and optionally, feature vectors corresponding to the appearance attributes of the target person. Similarly, an object's motion trajectory refers to the continuous motion path data formed by linking multiple detection boxes belonging to the same object target in a video sequence corresponding to a video interval along the temporal dimension. Its data structure is similar to that of a person's motion trajectory, including an object trajectory ID and a temporally sequenced bounding box sequence.

[0034] Specifically, for human and object targets detected from video sequences corresponding to video intervals, a pre-trained multi-target tracking model, such as an association algorithm based on appearance features and motion models, can be used to associate detection boxes belonging to the same target at different times. This creates a continuous human motion trajectory for each human target and a continuous object motion trajectory for each object target. Each trajectory contains a unique trajectory identifier, a timestamp sequence, and a corresponding bounding box coordinate sequence. This embodiment generates a continuous trajectory for each target, containing a unique trajectory identifier, a timestamp sequence, and a bounding box coordinate sequence, thus generating complete motion trajectories for both human and object targets. This integrates the detection results across frames, forming a coherent and accurate motion data chain. This provides a stable and reliable temporal data foundation for analyzing the evolution and interaction patterns of human and object target interactions over time, which is beneficial for improving the accuracy and interpretability of interaction behavior recognition over long time spans.

[0035] In one specific implementation, a multi-target tracking algorithm such as DeepSORT (Deep Simple Online and Realtime Tracking) is employed. For people, a deep learning network is used to extract deep appearance features (such as vector representations of clothing and body shape). Combined with motion vectors, such as position changes predicted based on Kalman filtering, detection boxes belonging to the same person in adjacent frames are associated and assigned a unique trajectory ID_1. Similarly, for detected cigarette boxes, a similar association logic is used to generate a trajectory ID_Obj_A. Ultimately, a complete spatiotemporal trajectory sequence is formed, such as "Trajectory ID_1: {Frame 1: Box coordinates, Frame 2: Box coordinates, ...}" and "Trajectory ID_Obj_A: {Frame 1: Box coordinates, Frame 2: Box coordinates, ...}".

[0036] S104. Based on the movement trajectory of the person and the movement trajectory of the object, determine the spatiotemporal overlap relationship between the person target and the object target.

[0037] Among them, the spatiotemporal overlap relationship refers to the relationship between the degree of spatial overlap and the time. In this embodiment, the spatiotemporal overlap relationship is a composite index used to quantitatively describe the dynamic interaction state between two or more targets in the surveillance video in terms of spatial proximity and temporal continuity.

[0038] Specifically, based on the movement trajectories of a person and an object, the spatial overlap index between the bounding boxes of the person and the associated object at the same time point is calculated. The sequence data of the spatial overlap index evolving over time is analyzed using a time-series analysis algorithm to extract change features, such as persistently high values, trend stability, or peak patterns. Based on preset logical rules, the spatiotemporal overlap relationship pattern between the person and the associated object is determined, thus establishing the spatiotemporal overlap relationship. This embodiment, by quantitatively analyzing the continuous changes in the spatiotemporal overlap relationship between the person and the object, enables behavior recognition to move from perceiving instantaneous postures to logically determining long-term interaction processes. This effectively addresses the shortcomings of traditional short-term action models in expressing long-term interaction relationships, improving the rationality and accuracy of subsequent interactive behavior recognition.

[0039] S105. Based on the spatiotemporal overlap relationship, identify the interaction behavior tags between the person target and the object target.

[0040] Among them, the interactive behavior label is the result of interactive behavior recognition, such as moving, pushing, or delivering.

[0041] The inventors discovered that in related technologies, action recognition often employs time-segment-based action recognition models, such as Two-Stream Networks (Two-Stream), Temporal Segmentation Networks (TSN), and 3D Convolutional Networks (I3D). These models primarily capture short-term (usually within a few seconds) changes in human posture, making it difficult to model and recognize complex behaviors specific to certain industries, such as "carrying cigarette boxes," "pushing a trolley," and "delivering cigarette packs," which rely on long-term spatiotemporal interactions between people and objects. Furthermore, their recognition results lack clear business semantics and interpretability.

[0042] Specifically, spatiotemporal overlap relationships can be mapped into interactive behavior tags with clear business semantics based on preset recognition rules. In this embodiment, the determination method based on spatiotemporal relationship quantification improves the accuracy of behavior recognition results, enhances the precision of video analysis, makes the behavior recognition results interpretable, and improves the clarity of business semantics and physical interpretability of interactive behavior tags.

[0043] S106. Integrate the facial image, appearance attributes, movement trajectory, and interactive behavior tags of the target person to generate a corresponding record of tobacco industry-related criminal behavior.

[0044] Here, a face image refers to a face image obtained by recognizing facial features of a person. Appearance attributes refer to the visual characteristics of the person, such as clothing features, items and accessories carried, hairstyle, etc.

[0045] Specifically, the facial images, appearance attributes, movement trajectories, and interactive behavior tags of the target person are associated and integrated to form a unified record of the target person, namely a structured and searchable record of the investigative behavior. This embodiment provides effective identification basis through other modalities such as appearance attributes, trajectory continuity, and behavioral patterns, and realizes technical compensation for weak modal information, making the identification results more stable and reliable, and improving the accuracy and efficiency of surveillance video analysis in the tobacco industry.

[0046] In summary, the tobacco industry surveillance video analysis method provided by this invention extracts video intervals containing valid human activities from the original surveillance video stream of the tobacco industry, avoiding redundant analysis of static segments without motion content. This reduces the computational load of subsequent video analysis and improves efficiency. Within the video intervals, simultaneous detection of human targets and preset tobacco industry object targets achieves the acquisition of human targets and associated object targets under a unified spatiotemporal reference, ensuring the spatiotemporal consistency of human and object target data. This improves the accuracy and efficiency of subsequent analysis of the interaction behavior and trajectory association between human targets and key objects. Tracking the detected human and object targets separately generates their corresponding human and object motion trajectories, integrating cross-frame detection results to form a coherent and accurate motion data chain, which improves the rationality and accuracy of subsequent interaction behavior recognition. Based on human... By analyzing the motion trajectories of people and objects, the spatiotemporal overlap between the target person and the target object is determined. Quantitative analysis of the continuous changes in this overlap allows behavior recognition to evolve from perceiving instantaneous postures to making logical judgments about long-term interaction processes, improving the rationality and accuracy of subsequent interactive behavior recognition. Based on the spatiotemporal overlap, interactive behavior tags between the target person and associated target object are identified, enhancing the accuracy of behavior recognition results and improving the precision of video analysis. This makes the behavior recognition results interpretable and improves the business semantic clarity and physical interpretability of the interactive behavior tags. By integrating the target person's facial image, appearance attributes, motion trajectory, and interactive behavior tags, corresponding records of tobacco industry-related behaviors are generated. Appearance attributes, trajectory continuity, behavioral patterns, and other modalities provide effective identification evidence, achieving technical compensation for weak modal information and making the recognition results more stable and reliable.

[0047] In some embodiments of the present invention, step S101 includes: performing adaptive background modeling and motion region detection on the video frame sequence of the original monitoring video stream; and determining, based on the motion region detection results, continuous video intervals in the original monitoring video stream where the motion region exceeds a preset threshold as the video intervals containing valid human activity.

[0048] Adaptive background modeling is a technique for dynamically estimating the background of a video scene. It can handle complex situations such as changes in lighting and slight movements of background objects. Its algorithm can be a Gaussian mixture model.

[0049] Motion region detection can employ the inter-frame difference method. By calculating the difference between corresponding pixels in two adjacent frames or between the motion region and the background model frame, pixels with differences exceeding a preset threshold are marked as foreground moving pixels, thus initially locating the region where the moving object is located. Setting an area threshold is key to elevating motion analysis from pixel-level to region-level judgment. This filters out small, isolated motion regions caused by noise and lighting fluctuations, ensuring that only continuous motion of a certain scale, potentially caused by activities such as people or vehicles, triggers region marking. This improves the correlation between the filtered results and "valid human activities."

[0050] The motion region detection results are the video streams of valid human activities that have been preliminarily screened.

[0051] Specifically, a motion analysis-based pre-screening method can be employed. This involves adaptively modeling the background of the original surveillance video stream's frame sequence to dynamically learn and update the understanding of the surveillance scene's background. This adapts to environmental interference such as slow changes in lighting and slight swaying of leaves, allowing for more accurate separation of moving objects in the foreground. Next, motion region detection is performed. For example, by calculating the differences between consecutive frames (inter-frame difference), regions exhibiting pixel changes are identified and judged as potential moving targets. Subsequently, a quantized judgment mechanism is introduced. Based on the motion region detection results, such as calculating the total area of ​​moving pixel regions in each frame and comparing it with a pre-set threshold, when a series of consecutive frames are identified where the area of ​​moving regions consistently exceeds the pre-set threshold, this consecutive time period is determined to be a video interval containing valid human activity. By judging the motion region area threshold, invalid static segments are pre-emptively eliminated, fundamentally avoiding the consumption of computational resources. This allows the system to complete the analysis of large amounts of video in a shorter time and with lower hardware overhead, providing technical feasibility for large-scale historical video retrospective analysis in actual case-handling environments. In addition, the accuracy of pre-screening ensures that key activity segments are not missed, providing a high-quality data input foundation for subsequent analysis steps.

[0052] Understandably, this embodiment introduces a dynamic motion pre-screening mechanism before video analysis, employing adaptive background modeling and motion region detection. Subsequent recognition processes are executed only during the time periods when valid actions exist in the video, fundamentally avoiding the massive redundant computations caused by traditional frame-by-frame processing of the entire video. By pre-removing invalid static frames, the total number of analysis frames is reduced significantly, thereby substantially lowering the average processing load, increasing overall analysis speed, and enhancing the system's scalability in large-scale case video processing scenarios. Compared to existing technologies, this invention can complete more video tasks with lower resource consumption, demonstrating a significant efficiency advantage and improving the overall system processing efficiency.

[0053] In some embodiments of the present invention, step S102 includes: extracting video frames corresponding to the video interval containing valid human activities; calling a pre-trained target detection model to simultaneously detect human targets and object targets belonging to the preset tobacco industry in the video frames, wherein the object targets include at least one of cigarette boxes, handcarts, or cigarette packs.

[0054] The pre-trained object detection model can be based on at least one of YOLO, Faster R-CNN, or SSD object detection models. It is a model specifically trained or optimized for tobacco industry application scenarios, enabling it to simultaneously identify two types of targets: one is the general "person" target; the other is a pre-defined, specific object target strongly related to tobacco industry business. For example, in an analysis task targeting warehouse area videos, the model will simultaneously search for and locate all people and cigarette boxes or trolleys appearing in the scene in each frame; while in an analysis task targeting the area around retail counters, the model may simultaneously detect people and cigarette boxes.

[0055] Specifically, firstly, video frames corresponding to video intervals containing valid human activity are extracted; then, a pre-trained object detection model is invoked to simultaneously detect human targets and objects belonging to the preset tobacco industry within the video frames. This allows the object detection model to output the detection results of all targets of interest in the frame, i.e., human targets and object targets, in a single forward inference process. For each detected target, the object detection model typically outputs its precise bounding box coordinates (for localization), its category, and a confidence score to measure the reliability of the detection results.

[0056] This embodiment employs a synchronous detection model tailored to industry-specific objects, avoiding the need for multiple independent models or repeated detection and inference. This significantly reduces the processing time and computational resource consumption per frame. Furthermore, synchronous detection ensures that information about people and related objects is acquired within the same time and spatial coordinate system. Therefore, this aligned detection operation provides a precise and efficient initial data foundation for subsequent complex trajectory association and behavior determination, ensuring the feasibility and accuracy of subsequent analysis.

[0057] In some embodiments of the present invention, the person target and the object target include corresponding bounding boxes; step S103 includes: using a multi-target tracking model to associate the appearance features and / or motion vectors of the bounding boxes of the person target and the object target in different video frames to form a person motion trajectory and an object motion trajectory that includes a trajectory ID, a timestamp, and a serialized bounding box.

[0058] Among them, the multi-target tracking model is an algorithmic framework used to maintain multiple target identities in a video sequence. For example, it can be a multi-target tracking algorithm that combines deep appearance features with motion vectors, and can be trained using DeepSORT or its variants.

[0059] Appearance attributes can be extracted from the bounding box image of a person using a deep convolutional neural network, generating a high-dimensional feature vector. This vector encodes the visual representation information of the person, such as clothing color, texture, body contour, or stable visual attributes like the shape and color of an object. Ideally, the appearance feature vectors of the same target in different frames should be relatively close in the feature space, while the feature vectors of different targets should be relatively far apart.

[0060] Motion vectors describe the motion state of a person or target on the image plane and can be predicted and updated based on motion models such as Kalman filtering. This model predicts the target's expected position in the current frame based on its position and velocity in the previous moment. The consistency of motion states can be measured by calculating the similarity (such as overlap or distance) between the new detection box's position and the predicted position.

[0061] Specifically, after detecting people and objects in the initial frame, a pre-trained and deployed multi-object tracking model is invoked. For each new input video frame, the bounding boxes of the detected people or objects in the current frame are received. Using established short-term historical trajectory information, it is determined which object in the existing trajectory is most likely to belong to the same entity as a new detection box in the current frame. By combining the appearance features and / or motion vectors of the bounding boxes of people and objects, the same physical entity appearing in the scene at different times is associated, forming a continuous motion trajectory in the time dimension. This ensures the continuity of the identities of people and objects on the time axis, fundamentally improving the feasibility and accuracy of subsequent behavior recognition modules.

[0062] In some embodiments of the present invention, such as Figure 2 As shown, step S104 includes: S201. Obtain the motion trajectory of the person and the motion trajectory of the object, and the bounding box of the corresponding timestamp within the time series in which the person target is continuously tracked. S202. Calculate the cross-union ratio change data between the bounding box of the person target and the bounding box of the object target to obtain the spatiotemporal overlap relationship.

[0063] The Intersection over Union (IoU) ratio is a metric for calculating the degree of overlap between two bounding boxes. It is the ratio of their intersection area to their union area, ranging from 0 to 1. A higher IoU value indicates a tighter spatial overlap. The spatiotemporal overlap relationship is represented by the IoU variation between the bounding boxes of the person and the object targets.

[0064] Specifically, from the generated human and object motion trajectories, corresponding data within the same time series are extracted. First, the timelines are aligned to ensure that data points within the same time period and at the same timestamp are obtained. For each matching timestamp, the bounding box coordinates of the human target in the human motion trajectory and the bounding box coordinates of the object target in the object motion trajectory are read synchronously. Then, the intersection-union ratio (IUGR) between the human and object target bounding boxes is calculated time-by-time. The continuous change of this ratio throughout the interaction period is calculated and recorded in chronological order, thus forming a complete set of IUGR change data. This objectively and quantitatively depicts the dynamic process of the approach, contact, and separation of human and object in space over time.

[0065] Understandably, this embodiment, by accurately acquiring and calculating the intersection-union ratio (IU / R) change data of bounding box sequences, provides an objective quantitative standard for behavior determination. This is in contrast to traditional video behavior analysis, which relies on overall pattern recognition of short-term poses or optical flow, making it difficult to accurately capture and define complex events such as "person manipulating objects" that depend on long-term, dynamic spatial relationships. This reduces the error of subjective judgment and improves the accuracy and consistency of the analysis results. Furthermore, by quantitatively tracking long-term spatial relationships, it is possible to understand and determine industry-specific behaviors with long durations (such as a moving process lasting several seconds), enhancing the ability to identify complex, long-term interactive behaviors. Simultaneously, this relationship definition method based on geometric calculation has a clear and traceable logical chain, ensuring that the final behavior recognition result is no longer a "black box" output, but possesses stronger interpretability and evidentiary support, meeting the high standards for the reliability of analytical conclusions in tobacco industry case analysis.

[0066] In one specific implementation, a person's "trajectory ID_1" and a cigarette box's "trajectory ID_Obj_A" are selected. Based on the bounding box sequence of both at the same timestamp, the "Intersection over Union (IoU)" of the person's bounding box and the cigarette box's bounding box is calculated in each frame (or every few frames). The IoU value is between 0 and 1, where 0 indicates no overlap and 1 indicates complete overlap. The system records the sequence data of the IoU value changing over time from time T1 to T2, for example: [0, 0.1, 0.6, 0.8, 0.9, 0.85, …]. This changing sequence represents the dynamic relationship of the spatial proximity between the two.

[0067] In some embodiments of the present invention, the interactive behavior label includes at least one of carrying, pushing, and delivering; step S105 includes: when the cross-union ratio change data is continuously higher than a first threshold and the duration exceeds a first time threshold, it is determined that a carrying behavior has occurred; when the cross-union ratio change data is maintained within a second threshold range and the synchronous change of the movement trajectories of the person target and the object target continues to exceed the second time threshold, it is determined that a pushing behavior has occurred; when the cross-union ratio change data shows a peak change in a short period of time and conforms to the transmission trajectory pattern, it is determined that a delivery behavior has occurred.

[0068] The first threshold can be 0.7, the second threshold can be in the range of [0.3, 0.6], the first time threshold can be 3S, and the second time threshold can be 3S.

[0069] Specifically, for the determination of transport behavior, it is determined that transport behavior has occurred when the change data of the cross-union ratio (IoU) between the target person and the associated object (such as a cigarette box) is continuously higher than a set first threshold (e.g., IoU>0.7) and the duration of this high overlap state exceeds a first time threshold (e.g., 3 seconds).

[0070] For the determination of the pushing behavior, it is set that when the intersection over union change data remains within a specific second threshold range (for example, 0.3 < IoU < 0.6), and in this state, the motion trajectories of the person target and the object target (such as a trolley) exhibit highly synchronized changes that last longer than a second time threshold, the behavior is determined as a pushing behavior.

[0071] For the determination of the delivery behavior, a sharp spatial relationship change and article transfer occurring within a short period of time are identified. It is set that when the intersection over union between the person target and the object target shows a significant peak change within a very short time window, and the change pattern conforms to the rule of transfer from one motion trajectory to another, the delivery behavior is determined to have occurred.

[0072] This embodiment adopts a spatiotemporal relationship-driven behavior recognition mechanism, and uses the change of the intersection over union (IoU) between the person bounding box and the object bounding box over time as the judgment basis, so as to realize the interpretable determination of industry-specific behaviors such as carrying cigarette boxes, pushing trolleys, and delivering cigarette packs. Compared with the traditional behavior recognition method based on short-time action segments, it can accurately express the long-term interaction relationship between a person and an object, is not affected by instantaneous posture changes or short-term occlusions, and can output reliable, stable behavior recognition results with clear logical basis in complex monitoring environments.

[0073] It is understandable that, by introducing the determination rules based on continuous spatiotemporal overlap relationship and motion synchronization, this embodiment converts business knowledge into clear and computable logical judgments. This not only greatly improves the accuracy and robustness of recognizing the above-mentioned key industry behaviors in complex monitoring environments, and reduces misjudgments caused by instantaneous posture changes or short-term occlusions, but also its determination process is based on objective and quantifiable data, namely IoU values, duration, and trajectory synchronization data, so that the behavior recognition results have good interpretability and traceability. For the tobacco inspection business that uses video analysis results as clues or evidence for case investigation and judgment, this provides higher credibility and practical value.

[0074] In some embodiments of the present invention, after step S104, the method further includes: intercepting an original video segment from the original monitoring video stream according to the time interval corresponding to the person motion trajectory; and performing masking processing on the original video segment to generate a corresponding masked video segment.

[0075] The masking processing is used to mask the background area outside the person target bounding box, that is, only the pixels inside the person target bounding box are retained, and the pixels in the background area outside the bounding box are set to a specific value.

[0076] Specifically, original video segments are extracted from the original surveillance video stream. For each frame of the original video segment, the area within the bounding box of the person's movement trajectory at the same moment is determined as the foreground, and the area outside the bounding box is determined as the background. Subsequently, the processing only retains the foreground pixels, while the pixel values ​​of the background area are uniformly set to pure black or other neutral values ​​(such as 0), thereby generating a new video segment, namely the masked video segment. Understandably, this embodiment, by generating "original video segments" and "masked segments," enables subsequent recognition models to focus solely on the target object in an environment free from background interference, improving the stability and accuracy of face recognition, appearance attribute recognition, and behavior recognition models, and maintaining a high recognition rate even under adverse conditions such as changes in lighting, occlusion, and complex backgrounds.

[0077] In some embodiments of the present invention, the steps for determining the face image and / or appearance attributes of the target person include: calling a pre-trained face recognition model to extract and cluster the face features of the target person in the masked video segment to obtain the face image; calling a pre-trained visual classification model to identify the appearance information of the target person in the masked video segment to obtain the appearance attributes; the steps for determining the spatiotemporal overlap relationship include: determining the spatiotemporal overlap relationship based on the masked video segments corresponding to the movement trajectory of the person and the movement trajectory of the object, respectively.

[0078] Specifically, a pre-trained face recognition model is invoked to process the masked video clip. This model performs multi-frame face detection and localization within the masked video clip, extracting high-dimensional feature vectors from the detected face regions in each frame. Subsequently, these multi-frame feature vectors from the same trajectory are aggregated and analyzed (e.g., through clustering or average pooling) to generate a more representative, stable, and unified facial feature representation. Simultaneously, the frame with the best clarity can be selected from the multiple frames as a representative image of the target person's trajectory. By utilizing temporal information to compensate for potential defects in single-frame images, such as blurriness or poor angles, the robustness of the identity features is improved.

[0079] A pre-trained visual classification model is used to analyze target regions of people in masked video clips. This model, trained on a large amount of labeled data, can output multiple attribute labels for the input image of the person's region at once. These labels include, but are not limited to, gender, age group, clothing color (e.g., shirt color), and binary states such as whether the person is carrying a backpack, hat, or mask. Because the recognition is performed on masked video clips with complex backgrounds removed, the model can focus more on the visual features of the person themselves, reducing false positives caused by background noise and thus improving the accuracy of appearance attribute recognition.

[0080] Based on the movement trajectories of people and objects, corresponding video segments are obtained for each. When analyzing the interaction between people and objects, the spatiotemporal relationship between the bounding boxes of the person and object targets in the masked video segments is calculated. Since the background is occluded, the spatial positional changes between the target subjects can be reflected more purely, avoiding misjudgments caused by interfering "false overlap" due to objects of similar color or texture in the background. This makes changes in metrics such as Intersection over Union (IoU) more accurately reflect the actual interaction process. This ensures that even in complex scenes (such as brief occlusion, multiple people and multiple objects), the system can continuously track the relationship between specific people and specific objects, overcoming the shortcomings of single-frame detection or short-term tracking that easily lose targets and cannot analyze long-term interactions.

[0081] Understandably, by constructing a multimodal feature collaborative recognition system, which comprehensively utilizes facial features, appearance attributes, behavioral features, trajectory features, and spatiotemporal interaction information between people and objects, each modality compensates for and enhances the others during the recognition process. When a person is wearing a mask, is at a side angle, or the facial image is unclear, traditional single-modal recognition systems often fail to reach the correct conclusion. However, the embodiments of this application can continue to provide effective recognition evidence through other modalities such as appearance attributes, trajectory continuity, and behavioral patterns, achieving technical compensation for weak modal information. This improves the robustness and adaptability of recognition in complex monitoring environments, making the recognition results more stable and reliable.

[0082] In some embodiments of the present invention, after step S106, the method further includes: responding to the received retrieval request, querying based on the tobacco industry-related criminal behavior records, wherein the retrieval conditions corresponding to the retrieval request include at least one of: feature vector similarity retrieval based on the face image, filtering based on the appearance attributes, and location retrieval based on the interaction behavior tags; and outputting a list of criminal behavior records and corresponding video clips that match the retrieval conditions.

[0083] Specifically, when a user (such as an investigator) initiates a search request, the system directly performs an efficient query on the pre-constructed records of relevant conduct. Search requests can be initiated through the system's interactive interface, enabling multi-dimensional and freely combinable search conditions. The system outputs a list of relevant conduct records matching the search conditions and corresponding video clips. Understandably, this embodiment integrates multi-modal recognition results such as facial features, appearance attributes, behavioral tags, and trajectory information into structured data through a unified person record fusion mechanism. Based on this, it achieves multi-dimensional combined retrieval, including vector search based on facial features, multi-condition filtering based on attribute features, and precise positioning based on behavioral features. Unlike existing technologies that only support single-modal retrieval, this system can perform cross-modal searches based on the user's comprehensive conditions, thereby achieving more accurate and faster target person or behavioral clip positioning. The composite retrieval capability of this embodiment significantly improves case analysis efficiency, aids in clue reconstruction and event inference, and has high practical value in tobacco industry analysis scenarios.

[0084] In one specific implementation, such as Figure 3 The diagram shown illustrates the business process in tobacco case analysis, specifically including: I. Case Creation and Video Data Acquisition Process In actual platform operation, users first access the system homepage through a browser, complete user registration, and log in. After successful login, they select "Video Upload" in the left navigation bar and click "Create Case," opening the case information entry window. Users enter the case name and description, and click "OK." The system then creates a new case record in the database and generates a case ID as an index for subsequent video and recognition results. In the "Video Upload" interface, users select the newly created case, click the upload area, and choose a surveillance video file from their local machine. The system verifies the uploaded file; if successful, it stores the video in the server object storage and writes video metadata to the database, including camera number, video time period, encoding format, uploader, and time.

[0085] II. Video Dynamic Pre-screening Process When a user enters the "Video Analysis" interface and selects a case to initiate analysis, the model scheduling system of this invention automatically performs pre-screening processing on the video. The system first reads the video frame sequence, constructs background frames using an adaptive background model, and calculates motion regions using inter-frame differences. When the detected motion region area exceeds a threshold, the current frame is marked as a valid frame, and a continuous valid interval is further determined. This pre-screening process is seamless for the user, but significantly reduces the computational load of subsequent depth models in the background, resulting in a significant improvement in analysis speed.

[0086] III. Target Detection and Continuous Trajectory Tracking Process After obtaining the valid frame range, the system invokes the object detection model deployed on the GPU service to detect people in each frame and output bounding boxes, confidence scores, and categories. The system then employs a multi-object tracking algorithm combining deep appearance features and motion vectors to associate the detection results of adjacent frames into continuous trajectories, generating unique trajectory IDs. The system writes the trajectory information to a database, allowing users to view the parsing status of each video (unparsed, parsing, parsing complete) in "Video Management," and providing a foundation for subsequent end-to-end recognition tasks.

[0087] IV. Trajectory Segment Generation and Video Mask Processing Flow The system extracts corresponding original video segments based on the time range of each trajectory, and simultaneously generates masked video segments. The masked video segments retain only the pixels within the bounding box of the person, while all other pixels are blacked out. The masked videos enhance the stability of subsequent recognition models in complex backgrounds. The generated segments will be used for tasks such as behavior analysis, face recognition, and attribute recognition. The segment indexes are also written to a database so that users can directly access the corresponding video segments when clicking "View Video Segment" in the analysis results.

[0088] V. Multimodal Recognition Process Based on trajectory fragments, the system performs the following three types of recognition tasks, ultimately forming a "person record" for intelligent retrieval and case review: (a) Facial Recognition Process The system performs multi-frame face detection within the masked segment, extracts high-dimensional vectors for each detected face region, and aggregates the vectors from multiple frames to generate a unified facial feature representation. Simultaneously, the system selects the frame with the best clarity as a trajectory representative image for the "review face image" function in case review.

[0089] (II) Appearance Attribute Recognition Process The system crops the data of the person and feeds it into a multi-label attribute model to identify attributes such as gender, age group, clothing color, and whether they are carrying a backpack / hat / mask. Users can manually review these attributes on the case review interface; if the identification is inaccurate, they can click to modify and "save results".

[0090] (III) Industry-Specific Behavior Identification Process The system determines whether behaviors such as handing out cigarettes, moving boxes, or pushing carts exist based on the change in IoU between the character's bounding box and the bounding box of objects (such as cigarette boxes, trolleys, cigarette packs, etc.) over time, combined with rules such as continuous frame count and action duration.

[0091] Users can view behavioral tags on the case review page and confirm or modify them based on video clips. The above multimodal recognition results are uniformly written into the person record, forming searchable structured data.

[0092] VI. Intelligent Search and Video Clip Display Process Users upload facial images on the video parsing interface. The system calculates their feature vectors and performs vector retrieval with all records of people, returning the result with the highest similarity.

[0093] Users can select behavioral events (such as offering cigarettes, moving boxes, pushing carts, etc.) as search criteria, and the system will perform multimodal composite matching and present the final list of video clips.

[0094] Users can click "View Video Clip" to directly view the identified original video segments, masked video clips, and keyframe screenshots to assist in analyzing the suspect's behavior.

[0095] VII. Case Management and Review Process In the "Case List", users can view the status of all videos for a particular case.

[0096] Once the recognition results are generated, the user proceeds to "Case Video Review," which involves: face verification, attribute verification, behavior verification, video content approval, batch approval or batch rejection, and the review results being written to the case record. After all videos have been reviewed, the user can view the full set of video clips and recognition results arranged chronologically. This review process ensures the authenticity of the recognition results and their reliability as evidence in the industry.

[0097] VIII. Operational Audit Process To meet regulatory requirements, the system provides an "Operation Audit" module that records all key user operations, including uploading videos, deleting cases, modifying review results, and changing attributes. The system supports filtering by operation type, operation object, and operator, and writes audit logs to an immutable archive storage to ensure the integrity and traceability of the operation chain.

[0098] In one specific implementation, such as Figure 4 The diagram shown is an overall structural framework of a surveillance video analysis and retrieval system for the tobacco industry. Details are as follows: The front-end service layer (user interaction layer) comprises the following core modules: case management, AI intelligent analysis, AI intelligent search, review management, and operation auditing. This layer provides the user interface for all business functions to end users (such as inspectors). It is responsible for receiving user instructions (such as uploading videos, initiating searches, and reviewing results), sending processing requests to the backend, and presenting the structured data and video clips returned by the backend to the user in a visual format (such as lists, timelines, and video players).

[0099] The backend service layer (core algorithm and business logic layer) includes the following core modules: face detection and feature extraction module, person attribute recognition module, target detection module, target tracking service module, enhanced detection service module, and business service module.

[0100] The target detection module and the target tracking service module work together to simultaneously detect and track human targets and preset tobacco industry objects.

[0101] The enhanced detection service module corresponds to the preprocessing step of generating masked video clips to eliminate background interference.

[0102] The face detection and feature extraction module and the person attribute recognition module correspond to the face recognition and appearance attribute recognition tasks in "multimodal collaborative recognition".

[0103] Deployment and Operations Layer: Docker Compose Deployment: The system uses containerization technology for packaging and deployment, ensuring consistency of the algorithm environment, rapid scaling of services, and maintainability of the system.

[0104] Data persistence layer, database: used to store structured data, including case metadata, video indexes, and unified person records (i.e. records of actions involved in the case).

[0105] To better implement the tobacco industry surveillance video analysis method in this invention embodiment, based on the tobacco industry surveillance video analysis method, correspondingly, as follows: Figure 5 As shown, this embodiment of the invention also provides a video surveillance analysis system for the tobacco industry. The video surveillance analysis system 500 for the tobacco industry includes: Extraction unit 501 is used to extract video segments containing valid human activities from the original surveillance video stream of the tobacco industry; Detection unit 502 is used to simultaneously detect human targets and preset tobacco industry objects within the video range; The tracking unit 503 is used to track the detected human target and the object target respectively, and generate their respective human motion trajectory and object motion trajectory; The determining unit 504 is used to determine the spatiotemporal overlap relationship between the person target and the object target based on the person's movement trajectory and the object's movement trajectory; The identification unit 505 is used to identify the interaction behavior tags between the person target and the object target based on the spatiotemporal overlap relationship; The generation unit 506 is used to fuse the facial image, appearance attributes, movement trajectory, and interactive behavior tags of the target person to generate a corresponding record of tobacco industry-related criminal behavior.

[0106] The tobacco industry surveillance video analysis system 500 provided in the above embodiments can implement the technical solutions described in the above embodiments of the tobacco industry surveillance video analysis method. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the tobacco industry surveillance video analysis method, which will not be repeated here.

[0107] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the tobacco industry monitoring video analysis methods provided in the above-described method embodiments.

[0108] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0109] The above provides a detailed description of the monitoring video analysis method and system for the tobacco industry provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for analyzing surveillance video in the tobacco industry, characterized in that, The method includes: Extract video segments containing valid human activities from the original surveillance video streams of the tobacco industry; Within the video range, human targets and preset tobacco industry objects are simultaneously detected, and the human targets and object targets include corresponding bounding boxes; The detected human and object targets are tracked separately to generate their respective motion trajectories. Based on the time interval corresponding to the person's movement trajectory, an original video segment is extracted from the original monitoring video stream. The original video segment is then masked to generate a corresponding masked video segment. The processing method is to retain only the pixels within the person's bounding box, while uniformly setting the background area pixels outside the bounding box to pure black. Based on the movement trajectory of the person and the movement trajectory of the object, the spatiotemporal overlap relationship between the person target and the object target is determined, including: acquiring the movement trajectory of the person and the movement trajectory of the object, and the bounding boxes corresponding to the timestamps in the time series in which the person target is continuously tracked; calculating the intersection-union ratio (IUGR) change data between the bounding boxes of the person target and the bounding boxes of the object target to obtain the spatiotemporal overlap relationship; Based on the spatiotemporal overlap relationship, interaction behavior tags between the person target and the object target are identified, and the interaction behavior tags include at least one of carrying, pushing, and delivering; including: When the crossover ratio change data is continuously higher than the first threshold and the duration exceeds the first time threshold, it is determined that a transfer behavior has occurred; When the crossover ratio change data remains within the second threshold range and the synchronous change of the movement trajectories of the human target and the object target continues to exceed the second time threshold, it is determined that a pushing behavior has occurred. When the crossover ratio change data shows a peak change in a short period of time and conforms to the transmission trajectory pattern, it is determined that a delivery behavior has occurred; The facial features and appearance information of the target person are extracted from the masked video segment to determine the facial image and appearance attributes; the facial image, appearance attributes, movement trajectory, and interaction behavior tags of the target person are fused to generate a corresponding record of tobacco industry-related criminal behavior.

2. The method according to claim 1, characterized in that, The step of extracting video segments containing valid human activities from the original surveillance video stream of the tobacco industry includes: Adaptive background modeling and motion region detection are performed on the video frame sequence of the original monitoring video stream; Based on the motion area detection results, the continuous video intervals in the original monitoring video stream where the motion area exceeds a preset threshold are determined as the video intervals containing valid human activity.

3. The method for analyzing surveillance video in the tobacco industry according to claim 1, characterized in that, The simultaneous detection of human targets and preset tobacco industry objects within the video range includes: Extract the video frames corresponding to the video interval containing valid human activities; A pre-trained target detection model is invoked to simultaneously detect human targets and object targets belonging to the preset tobacco industry in the video frame. The object targets include at least one of cigarette boxes, handcarts, or cigarette packs.

4. The method for analyzing surveillance video in the tobacco industry according to claim 1, characterized in that, The step of tracking the detected human and object targets, and generating their respective corresponding human and object motion trajectories, includes: A multi-target tracking model is used to associate the appearance features and / or motion vectors of the bounding boxes of human targets and object targets in different video frames to form human motion trajectories and object motion trajectories that include trajectory IDs, timestamps and serialized bounding boxes.

5. The method for analyzing surveillance video in the tobacco industry according to claim 1, characterized in that, The steps for determining the facial image and / or appearance attributes of the target person include: A pre-trained face recognition model is invoked to extract and cluster the facial features of the target person in the masked video segment, thereby obtaining the face image; A pre-trained visual classification model is invoked to identify the appearance information of the human target in the masked video segment, thereby obtaining the appearance attributes; The steps for determining the spatiotemporal overlap relationship include: The spatiotemporal overlap relationship is determined based on the masked video segments corresponding to the movement trajectories of the person and the object, respectively.

6. The method for analyzing surveillance video in the tobacco industry according to any one of claims 1-5, characterized in that, After generating the corresponding record of tobacco industry-related offenses, it also includes: In response to a received search request, a query is performed based on the records of tobacco industry-related offenses. The search conditions corresponding to the search request include at least one of the following: feature vector similarity search based on the facial image, filtering based on the appearance attributes, and location search based on the interaction behavior tags. Output a list of records of the criminal acts that match the search criteria and the corresponding video clips.

7. A video surveillance analysis system for the tobacco industry, characterized in that, include: The extraction unit is used to extract video segments containing valid human activities from the original surveillance video stream of the tobacco industry; The detection unit is used to simultaneously detect human targets and preset tobacco industry object targets within the video range, wherein the human targets and object targets include corresponding bounding boxes; The tracking unit is used to track the detected human target and the object target respectively, and generate their respective corresponding human motion trajectory and object motion trajectory; The masking unit is used to extract the original video segment from the original monitoring video stream according to the time interval corresponding to the movement trajectory of the person, and perform masking processing on the original video segment to generate the corresponding masked video segment. The processing method is to only retain the pixels inside the bounding box of the person, while uniformly setting the background area pixels outside the bounding box to pure black. The determining unit is configured to determine the spatiotemporal overlap relationship between the human target and the object target based on the human movement trajectory and the object movement trajectory, including: acquiring the bounding boxes of the human movement trajectory and the object movement trajectory, corresponding to the timestamps in the time series in which the human target is continuously tracked; calculating the intersection-union ratio (IUGR) change data between the bounding boxes of the human target and the bounding boxes of the object target to obtain the spatiotemporal overlap relationship; The identification unit is configured to identify interaction behavior tags between the person target and the object target based on the spatiotemporal overlap relationship, wherein the interaction behavior tags include at least one of carrying, pushing, and delivering; including: When the crossover ratio change data is continuously higher than the first threshold and the duration exceeds the first time threshold, it is determined that a transfer behavior has occurred; When the crossover ratio change data remains within the second threshold range and the synchronous change of the movement trajectories of the human target and the object target continues to exceed the second time threshold, it is determined that a pushing behavior has occurred. When the crossover ratio change data shows a peak change in a short period of time and conforms to the transmission trajectory pattern, it is determined that a delivery behavior has occurred; The generation unit is used to extract the facial features and appearance information of the target person from the masked video clip to determine the facial image and appearance attributes; and to fuse the facial image, appearance attributes, movement trajectory, and interaction behavior tags of the target person to generate a corresponding record of tobacco industry-related criminal behavior.

Images