Smoking behavior detection method, system and device based on timing trajectory constraint

Abstract

The application provides a smoking behavior detection method, system and device based on a time sequence trajectory constraint, wherein the smoking behavior detection method based on the time sequence trajectory constraint comprises: performing smoking behavior detection on a video frame to obtain a candidate smoking target; performing cross-frame association on the candidate smoking target based on appearance feature similarity and spatial position overlap of the candidate smoking target to establish a tracking trajectory; performing multi-frame smoking behavior confirmation on the tracking trajectory, and determining that a person corresponding to the tracking trajectory has a smoking behavior when a preset confirmation condition is met; and performing alarm reporting on the tracking trajectory determined to have the smoking behavior. The above scheme improves the single-frame instantaneous determination of the smoking behavior detection to cross-frame association tracking based on the appearance feature similarity and the spatial position overlap, and performs multi-frame hit confirmation and single-event alarm on the tracking trajectory, thereby effectively suppressing accidental false detection caused by slender similar objects, and improving the reliability of the smoking behavior detection result.

Description

Technical Field

[0001] This application relates to the field of intelligent security technology, and more specifically, to a method, system, and device for detecting smoking behavior based on time-series trajectory constraints. Background Technology

[0002] In recent years, with the continuous advancement of smart city and smart park construction, video surveillance networks have been widely deployed in public areas such as industrial parks, transportation hubs, medical institutions, educational institutions, and commercial complexes. Intelligent behavior analysis technology based on video images has become a crucial support for public safety supervision and risk warning. Simultaneously, regulations on smoking control in public places are becoming increasingly stringent, with strengthened enforcement in key areas such as hospitals, schools, stations, office buildings, chemical industrial parks, hazardous materials storage areas, and gas stations. In these scenarios, smoking not only violates public health regulations but, in special working environments containing flammable and explosive materials, can also lead to major safety accidents such as fires or explosions. Therefore, real-time monitoring and early warning of illegal smoking are of significant practical importance. Against this backdrop, utilizing computer vision technology to replace traditional manual monitoring methods and achieve automated detection of smoking behavior has become an important research direction in the field of intelligent security.

[0003] Currently, the industry generally uses deep learning-based object detection models to identify smoking behavior in surveillance videos. The basic processing flow is as follows: target detection is performed on a single frame of the video image to obtain the location result of the cigarette target, and a smoking behavior judgment is directly output based on the detection result. However, because cigarette targets in surveillance videos are usually small in size and slender in shape, and are easily affected by factors such as occlusion, resolution, background, and changes in lighting, the detection results obtained by the existing single-frame detection mechanism can only reflect the instantaneous target state in the current frame. This leads to slender and similar objects (such as straws, ballpoint pens, chopsticks, lollipops, and fingers) having high similarity to cigarette targets in local appearance features, which can easily be misidentified as smoking behavior, resulting in a large number of false alarms. Summary of the Invention

[0004] The purpose of this application is to provide a smoking behavior detection method, system, and device based on time-series trajectory constraints to solve the above-mentioned problems.

[0005] In a first aspect, embodiments of this application provide a smoking behavior detection method based on temporal trajectory constraints. The method includes: detecting smoking behavior in video frames to obtain candidate smoking targets; establishing a tracking trajectory by cross-frame association of the candidate smoking targets based on the similarity of their appearance features and the overlap of their spatial positions; confirming smoking behavior on the tracking trajectory across multiple frames; determining that the person corresponding to the tracking trajectory has engaged in smoking behavior when a preset confirmation condition is met; and reporting an alarm to the tracking trajectory for which smoking behavior has been determined.

[0006] In the implementation of the above scheme, by improving the smoking behavior detection from single-frame instantaneous judgment to cross-frame correlation tracking based on appearance feature similarity and spatial position overlap, and performing multi-frame hit confirmation and single event alarm on the tracking trajectory, the occasional false detection caused by slender similar objects is effectively suppressed, thereby improving the reliability of smoking behavior detection results.

[0007] In one implementation of the first aspect, the step of establishing a tracking trajectory by cross-frame association of the candidate smoking targets based on the similarity of their appearance features and the overlap of their spatial locations includes: extracting human appearance features of the personnel region to which the candidate smoking target belongs; calculating the similarity between the human appearance features and the appearance features of each existing tracking trajectory; calculating the intersection-union ratio (IUGR) between the detection boxes of the candidate smoking target and the detection boxes of each existing tracking trajectory; when both the similarity and the IUGR meet corresponding preset conditions, matching and associating the candidate smoking target with the existing tracking trajectory and updating the state information of the existing tracking trajectory; when neither the similarity nor the IUGR meets the corresponding preset conditions, creating a new tracking trajectory for the candidate smoking target.

[0008] In the implementation of the above scheme, by jointly utilizing the similarity of appearance features and the overlap of spatial location to associate candidate smoking targets across frames, it is possible to maintain the consistency of the identity of the tracking trajectory in occluded scenes and cross-pedestrian scenes, avoid misassociation of trajectories between different people, and thus provide a stable temporal verification object for subsequent multi-frame smoking behavior confirmation, thereby improving the accuracy of multi-frame smoking behavior confirmation.

[0009] In one implementation of the first aspect, the step of confirming smoking behavior on the tracking trajectory across multiple frames includes: maintaining a hit count for smoking behavior for each tracking trajectory; incrementing the hit count when a candidate smoking target corresponding to the tracking trajectory is detected as smoking behavior in the current frame; and determining that the person corresponding to the tracking trajectory has smoking behavior when the hit count reaches a preset threshold.

[0010] In the implementation of the above scheme, the hit count of smoking behavior is maintained by tracking the trajectory, and smoking behavior is only determined when the cumulative hit count reaches a preset threshold. This improves the single-frame instantaneous detection to multi-frame temporal joint verification, effectively filtering out occasional false detections caused by occlusion, low resolution or complex background, and further improving the reliability of smoking behavior detection results.

[0011] In one implementation of the first aspect, the step of detecting smoking behavior in a video frame to obtain a candidate smoking target includes: performing personnel target detection on the video frame to obtain a personnel detection box; cropping a personnel region image from the video frame based on the personnel detection box; and performing cigarette target detection on the personnel region image to obtain the candidate smoking target.

[0012] In the implementation of the above scheme, by first performing personnel target detection on the video frame and cropping it to obtain the personnel area image, and then performing cigarette target detection in this local area, the search range is focused from the entire monitoring screen to the local area of ​​the personnel, reducing the interference of complex background on cigarette target detection and improving the detection accuracy of candidate smoking targets.

[0013] In one implementation of the first aspect, the step of performing alarm reporting on the tracking trajectory where smoking behavior is determined to exist includes: maintaining a reporting status identifier for each tracking trajectory, the reporting status identifier including a non-reported state and a reported state; when the tracking trajectory is determined to have smoking behavior and the reporting status identifier is in the non-reported state, performing alarm reporting and updating the reporting status identifier to the reported state; when the tracking trajectory is determined to have smoking behavior and the reporting status identifier is in the reported state, not performing alarm reporting.

[0014] In the implementation of the above scheme, by maintaining a reporting status identifier for each tracking trajectory, when smoking behavior is first detected, an alarm is reported and marked as reported. Even if smoking behavior is continuously detected on the same tracking trajectory, the alarm will not be triggered again. The alarm granularity is improved from the single frame level to the event level, avoiding the redundancy of alarm information caused by repeated reporting by the same person in multiple consecutive frames, and reducing the backend management cost.

[0015] In one implementation of the first aspect, the method further includes: maintaining a mismatch count for each of the tracking trajectories; incrementing the mismatch count for existing tracking trajectories that have not been successfully matched during the cross-frame association process; deleting the corresponding tracking trajectory when the mismatch count reaches a preset threshold; matching the newly created tracking trajectory with the deleted historical tracking trajectory based on appearance feature similarity and spatial position overlap; and restoring the trajectory information of the historical tracking trajectory to the newly created tracking trajectory when the newly created tracking trajectory successfully matches the historical tracking trajectory.

[0016] In the implementation of the above scheme, by maintaining a mismatch count and automatically deleting old trajectories when consecutive mismatches exceed a preset threshold, it also supports the restoration of historical trajectory information based on appearance features and spatial location overlap when personnel briefly leave and re-enter. This controls the memory usage of the tracking trajectory list while avoiding repeated alarm reports caused by the same person being repeatedly created as a new target due to trajectory interruption.

[0017] In one implementation of the first aspect, the method further includes: performing face detection on the personnel area to which the candidate smoking target belongs to obtain face information; and associating the face information with the corresponding tracking trajectory.

[0018] In the implementation of the above solution, by associating facial information with the tracking trajectory, the smoking behavior alarm output carries the corresponding person's identity identifier, which makes it easier for backend administrators to quickly locate the violator and trace historical behavior records.

[0019] In one implementation of the first aspect, the step of performing face detection on the personnel area to which the candidate smoking target belongs to obtain face information includes: performing face localization on the personnel area to obtain face detection boxes; based on the comparison result of the height and / or width of the face detection boxes with a preset size threshold, removing face detection boxes that do not meet the size requirements; based on the comparison result of the horizontal offset of the nose tip relative to the midpoint of the eyes with a preset offset threshold, removing face detection boxes that do not meet the side profile posture requirements; based on the comparison result of the relative position ratio of the nose tip in the vertical direction of the eyes and mouth with a preset pitch range, removing face detection boxes that do not meet the pitch posture requirements; and obtaining face information based on the face detection boxes that are retained after filtering.

[0020] In the implementation of the above scheme, the detected faces are screened based on face size and pose angle, eliminating faces with small faces, side faces, and tilt poses that do not meet the recognition conditions, thereby improving the accuracy of the association between face information and tracking trajectory.

[0021] Secondly, embodiments of this application also provide a smoking behavior detection system based on time-series trajectory constraints, including a host computer and a video acquisition device communicatively connected to the host computer, wherein: The video acquisition device is used to acquire surveillance video frames; The host computer is used to detect smoking behavior in the video frames and obtain candidate smoking targets; based on the similarity of appearance features and the overlap of spatial positions of the candidate smoking targets, it performs cross-frame association of the candidate smoking targets to establish a tracking trajectory; it confirms smoking behavior in the tracking trajectory across multiple frames, and when a preset confirmation condition is met, it determines that the person corresponding to the tracking trajectory has smoking behavior; and it performs alarm reporting for the tracking trajectory where smoking behavior is determined to exist.

[0022] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a communication bus, wherein the processor and the memory communicate with each other through the communication bus; the memory stores computer program instructions that can be executed by the processor, and the computer program instructions are read and executed by the processor to perform the method provided in the first aspect or any possible implementation of the first aspect.

[0023] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the method provided in the first aspect or any possible implementation thereof.

[0024] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the method provided by the first aspect or any possible implementation of the first aspect.

[0025] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims and drawings. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1A flowchart illustrating the smoking behavior detection method based on temporal trajectory constraints provided in this application embodiment; Figure 2 A flowchart illustrating a smoking behavior detection method based on temporal trajectory constraints in an application scenario provided in this application embodiment; Figure 3 A schematic diagram of the architecture of a smoking behavior detection system based on time-series trajectory constraints provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The embodiments of the technical solution of this application will be described in detail below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of this application, and are therefore merely examples and should not be used to limit the scope of protection of this application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and the foregoing description of the accompanying drawings are intended to cover non-exclusive inclusion.

[0029] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0030] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0031] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0032] In existing technologies, mainstream smoking behavior detection schemes use deep learning-based object detection models to directly locate and identify cigarette targets in surveillance footage. These schemes typically use models such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), or Faster R-CNN (Faster Region-based Convolutional Neural Network) as the basic detection network. Video frames are input into a pre-trained cigarette target detection model, which directly outputs the cigarette target bounding box and its corresponding confidence score, and then determines whether smoking behavior exists based on the detection results. However, because cigarette targets in surveillance footage are usually small and elongated, and surveillance videos generally suffer from resolution limitations, target occlusion, and complex background interference, existing direct detection methods struggle to reliably capture the fine-grained features of cigarette targets across the entire image. They are prone to misidentifying elongated objects with similar local appearance features to cigarettes, such as straws, pens, chopsticks, and fingers, as smoking targets, leading to insufficient reliability of the detection results.

[0033] Another existing approach incorporates human keypoint estimation and action recognition technology. By locating keypoints on a person's hands and mouth, it analyzes the movement pattern of raising the hand to the mouth area to help determine whether smoking has occurred. This approach relies on the precise localization capability of human keypoint detection networks for minute movements. However, in real-world surveillance scenarios, the distance between the target and the camera can lead to insufficient imaging resolution, or changes in the person's posture can obscure keypoints on the hands, causing errors in keypoint localization and affecting the accuracy of subsequent action recognition. Furthermore, this approach focuses on action pattern analysis and lacks the ability to discriminate target-level semantic information. That is, it can identify that a person is raising their hand, but it cannot distinguish whether the hand is holding a cigarette, a lollipop, or a pen-like object. Therefore, it still suffers from a high false detection rate in scenarios where the actions are similar but the targets are different.

[0034] Furthermore, at the alarm mechanism level, existing solutions generally adopt a single-frame detection trigger alarm processing mode. When a suspected cigarette target is detected in a video frame, a smoking alarm is immediately output. The detection result of this mechanism only reflects the instantaneous state of the current frame and lacks the ability to continuously verify the detection result over time. It is easy to directly convert occasional false detections into alarm outputs. At the same time, since a cross-frame target identity association mechanism has not been established, when the same person is repeatedly detected in multiple consecutive video frames, each frame's detection result is reported as an independent alarm event, resulting in duplicate alarms for the same smoking behavior. This causes the backend management platform to receive a large number of redundant alarm messages, increasing the cost of manual review and the operational burden.

[0035] In view of this, this application provides a smoking behavior detection method based on temporal trajectory constraints. This method improves the smoking behavior detection from single-frame instantaneous judgment to cross-frame correlation tracking based on appearance feature similarity and spatial position overlap, and performs multi-frame hit confirmation and single event alarm on the tracking trajectory, effectively suppressing occasional false detections caused by slender similar objects, thereby improving the reliability of smoking behavior detection results.

[0036] Please see Figure 1 The illustrated diagram shows a flowchart of a smoking behavior detection method based on time-series trajectory constraints provided in an embodiment of this application. This smoking behavior detection method based on time-series trajectory constraints can be applied to electronic devices, which may include physical devices such as servers, PCs, tablets, or smartphones, or virtual devices such as virtual machines or containers. The electronic device can be a single device, a combination of multiple devices, or a cluster of a large number of devices. The aforementioned smoking behavior detection method based on time-series trajectory constraints may include: Step S110: Detect smoking behavior in the video frames to obtain candidate smoking targets.

[0037] Optionally, step S110 includes: performing personnel target detection on the video frame to obtain personnel detection boxes; cropping personnel region images from the video frame based on the personnel detection boxes; and performing cigarette target detection on the personnel region images to obtain candidate smoking targets.

[0038] In security scenarios such as park surveillance, surveillance cameras typically have a large coverage area, and the proportion of people in the overall video frame is relatively small, while cigarette targets are even smaller. Directly detecting cigarette targets on the raw surveillance image is prone to missed detections and false detections due to the low pixel ratio of the target and complex background interference. Therefore, this application's embodiment adopts a two-stage detection architecture: first, it locates people in the video frame; then, based on the location results, it crops out a local area of ​​the person and performs fine-grained cigarette detection within that local area. This focuses the search range from the entire image to the area near the person, improving the accuracy and stability of small target detection.

[0039] In the personnel detection stage, a deep learning-based target detection model can be used to process the input video frames to obtain person boxes corresponding to each person in the frame. These boxes identify the spatial location of each person in the frame. Taking the YOLOv8l model as an example, this model, through multi-scale feature extraction and feature fusion, can capture the semantic information of personnel targets at different resolutions and output person boxes containing personnel category confidence and spatial location coordinates. Subsequently, based on the coordinate information of the person boxes, the corresponding person region area (ROI) is cropped from the original video frames as input for subsequent fine-grained detection.

[0040] Then, multiple person region sub-images P_ROI are input into the self-trained cigarette target detection model to perform fine-grained cigarette detection on the local areas of people, obtaining suspected cigarette detection boxes and corresponding cigarette region images (denoted as S_ROI). By performing a two-stage processing flow of first detecting people and then detecting cigarettes within the local areas of people, the search range can be effectively narrowed, the interference of complex backgrounds on the detection results of small cigarette targets can be reduced, thereby improving the accuracy of smoking behavior detection.

[0041] Step S120: Based on the similarity of appearance features and the overlap of spatial location of candidate smoking targets, cross-frame association is performed on candidate smoking targets to establish a tracking trajectory.

[0042] Optionally, step S120 includes: extracting the human appearance features of the area to which the candidate smoking target belongs; calculating the similarity between the human appearance features and the appearance features of each existing tracking trajectory; calculating the intersection-union ratio (IUR) between the detection boxes of the candidate smoking target and the detection boxes of each existing tracking trajectory; when both the similarity and IUR meet the corresponding preset conditions, matching and associating the candidate smoking target with the existing tracking trajectory, and updating the status information of the existing tracking trajectory; when the similarity and IUR do not simultaneously meet the corresponding preset conditions, creating a new tracking trajectory for the candidate smoking target.

[0043] It is understandable that, due to the movement, occlusion, and posture changes of people in the surveillance footage, relying solely on single-frame detection results cannot establish cross-frame target correspondences, easily leading to problems such as the same person being counted repeatedly or the trajectories of different people being confused. Therefore, cross-frame association can be performed on candidate smoking targets in consecutive frames to establish and maintain an independent tracking trajectory for each detected smoking person. The method for establishing the tracking trajectory is as follows: From the personnel region sub-image, human appearance features are extracted using deep convolutional neural networks such as ResNet (Residual Network). After L2 normalization, these features are mapped to a feature vector of a predetermined dimension, such as a 256-dimensional feature vector, denoted as . .vector The visual features of candidate targets in terms of clothing, posture, and overall appearance are characterized. L2 normalization scales the feature vectors to a unit modulus, eliminating modulus fluctuations caused by differences in image brightness and scale, ensuring that subsequent similarity calculations focus only on the consistency of feature directions. It should be noted that human appearance features can be directly extracted from the personnel region sub-image P_ROI. In actual surveillance scenarios, although the personnel region sub-image may contain some background information, since the detection area is usually deployed within a fixed surveillance scene, the impact of background interference on feature discrimination is relatively limited, and the extracted human appearance features are sufficient to distinguish different personnel.

[0044] In obtaining feature vectors Then, calculate the eigenvectors. The cosine similarity between the candidate target and the appearance feature vectors of existing trajectories is used to measure the semantic consistency of the candidate target with historical trajectories in terms of appearance. The expression for calculating the cosine similarity is:

[0045] In the formula, Indicates the first Normalized human appearance feature vectors of candidate targets; Indicates the first A human appearance feature vector with an existing trajectory; This represents the vector dot product operation; and These represent the eigenvectors. and eigenvectors The L2 norm; Representing the eigenvector With feature vectors cosine similarity, The range of values ​​is , The closer the value of is to 1, the more it indicates the th The candidate target and the first The higher the similarity in human appearance between targets corresponding to existing trajectories.

[0046] Besides the similarity of human appearance features, spatial location information is another important basis for judging the consistency of targets across frames. The intersection-union ratio (IUR) of candidate target detection boxes and existing trajectory-corresponding detection boxes is calculated to reflect the degree of spatial overlap between the two targets. The IUR calculation expression is:

[0047] In the formula, Indicates the first Detection boxes for each candidate target; Indicates the first The latest detection box corresponding to an existing trajectory; This represents the intersection region of two detection boxes on the image plane; This represents the union region of two detection boxes on the image plane. and Defined by pixel coordinates in the image coordinate system; The range of values ​​is , The closer the value is to 1, the higher the spatial overlap between the two detection boxes.

[0048] In the trajectory association determination stage, a strategy combining cosine similarity and intersection-union ratio (IUU) constraints is employed. Association is determined only when both appearance feature similarity and spatial overlap simultaneously meet their respective preset thresholds. and Only after this process is complete can a candidate target be determined to belong to the same person as an existing trajectory. By employing this joint determination mechanism, it is possible to prevent different people with similar appearances but separate spatial locations from being incorrectly associated with the same trajectory. For example, in a cross-pedestrian scenario, if two people with similar clothing are walking towards each other, relying solely on appearance similarity can easily lead to false associations. However, by introducing spatial overlap constraints, due to the extremely low intersection-union ratio of the two detection boxes, they will be correctly identified as different targets and separate trajectories will be created. In typical park monitoring scenarios, and The value is usually set to 0.8, but it can be flexibly adjusted according to the density of people and the degree of obstruction in actual deployment.

[0049] Furthermore, it is understood that the above cosine similarity calculation and intersection-union ratio calculation can be performed serially or in parallel: (1) When using the serial approach, the cosine similarity can be calculated first, and the intersection-union ratio (CUNR) can be calculated only after the cosine similarity is greater than the preset similarity threshold. Alternatively, the CUNR can be calculated first, and the cosine similarity can be calculated only after the CUNR is greater than the preset CUNR threshold. This serial approach can save certain computational resources.

[0050] (2) When using parallel computing, the cosine similarity and crossover-union ratio are calculated in parallel, and then a joint judgment is made based on the cosine similarity and crossover-union ratio. This parallel scheme can speed up the processing efficiency.

[0051] For candidate targets that meet the joint constraints, they are matched and associated with the corresponding existing tracking trajectories, and the status information of the trajectory is updated, including human appearance feature vectors, the latest detection box position information, face information, and smoking confidence, so that the trajectory record can maintain consistency with the current detection state over time. For candidate targets that fail to meet both similarity and intersection-union (IUU) constraints, they are determined to be new targets that appear for the first time or have not been tracked before, and a new tracking trajectory is automatically created for them. When a new trajectory is created, its mismatch counter CompareNum and hit counter TrackNum are both initialized to 0, and the reporting flag ReportFlag is initialized to False, indicating that the trajectory has not yet triggered a smoking alarm.

[0052] The aforementioned dual-threshold trajectory association mechanism effectively improves the stability of target identity association in occluded scenarios through dual verification of appearance features and spatial location. Experiments show that the association accuracy of the dual-threshold trajectory association mechanism in occluded scenarios is improved by more than 35% compared with the single criterion, and the false association rate is reduced by more than 60% in cross-pedestrian scenarios.

[0053] Step S130: Confirm smoking behavior across multiple frames of the tracked trajectory. When the preset confirmation conditions are met, determine that the person corresponding to the tracked trajectory has engaged in smoking behavior.

[0054] Optionally, the above-mentioned confirmation of smoking behavior on the tracking trajectory in multiple frames includes: maintaining a hit count of smoking behavior for each tracking trajectory; incrementing the hit count when the candidate smoking target corresponding to the tracking trajectory is detected as smoking behavior in the current frame; and determining that the person corresponding to the tracking trajectory has smoking behavior when the hit count reaches a preset threshold.

[0055] The above scheme can use a hit counter, TrackNum, to maintain a hit count for smoking behavior on each tracking track. This count represents the cumulative number of times the corresponding tracking track has been detected as exhibiting smoking behavior since its creation. During the processing of each frame, when a tracking track is determined by the detection model to have exhibited smoking behavior in the current frame, the hit count of that tracking track is incremented.

[0056] When the hit count of the TrackNum hit counter reaches the preset confirmation threshold, it is determined that the person corresponding to the tracked trajectory has actually smoked. The preset confirmation threshold can be configured according to the business tolerance. For example, in a certain application scenario, the confirmation threshold is set to 3. In this case, the final smoking behavior confirmation result will be output only after the candidate target has been detected smoking 3 times.

[0057] In actual smoking behavior, individuals may exhibit intermittent patterns of smoking, pausing, and resuming smoking. During these periods, due to hand obstruction, body rotation, or brief absence from the scene, the cigarette target may not be detected for several frames. For such intermittent smoking scenarios, after detecting a smoking individual in the current frame, a search and matching process is prioritized within the ReID feature list of existing tracking trajectories. If the individual's appearance features successfully match a historical trajectory, the hit count for that historical trajectory continues to accumulate, maintaining the temporal continuity of the smoking behavior record. If the individual fails to match any historical trajectories due to prolonged obstruction or absence from the scene, a new tracking trajectory is created for them, and the hit count is recalculated from zero.

[0058] Step S140: Execute an alarm report for the tracking trajectory where smoking behavior is determined to exist.

[0059] Optionally, step S140 includes: maintaining a reporting status identifier for each tracking trajectory, the reporting status identifier including a non-reported status and a reported status; when a tracking trajectory is determined to have smoking behavior and the reporting status identifier is in the non-reported status, an alarm is reported and the reporting status identifier is updated to the reported status; when a tracking trajectory is determined to have smoking behavior and the reporting status identifier is in the reported status, no alarm is reported.

[0060] A ReportFlag is maintained for each tracking trajectory. This flag records whether a smoking alarm has been triggered for the corresponding tracking trajectory. The ReportFlag has two possible values: not reported and reported. During the initial creation of a tracking trajectory, the flag defaults to the not reported state, indicating that the tracking trajectory has not yet output smoking alarm information. When a tracking trajectory is determined to have smoking behavior after multiple frame confirmations, and the ReportFlag for that tracking trajectory is still in the not reported state, the alarm reporting process is triggered. The smoking alarm information corresponding to that tracking trajectory is output to the backend management platform, and the ReportFlag for that tracking trajectory is updated to the reported state.

[0061] For tracking trajectories where the reporting status flag is already set to reported, even if the person continues to be detected smoking in subsequent frames and the hit count continues to increase, no further alarm reporting will be triggered. This alarm mechanism elevates the alarm granularity from the single-frame level to the event level, ensuring that only one alarm record is output for the same smoking behavior of the same person, avoiding redundancy of alarm information caused by repeated detection in multiple consecutive frames.

[0062] When a tracking trajectory is deleted due to multiple consecutive frames of unsuccessful matching causing the mismatch counter CompareNum to exceed a preset threshold, the reporting status flag of that trajectory and its associated trajectory information will also be deleted. If the corresponding person re-enters the monitoring range after a period of time and is detected smoking, since the historical trajectory has been deleted, the above scheme can use this target as a new candidate target to create a new tracking trajectory, and the reporting status flag ReportFlag of the new trajectory will be restored to the non-reported state. When the new trajectory meets the smoking behavior confirmation conditions, an alarm report can be triggered again, thereby identifying the reappearance of the person as a new smoking event.

[0063] Optionally, the above-mentioned smoking behavior detection method based on temporal trajectory constraints further includes: maintaining a mismatch count for each tracking trajectory; incrementing the mismatch count for existing tracking trajectories that have not been successfully matched during cross-frame association; deleting the corresponding tracking trajectory when the mismatch count reaches a preset threshold; matching the newly created tracking trajectory with the deleted historical tracking trajectory based on appearance feature similarity and spatial position overlap; and restoring the trajectory information of the historical tracking trajectory to the newly created tracking trajectory when the newly created tracking trajectory is successfully matched with the historical tracking trajectory.

[0064] A mismatch counter, CompareNum, is used to maintain a mismatch count for each tracking trajectory. This count records the cumulative number of times the corresponding trajectory has failed to be successfully associated in the current frame and several consecutive previous frames. The mismatch count is incremented only for existing trajectories that have not been associated with any candidate target in the current frame. For trajectories that have successfully been associated in the current frame, their mismatch counts are not incremented; instead, a status update operation is performed. After each frame is processed, the mismatch counts of each existing tracking trajectory are checked. When the mismatch count of a trajectory exceeds a preset maximum value, the trajectory is automatically deleted. The configuration of the preset maximum value is related to the video frame extraction frequency and the tolerance of the business scenario for target disappearance. For example, when the frame extraction frequency is 2 frames per second and the preset maximum value is set to 10, it means that if the target is not tracked for approximately 5 seconds within the observation window, it is determined to have left the monitoring range, and its corresponding trajectory information is automatically deleted.

[0065] Deleted trajectory information can be retained in the recycle bin for a certain period before being permanently deleted. This period can be adaptively set based on the business scenario. When creating a new tracking trajectory for a candidate target, a search and matching process is performed in the recycle bin based on appearance feature similarity and spatial location overlap. This matching process still adopts the same dual constraint strategy as cross-frame association, that is, if both appearance feature similarity and spatial location overlap are greater than a preset threshold, it is determined that the newly created trajectory and a certain historical trajectory in the recycle bin belong to the same person. After a successful match, the corresponding trajectory information is restored from the recycle bin and migrated to the newly created trajectory. The migrated information may include key status data such as historical hit counts and reported status identifiers, so that the new trajectory inherits the person's previous behavior records and alarm status.

[0066] By combining automatic trajectory deletion with historical trajectory restoration, the system maintains a lightweight active trajectory list while ensuring the continuity of behavior records when personnel briefly leave and re-enter the monitoring range. This avoids fragmentation of smoking behavior records caused by trajectory interruption and duplicate alarms caused by status reset. Within a preset time period, the same smoking event can still be identified as a continuous behavior or a reported event after personnel leave and re-enter the monitoring area, rather than an independent new event.

[0067] Optionally, the above-mentioned smoking behavior detection method based on temporal trajectory constraints further includes: performing face detection on the personnel area to which the candidate smoking target belongs to obtain face information; and associating the face information with the corresponding tracking trajectory.

[0068] After detecting a candidate smoking target, face detection is performed on the personnel region sub-image to further obtain the identity information of the smoker. Face detection and cigarette detection can be processed in a serial manner, that is, the face recognition process for that personnel region sub-image is triggered only if the current frame determines that a person is smoking. If no smoking behavior is detected, the face detection step is skipped to avoid performing unnecessary identity recognition calculations for non-smokers.

[0069] In the face detection phase, face recognition models are used to locate and extract features from faces within a personnel region sub-image. Taking the ArcFace model as an example, this model learns high-dimensional discriminative features of face images through a deep convolutional network, mapping faces to a compact feature space. This is then compared with a pre-built face database to output the corresponding personnel identification and matching confidence score. It should be noted that the face data collected in the aforementioned face database was collected only after the user was clearly aware of the purpose of the face data collection and explicitly consented. For example, before the face feature information is entered into the database, explicit authorization from the data subject is obtained through written agreements, electronic authorization letters, or other explicit consent mechanisms, informing them of the purpose of face data collection, the scope of use of the face data, and the storage period. Furthermore, the collected face data is strictly encrypted, access controls are implemented, and anonymization is carried out in accordance with relevant regulations and data security management standards, and the scope of use of the face data is strictly controlled.

[0070] Optionally, the above-mentioned face detection in the personnel area to which the candidate smoking target belongs, to obtain face information, includes: performing face localization in the personnel area to obtain face detection boxes; based on the comparison result of the height and / or width of the face detection boxes with a preset size threshold, removing face detection boxes that do not meet the size requirements; based on the comparison result of the horizontal offset of the nose tip relative to the midpoint of the eyes with a preset offset threshold, removing face detection boxes that do not meet the side profile posture requirements; based on the comparison result of the relative position ratio of the nose tip in the vertical direction of the eyes and mouth with a preset pitch range, removing face detection boxes that do not meet the pitch posture requirements; and obtaining face information based on the face detection boxes that are retained after filtering.

[0071] In surveillance scenarios, due to the distance between people and the camera, some faces are too small to be effectively identified by facial features. To address this, the height and / or width of the face detection bounding box are compared to a preset size threshold. If either dimension is smaller than the preset pixel threshold, the face is deemed not to meet the minimum imaging size requirement and is discarded. The preset pixel threshold can be set based on practical engineering experience to ensure that the retained face detection bounding boxes have sufficient texture detail to support subsequent feature extraction and identity verification.

[0072] For profile poses, the left and right rotation angles of the face are quantified by calculating the horizontal offset of the nose tip relative to the midpoints of the eyes. The midpoints of the eyes are determined by the average of the coordinates of the left and right eye centers. The horizontal offset of the nose tip relative to this midpoint is normalized by dividing the distance between the eyes to obtain the horizontal offset ratio. This horizontal offset ratio is compared with a preset offset ratio threshold. When the offset ratio exceeds the preset threshold, the face is determined to be in a profile pose and is discarded. The preset offset ratio threshold is generally set between 0.30 and 0.35. The smaller the preset offset ratio threshold is set, the stricter the profile pose selection criteria are, meaning a lower tolerance for profile poses.

[0073] For head-up and head-down postures, the pitch angle of the face is quantified by calculating the relative position ratio of the nose tip in the vertical direction between the eyes and mouth. Using the vertical line segment formed by the midpoint of the line connecting the eyes and the center of the mouth as a reference, the relative position ratio of the nose tip in this vertical direction is calculated. This relative position ratio is compared to a preset pitch range. When the relative position ratio is lower than the head-up position ratio threshold, it is determined to be a head-up posture and discarded; when the relative position ratio is higher than the head-down position ratio threshold, it is determined to be a head-down posture and discarded. The head-up position ratio threshold is generally set to 0.35, and the head-down position ratio threshold is generally set to 0.65. In specific monitoring scenarios, due to differences in camera installation height or viewing angle, the head-up and head-down position ratio thresholds can be adjusted appropriately according to actual needs to adapt to the imaging geometry in different scenarios.

[0074] When multiple faces exist within a single person region image, they are sorted by face size, and the face with the largest area is selected as the face corresponding to that person region. This area-first principle ensures that the selected face belongs to the person in question. After the above size and pose filtering, for faces that are rejected or not found in the face database, a downgraded processing strategy is adopted. The face name is uniformly marked as "Unknown," and the confidence level is set to -1 to avoid mismatches of low-quality or unfamiliar faces interfering with subsequent person identity association.

[0075] To facilitate understanding of the working principle of the smoking behavior detection method based on time-series trajectory constraints, this application also provides specific application examples of this method in a certain application scenario. For example... Figure 2 As shown, in this application scenario, the above-mentioned smoking behavior detection method based on time-series trajectory constraints mainly protects: Step 1: Detect smoking behavior in video frames to obtain candidate smoking targets; In the park surveillance scenario, the YOLOv8l model is first used to detect people in the input video frames, obtaining the person boxes corresponding to each person in the image. Based on the coordinate information of the person boxes, the person region sub-images P_ROI are cropped from the original video frames. Multiple P_ROIs are input into a self-trained cigarette target detection model to perform fine-grained cigarette detection on the local regions of people, obtaining the suspected cigarette detection boxes and the corresponding cigarette region images, denoted as S_ROI.

[0076] For suspected smokers, the ArcFace model is used to perform facial recognition on the P_ROI. Cigarette detection and facial recognition are processed sequentially, meaning that facial recognition is only triggered when smoking is detected. Before facial recognition, the faces are first screened for quality: Small face filtering rule: if either the height or width of the face detection box is less than 40 pixels, the face is deemed not to meet the imaging size requirements and is removed; Side face filtering rule: calculate the horizontal offset of the nose tip relative to the midpoint of the eyes. If this offset exceeds a preset offset threshold, it is considered a side face and is removed. In practice, this offset threshold is generally set between 0.30 and 0.35; Head tilting and head-down filtering rule: calculate the relative position ratio of the nose tip in the vertical direction between the eyes and mouth. If this ratio is lower than the head tilting threshold of 0.35 or higher than the head-down threshold of 0.65, it is considered an abnormal tilting posture and is removed. In specific scenarios, the head-down threshold can be appropriately increased as needed. When multiple faces exist within a single person region image, they are sorted by face size, and the face with the largest area is selected as the face corresponding to that person region. For face data that fails the quality screening or is not in the face database, a downgraded processing strategy is adopted, and the face name is uniformly identified as "Unknown" with a confidence level of -1.

[0077] Step 2: Based on the similarity of appearance features and the overlap of spatial location of candidate smoking targets, cross-frame association is performed on candidate smoking targets to establish a tracking trajectory; For detected candidate smoking targets, ResNet18 is used to extract human appearance features from the P_ROI, which are then mapped to a 256-dimensional feature vector after L2 normalization. The cosine similarity is calculated between the appearance feature vectors of the existing tracking trajectories in the Tracker. Simultaneously, the intersection-union ratio (IUU) of the detection boxes of candidate smoking targets and the latest detection boxes of existing tracking trajectories is calculated. When cosine similarity Greater than the preset similarity threshold And intersection and comparison Greater than the preset intersection-union ratio threshold When a candidate smoking target is determined to belong to the same person as an existing tracking trajectory, the candidate smoking target is matched and associated with the existing tracking trajectory, and the status information of the existing tracking trajectory is updated, including appearance feature vector, detection box position information, face information, and smoking confidence score. If the cosine similarity and intersection-union ratio do not simultaneously meet their respective preset conditions, a new tracking trajectory is created for the candidate smoking target. In a typical park monitoring scenario, and The value can be set to 0.8, and can be flexibly adjusted according to the scene's pedestrian density and occlusion level during actual deployment. When a new trajectory is created, its mismatch count CompareNum and hit count TrackNum are both initialized to 0, and the reporting status flag ReportFlag is initialized to False, that is, no reporting status.

[0078] Step 3: Confirm smoking behavior in multiple frames of the tracking trajectory. When the preset confirmation conditions are met, it is determined that the person corresponding to the tracking trajectory has smoked. For each tracking track, a hit count (TrackNum) for smoking behavior is maintained. The candidate smoking target information input to the Tracker consists of data on individuals already identified as smoking in the current frame, without secondary filtering based on the confidence level of the cigarette detection boxes. When a candidate smoking target corresponding to a tracking track is detected as smoking in the current frame, the hit count (TrackNum) for that tracking track is incremented. When the hit count (TrackNum) reaches a preset confirmation threshold (N), it is determined that the person corresponding to that tracking track has engaged in smoking behavior. For example, if N=3, in a scenario with a continuous frame rate of 2 frames per second, it means that the target needs to be detected smoking behavior a total of 3 times within an observation window of approximately 1.5 seconds before a final confirmation result is output. For intermittent smoking scenarios caused by hand covering, body turning, or brief departure during the smoking process, after detecting a smoking person in the current frame, priority is given to searching and matching in the ReID feature list of existing tracking trajectories; if the match is successful, the hit count continues to accumulate based on the existing hit count; if the match fails, a new tracking trajectory is created and the hit count is accumulated from zero again.

[0079] Step 4: Execute alarms and report them for the tracking trajectories where smoking behavior is determined to occur; Each tracking track maintains a ReportFlag, which includes a False (no reporting) and a True (reported) status. When a tracking track is determined to have smoking behavior and the ReportFlag is False, an alarm is reported and the ReportFlag is updated to True. When a tracking track is determined to have smoking behavior and the ReportFlag is True, no alarm is reported. Once the person corresponding to the tracking track stops smoking, the tracking track enters the subsequent mismatch elimination process because smoking is no longer detected. If the person continues to smoke after an alarm has been triggered, the hit count TrackNum continues to increase from the original value, but due to the locking of the ReportFlag, a duplicate alarm will not be triggered again.

[0080] In addition, a mismatch count, CompareNum, is maintained for each tracking trajectory. During cross-frame association, the mismatch count, CompareNum, is incremented only for existing tracking trajectories that have not been successfully matched in the current frame. When the mismatch count, CompareNum, reaches a preset maximum value, M, the corresponding tracking trajectory is deleted. In a scenario with a frame sampling frequency of 2 frames per second, M is typically set to 10, meaning that if the target is not tracked for approximately 5 seconds, it is considered to have left the monitoring range, and its corresponding trajectory information is removed from the active trajectory list. For newly created tracking trajectories, based on the similarity of appearance features and the overlap of spatial positions, the newly created tracking trajectory is matched with deleted historical tracking trajectories. When a match is successful, the trajectory information of the historical tracking trajectory is restored to the newly created tracking trajectory, so that when personnel briefly leave the area and re-enter the monitoring range, they can inherit historical behavior records and alarm statuses.

[0081] like Figure 3 As shown, based on the same inventive concept, this application also provides a smoking behavior detection system 200 based on time-series trajectory constraints, including a host computer 210 and a video acquisition device 220 communicatively connected to the host computer 210, wherein: Video capture device 220 is used to capture surveillance video frames; The host computer 210 is used to detect smoking behavior in video frames and obtain candidate smoking targets; based on the similarity of appearance features and the overlap of spatial positions of candidate smoking targets, it performs cross-frame association of candidate smoking targets to establish a tracking trajectory; it confirms smoking behavior in multiple frames on the tracking trajectory, and when the preset confirmation conditions are met, it determines that the person corresponding to the tracking trajectory has smoking behavior; it executes alarm reporting for the tracking trajectory that has been determined to have smoking behavior.

[0082] It should be noted that the host computer 210 described above can implement any one of the functions of the smoking behavior detection method based on time-series trajectory constraints provided in the embodiments of this application. For the method embodiment section, please refer to the method embodiment section for the implementation method and working principle. The system embodiment section will not repeat the description.

[0083] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of this application. (Refer to...) Figure 4 The electronic device 300 includes a processor 310, a memory 320, and a communication interface 330. These components are interconnected and communicate with each other via a communication bus 340 and / or other forms of connection mechanism (not shown).

[0084] The memory 320 includes one or more (only one is shown in the figure), which may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The processor 310 and other possible components may access the memory 320 to read and / or write data therein.

[0085] Processor 310 includes one or more (only one is shown in the figure), which can be an integrated circuit chip with signal processing capabilities. The processor 310 described above can be a general-purpose processor, including a central processing unit (CPU), a microcontroller unit (MCU), a network processor (NP), or other conventional processors; it can also be a special-purpose processor, including a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0086] The communication interface 330 includes one or more (only one is shown in the figure) and can be used to communicate directly or indirectly with other devices to exchange data. For example, the communication interface 330 can be an Ethernet interface; it can be a mobile communication network interface, such as an interface for 3G, 4G, or 5G networks; or it can be other types of interfaces with data transmission and reception functions.

[0087] One or more computer program instructions may be stored in the memory 320. The processor 310 may read and run these computer program instructions to implement the smoking behavior detection method based on time-series trajectory constraints provided in the embodiments of this application, as well as other desired functions.

[0088] Understandable. Figure 4 The structure shown is for illustrative purposes only; the electronic device 300 may also include components that are more advanced than those shown. Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown. Figure 4 The components shown can be implemented using hardware, software, or a combination thereof. For example, electronic device 300 can be a single server (or other device with computing power), a combination of multiple servers, a cluster of a large number of servers, etc., and can be either a physical device or a virtual device.

[0089] This application also provides a computer-readable storage medium storing computer program instructions. These instructions are read and executed by a processor to perform the smoking behavior detection method based on time-series trajectory constraints provided in this application. For example, the computer-readable storage medium can be implemented as follows: Figure 4 The memory 320 in the electronic device 300, or a separate storage product (such as a USB flash drive, portable hard drive, etc.).

[0090] This application also provides a computer program product, which includes computer program instructions. These computer program instructions are read and executed by a processor to perform the smoking behavior detection method based on time-series trajectory constraints provided in this application. For example, these computer program instructions can be stored in... Figure 4 The memory 320 in the electronic device 300 is located inside the memory, or it is stored in a separate storage product (such as a USB flash drive, portable hard drive, etc.).

[0091] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0092] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0093] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0094] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for detecting smoking behavior based on temporal trajectory constraints, characterized in that, The method includes: Smoking behavior detection is performed on video frames to obtain candidate smoking targets; Based on the similarity of appearance features and the overlap of spatial location of the candidate smoking targets, cross-frame association is performed on the candidate smoking targets to establish a tracking trajectory; The tracking trajectory is used to confirm smoking behavior across multiple frames. When a preset confirmation condition is met, it is determined that the person corresponding to the tracking trajectory has engaged in smoking behavior. An alarm will be reported for any tracking trajectory where smoking is detected.

2. The smoking behavior detection method based on time-series trajectory constraints according to claim 1, characterized in that, The step of establishing a tracking trajectory by performing cross-frame association on the candidate smoking targets based on the similarity of their appearance features and the overlap of their spatial locations includes: Extract the human appearance features of the personnel region to which the candidate smoking targets belong; Calculate the similarity between the human appearance features and the appearance features of each existing tracking trajectory; Calculate the intersection-union ratio (IUU) between the detection bounding boxes of the candidate smoking targets and the detection bounding boxes of each existing tracking trajectory; When both the similarity and the intersection-union ratio meet the corresponding preset conditions, the candidate smoking target is matched and associated with the existing tracking trajectory, and the status information of the existing tracking trajectory is updated. When the similarity and the intersection-union ratio do not simultaneously meet the corresponding preset conditions, a new tracking trajectory is created for the candidate smoking target.

3. The smoking behavior detection method based on time-series trajectory constraints according to claim 1, characterized in that, The step of confirming smoking behavior across multiple frames of the tracked trajectory includes: Maintain a hit count for smoking behavior for each of the aforementioned tracking trajectories; When the candidate smoking target corresponding to the tracking trajectory is detected as smoking behavior in the current frame, the hit count is incremented; When the hit count reaches a preset threshold, it is determined that the person corresponding to the tracked trajectory has engaged in smoking behavior.

4. The smoking behavior detection method based on time-series trajectory constraints according to claim 1, characterized in that, The step of detecting smoking behavior in video frames to obtain candidate smoking targets includes: Perform personnel target detection on the video frames to obtain personnel detection boxes; The personnel region image is obtained by cropping from the video frame based on the personnel detection box; Cigarette target detection is performed on the personnel area image to obtain the candidate smoking targets.

5. The smoking behavior detection method based on temporal trajectory constraints according to any one of claims 1 to 4, characterized in that, The step of reporting an alarm to the tracking trajectory where smoking behavior is determined to exist includes: Maintain a reporting status identifier for each of the tracking trajectories, the reporting status identifier including a non-reported status and a reported status; When the tracking trajectory is determined to contain smoking behavior and the reporting status is not reported, an alarm is reported and the reporting status is updated to reported. When the tracking trajectory is determined to contain smoking behavior and the reporting status is marked as reported, no alarm is reported.

6. The smoking behavior detection method based on time-series trajectory constraints according to any one of claims 1 to 4, characterized in that, The method further includes: Maintain a mismatch count for each of the aforementioned tracking trajectories; During the cross-frame association process, the mismatch count is incremented for existing tracking trajectories that have not been successfully matched; When the mismatch count reaches a preset threshold, the corresponding tracking trajectory is deleted; For newly created tracking trajectories, based on appearance feature similarity and spatial position overlap, the newly created tracking trajectory is matched with deleted historical tracking trajectories; When the newly created tracking trajectory successfully matches the historical tracking trajectory, the trajectory information of the historical tracking trajectory is restored to the newly created tracking trajectory.

7. The smoking behavior detection method based on time-series trajectory constraints according to claim 1, characterized in that, The method further includes: Face detection is performed on the personnel area to which the candidate smoking target belongs to obtain face information; The facial information is associated with the corresponding tracking trajectory.

8. The smoking behavior detection method based on time-series trajectory constraints according to claim 7, characterized in that, The step of performing face detection on the personnel area to which the candidate smoking target belongs, and obtaining face information, includes: Face localization is performed on the personnel area to obtain face detection bounding boxes; Based on the comparison results between the height and / or width of the face detection box and the preset size threshold, face detection boxes that do not meet the size requirements are removed. Based on the comparison of the horizontal offset of the nose tip relative to the midpoint of the eyes with the preset offset threshold, face detection boxes that do not meet the side profile pose requirements are removed. Based on the comparison of the relative position ratio of the tip of the nose in the vertical direction of the eyes and mouth with the preset pitch range, face detection boxes that do not meet the pitch posture requirements are removed. Face information is obtained based on the face detection bounding boxes retained after filtering.

9. A smoking behavior detection system based on time-series trajectory constraints, characterized in that, It includes a host computer and a video acquisition device that is communicatively connected to the host computer, wherein: The video acquisition device is used to acquire surveillance video frames; The host computer is used to detect smoking behavior in the video frames and obtain candidate smoking targets; based on the similarity of appearance features and the overlap of spatial positions of the candidate smoking targets, it performs cross-frame association of the candidate smoking targets to establish a tracking trajectory; it confirms smoking behavior in the tracking trajectory across multiple frames, and when a preset confirmation condition is met, it determines that the person corresponding to the tracking trajectory has smoking behavior; and it performs alarm reporting for the tracking trajectory where smoking behavior is determined to exist.

10. An electronic device, characterized in that, include: A processor, a memory, and a communication bus, wherein the processor and the memory communicate with each other via the communication bus; The memory stores program instructions that can be executed by the processor, and the processor can execute the method as described in any one of claims 1 to 8 by calling the program instructions.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images