Smoking behavior detection method and device, equipment and storage medium
By extracting pose features from continuous video frames for temporal pattern matching and object detection in smoking behavior detection, the dynamic evolution of smoking behavior is captured. By adopting a collaborative mechanism of pose temporal chain and object dynamic verification, the problem of high false positive rate is solved, and high-precision smoking behavior detection is achieved.
Patent Information
- Application Number
- CN202610483465.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-13
- Publication Date
- 2026-05-15
AI Technical Summary
Existing smoking behavior detection technologies have a high false alarm rate in high-risk scenarios and public places, and cannot effectively capture the complete smoking behavior chain, resulting in a high false alarm rate.
By extracting pose features from consecutive video frames for temporal pattern matching and combining them with a target object detection model, the dynamic evolution of smoking behavior is captured. A collaborative mechanism of pose temporal chain and object dynamic verification is adopted to ensure the rigor of the behavioral logic in the time dimension and to perform double verification to reduce false judgments.
It significantly reduced the false positive rate of smoking behavior, achieved high-precision and low-false-positive smoking behavior detection, and provided reliable technical support for fire safety and public management.
Smart Images
Figure CN122049991A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image recognition, and in particular to a method, apparatus, device, and storage medium for detecting smoking behavior. Background Technology
[0002] Smoking behavior detection is an important application area in video analytics, particularly in high-risk scenarios such as chemical plants, gas stations, and warehouses for fire safety, and in the civilized and regulated management of public places. Recent technological trends have attempted to incorporate pose estimation, trying to aid in judgment by analyzing human movements. However, most existing fusion solutions are rather rigid: some only use pose to determine "whether the hand is near the mouth," ignoring whether there is a cigarette in the hand (leading to false alarms for eating or making phone calls); others, while detecting both people and cigarettes simultaneously, lack a rigorous temporal logical connection, merely superimposing "person detection boxes + cigarette detection boxes."
[0003] In conclusion, how to capture the complete smoking behavior chain to reduce the false positive rate of smoking behavior is an urgent problem to be solved. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide a smoking behavior detection method, apparatus, device, and storage medium capable of capturing the complete smoking behavior chain to reduce the false positive rate of smoking behavior detection. The specific solution is as follows: Firstly, this application discloses a method for detecting smoking behavior, including: The target video frame corresponding to the target event to be detected is obtained, and the pose features are extracted based on each target video frame to determine the target pose corresponding to each target video frame based on the pose features; the target video frame is a set of consecutive video frames, and the number of target video frames is not less than a preset number of frames; Based on the timestamps corresponding to each target video frame and the target pose, temporal pattern matching is performed on the target video frames to obtain the corresponding pattern matching results. When the pattern matching results indicate that the target event is a suspected smoking event, the target start frame and target end frame are determined from the target video frames. Based on the target pose, the video frame to be verified and the corresponding dynamic region of interest are determined from all the target video frames, and the target object detection model is used to perform object detection on the dynamic region of interest to obtain the corresponding target detection results. Based on the target start frame, the target end frame and the target detection results, the smoking behavior detection results corresponding to the target event are generated; the target detection results include cigarette detection results and handheld cigarette posture detection results.
[0005] Optionally, the step of extracting pose features based on each of the target video frames includes: Human pose estimation is performed based on the target video frame to determine target skeletal points; the target skeletal points include the shoulder, elbow, wrist, and head. Based on the target skeletal points, the pose features corresponding to the target video frame are extracted; the pose features include relative distance features and joint angle features; the relative distance features are the normalized distance from the wrist to the mouth and nose determined based on the torso length, and the joint angle features are the vector angle formed by the shoulder, elbow, and wrist.
[0006] Optionally, determining the target pose corresponding to each target video frame based on the pose features includes: The target pose corresponding to each target video frame is determined using a target state machine based on the pose features; the target state machine is a finite state machine constructed based on anatomical features.
[0007] Optionally, the target pose includes a natural pose, a triggered pose, and a transition pose; The natural posture includes a joint angle feature greater than a first angle threshold, the triggered posture includes a relative distance feature less than a first preset distance, and the transition posture includes a joint angle feature less than a second angle threshold and a relative distance feature greater than a second preset distance.
[0008] Optionally, the step of performing temporal pattern matching on the target video frames based on the timestamps corresponding to each target video frame and the target pose to obtain the corresponding pattern matching results includes: Based on the timestamp corresponding to each target video frame and the target pose, the target video frame is converted into a target character sequence; The target character sequence is subjected to temporal pattern matching based on a preset action pattern library. When the target character sequence matches a smoking action sequence in the preset action pattern library, a pattern matching result is generated that indicates the target event is a suspected smoking event.
[0009] Optionally, determining the video frame to be verified and its corresponding dynamic region of interest from all the target video frames based on the target pose includes: If the target pose corresponding to the target video frame is a trigger pose or a transition pose, then the target video frame and its adjacent video frames are determined as video frames to be verified. The dynamic region of interest corresponding to the video frame to be verified is determined based on the key point coordinates corresponding to the target skeleton points in the video frame to be verified.
[0010] Optionally, generating the smoking behavior detection result corresponding to the target event based on the target start frame, the target end frame, and the target detection result includes: Determine the target time window corresponding to the target action closed loop based on the target start frame and the target end frame; Based on the target detection results, the target cumulative value corresponding to the cigarette counter within the target time window is determined, and when the target cumulative value is greater than a preset cumulative value, a smoking behavior detection result representing the target event as a smoking event is generated.
[0011] Secondly, this application discloses a smoking behavior detection device, comprising: The pose determination module is used to acquire the target video frame corresponding to the target event to be detected, and extract pose features based on each target video frame to determine the target pose corresponding to each target video frame based on the pose features; the target video frame is a set of consecutive video frames, and the number of target video frames is not less than a preset number of frames; The video frame determination module is used to perform temporal pattern matching on the target video frames based on the timestamps corresponding to each target video frame and the target pose to obtain the corresponding pattern matching results, and when the pattern matching results indicate that the target event is a suspected smoking event, determine the target start frame and the target end frame from the target video frames. The detection result acquisition module is used to determine the video frame to be verified and the corresponding dynamic region of interest from all the target video frames based on the target pose, and to use the target object detection model to perform object detection on the dynamic region of interest to obtain the corresponding target detection results, so as to generate the smoking behavior detection results corresponding to the target event based on the target start frame, the target end frame and the target detection results; the target detection results include cigarette detection results and handheld cigarette posture detection results.
[0012] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned smoking behavior detection method.
[0013] Fourthly, this application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned smoking behavior detection method.
[0014] In this application, when detecting smoking behavior, target video frames corresponding to the target event to be detected are acquired, and pose features are extracted based on each target video frame to determine the target pose corresponding to each target video frame based on the pose features; the target video frames are a set of consecutive video frames, and the number of target video frames is not less than a preset number of frames; temporal pattern matching is performed on the target video frames based on the timestamps corresponding to each target video frame and the target pose to obtain the corresponding pattern matching results, and when the pattern matching results indicate that the target event is a suspected smoking event, the target start frame and the target end frame are determined from the target video frames; based on the target pose, the video frames to be verified and the corresponding dynamic regions of interest are determined from all the target video frames, and the target object detection model is used to perform object detection on the dynamic regions of interest to obtain the corresponding target detection results, so as to generate the smoking behavior detection results corresponding to the target event based on the target start frame, the target end frame and the target detection results; the target detection results include cigarette detection results and handheld cigarette posture detection results. As can be seen, this application captures the dynamic evolution of smoking behavior (such as the complete action sequence of the hand from taking a cigarette to smoking) by extracting the pose features of continuous target video frames (the number of which is not less than a preset number) and performing temporal pattern matching. This ensures the rigor of the behavioral logic in the temporal dimension, thereby significantly reducing false alarms caused by isolated frame analysis (effect derivation). After identifying suspected events through temporal matching, the region of interest (ROI) of the video frame to be verified is dynamically generated based on the target pose, and a target object detection model is introduced for fine-grained detection (including cigarette detection and handheld cigarette pose detection). This dual verification mechanism eliminates interference from non-smoking behaviors, reducing the false positive rate to a practical level. The dynamic ROI is adaptively adjusted according to pose changes, focusing on local areas of key frames (such as the interaction position between the hand and mouth), reducing the computational overhead of full-frame detection. This application achieves high-precision, low-false-positive detection of smoking behavior through a collaborative mechanism of "pose temporal chain + object dynamic verification," providing reliable technical support for fire safety and public management. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0016] Figure 1 This is a flowchart of a smoking behavior detection method disclosed in this application; Figure 2 This is a schematic diagram of a specific smoking behavior detection method disclosed in this application; Figure 3 This is a schematic block diagram illustrating the logical principle of mutual verification between action screening and object measurement disclosed in this application; Figure 4 This is a schematic diagram of the structure of a smoking behavior detection device disclosed in this application; Figure 5 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation
[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] Smoking behavior detection is an important application area in video analytics, particularly in high-risk scenarios such as chemical plants, gas stations, and warehouses for fire safety and in the civilized and regulated management of public places. Recent technological trends have attempted to incorporate human pose estimation, trying to aid in judgment by analyzing human movements. However, most existing fusion solutions are rather rigid: some only use pose to determine "whether the hand is near the mouth," ignoring whether there is a cigarette in the hand (leading to false alarms for eating or making phone calls); others, while detecting both people and cigarettes simultaneously, lack a rigorous temporal logical connection, merely superimposing "person detection boxes + cigarette detection boxes." To address these technical problems, this application discloses a smoking behavior detection method capable of capturing the complete smoking behavior chain to reduce the false alarm rate.
[0019] See Figure 1 As shown in the figure, an embodiment of the present invention discloses a method for detecting smoking behavior, including: Step S11: Obtain the target video frame corresponding to the target event to be detected, and extract pose features based on each target video frame to determine the target pose corresponding to each target video frame based on the pose features; the target video frame is a set of consecutive video frames, and the number of target video frames is not less than a preset number of frames.
[0020] In this embodiment, as Figure 2As shown, the smoking behavior detection system, after acquiring the target video frame corresponding to the target event to be detected, extracts the posture features corresponding to each target video frame, and then determines the target posture corresponding to each target video frame based on the posture features. Extracting posture features based on each target video frame includes: estimating human posture based on the target video frame to determine target skeletal points; the target skeletal points include shoulder, elbow, wrist, and head; extracting posture features corresponding to the target video frame based on the target skeletal points; the posture features include relative distance features and joint angle features; the relative distance feature is the normalized distance from the wrist to the mouth and nose determined based on the torso length, and the joint angle feature is the vector angle formed by the shoulder, elbow, and wrist. In a specific embodiment, determining the target posture corresponding to each target video frame based on the posture features includes: using a target state machine to determine the target posture corresponding to each target video frame based on the posture features; the target state machine is a finite state machine constructed based on anatomical features. The target posture includes a natural posture, a triggered posture, and a transitional posture; wherein, a natural posture includes joint angle features greater than a first angle threshold, a triggered posture includes relative distance features less than a first preset distance, and a transitional posture includes joint angle features less than a second angle threshold and relative distance features greater than a second preset distance. In other words, this embodiment first performs coarse-grained pose feature extraction and state modeling, including: estimating human pose from video frames, extracting key skeletal points such as shoulders, elbows, wrists, and heads, and constructing a finite state machine (FSM) based on anatomical features. Instead of directly using coordinates, it calculates rotationally invariant geometric features, such as using the normalized distance from the wrist to the nose (based on torso length) as a relative distance feature, and the angle between the shoulder-elbow-wrist vectors as a joint angle feature determining the arm bending angle. This divides continuous motion into discrete states, including a natural state where the arm hangs naturally (state A) or is slightly bent (state B), a triggered state where the wrist is extremely close to the mouth and nose area (suspected smoking core frame, corresponding to state D), and a transitional state where the arm is noticeably bent but not touching the face (state C). It is understood that the target video frames are a set of consecutive video frames, and the number of target video frames is not less than a preset number of frames.
[0021] Step S12: Perform temporal pattern matching on the target video frames based on the timestamps corresponding to each target video frame and the target pose to obtain the corresponding pattern matching results. When the pattern matching results indicate that the target event is a suspected smoking event, determine the target start frame and the target end frame from the target video frames.
[0022] In this embodiment, the smoking behavior detection system performs temporal pattern matching on target video frames based on the timestamps and target poses corresponding to each target video frame to obtain corresponding pattern matching results. This can include: converting target video frames into target character sequences based on the timestamps and target poses corresponding to each target video frame; performing temporal pattern matching on the target character sequences based on a preset action pattern library; and generating a pattern matching result characterizing the target event as a suspected smoking event when the target character sequence matches a smoking action sequence in the preset action pattern library. In other words, this embodiment utilizes an "action filter" based on temporal pattern matching to convert the pose states of consecutive frames into character sequences (e.g., ...AABBCDC...), and sets valid smoking action regular expressions or pattern libraries for temporal logic verification (e.g., Rising(B->C) -> Holding(D) -> Falling(C->B)). Only when a complete, biomechanically consistent "raise hand-inhale-put down" closed-loop sequence is detected is it determined to be a "suspected smoking action event," and the start and end frames of the event are recorded. This step can act as a coarse filter, effectively filtering out non-smoking actions such as yawning (no loop) and constantly resting one's chin on one's hand (no dynamic change). The state machine logic is clearly visible, making it easy to fine-tune the rules for specific scenarios (such as specific gestures that cause frequent false alarms in factories) without retraining the entire model.
[0023] Step S13: Based on the target pose, determine the video frame to be verified and the corresponding dynamic region of interest from all the target video frames, and use the target object detection model to perform object detection on the dynamic region of interest to obtain the corresponding target detection results, so as to generate the smoking behavior detection results corresponding to the target event based on the target start frame, the target end frame and the target detection results; the target detection results include cigarette detection results and handheld cigarette posture detection results.
[0024] In this embodiment, as Figure 3 As shown, the smoking behavior detection system also determines the video frame to be verified and its corresponding dynamic region of interest from all target video frames based on the target pose. Specifically, this includes: if the target pose corresponding to the target video frame is a trigger pose or a transition pose, then the target video frame and its adjacent video frames are determined as the video frames to be verified; the dynamic region of interest corresponding to the video frame to be verified is determined based on the key point coordinates corresponding to the target skeleton points in the video frame to be verified. Based on the target start frame, target end frame, and target detection results, a smoking behavior detection result corresponding to the target event is generated, including: determining the target time window corresponding to the target action loop based on the target start frame and target end frame; determining the target cumulative value corresponding to the cigarette counter within the target time window based on the target detection results; and generating a smoking behavior detection result representing the target event as a smoking event when the target cumulative value is greater than a preset cumulative value.
[0025] In one specific implementation, when the target state machine enters "State C" or "State D" and its neighboring frames, the system automatically calculates a local region encompassing the face and hand's range of motion based on the key point coordinates of the head and hands at that moment, achieving dynamic ROI (Region of Interest) cropping. Then, coordinate mapping is performed, transforming the global coordinate system into a local ROI coordinate system to eliminate background noise interference (such as lines on a background wall or distant objects). Next, a high-precision target object detection model is used to perform local enhancement detection on the cropped dynamic ROI, specifically identifying "cigarettes" or "cigarette-holding postures" to obtain corresponding target detection results, including cigarette detection results and cigarette-holding posture detection results. This cascade mechanism, where coarse-grained actions (postures) trigger fine-grained perception (object detection), activates local ROI detection only under specific action states, achieving an optimal balance between computational power and accuracy. Simultaneously, using posture guidance to generate local ROIs is equivalent to optically zooming the image, significantly improving the pixel ratio and detection clarity for small targets like cigarettes. Specifically, unlike single-frame determination, this embodiment maintains a "smoke counter" throughout the lifecycle of the action sequence to achieve dynamic accumulation of confidence. Simultaneously, this embodiment introduces an environmental evaluation factor (the mean and variance of the detection box confidence) to automatically lower the single-frame detection threshold in ambiguous environments, while relying on the accumulated frame count to ensure accuracy.
[0026] Understandably, the alarm triggering of the smoking behavior detection result corresponding to the target event must simultaneously satisfy both temporal and spatial empirical conditions. The temporal condition refers to the posture sequence matching module confirming the completion of a complete smoking action loop that conforms to the definition, thereby effectively filtering out static false alarms from long objects. The spatial empirical condition refers to the cumulative value of the "cigarette counter" within the ROI area exceeding a set threshold within the time window of the action loop, thereby effectively filtering out false actions without smoke, such as touching the mouth or biting the fingers. This detection logic, which uses "temporal action completeness" and "spatial object existence" as mutually necessary conditions, completely solves the two major traditional problems of "likes smoke but no action" (such as holding a pen without moving) and "likes action but no smoke" (such as eating fries or touching the nose) by requiring simultaneous matching of "action" and "object," distinguishing it from single object detection or action recognition.
[0027] As can be seen, this application captures the dynamic evolution of smoking behavior (such as the complete action sequence of the hand from taking a cigarette to smoking) by extracting the pose features of continuous target video frames (the number of which is not less than a preset number) and performing temporal pattern matching. This ensures the rigor of the behavioral logic in the temporal dimension, thereby significantly reducing false alarms caused by isolated frame analysis (effect derivation). After identifying suspected events through temporal matching, the region of interest (ROI) of the video frame to be verified is dynamically generated based on the target pose, and a target object detection model is introduced for fine-grained detection (including cigarette detection and handheld cigarette pose detection). This dual verification mechanism eliminates interference from non-smoking behaviors, reducing the false positive rate to a practical level. The dynamic ROI is adaptively adjusted according to pose changes, focusing on local areas of key frames (such as the interaction position between the hand and mouth), reducing the computational overhead of full-frame detection. This application achieves high-precision, low-false-positive detection of smoking behavior through a collaborative mechanism of "pose temporal chain + object dynamic verification," providing reliable technical support for fire safety and public management.
[0028] See Figure 4 As shown, this application discloses a smoking behavior detection device, comprising: The pose determination module 11 is used to acquire the target video frame corresponding to the target event to be detected, and extract pose features based on each target video frame to determine the target pose corresponding to each target video frame based on the pose features; the target video frame is a set of consecutive video frames, and the number of target video frames is not less than a preset number of frames; The video frame determination module 12 is used to perform temporal pattern matching on the target video frames based on the timestamps corresponding to each target video frame and the target pose to obtain the corresponding pattern matching results, and when the pattern matching results indicate that the target event is a suspected smoking event, determine the target start frame and the target end frame from the target video frames. The detection result acquisition module 13 is used to determine the video frame to be verified and the corresponding dynamic region of interest from all the target video frames based on the target posture, and to use the target object detection model to perform object detection on the dynamic region of interest to obtain the corresponding target detection results, so as to generate the smoking behavior detection results corresponding to the target event based on the target start frame, the target end frame and the target detection results; the target detection results include cigarette detection results and handheld cigarette posture detection results.
[0029] In one specific embodiment, the attitude determination module 11 may include: A skeleton point determination unit is used to perform human pose estimation based on the target video frame to determine target skeleton points; the target skeleton points include the shoulder, elbow, wrist, and head; The feature extraction unit is used to extract the pose features corresponding to the target video frame based on the target skeleton points; the pose features include relative distance features and joint angle features; the relative distance features are the normalized distance from the wrist to the mouth and nose determined based on the torso length, and the joint angle features are the vector angle formed by the shoulder, elbow, and wrist.
[0030] In one specific embodiment, the attitude determination module 11 may include: The pose determination unit is used to determine the target pose corresponding to each target video frame based on the pose features using a target state machine; the target state machine is a finite state machine constructed based on anatomical features.
[0031] In one specific embodiment, the video frame determination module 12 may include: A sequence acquisition unit is used to convert the target video frames into target character sequences based on the timestamps corresponding to each target video frame and the target pose. The sequence matching unit is used to perform temporal pattern matching on the target character sequence based on a preset action pattern library, and generate a pattern matching result that characterizes the target event as a suspected smoking event when the target character sequence matches a smoking action sequence in the preset action pattern library.
[0032] In one specific embodiment, the detection result acquisition module 13 may include: If the target pose corresponding to the target video frame is a trigger pose or a transition pose, then the target video frame and its adjacent video frames are determined as video frames to be verified. The dynamic region of interest corresponding to the video frame to be verified is determined based on the key point coordinates corresponding to the target skeleton points in the video frame to be verified.
[0033] In one specific embodiment, the detection result acquisition module 13 may include: Determine the target time window corresponding to the target action closed loop based on the target start frame and the target end frame; Based on the target detection results, the target cumulative value corresponding to the cigarette counter within the target time window is determined, and when the target cumulative value is greater than a preset cumulative value, a smoking behavior detection result representing the target event as a smoking event is generated.
[0034] Furthermore, embodiments of this application also disclose an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.
[0035] Figure 5This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the smoking behavior detection method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0036] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0037] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon can include an operating system 221, computer programs 222, etc., and the storage method can be temporary storage or permanent storage.
[0038] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the smoking behavior detection method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.
[0039] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned smoking behavior detection method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.
[0040] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0041] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0042] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0043] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0044] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for detecting smoking behavior, characterized in that, include: The target video frame corresponding to the target event to be detected is obtained, and the pose features are extracted based on each target video frame to determine the target pose corresponding to each target video frame based on the pose features; the target video frame is a set of consecutive video frames, and the number of target video frames is not less than a preset number of frames; Based on the timestamps corresponding to each target video frame and the target pose, temporal pattern matching is performed on the target video frames to obtain the corresponding pattern matching results. When the pattern matching results indicate that the target event is a suspected smoking event, the target start frame and target end frame are determined from the target video frames. Based on the target pose, the video frame to be verified and the corresponding dynamic region of interest are determined from all the target video frames. The target object detection model is used to perform object detection on the dynamic region of interest to obtain the corresponding target detection results. Based on the target start frame, the target end frame and the target detection results, the smoking behavior detection results corresponding to the target event are generated. The target detection results include cigarette detection results and cigarette handheld posture detection results. The step of performing temporal pattern matching on the target video frames based on the timestamps corresponding to each target video frame and the target pose to obtain the corresponding pattern matching results includes: Based on the timestamp corresponding to each target video frame and the target pose, the target video frame is converted into a target character sequence; The target character sequence is subjected to temporal pattern matching based on a preset action pattern library. When the target character sequence matches a smoking action sequence in the preset action pattern library, a pattern matching result is generated that indicates the target event is a suspected smoking event.
2. The smoking behavior detection method according to claim 1, characterized in that, The extraction of pose features based on each of the target video frames includes: Human pose estimation is performed based on the target video frame to determine target skeletal points; the target skeletal points include the shoulder, elbow, wrist, and head. Based on the target skeletal points, the pose features corresponding to the target video frame are extracted; the pose features include relative distance features and joint angle features; the relative distance features are the normalized distance from the wrist to the mouth and nose determined based on the torso length, and the joint angle features are the vector angle formed by the shoulder, elbow, and wrist.
3. The smoking behavior detection method according to claim 1, characterized in that, Determining the target pose corresponding to each target video frame based on the pose features includes: The target pose corresponding to each target video frame is determined using a target state machine based on the pose features; the target state machine is a finite state machine constructed based on anatomical features.
4. The smoking behavior detection method according to claim 1, characterized in that, The target pose includes a natural pose, a triggered pose, and a transitional pose; The natural posture includes a joint angle feature greater than a first angle threshold, the triggered posture includes a relative distance feature less than a first preset distance, and the transition posture includes a joint angle feature less than a second angle threshold and a relative distance feature greater than a second preset distance.
5. The smoking behavior detection method according to claim 4, characterized in that, The step of determining the video frame to be verified and its corresponding dynamic region of interest from all the target video frames based on the target pose includes: If the target pose corresponding to the target video frame is a trigger pose or a transition pose, then the target video frame and its adjacent video frames are determined as video frames to be verified. The dynamic region of interest corresponding to the video frame to be verified is determined based on the key point coordinates corresponding to the target skeleton points in the video frame to be verified.
6. The smoking behavior detection method according to any one of claims 1 to 5, characterized in that, The step of generating the smoking behavior detection result corresponding to the target event based on the target start frame, the target end frame, and the target detection result includes: Determine the target time window corresponding to the target action closed loop based on the target start frame and the target end frame; Based on the target detection results, the target cumulative value corresponding to the cigarette counter within the target time window is determined, and when the target cumulative value is greater than a preset cumulative value, a smoking behavior detection result representing the target event as a smoking event is generated.
7. A smoking behavior detection device, characterized in that, include: The pose determination module is used to acquire the target video frame corresponding to the target event to be detected, and extract pose features based on each target video frame to determine the target pose corresponding to each target video frame based on the pose features; the target video frame is a set of consecutive video frames, and the number of target video frames is not less than a preset number of frames; The video frame determination module is used to perform temporal pattern matching on the target video frames based on the timestamps corresponding to each target video frame and the target pose to obtain the corresponding pattern matching results, and when the pattern matching results indicate that the target event is a suspected smoking event, determine the target start frame and the target end frame from the target video frames. The detection result acquisition module is used to determine the video frame to be verified and the corresponding dynamic region of interest from all the target video frames based on the target pose, and to use the target object detection model to perform object detection on the dynamic region of interest to obtain the corresponding target detection result, so as to generate the smoking behavior detection result corresponding to the target event based on the target start frame, the target end frame and the target detection result; The target detection results include cigarette detection results and handheld cigarette posture detection results; The video frame determination module specifically includes: A sequence acquisition unit is used to convert the target video frames into target character sequences based on the timestamps corresponding to each target video frame and the target pose. The sequence matching unit is used to perform temporal pattern matching on the target character sequence based on a preset action pattern library, and generate a pattern matching result that characterizes the target event as a suspected smoking event when the target character sequence matches a smoking action sequence in the preset action pattern library.
8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the smoking behavior detection method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the smoking behavior detection method as described in any one of claims 1 to 6.