Smoking behavior detection method, device, equipment and medium

Abstract

The invention discloses a smoking behavior detection method, device and equipment and a medium, and relates to the technical field of image processing, and the smoking behavior detection method based on attitude estimation can significantly simplify the deployment process of a system. The method mainly depends on a computer vision technology, so that detection can be realized through a common camera, and complicated hardware equipment or a special sensor is not needed. The system can be easily integrated into an existing monitoring system, the environment does not need to be greatly improved, and the deployment difficulty and time are greatly reduced. Compared with the traditional smoking detection technology, such as a smoke sensor or a high-precision gas analyzer, the method has lower cost. The price of a computer vision algorithm and a common camera is far lower than that of high-end sensor equipment. By using the existing consumption-level hardware and an open-source software platform, the construction cost of the whole system can be effectively reduced, so that the smoking behavior detection technology can be more widely applied to various places.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a smoking behavior detection method, device, equipment and medium based on posture estimation. Background Art

[0002] In many places, smoking is prohibited in order to prevent the harm caused by smoking. However, many places now use real-time observation of surveillance videos by human eyes to determine whether there is smoking, which is inefficient and has the risk of untimely human detection. Therefore, a real-time alarm method for smoking behavior based on surveillance videos is needed.

[0003] With the development of computer vision technology, a series of algorithms have been proposed to detect smoking behavior. These methods can automatically process surveillance video data and can be roughly divided into two categories: one is the target detection method, which detects whether cigarette butts appear in the image to determine whether smoking occurs. Unfortunately, cigarette butts are small targets in surveillance videos, so the detection accuracy is not high and it is easy to make false detections. The other is the posture estimation method, which uses the position of key points of the body to determine whether there is a smoking action, thereby determining whether smoking occurs. In actual scenes, many actions are similar to smoking actions, which are prone to false judgments.

[0004] Patent document CN111507416A records a real-time detection method for smoking behavior based on deep learning. The patent provides a set of target detection processes, including data set construction, data augmentation, model construction, smoking judgment, etc. When cigarette butts are detected in multiple images, it is judged that there is smoking behavior. However, this method still cannot get rid of the defect of high false detection rate in complex scenes, and has high requirements for image resolution.

[0005] Patent document CN110674717A records a smoking monitoring system based on posture recognition. This patent extracts and recognizes the characteristic points of smokers in the collected image through an image acquisition module, an acquisition and division module, and a posture feature simulation module, and counts the distance between characteristic points in the smoking posture through a screening and positioning module, a posture correction module, a posture recognition and comparison module, and a posture evaluation and processing module. The smoking status of smokers can be monitored according to the smoking posture of smokers. However, judging the smoking frequency per unit time based on the distance between characteristic points can only meet the conditions in specific occasions, and is not robust in general scenarios.

[0006] Patent document CN114202646A records a method and system for infrared image smoking detection based on deep learning. The method discloses a method and system for infrared image smoking detection based on deep learning, including hardware environment, data collection, data set preprocessing and production, smoke point feature extraction, smoke point feature fusion, smoke point detection and smoke point alarm. The invention fuses visible light with infrared images, combines the smoking behavior detection method based on deep learning, and enriches the sample semantic features and positioning features through the FPN+PAN structure. The method focuses on detecting smoke points through infrared images, but in surveillance videos, smoke points account for a very small proportion of the image, which is prone to missed detection.

[0007] Patent document CN108629282A describes a smoking detection method, storage medium and computer. The method synthesizes smoking images based on face image data and merges them with non-smoking images into a smoking detection training set; labels the smoking images based on whether there is smoke in each image in the smoking detection training set; and trains a preset smoking detection neural network model using the smoking detection training set based on the labels until the model converges. This method is only applicable to high-definition face images, and the error in judging whether smoking is present by classification in surveillance video data is too large.

[0008] Patent document CN111797757A records a method and system for monitoring smoking behavior, which discloses a method for monitoring smoking behavior, including: obtaining a video image, identifying the facial features of the target person and the features of the cigarette according to the video image, and identifying the position of the mouth of the human body from the facial features, and judging whether one end of the cigarette is in the mouth of the human body, if so, there is smoking behavior. This method requires high-resolution images to accurately detect whether smoking is present, and the accuracy of judging the face in the monitoring video is not high, and the error of further judging the mouth and cigarette will be greater.

[0009] However, the solutions recorded in the current patent documents generally have the following disadvantages:

[0010] (1) Over-reliance on certain specific features or conditions, such as faces and cigarette positions, makes it difficult to ensure the reliability and stability of these features in actual complex scenarios.

[0011] (2) There is a high rate of false detection or missed detection, which cannot meet the requirements of detection accuracy in practical applications. The reasons for this are mainly due to the low accuracy of the trained model, the complexity of the detection scene, and the small size of the target.

[0012] (3) Lack of comprehensive consideration and adaptability to multiple complex environmental factors, and insufficient robustness. Summary of the invention

[0013] In view of the above problems, the present invention provides a smoking behavior detection method, device, equipment and medium for overcoming the above problems or at least partially solving the above problems.

[0014] The present invention provides the following scheme:

[0015] A smoking behavior detection method, comprising:

[0016] Acquire a video stream of the monitored area, and extract at least one frame of image from the video stream as an image to be tested;

[0017] Input the image to be tested into a posture estimation model, wherein the posture estimation model is used to predict the coordinates of a plurality of target key points, so as to obtain a plurality of key point part images corresponding to each of the target key points by cutting the coordinates of each of the target key points; the target key points include at least a nose, a left hand, and a right hand;

[0018] Inputting the plurality of key point images into a target detection model, the target detection model being used to output cigarette detection results corresponding to each of the target key points according to the plurality of key point images;

[0019] Determining that a cigarette exists in the cigarette detection result corresponding to any one of the target key points, then determining that smoking behavior exists in the image to be tested;

[0020] Determine whether to trigger an alarm based on the long-term strategy combined with the smoking behavior.

[0021] Preferably: the posture estimation model includes a deep convolutional neural network, which includes multiple convolutional layers and pooling layers for extracting features in the image, a fully connected layer for associating the extracted features with key point coordinates, residual connections, multi-scale feature fusion and an attention mechanism.

[0022] Preferably: the target detection model includes a CSPDarkNet53 backbone network; the target detection model uses a feature pyramid with an attention mechanism as a feature fusion network, and uses a multi-scale prediction head as the output structure of the model.

[0023] Preferably, the upper left and lower right coordinate points of the clipping are expressed by the following formula:

[0024]

[0025]

[0026] In the formula, x and y are the horizontal and vertical coordinates of the key points, respectively, and w and h are the width and height of the human body detection frame, respectively. Represents the clip function.

[0027] Preferably: a plurality of the key point images are combined into a tensor and then input into the target detection model.

[0028] Preferably: the long-term strategy includes extracting a target number of images of consecutive frames from the video stream as images to be tested, determining a measurement value between the number of images to be tested in which smoking behavior is detected and the target number, and determining whether to trigger an alarm according to a relationship between the measurement value and a preset ratio; the measurement value is obtained by the following calculation formula:

[0029]

[0030] Where: p represents the metric value, n represents the number of targets, τ i Represents the indicator function.

[0031] 7. The smoking behavior detection method according to claim 6, characterized in that the alarm decision mode is expressed by the following formula:

[0032]

[0033] Wherein: 0.85 means that 85% of the target number in all the images to be tested are detected to have smoking behavior.

[0034] A smoking behavior detection device, used to perform the above-mentioned smoking behavior detection method, the device comprising:

[0035] The image acquisition unit to be tested is used to acquire a video stream of the monitoring area and extract at least one frame of image from the video stream as the image to be tested;

[0036] A key point detection unit, used for inputting the image to be tested into a posture estimation model, wherein the posture estimation model is used for predicting the coordinates of a plurality of target key points, so as to obtain a plurality of key point part images corresponding to each of the target key points by cutting according to the coordinates of each of the target key points; the target key points include at least a nose, a left hand, and a right hand;

[0037] A target detection unit, used for inputting the plurality of key point images into a target detection model, wherein the target detection model is used for outputting cigarette detection results corresponding to each of the target key points according to the plurality of key point images;

[0038] a smoking behavior judgment unit, configured to determine that if a cigarette exists in the cigarette detection result corresponding to any one of the target key points, then determine that smoking behavior exists in the image to be tested;

[0039] The alarm determination unit is used to determine whether to trigger an alarm based on the long-term strategy in combination with the smoking behavior.

[0040] A smoking behavior detection device, the device comprising a processor and a memory:

[0041] The memory is used to store program code and transmit the program code to the processor;

[0042] The processor is used to execute the above-mentioned smoking behavior detection method according to the instructions in the program code.

[0043] A computer-readable storage medium is used to store program codes, and the program codes are used to execute the above-mentioned smoking behavior detection method.

[0044] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0045] The embodiments of the present application provide a smoking behavior detection method, device, equipment and medium. The smoking behavior detection method based on posture estimation can significantly simplify the deployment process of the system. Since the method mainly relies on computer vision technology, detection can be achieved through ordinary cameras without the need for complex hardware equipment or special sensors. This method can be easily integrated into the existing monitoring system without major changes to the environment, greatly reducing the difficulty and time of deployment.

[0046] Compared with traditional smoking detection technologies, such as smoke sensors or high-precision gas analyzers, this method has lower costs. Computer vision algorithms and ordinary cameras are much cheaper than high-end sensor devices. Using existing consumer-grade hardware and open source software platforms can effectively reduce the construction cost of the overall system, making smoking behavior detection technology more widely used in various places, including those with limited budgets.

[0047] Through advanced algorithms and high-speed processing capabilities, the method can quickly respond to and identify smoking actions, ensuring that feedback is given as soon as smoking behavior occurs. This real-time nature is crucial for maintaining a non-smoking environment in public places and implementing timely intervention, helping to improve the effectiveness and reliability of detection.

[0048] This method is not only suitable for factory scenarios, but can also be flexibly applied to other scenarios that require personnel monitoring and management, such as schools, communities, construction sites, etc., with good scalability and adaptability.

[0049] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 is a flow chart of a smoking behavior detection method provided by an embodiment of the present invention;

[0052] Figure 2 is a structural block diagram of a posture estimation model provided by an embodiment of the present invention;

[0053] Figure 3 is a structural block diagram of a target detection model provided by an embodiment of the present invention;

[0054] Figure 4 is a model training flow chart provided by an embodiment of the present invention;

[0055] Figure 5 is a flow chart of smoking behavior detection provided by an embodiment of the present invention;

[0056] Figure 6 is a schematic diagram of a smoking behavior detection device provided by an embodiment of the present invention;

[0057] Figure 7 It is a schematic diagram of a smoking behavior detection device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0059] See also Figure 1 , is a smoking behavior detection method provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0060] S101: Acquire a video stream of a monitored area, and extract at least one frame of image from the video stream as an image to be tested;

[0061] S102: Input the image to be tested into a posture estimation model, and the posture estimation model is used to predict the coordinates of several target key points, so as to obtain several key point part images corresponding to each target key point by cropping according to the coordinates of each target key point; the target key points include at least a nose, a left hand, and a right hand; in specific implementation, the embodiment of the present application can provide that the posture estimation model includes a deep convolutional neural network, and the deep convolutional neural network includes multiple convolutional layers and pooling layers for extracting features from the image, a fully connected layer for associating the extracted features with the key point coordinates, a residual connection, multi-scale feature fusion and an attention mechanism.

[0062] The upper left and lower right coordinate points of the clipping are expressed as follows:

[0063]

[0064]

[0065] In the formula, x and y are the horizontal and vertical coordinates of the key points, respectively, and w and h are the width and height of the human body detection frame, respectively. Represents the clip function.

[0066] S103: Inputting several of the key point part images into the target detection model, the target detection model is used to output the cigarette detection results corresponding to each of the target key points according to the several key point part images; in specific implementation, the embodiment of the present application can provide that the target detection model includes a CSPDarkNet53 backbone network; the target detection model uses a feature pyramid with an attention mechanism as a feature fusion network, and uses a multi-scale prediction head as the output structure of the model.

[0067] In order to reduce time consumption, the embodiment of the present application can also provide a method of merging several key point images into a tensor and then inputting the tensor into the target detection model.

[0068] S104: if it is determined that a cigarette exists in the cigarette detection result corresponding to any one of the target key points, it is determined that smoking behavior exists in the image to be tested;

[0069] S105: Determine whether to trigger an alarm based on the long-term strategy and the smoking behavior.

[0070] In order to further improve the accuracy of the system alarm, a long-term strategy is adopted. When implemented specifically, the embodiment of the present application may also provide that the long-term strategy includes extracting a target number of images of consecutive frames from the video stream as images to be tested, determining a measurement value between the number of images to be tested in which smoking behavior is detected and the target number, and determining whether to trigger an alarm based on the relationship between the measurement value and a preset ratio; the measurement value is obtained by the following formula:

[0071]

[0072] Where: p represents the metric value, n represents the number of targets, τ i Represents the indicator function.

[0073] Adopt a long-term strategy.

[0074] Furthermore, the alarm decision mode is expressed by the following formula:

[0075]

[0076] Wherein: 0.85 means that 85% of the target number in all the images to be tested are detected to have smoking behavior.

[0077] In order to improve the accuracy and robustness of smoking behavior detection, the smoking behavior detection method provided in this application combines the target detection method with the posture estimation method, which is an effective strategy. Combining the advantages of these two methods can form a comprehensive detection system to overcome the limitations of a single method.

[0078] The smoking behavior detection method based on posture estimation provided in the embodiment of the present application forms a complete technical process framework from building a model structure to organizing data sets for model training, and then to real-time automatic monitoring of surveillance videos to detect whether smoking behavior occurs.

[0079] Construct pose estimation and target detection models. These two models have the advantages of being lightweight and low-latency, and have both speed and accuracy in the inference stage. Construct a training data set, process surveillance videos and images crawled from the Internet according to certain rules, enrich the scene, and improve the robustness of the model. Use the pose estimation and target detection cascade method to first detect key points, then capture key point images and then perform cigarette butt detection, avoiding detection interference caused by complex backgrounds and further improving the accuracy of smoking behavior recognition. Adopt a long-term decision-making strategy to comprehensively judge the proportion of frames in which smoking behavior occurs in a certain number of frames to the total number of frames, avoiding the defect of false detection alarm of single-frame images.

[0080] The smoking behavior detection method provided in this application is introduced in detail below.

[0081] The embodiment of the present application provides a smoking behavior detection method based on posture estimation, which aims to detect smoking behavior in real time and accurately through video surveillance data and issue an alarm, thereby reducing the harm caused by smoking behavior in non-smoking areas.

[0082] S1: Build pose estimation and object detection networks.

[0083] S11 pose estimation uses a deep convolutional neural network (DCNN) to build a pose estimation model. The model contains multiple convolutional layers and pooling layers to extract features from the image. In the last part of the model, the extracted features are associated with the key point coordinates through a fully connected layer. At the same time, in order to improve the detection accuracy, residual connections, multi-scale feature fusion, and attention mechanisms are introduced to enable the model to better capture subtle changes in human pose. The pose estimation model structure is shown in Figure 1. Figure 2 shown.

[0084] S12: The target detection model uses CSPDarkNet53 as the backbone network. While maintaining the model depth and parameter volume, it greatly improves the feature extraction capability and can learn richer and more effective visual features, thereby significantly improving the detection accuracy. The feature pyramid with attention mechanism is used as the feature fusion network. This feature pyramid fusion mechanism enhances the model's perception of targets of different scales and improves the robustness of detection. Using a multi-scale prediction head as the output structure of the model, it is possible to perform target detection prediction on feature maps of different resolutions and can better adapt to targets of different sizes. The structure of the target detection model is as follows: Figure 3 shown.

[0085] S2: Build the dataset.

[0086] The performance of the smoking behavior target detection model determines the quality of the detection effect. Training a high-quality detection model is a key step in the entire process. First, organize the data set, download open source smoking pictures, and perform frame extraction and annotation on the videos you collected. Combine the two data sets in YOLO format; then configure the relevant training environment; and finally start training the model. See the training flow chart. Figure 4 .

[0087] S21: Acquire and crop the image.

[0088] 1. Get the factory surveillance video, use OpenCV to extract a picture every ten frames, and then filter out the pictures containing smoking.

[0089] 2. Download the open source smoking dataset to enrich smoking scenarios and expand the dataset.

[0090] 3. For the open source data set, crop the image containing the cigarette butt with a width and height 0.35 times that of the human detection frame centered on the annotation frame; for the images obtained by yourself, use LabelImg to mark the parts of the cigarette in the image, and then use the method of cropping the open source data set images to crop the labeled images.

[0091] 4. Remove low-quality images from the cropped images to avoid misleading the model and causing hallucinations during training.

[0092] S22: Preprocessing.

[0093] First, all images are expanded by flipping, mirroring, etc., then the images are scaled to 255, and finally each pixel is divided by 255 for normalization.

[0094] S23: Construct YOLO format integrated dataset.

[0095] The original annotation information is transformed according to the cropping rules, and the annotation confidence of the images of the two data sets is written into a txt file in the format of [cx y wh], and the training and test data sets are divided.

[0096] S24: Configure the environment and train.

[0097] The main environment configuration is python3.8, torch2.0, cuda12.2. The embodiment of the present application is to fine-tune the pre-trained model, and the fine-tuning strategy is: using a logarithmic learning rate, the backbone network learning rate is reduced to one tenth of the original learning rate, and the output head learning rate remains unchanged; the backbone network is frozen for the first 20 epochs and unfrozen for the next 50 epochs, and the output head updates the parameters normally during the entire training process.

[0098] The loss function is shown in formulas (1), (2), and (3):

[0099]

[0100]

[0101] Loss=Loss cls +γLoss reg (3)

[0102] Here, γ is the equilibrium parameter.

[0103] S3: Reasoning and decision making.

[0104] S31: Predict key points.

[0105] The preprocessed image is input into the posture estimation model, which outputs 17 key points and detection frames. Under normal circumstances, smoke appears in the nose, left and right hands, so the parts we focus on are also these three key points.

[0106] S32: Crop the key point image.

[0107] The three key points of the nose, left and right hands are obtained from the output of the posture estimation model. The upper left and lower right coordinate points of the crop can be expressed by formula (4):

[0108]

[0109] Among them, x and y are the horizontal and vertical coordinates of the key points, w and h are the width and height of the human body detection frame, respectively. Represents the clip function, and its value range is limited to [0,w].

[0110] The three key points can be cropped out by using the coordinates. The advantage of this cropping is that it avoids false detection caused by complex backgrounds during the detection process, and only detects where smoke may exist, thereby improving the accuracy of detection and reducing the false detection rate caused by the background.

[0111] S33: Object detection.

[0112] Through step S32, we obtain images of the nose, left and right hands of all people in a picture. The sizes of these images are scaled to 255, and all the images are merged into one tensor after preprocessing, which is sent to the target detection model at one time to avoid the time-consuming reasoning caused by outputting images one by one.

[0113] S34: Decision making.

[0114] The target detection model will output the detection results of the nose and left and right hand images. We set the filtering threshold to 0.6. Detection frames smaller than the threshold will be ignored. It is considered that as long as smoke is detected in a key point image, smoking behavior exists in the original image of the frame.

[0115] In order to ensure the accuracy of the alarm, a long-term strategy is adopted. The process is as follows Figure 5 The measurement method is shown in formula (5), and the decision method is shown in formula (6). If smoking behavior is detected in 85% of the images in 30 consecutive surveillance images, it is considered that someone is smoking and an alarm is triggered. This strategy effectively avoids the defect of false detection alarm of single-frame images.

[0116]

[0117] where τ i is an indicator function, which is 1 if smoking behavior is detected in the i-th frame image, otherwise it is 0.

[0118]

[0119] The method provided in the embodiment of the present application can quickly and accurately infer during video surveillance by constructing a lightweight posture estimation and target detection model, providing a basis for the real-time nature of the process. Build a more sophisticated data set, and crop the three possible parts of the nose, left and right hands according to the predicted key points according to the rules, focus on the detection of these three parts, and improve the focus of the model's targeted detection. Image tensor parallelism reduces the reasoning time of the detection process and avoids reasoning for a single image one by one. Using a cascade method, the entire process is gradually refined, from the entire surveillance video image to the three key point images, to improve the accuracy of smoking behavior detection, and through a long-term decision-making strategy, a complete closed-loop process is formed.

[0120] In summary, the smoking behavior detection method provided by this application, the smoking behavior detection method based on posture estimation, can significantly simplify the deployment process of the system. Since this method mainly relies on computer vision technology, detection can be achieved through ordinary cameras without the need for complex hardware equipment or dedicated sensors. This method can be easily integrated into existing monitoring systems without major modifications to the environment, greatly reducing the difficulty and time of deployment.

[0121] Compared with traditional smoking detection technologies, such as smoke sensors or high-precision gas analyzers, posture estimation-based methods have lower costs. Computer vision algorithms and ordinary cameras are much cheaper than high-end sensor devices. Using existing consumer-grade hardware and open source software platforms can effectively reduce the construction cost of the overall system, making smoking behavior detection technology more widely used in various places, including those with limited budgets.

[0122] Through advanced algorithms and high-speed processing capabilities, the method can quickly respond to and identify smoking actions, ensuring that feedback is given as soon as smoking behavior occurs. This real-time nature is crucial for maintaining a non-smoking environment in public places and implementing timely intervention, helping to improve the effectiveness and reliability of detection.

[0123] This method is not only suitable for factory scenarios, but can also be flexibly applied to other scenarios that require personnel monitoring and management, such as schools, communities, construction sites, etc., with good scalability and adaptability.

[0124] See also Figure 6 , the embodiment of the present application can also provide a smoking behavior detection device, such as Figure 6 As shown, the device for executing the above-mentioned smoking behavior detection method may include:

[0125] The image acquisition unit 601 is used to acquire a video stream of the monitoring area and extract at least one frame of image from the video stream as the image to be tested;

[0126] A key point detection unit 602 is used to input the image to be tested into a posture estimation model, and the posture estimation model is used to predict the coordinates of a plurality of target key points, so as to obtain a plurality of key point part images corresponding to each of the target key points by cutting according to the coordinates of each of the target key points; the target key points include at least a nose, a left hand, and a right hand;

[0127] The target detection unit 603 is used to input the key point images into a target detection model, and the target detection model is used to output the cigarette detection results corresponding to each target key point according to the key point images;

[0128] A smoking behavior judgment unit 604 is used to determine that if a cigarette exists in the cigarette detection result corresponding to any one of the target key points, then determine that smoking behavior exists in the image to be tested;

[0129] The alarm determination unit 605 is used to determine whether to trigger an alarm based on the long-term strategy and the smoking behavior.

[0130] The embodiment of the present application may also provide a smoking behavior detection device, the device comprising a processor and a memory:

[0131] The memory is used to store program code and transmit the program code to the processor;

[0132] The processor is used to execute the steps of the above-mentioned smoking behavior detection method according to the instructions in the program code.

[0133] like Figure 7 As shown, a smoking behavior detection device provided in an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 communicate with each other through the communication bus 13.

[0134] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0135] The processor 10 may call a program stored in the memory 11 . Specifically, the processor 10 may execute the operations in the embodiment of the smoking behavior detection method.

[0136] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0137] Acquire a video stream of the monitored area, and extract at least one frame of image from the video stream as an image to be tested;

[0138] Input the image to be tested into a posture estimation model, wherein the posture estimation model is used to predict the coordinates of a plurality of target key points, so as to obtain a plurality of key point part images corresponding to each of the target key points by cutting the coordinates of each of the target key points; the target key points include at least a nose, a left hand, and a right hand;

[0139] Inputting the plurality of key point images into a target detection model, the target detection model being used to output cigarette detection results corresponding to each of the target key points according to the plurality of key point images;

[0140] Determining that a cigarette exists in the cigarette detection result corresponding to any one of the target key points, then determining that smoking behavior exists in the image to be tested;

[0141] Determine whether to trigger an alarm based on the long-term strategy combined with the smoking behavior.

[0142] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required for at least one function (such as a file creation function, a data reading and writing function), etc.; the data storage area can store data created during use, such as initialization data, etc.

[0143] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0144] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0145] Of course, it should be noted that Figure 7 The structure shown does not constitute a limitation on the smoking behavior detection device in the embodiment of the present application. In actual applications, the smoking behavior detection device may include Figure 7 More or fewer components than shown, or combinations of certain components.

[0146] The embodiment of the present application may also provide a computer-readable storage medium, wherein the computer-readable storage medium is used to store program codes, and the program codes are used to execute the steps of the above-mentioned smoking behavior detection method.

[0147] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0148] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0149] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0150] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A smoking behavior detection method, characterized in that: include: Acquire a video stream of the monitored area, and extract at least one frame of image from the video stream as an image to be tested; Input the image to be tested into a posture estimation model, wherein the posture estimation model is used to predict the coordinates of a plurality of target key points, so as to obtain a plurality of key point part images corresponding to each of the target key points by cutting the coordinates of each of the target key points; the target key points include at least a nose, a left hand, and a right hand; Inputting the plurality of key point images into a target detection model, the target detection model being used to output cigarette detection results corresponding to each of the target key points according to the plurality of key point images; Determining that a cigarette exists in the cigarette detection result corresponding to any one of the target key points, then determining that smoking behavior exists in the image to be tested; Determine whether to trigger an alarm based on the long-term strategy combined with the smoking behavior.

2. The smoking behavior detection method according to claim 1, characterized in that: The posture estimation model includes a deep convolutional neural network, which includes multiple convolutional layers and pooling layers for extracting features in an image, a fully connected layer for associating the extracted features with key point coordinates, residual connections, multi-scale feature fusion and an attention mechanism.

3. The smoking behavior detection method according to claim 1, characterized in that: The target detection model includes a CSPDarkNet53 backbone network; the target detection model adopts a feature pyramid with an attention mechanism as a feature fusion network, and uses a multi-scale prediction head as the output structure of the model.

4. The smoking behavior detection method according to claim 1, characterized in that: The upper left and lower right coordinate points of the clipping are expressed as follows: In the formula, x and y are the horizontal and vertical coordinates of the key points, respectively, and w and h are the width and height of the human body detection frame, respectively. Represents the clip function.

5. The smoking behavior detection method according to claim 1, characterized in that: The images of several key point parts are combined into a tensor and then input into the target detection model.

6. The smoking behavior detection method according to claim 1, characterized in that: The long-term strategy includes extracting a target number of images of consecutive frames from the video stream as images to be tested, determining a measurement value between the number of images to be tested in which smoking behavior is detected and the target number, and determining whether to trigger an alarm based on the relationship between the measurement value and a preset ratio; the measurement value is obtained by the following calculation formula: Where: p represents the metric value, n represents the number of targets, τ i Represents the indicator function.

7. The smoking behavior detection method according to claim 6, characterized in that: The alarm decision mode is expressed by the following formula: Wherein: 0.85 means that 85% of the target number in all the images to be tested are detected to have smoking behavior.

8. A smoking behavior detection device, characterized in that: Used to perform the smoking behavior detection method according to any one of claims 1 to 7, the device comprises: The image acquisition unit to be tested is used to acquire a video stream of the monitoring area and extract at least one frame of image from the video stream as the image to be tested; A key point detection unit, used for inputting the image to be tested into a posture estimation model, wherein the posture estimation model is used for predicting the coordinates of a plurality of target key points, so as to obtain a plurality of key point part images corresponding to each of the target key points by cutting according to the coordinates of each of the target key points; the target key points include at least a nose, a left hand, and a right hand; A target detection unit, used for inputting the plurality of key point images into a target detection model, wherein the target detection model is used for outputting cigarette detection results corresponding to each of the target key points according to the plurality of key point images; a smoking behavior judgment unit, configured to determine that if a cigarette exists in the cigarette detection result corresponding to any one of the target key points, then determine that smoking behavior exists in the image to be tested; The alarm determination unit is used to determine whether to trigger an alarm based on the long-term strategy and the smoking behavior.

9. A smoking behavior detection device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the smoking behavior detection method according to any one of claims 1 to 7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the smoking behavior detection method according to any one of claims 1 to 7.

Images